Sunday, March 10, 2019

Rust for linear algebra and neural networks

From time to time I do a little bit of fooling around with algorithms and AI, I suppose it's a break from what I do in work, which is more about building business applications. I don't have a real goal in mind, I don't think I'm going to stumble about an algorithm for general intelligence or something, but I like to understand how things work at a low level.

So I've writing a bit of Rust code to implement some linear algebra (some matrix and vector operations) and use that to build some small neural networks. You can find the current state of that code here. Of course, if you really want to use production ready code for linear algebra you should check https://www.rustsim.org/ and if neural networks and AI in Rust are what you're after please check https://github.com/AtheMathmo/rusty-machine. Note that rusty-machine is still in a state of flux since it's using its own linear algebra library but there's talk about using nalgebra from rustsim hopefully that will bring together all these initiatives in one unified Rust platform for numeric operations and AI.

I have to say I'm still impressed with Rust. I like that you can do a lot of things with a functional approach, while still being able to update variable values or do other imperative operations when it makes sense. It does feel a lot like "pragmatic Haskell" at times. So far I haven't struggled with the borrow checker, but the code I've written so far is fairly simple and self-contained.

As always, I'm still wondering if there are some projects that it would make sense for me to contribute too. I'm certainly not a AI or math expert, but if you have a Rust project that could do with some contributions, let me know!

Happy Rust Hacking!

Sunday, February 10, 2019

Rust helped me escape Cthulhu!

I just downloaded the game Lovecraft Quest on my Android tablet, and was enjoying the puzzles, until I got to the final one, and that got me, ahem, *frustrated*. I found a walk-through where the reviewers admitted to more or less clicking randomly on the screen for one hour until it worked:

I actually managed to solve the puzzle the first time quickly with sheer luck, but on the second run I was stumped. So I followed the only reasonable course of action: let the computer solve the puzzle for me!

The puzzle is as follows: there is a square (4 rows, 4 columns) of gold bars that can be either in horizontal or vertical position. You can make one bar pivot, but when you do that all the bars in the same row and column pivot too! You need to get all bars to the vertical position for the door to open to reveal... Cthulhu itself!

So let us use Rust for this puzzle, and see if a brute force approach can work. 
Let's first define a few types. A grid is a 4x4 array of booleans and a position is a tuple. We'll have positions ranging from (0,0) to (3,3). A grid is solved is all elements are true (so true means vertical position for a bar).
Then let's implement the swapping operation. We'll take one Grid reference and return a new Grid with the proper bars swapped.
Note that we swap again the position we chose since we swapped it twice already, one for the columns and once for the rows. Not very elegant maybe but simple

The main solving function is as follows: we're given a grid and we return a list of positions to swap to solve the puzzle. If the grid is already solved we return an empty vec. Otherwise we keep two structures: a map of grids to a vector of positions (showing how we got to that particular grid from the start grid) and a list of grids to process. We use a Deque here so we can put new grids at the end but process grids from the start, to do a breadth-first traversal of all possible grids.
The try_all function will do all possible swaps on the given grid and store all the results with the path used to get to them. We store for all grids the shortest path we used to get to it. New grids get added at the end of the todo list. We return the path if we end up with a winning grid.
To simplify calling the program, we'll just pass a string made of 16 characters, with 1s and 0s indicating the position of the bars we see on the screen. Parsing this string into a Grid instance is easy enough with the help of chunks():
And we can pass then that string as a argument to our binary program, and print the result:
In my case, the program ran and gave me the solution: only 12 moves!

This Rust code is probably fairly naive in places, since I only have a couple of weeks of evenings learning it. I'm not too happy with the number of clone() calls I had to add to make the borrow checker happy, on top of the ones I thought I needed anyway. Maybe immutable structures would in fact be easier to use here! Of course this code could be made parallel but since it gave me my answer in a few seconds I didn't need to improve it further.

Happy Rust Hacking!


Sunday, February 03, 2019

Pedal to the Metal: starting with Rust


After hearing for years about Rust, I finally decided to give it a go. This is my journey so far

I'm currently using Visual Studio Code with the Rustextension. It's sufficient for the little exercises I've been doing so far, but I notices that code completion is not very accurate and does not seem to give me everything that's possible.  I've installed the IntelliJ Rust extension in my community IDEA, I'll see how well it works when I start doing bigger projects.

Rust is obviously a different kettle of fish than the last language I picked up, which was Go. It's a lot more complex and rich and hopefully powerful, and consequently takes a lot more time to master. It feels a bit like a cross between C for the low level aspects and Haskell for the structural and functional parts. I was a bit surprised straight away to read about variable shadowing since it seems to take away the security somehow (you cannot mutate a variable but you can redefine it), but from what I read it doesn't seem to be an issue practically since the compiler holds your hand at all time.

Most languages I've worked with have a Garbage Collector, so I'd say dealing with the borrow checker is going to be fun in larger code bases, but so far in the small exercises I've done it hasn't been an issue.

Hopefully I can get productive with Rust quickly, I'd like to contribute to projects like Rusty-Machine to do neural networks and ML with Rust, which would seem a perfect match with its performance and safety. We'll see!

Happy Rust Hacking!


Saturday, January 12, 2019

A little Go project: Cassandra + Elastic + GraphQL


I've put a littleproject on Github written in Go. It's a little database system that allows writing and reading objects, and do GraphQL queries on them. The main principle is that objects have a arbitrary String ID that is like a path, separated by slashes, and deleting an item with a given ID deletes also items with "children" ids.

For fun, I used Cassandra as the main data store, since it's optimized for fast writes and loads of data; we never delete anything, we just keep every state of an object as history. Text searches are done on the last version of objects using Elastic. GraphQL support allows to follow object namespace relationships.


I've used the following main libraries:

I also provide Dockerfiles to deploy either just the database systems for testing, or the whole application.

I'm not sure this can be useful as is for anybody, but this was for me to try Go on a real-life project. Apart from the gotchas I've wrote about before, it was a rather painless and enjoyable experience.

Happy Go Hacking!

Saturday, January 05, 2019

Go Gotchas

As I wrote my first non trivial project in Go, I had a few head-scratching moments. Nothing that a quick search could clear up, but worth keeping in mind...

Nil is not nil. 

You may run into that one quickly enough in tests. An instance of an interface can have a nil value but since it carries the interface type information, it does not equal to nil!! A lot has been written about it so make sure to read about it!

Redeclarations

The fact that you can use := to declare variables easily is a double edged sword, as declaring a variable with the same name as a previously declared one is allowed if it's in a different scope, and it shadows the original variable. So:



This code will only print True at the end. The "err" variable inside the if block has been redefined so even though it's not nil inside the if block, it IS nil once you exit it! So if you want to chain several operations that may return errors and then at the end handle whatever error occurred in a consistent way, you have to be careful.

Closures

This one gave more some Javascript PTSD:



This prints :
4
4
4

Which was not what I expected! One easy fix is to add a line saying i:=i before creating the func in the first loop, which by virtue of the previous gotcha redeclares a variable in the scope of the for loop body.


Errors

I'm not convinced in the end by the standard error reporting in Go. A lot has been written on that topic too. I feel that the code is peppered with error handling that gets in the way of readability and implementing cleanly the business logic. I'm sure there are some design tricks you can use, but you shouldn't have to resort to tricks to do clean error handling in a language, because at the end of the day it's how you handle errors that make up the quality of your software, not only how you handle the happy path.

These things you can watch out for and probably code around pretty easily, but they do impact a bit the ease of use of Go, even though I still think it's quite a simple elegant language to quickly be productive in.

Happy Go Hacking!

Saturday, December 15, 2018

1, 2, 3, Go!

OK, the title is an easy joke, but that's what happens when you give a silly name to a language… So I've started looking at Golang, since it seems to be the newish (ok, ok, not that new) kid on the block. And in fact the name is not an issue: I usually type "golang" to prefix my searches on the web and it works pretty well.

- I installed Go and the first striking thing are the instructions to read "how to write Go code". Having one way to structure the code (one workspace containing source control repositories, each containing packages) makes it very easy to get started - compared say to Java where different IDEs may have different approaches to what a "project" is. I wonder how this structure works when you write Go professionally, and you want to distinguish open sources dependencies from proprietary code. Also coming from Java where you're used to structure classes into packages, seeing Go repositories with a huge number of files inside one folder makes me shudder, but we'll see over time.
- I started of course with the Tour of Go. In a couple of evenings I went through it, proving how easy and small the base language is. Encouraging!
- I then started to get my hands dirty by writing a little app that hooks together JSON web services, Cassandra, Elastic Search and Docker. It was fairly easy to find libraries and manage to put them to use. I'll probably push that code to Github at some stage after some cleaning up.

So Go delivers on the promise to be a language easy to pick up and easy to get started. You can get from zero to productive in a few hours.

These are the features that stood up for me:

- Lack of generics. It's a trope in the computing world "Go doesn't have generics, ha ha ha". Of course straight away you see that arrays/slices and maps are typed, so already the basic use case for generics (safety in collections) is taken care of. Then you have functions as first-class citizens and interfaces, so there are plenty of techniques you can use to go around the lack of generics, so I'm not sure it's such a huge problem.
- Interfaces. It's interesting that there is no explicit declaration that a type implements an interface. If you define a Duck interface with a Quack method, every type that you use as the receiver for an implementation of a Quack function is considered a Duck. This is nice, but still tooling support to find out all types that implement a given interface will be a must ("OK, this function expects anything implementing Duck, what implementation can I use?").
- Pointers. I'm at the core a Java developer, so the idea of pointers makes me a bit uneasy. But I breathed easier when I saw "no pointer arithmetic". In fact, pointers in Go just make explicit if you pass a struct by value or by reference. I feel this adds a level of clarity that is worth the extra syntax and the initial repulsion the word "pointer" may awake in some.
- Errors. Go recommends a style where a function that may fail should return a tuple: normal value, error. The calling code can decide to ignore the error or deal with it. This is reminiscent of Either in Haskell. I'm still not sold that this is a better system than checked exceptions (but I seem to be the only one liking checked exceptions it seems), but I'll need more practice before I make my mind up.

So all in all I like a lot of design decisions that were made in Go, and it certainly makes for a pleasant programming experience. I ran into a few gotchas that I'll cover maybe in later posts, but nothing too stressful. So far, I have to say I enjoyed the experience!

Happy Go Hacking!

Saturday, December 08, 2018

A little RPG in React Native


I built a little role-playing game in React Native. I tried to build the type of game I like playing, with quests and interactions with non-playing characters, while learning some mobile UI programming tricks.  These are some of the main points that I took away:

  • The initial setup was a bit involved, probably because I'm not used to develop JavaScript applications. Getting the right code to properly do the plumbing between React, Redux,  React Navigation, tests, etc. was a bit of a hassle. Obviously this has to be done once and then you can concentrate on writing the functional code. As I noted recently, then upgrading to newer versions of your dependencies can be a hassle!
  • I was happy with my declarative approach to the game world. I don't think it would be manageable to do otherwise.  I define as JavaScript structures the world itself: the different places and how they communicate, the items they contain, etc., the non-playing characters and the possible interactions you can have with them, items and their characteristics, monsters and spells. This gives me an immutable representation of the start world, and the state of an actual game just has to store the modifications from that initial state
  • This of course could mean that I (or somebody else) could reuse the game mechanics for a totally different adventure!
  • I of course used of Redux to manage state. The state is only modified by Redux reducers, so everything the player does in the UI just becomes actions, not explicit state management operations. I combine several reducers, each reducer being responsible for modifying one section of the global state, which makes each reducer simpler, but fragment the code. For example when the player takes an item from a room, one reducer adds the item to the inventory, another removes the item from the room, so maybe there's some potential for bugs there
  • I found the integration between React and Redux easy to use: you simply define the mapping between the state and components properties, and between component interactions and Redux actions.
  • I abused a little bit Redux to implement combat: combat holds a kind a mini state for each fighting round, and then the result of a fighting round both is represented in the UI ("the rats bite you for 2 damage"), and impacts the global world.
  • I didn't spend too much time on the UI, just using basic React Native components. I found some of the layout hard to do correctly, and styling components is not intuitive when you're not a CSS expert. Maybe I should take a CSS/Design course, because you can tell from my UI designs that I'm a programmer :-)
  • All in all, writing in modern JavaScript was nicer than expected, even though I think using a stronger typed language would be better. If I had to do another similar project I would probably try to use Typescript. I liked especially the object spreads to update parts of structures and the use of consts.
  • I used Expo to test on a tablet, I didn't try to actually generate a final application outside of Expo, I haven't looked at all on how to distribute the game as an app.

All in all, if was an enjoyable experience, but of course designing a full game instead of just a prologue you can do in 5 minutes would be a different type of endeavor...