Sunday, March 10, 2019

Rust for linear algebra and neural networks

From time to time I do a little bit of fooling around with algorithms and AI, I suppose it's a break from what I do in work, which is more about building business applications. I don't have a real goal in mind, I don't think I'm going to stumble about an algorithm for general intelligence or something, but I like to understand how things work at a low level.

So I've writing a bit of Rust code to implement some linear algebra (some matrix and vector operations) and use that to build some small neural networks. You can find the current state of that code here. Of course, if you really want to use production ready code for linear algebra you should check https://www.rustsim.org/ and if neural networks and AI in Rust are what you're after please check https://github.com/AtheMathmo/rusty-machine. Note that rusty-machine is still in a state of flux since it's using its own linear algebra library but there's talk about using nalgebra from rustsim hopefully that will bring together all these initiatives in one unified Rust platform for numeric operations and AI.

I have to say I'm still impressed with Rust. I like that you can do a lot of things with a functional approach, while still being able to update variable values or do other imperative operations when it makes sense. It does feel a lot like "pragmatic Haskell" at times. So far I haven't struggled with the borrow checker, but the code I've written so far is fairly simple and self-contained.

As always, I'm still wondering if there are some projects that it would make sense for me to contribute too. I'm certainly not a AI or math expert, but if you have a Rust project that could do with some contributions, let me know!

Happy Rust Hacking!

Sunday, February 10, 2019

Rust helped me escape Cthulhu!

I just downloaded the game Lovecraft Quest on my Android tablet, and was enjoying the puzzles, until I got to the final one, and that got me, ahem, *frustrated*. I found a walk-through where the reviewers admitted to more or less clicking randomly on the screen for one hour until it worked:

I actually managed to solve the puzzle the first time quickly with sheer luck, but on the second run I was stumped. So I followed the only reasonable course of action: let the computer solve the puzzle for me!

The puzzle is as follows: there is a square (4 rows, 4 columns) of gold bars that can be either in horizontal or vertical position. You can make one bar pivot, but when you do that all the bars in the same row and column pivot too! You need to get all bars to the vertical position for the door to open to reveal... Cthulhu itself!

So let us use Rust for this puzzle, and see if a brute force approach can work. 
Let's first define a few types. A grid is a 4x4 array of booleans and a position is a tuple. We'll have positions ranging from (0,0) to (3,3). A grid is solved is all elements are true (so true means vertical position for a bar).
Then let's implement the swapping operation. We'll take one Grid reference and return a new Grid with the proper bars swapped.
Note that we swap again the position we chose since we swapped it twice already, one for the columns and once for the rows. Not very elegant maybe but simple

The main solving function is as follows: we're given a grid and we return a list of positions to swap to solve the puzzle. If the grid is already solved we return an empty vec. Otherwise we keep two structures: a map of grids to a vector of positions (showing how we got to that particular grid from the start grid) and a list of grids to process. We use a Deque here so we can put new grids at the end but process grids from the start, to do a breadth-first traversal of all possible grids.
The try_all function will do all possible swaps on the given grid and store all the results with the path used to get to them. We store for all grids the shortest path we used to get to it. New grids get added at the end of the todo list. We return the path if we end up with a winning grid.
To simplify calling the program, we'll just pass a string made of 16 characters, with 1s and 0s indicating the position of the bars we see on the screen. Parsing this string into a Grid instance is easy enough with the help of chunks():
And we can pass then that string as a argument to our binary program, and print the result:
In my case, the program ran and gave me the solution: only 12 moves!

This Rust code is probably fairly naive in places, since I only have a couple of weeks of evenings learning it. I'm not too happy with the number of clone() calls I had to add to make the borrow checker happy, on top of the ones I thought I needed anyway. Maybe immutable structures would in fact be easier to use here! Of course this code could be made parallel but since it gave me my answer in a few seconds I didn't need to improve it further.

Happy Rust Hacking!


Sunday, February 03, 2019

Pedal to the Metal: starting with Rust


After hearing for years about Rust, I finally decided to give it a go. This is my journey so far

I'm currently using Visual Studio Code with the Rustextension. It's sufficient for the little exercises I've been doing so far, but I notices that code completion is not very accurate and does not seem to give me everything that's possible.  I've installed the IntelliJ Rust extension in my community IDEA, I'll see how well it works when I start doing bigger projects.

Rust is obviously a different kettle of fish than the last language I picked up, which was Go. It's a lot more complex and rich and hopefully powerful, and consequently takes a lot more time to master. It feels a bit like a cross between C for the low level aspects and Haskell for the structural and functional parts. I was a bit surprised straight away to read about variable shadowing since it seems to take away the security somehow (you cannot mutate a variable but you can redefine it), but from what I read it doesn't seem to be an issue practically since the compiler holds your hand at all time.

Most languages I've worked with have a Garbage Collector, so I'd say dealing with the borrow checker is going to be fun in larger code bases, but so far in the small exercises I've done it hasn't been an issue.

Hopefully I can get productive with Rust quickly, I'd like to contribute to projects like Rusty-Machine to do neural networks and ML with Rust, which would seem a perfect match with its performance and safety. We'll see!

Happy Rust Hacking!


Saturday, January 12, 2019

A little Go project: Cassandra + Elastic + GraphQL


I've put a littleproject on Github written in Go. It's a little database system that allows writing and reading objects, and do GraphQL queries on them. The main principle is that objects have a arbitrary String ID that is like a path, separated by slashes, and deleting an item with a given ID deletes also items with "children" ids.

For fun, I used Cassandra as the main data store, since it's optimized for fast writes and loads of data; we never delete anything, we just keep every state of an object as history. Text searches are done on the last version of objects using Elastic. GraphQL support allows to follow object namespace relationships.


I've used the following main libraries:

I also provide Dockerfiles to deploy either just the database systems for testing, or the whole application.

I'm not sure this can be useful as is for anybody, but this was for me to try Go on a real-life project. Apart from the gotchas I've wrote about before, it was a rather painless and enjoyable experience.

Happy Go Hacking!

Saturday, January 05, 2019

Go Gotchas

As I wrote my first non trivial project in Go, I had a few head-scratching moments. Nothing that a quick search could clear up, but worth keeping in mind...

Nil is not nil. 

You may run into that one quickly enough in tests. An instance of an interface can have a nil value but since it carries the interface type information, it does not equal to nil!! A lot has been written about it so make sure to read about it!

Redeclarations

The fact that you can use := to declare variables easily is a double edged sword, as declaring a variable with the same name as a previously declared one is allowed if it's in a different scope, and it shadows the original variable. So:



This code will only print True at the end. The "err" variable inside the if block has been redefined so even though it's not nil inside the if block, it IS nil once you exit it! So if you want to chain several operations that may return errors and then at the end handle whatever error occurred in a consistent way, you have to be careful.

Closures

This one gave more some Javascript PTSD:



This prints :
4
4
4

Which was not what I expected! One easy fix is to add a line saying i:=i before creating the func in the first loop, which by virtue of the previous gotcha redeclares a variable in the scope of the for loop body.


Errors

I'm not convinced in the end by the standard error reporting in Go. A lot has been written on that topic too. I feel that the code is peppered with error handling that gets in the way of readability and implementing cleanly the business logic. I'm sure there are some design tricks you can use, but you shouldn't have to resort to tricks to do clean error handling in a language, because at the end of the day it's how you handle errors that make up the quality of your software, not only how you handle the happy path.

These things you can watch out for and probably code around pretty easily, but they do impact a bit the ease of use of Go, even though I still think it's quite a simple elegant language to quickly be productive in.

Happy Go Hacking!