EclipseFP 2.0.0 released!

Hello all,

I'm proud to announce version 2.0.0 of EclipseFP, the Eclipse plugin for Haskell coding. The change in the major version number shows that some big changes have been made, even if under the hood:
- the Antlr parsing has been removed
- the JFace based syntax highlighter has been removed
We now rely solely on Scion and the GHC API to provide lexer tokens! This fixes some long standing syntax highlighting bugs while giving us more options for coloring.
Also, we now communicate to the Scion library via standard streams instead of network calls, which should solve some headaches.

There are also loads of bug fixes and small enhancements peppered a bit everywhere, either things that people asked or things that I did for myself, since EclipseFP is (of course) my IDE of choice for my Haskell coding. The release notes can be found here.

Another good news is that I'm not alone in hacking at EclipseFP, scooter has helped a lot with Helios, Cabal 1.6, GHC 6.10 and Mac OS X support, and has a few new features coming up soon!

Just point your Eclipse to the EclipseFP update site, and happy Haskell hacking!!

Haskell Neural Network: plugging a space leak

First, the good news: last week when I posted about my little digit recognition program I had made a mistake in my test code: when I give a rounded eight to my network, it does recognize it, even though it's been trained only with a square eight! Yoohoo!


Also, a few people pointed to online database of handwritten digits so that I could get more data to train my network. I dully downloaded and started working with some files. Soon I knew I had a problem. Not only was the learning slow, but the memory usage of the program showed exponential growth, like using 400MB after a few iterations. Ha ha, I thought, so it's me against lazy evaluation again!


Well, I was right. A couple of strictness annotation in the neuron output calculation function helped performance, nearly making the code twice as fast, but didn't help the memory consumption. Going mad with ! (BangPatterns is your friend) didn't help any more. WTF?


Then I started to read a bit more about seq and Weak Head Normal Form. So when I put a BangPattern in front of a tuple, it evaluates the tuple but not what's inside it. And, lo and behold, if at each learning iteration I traced the network to the standard output, memory usage stayed constant! So I needed to tell Haskell to fully evaluate the network each time, so as not to keep thunks full of arithmetic operations hanging round. A distant memory of a package called deepseq provided me the final help I needed. I didn't use the package itself because I wanted to understand really what's going on, so I just roll out a few lines of my own:



And by calling deepSeq on my network in each iteration, my program runs in constant space! And I looked at the flat line of memory usage in my task manager, and I was pleased. Now, I can try to improve the speed of the dang thing, but that's another set of problems...

Digit recognition with a neural network. First attempt!

I bought myself "On Intelligence", and I thought as the first step on the long road to building an intelligent machine I would go back to neural networks. I have my own little library that seems to work (kind of), so I decided I was going to try to write a little number recognition program. I was inspired of course by ai-junkie, but it was a bit short on details...
So here is the problem: I have a 5x5 grid of LEDs that can be on or off, and I want to recognize the digit that they show. 
The input will be a list of 25 numbers, either 0 (off) or 1 (on)
The output will be a list of 10 numbers, either 0 or 1. The first position of a 1 indicates the result (so if the first element is a 1, it's number 0, etc).
I arbitrarily decide to set the number of hidden neurons to 50, twice the input neurons. So my topology is 25-50-10. 
I train the network with only one version of each digit. For example 3 would be:

*****
    *
 ****
    *
*****

Wrote a few lines of Haskell code to read the training sets (moving from the representation with *, spaces and new lines to the list of 1 and 0, so that 3 becomes: [1.0,1.0,1.0,1.0,1.0,0.0,0.0,0.0,0.0,1.0,0.0,1.0,1.0,1.0,1.0,0.0,0.0,0.0,0.0,1.0,1.0,1.0,1.0,1.0,1.0]).
Then I train the network. It learns after 200 iterations, and successfully recognizes the training sets (ok, so things are working as they should be).
Then the real test: can my network recognize small variants of the training set?
Lets's try with:

****
    *
 ***
    *
****

(a slightly rounded 3): the network answers 3!!
Works also for a slightly rounded 6. But then:

 ***
*   *
 ***
*   *
 ***

(rounded 8) gives me 6. Oh dear... I could probably add more cases in the training sets and hopefully resolve the ambiguities, but there are bigger issues (I know I know, I'm very candid, but it's one thing being told something in a book, and another to build something to actually show it to you). For example:

***
* *
***

Is a small zero that doesn't take the full 5 LED width. My network recognizes 0 when it fills the full square, not when it's smaller (tells me it's a 5).
And of course, 1 is a disaster. The training "1" is a vertical row of LEDs on the left. A vertical row of LEDs on the right gives me 9, usually (and I suppose, 9 has all the LEDs on the right on).
So not only is basic training not sufficient to detect simple variations, but also the fact that we deal with absolute positioning of leds mean than scaling and simple side translations are not supported. So I suppose the next step is to not work with absolute positions, but relative positions of "on" LEDs. Another day's work...

EclipseFP development: using the GHC Lexer

The new version of EclipseFP is slowly coming along. Slowly, because of some holidays, because of a lot of bug fixes, and mainly because of some big changes.
There were a few bugs revolving around syntax highlighting. The original code was using some lexing rules from the JFace packages in Eclipse, but it didn't cover some common enough cases of Haskell syntax, and of course was fairly hopeless at more complex stuff (things like Template Haskell). So I could either trod on and fix the lexing rules for everything, or I could use a different approach. So I decided to use Scion again, and to add support for lexing. Now Scion exposes a method that calls the GHC Lexer on an arbitrary Haskell source. We use the result to color highlight properly the Haskell source code. Of course I have to handle things like CPP preprocessing directives and literal source files myself, but that was ok. The two main issues I had to tackle were performance, and advanced lexing options.
Performance was poor at start, and there are two reasons: the underlying Scion code deals with JSON Strings while the socket code uses ByteStrings, and, I think, the Eclipse console. For the first issue, I'm using now AttoJson, so that Json marshalling is done with ByteStrings and try to use ByteString as much as possible in the code. The Eclipse console is a different matter. I think when too much data is written to it it struggles to update the console control, blocks the UI thread and the whole UI becomes unresponsive. For the moment, I have disabled server output in the console for commands (since now getting the tokens from the source code is quite a big response). Since lexing is also done in the UI thread, we've probably lost a bit in that area, but I feel the advantages (less Haskell knowledge in the Java code, more reliance on the GHC code, which I suspect knows about Haskell pretty well) outweigh the costs.
One funny thing with the Lexer I found: for example when you have TemplateHaskell code in your source file, and you have the TemplateHaskell pragma, the Lexer doesn't automatically set the TemplateHaskell flag. So I decided that I was going to enable all the lexer flags, since performance didn't seem to suffer. Be liberal in what you parse...


Anyway, I'm not going to release the next version right now, a bit of testing is still needed, and there are a few little things that I want to fix. The testing is easy, in a way: I use the development version of EclipseFP to hack the Scion code, so by eating my own dog food I hope that I can release something of value to Haskell programmers.
So don't worry, EclipseFP is coming soon!