EclipseFP 2.5.0 released

It's my pleasure to announce another release of EclipseFP, the Eclipse plugins for Haskell development.

In this feature, I've added support for cabal-dev, as demonstrated in this video.
I also have added a "organize imports" function on an opened source file. This cleans the import lists by only importing what's really used and formatting it in a manner similar to what stylish-haskell does.

I have also tried to improve the performance of the editor, and fixed all bugs we were made aware of. Syntax highlighting can now highlight more tokens in different colors (non alphanumeric symbols, different types of comment).

Full release notes can be seen here.

As usual, to upgrade just point your Eclipse to http://eclipsefp.sf.net/updates. Report any issue to the SourceForge forum.

Happy Haskell Hacking!

Porting my typeclass game to reactive-banana-sdl

Some time ago I wrote a tiny game called TypeClass, just to experiment with the SDL bindings for Haskell. I was always curious to learn more about FRP, but never really knew where to start. So I just decided to change TypeClass to use reactive-banana, since it looked like a nice library with a dedicated author. And there are SDL bindings too!
It took a bit efforts since I didn't find any sample in reactive-banana-sdl, but I finally got it working. The game is exactly the same from the outside, so it still not that exciting. But I feel I've made progress in understanding events and behaviors and how it all fits together.

I've put the game on github, so anybody else interested in using reactive-banana with SDL can at least have a sample. It requires my own version of reactive-banana.

Happy Haskell Hacking!

EclipseFP + cabal-dev

Some of EclipseFP users had a dream: use cabal-dev sandboxing capabilities to manage their Haskell projects in EclipseFP.  Well, we're getting there. I've uploaded a video of the current state of EclipseFP and cabal-dev integration. The workflow is as follows:
- Fill in the cabal-dev location in the preferences page. Once this is filled, all your Haskell packages will be sandboxed
- Create projects as you would normally
- In the Cabal editor, local projects will now be available as dependencies. When you select a project as a dependency, a project reference is created by Eclipse. The dependent project is then installed in the current project sandbox
- Building, autocomplete, GHCi sessions, etc. will use the project sandbox

Under the scenes, we're only using cabal-dev install --sandbox=.... Thanks Rogan for the help!

This will be part of EclipseFP 2.5.0. I urge adventurous hackers to get the current version from github and start testing!

Discovering the value of QuickCheck

I like tests. I like the confidence that you get when you have an extensive test suite so you know you'll be notified when things break. But so far I've used mainly frameworks like HUnit to explicitly state my assertions. I know what the code should produce given the inputs, and I encode that information in tests. So I didn't really see what QuickCheck brought to that. I'm still not that comfortable with the random aspect of it, but I understand that it can find corner cases for you, whereas with assertions the corner cases you need to think of beforehand, or add them when your users encounter them (ouch). But I'm starting to see the value of QuickCheck.

I was working inside the HTF source code, and the goal was to produce a textual representation of a diff between strings similar to what the diff utility does on Unix. I just wanted to have a fallback for machines that don't have diff installed. And Stefan had written a QuickCheck property to verify that the outputs of the pure Haskell code and the utility matched. I used the property to fall back to my comfortable way of working: fire QuickCheck, let it find a failing case, add it to a HUnit TestCase, and try to understand and fix the issue without breaking the rest. So QuickCheck was just a test case generator.

But then Sterling, the maintainer of Diff, convinced me that 100% matches between the code and the utility was an utopian goal, since diff does some tradeoffs for speed and compactness, and rewriting diff with all its quirks wasn't really worthwhile. And that's when I started to like QuickCheck. I thought: we want exact matches most of the time, but it's fine to have differences as long as what we generate is not too big compared to what the diff utility outputs. This is what QuickCheck classify function can do: I first verify if the size of the output is ok (less than 10% bigger that the utility output), and then I classify the test case as an exact match if the two outputs match. And then I use the cover function (not, ahem, covered in the manual ) to ensure that I get at least  90% exact match:


cover (haskDiff == utilDiff) 90 "exact match" $ 
  classify (haskDiff == utilDiff) "exact match"    
    (div ((length haskDiff)*100) (length utilDiff) < 110)


So the tests are random, yes, but I've pretty good confidence that the output function does what's it's supposed to do. 90% of the time :-).

EclipseFP and performance

OK, ok, performance in EclipseFP could be better. I'll try to present a bit the challenges and why things are the way they are.

You see, EclipseFP used scion for a while, and scion was a long running process, which ended up eating loads of memory, and had a few issues due to the GHC API maintaining state and not releasing memory (I've written about this elsewhere). So I rewrote the whole Haskell backend with BuildWrapper, and I took opposite approach: BuildWrapper is an short-lived executable. You start BuildWrapper with some parameters, it does its stuff and outputs some JSON answer. And that's fine for a lot of operations. But people have started to complain that when they edit a source file, the feedback loop is quite slow, and they keep seeing that the synchronization job keeps on running all the time. That's because every time Eclipse says that the editor has changed (not every keystroke, luckily), EclipseFP goes back to BuildWrapper to analyze the source. So it fires the BuildWrapper executable (and only this can be slow if you have an over zealous anti virus running), which in turn builds up a GHC session, analyses the AST, writes it to JSON, and tears everything down. For small projects and fast machines, this is acceptable. But I eat my own dogfood, of course, and when working on BuildWrapper itself, I noticed things could be too slow for comfort. Sometimes it would take 4 seconds to analyze completely a file.

So I went the middle route. I create another command in BuildWrapper that makes the executable stay around and listen for more commands. More specifically, there is now a long running build command, that can repeatedly analyze a file while staying in the GHC session. It won't have the same issues as the long running scion because it's tied to the file you're editing. When you close the file in Eclipse, the executable dies. When you change something in the Cabal file, the executable restarts to it can take into account new modules, flags or dependencies. The synchronization job now runs in something like a quarter of the time, and even faster.

I believe we have a happy medium: most operations are short-lived and we don't need to worry about memory usage or stray processes, but where performance matters (when you're in the flow and writing Haskell and don't want to wait for the editor to catch up) we have a long running process that's efficient, but not too persistent.

This will be part of the upcoming 2.5.0 release of EclipseFP. Of course people are welcome to get the source and test it for themselves before the official release!

EclipseFP 2.4.2 released

Hello, I've just released 2.4.2 to fix a bug that caused HLint suggestions to not appear any more.

As usual, update from http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!

EclipseFP 2.4.1 released

Hello, a quick release  to patch a bug in the preferences page, causing a NullPointerException, if you don't have buildwrapper set up at all.
There is also an enhancement done with a new call to BuildWrapper, which has been bumped to 0.6.4: autocompletion also shows local variables. This only works if your module could be compiled successfully (so that we could get a GHC AST to retrieve the local bindings). One area of work where EclipseFP could improve would be to be able to automatically recover from some errors to be able to generate the AST and the related information even in the face of errors.

As usual, point Eclipse to http://eclipsefp.sf.net/updates to update. Release notes are here.

Thanks to all the users for updating and telling me of issues, and also for all the kind words I received!

Happy Haskell Hacking!

EclipseFP 2.4.0 released!

Hello, I've just released version 2.4.0 of EclipseFP, the set of Haskell development plugins for Eclipse, along with BuildWrapper 0.6.3 and scion-browser 0.2.13.
The main change in that version is the support for HTF and the Haskell-specific test results view, which removes the dependency on the JDT Junit View. I've blogged about these changes earlier, here and here.
There are also improvements in GHCi support and various bug fixes.

The release notes can be viewed here. To install or update, just point your Eclipse to the update site http://eclipsefp.sf.net/updates.


Thanks to all users, and especially to all the people that submitted bug reports and requests for enhancements. Now, I'll also accept pull requests :-).
Happy Haskell Hacking!

EclipseFP and HTF: demo video

The next version of EclipseFP will integrate with the upcoming version of HTF, a great Haskell Framework for automated tests. As a preview of things to come, I've uploaded a video on YouTube.
In this video, I create a small library project, using the tutorial code from the HTF tutorial, then use the HTF Test Suite wizard in the Cabal editor to generate automatically the cabal test-suite stanza, the main module and a test module per library module I want to test.
The I paste in the actual HTF code, and run the test suite. The results show in the new Haskell Test Results View. I correct the mistakes, wait for the rebuild, and the tests pass!

This will be part of the EclipseFP 2.3.3 release (or maybe I'll call it 2.4), that will be released probably as soon as HTF version 0.10 is released.

I'm still experimenting with screen captures and YouTube video, so apologies if the video is not crystal clear. Best viewed on the large viewer, I guess.

Improved test support in EclipseFP

Last year during his GSoC, Alejandro Serrano did a great job of integrating test-framework results in the JUnit view. This has served me well in my own endeavors (buildwrapper has 50 something HUnit test cases).

But there were a few limitations:
- using the JUnit view added a dependency on JDT plugins
- the view only gave an error when double clicking on a test result because there was no link between a test and the source code it was in
- the test results only appeared once the full test suite had run

I have reviewed the options, and the upcoming version of EclipseFP will have the following changes:
- A specific view for Haskell test results. Basically a rip-off of the JUnit view, without the progress bar because it's a custom control the JUnit UI guys wrote and life's too short. It's got history so you can get back to previous runs, and shows errors or failures in the bottom text area.
- Integration with HTF. While the test-framework integration will still work, using HTF gives us:
   - Jump to location (from test result to test definition location or failure location), thanks to the HTF preprocessor
   - Running updates thanks to HTF generating JSON output after each test and not in one big file at the end.
   - Automated discovery of tests based on naming conventions
   - Easy reduction of test cases to run from a command line argument
   - Elapsed time of tests
   - while still retaining integration with HUnit and QuickCheck

Thanks Stefan Wehr for your great work! Make sure to read his tutorial on HTF.

I've ported the buildwrapper test suite to HTF, and voila!

Since HTF relies on a few pragmas, I'm now going to work on a couple of wizards to simplify creation of the HTF Cabal test-suite and a HTF module. Watch this space!

I've finally figured out the UI the users want

After years of struggling with "we'd like to be able to tweak X and Y, and be able to do Z as well, but it has to be really intuitive and easy to use". Yes, it's the responsibility of the developers (or of the user interface design guru if you're lucky enough to have one) to provide a powerful yet easy to use interface, with intuitive navigation, sensible defaults, expert modes, etc, but sometimes it just gets too much. If you want to be the one that decides when to change gears, you have to know how to use the clutch.

EclipseFP: first timid step towards live programming

The title is sensationalist. All I've done is tweak a bit how we interface between GHCi and the Eclipse debugging framework. What that allows now (in the development version of EclipseFP) is to have expressions (in the Debug Expressions View) evaluated even if you're not stopped at a breakpoint.
So when you work directly in GHCi, you type in expressions to check the results. But when you change some code and hit :reload, your bindings are lost, you need to retype the expressions. EclipseFP does all that for you.

I've posted a video of a simple session in action on YouTube. It's my first video ever! It just shows a trivial arithmetic function being changed and how the result refreshes as soon as you select back the running program in the Debug view. Best viewed large...

EclipseFP 2.3.2 released

I decided to release a new version of EclipseFP, 2.3.2. There also new versions of buildwrapper (0.6.2) and scion-browser (0.2.11) on hackage, even though they are not strictly required (but they compile under GHC 7.6).
This new minor releases offers some bug fixes and performance improvements. Hopefully the installation has been made easier (it is possible to install all helper executables from one place). The debugging mode has also been enhanced, with GHCi history support and the possibility to select expressions in the source code and add them to the Expressions view or see their current value in a popup (when stopped at a breakpoint).

As usual, just point your Eclipse to http://eclipsefp.sf.net/updates. The full release notes can be found here.

Thanks to all the users, bug reporters and contributors!

Happy Haskell Hacking!!

OpenAlchemist: the start of an AI in Haskell

Lately I've been quite addicted to OpenAlchemist, a Tetris-like game developed by I think a couple of French developers (cocorico!). The code is in C++, there doesn't seem to be a clear API to integrate with the engine, the forums are closed, but I still wanted to write an AI for the game..

So I've written a little something in Haskell and posted it on Github just in case it's of interest to somebody. I basically use the Win32 API to capture the game window and save it as a BMP, then I use the bmp library to open the BMP, and some home grown image recognition to see which shapes are displayed on the screen. Then comes the "easy" part, checking the possible combinations and seeing which one yields the higher score. So far the AI hasn't beaten my own high score, so there's still work to do!

I'm still struggling with shape recognition sometimes, so this is an area that needs an improvement. I probably need to read up a bit on the proper algorithms that people have found to do this kind of things instead of just writing my own, but hey, one needs to have fun!

Using GHCi History in the EclipseFP Debugger

I've been asked if the EclipseFP debugger could provide stack traces. Links to great presentations about the subject were even presented! Simon Marlow's explanation of what can be done now regarding stack traces was great, but doesn't solve my problem: the debugging session in EclipseFP uses the GHCi debugger, basically sending GHCi commands and parsing the ouput, and Simon's new solutions require -prof flags, and hence don't work with GHCi.
For the time being I've done something that is not stack traces, but can provide some context when debugging, using the GHCi history. If when invoking a function in GHCi function, you start with :trace, then the :hist command gives you the last 50 evaluation steps. EclipseFP now calls :hist automatically on reaching a breakpoint and parses its output. It represents them as stack frames in the debug view, even that's not what they are. This way, clicking on them opens and select the relevant code, so hopefully you can get an idea of where you are and what you code is doing.
A screenshot showing the history on the program given as an example in the debugging article of the Monad Reader (PDF warning!):

This will be part of the EclipseFP 2.3.2 release, hopefully this will be helpful to some.

Making working from home an attractive option for developers (and their employers)

Warning! This is not a technical post about EclipseFP or Haskell in particular, but rather some ramblings...

I’m a software programmer. I work mainly from home. I love it. But I’m aware that very few job postings offer the possibility to work remotely, and that if ever I have or want to change jobs, I will need to move somewhere else, where there’s a decent job market, to be able to work.

Now, I suppose it’s easy for me to work from home, for a variety of reasons:

•           I have been working on the same project for a few years with success, so my management can trust me to do the work well;
•           I manage a small team of people I’ve known for a long time and that have worked, like me, for years on the project, so instant messaging, emails, a few phone calls and the occasional meeting in person is sufficient interaction: they know what I expect from them, and for better or worse work mainly in their comfort zone;
•           I work in a company that has grown mainly by acquisitions so has development centers all over the globe, in which taking time zones into account for meetings is natural, for example;
•           I like to think of myself as self-motivated, organized and committed, so I don’t need somebody behind my back telling me to work harder ;
•           I enjoy the technical work so I don’t want to be promoted to a management only role, where office politics and being physically close to the movers and shakers of the company matter.

A few of these reasons are specific to me and my situation, but it seems that generally speaking, a few factors could be seen as promoting working from home:

•           The technology is here. At least in the western world, most houses have fast internet access. We have email, instant messaging, phone and video over the internet. We have distributed source control management. We have web interfaces for most tools we may use. We have cloud computing and virtualization so you can log into a remote machine and work on it as if it were local.
•           With the wave of outsourcing, managers should be used to have remote workers, even in different time zones. Of course there was a backlash against outsourcing, but I think the bad experiences with it were more down to the unreasonable expectations (pay less money, get more work done) than to the actual distance between managers and workers.
•           Companies may want to be seen as environment friendly. Encouraging people to commute and/or move to big cities is not a good thing in that regard. I barely use the car during the week now that I can work in my slippers.

Of course I’m well aware of all the objections a company can raise.

•           How could I pay a good salary to somebody I don’t see? But are you only judging the value of an employee by the number of hours she puts in? The end result, working code in a shipping product, can be evaluated the same.
•           How can I build a team spirit if nobody is in the same room? Well, we’re talking computer programmers here. Of course social interactions are good (and I get plenty of that in my personal life, thank you), but I strive on writing good code, making a successful product, and feeling I can satisfy customers and win new ones. I don’t need to be in the same office as the sales people to have that motivation. Meeting the people in the office from time to time is of course OK, the cost of travel and some hotel nights is nothing compared to the office lease and fuel savings.
•           How safe is my code going to be if programmers log on from any kind of location? Well, it happens that I have to travel (to a customer, to another office) so the problem is there even if your employees are not traveling remotely most of the time. Use efficient security procedures (the ones that don’t get in the way of getting work done so that people don’t get so fed up with them that they try to bypass them at all costs) and encrypted connections, I suppose…
•           How do I train my new or junior staff? That is a real issue. If somebody needs to be hand-held to get started, it won’t be doable remotely. So you may want to organize a few face to face sessions between new recruits and senior employees at the beginning. But when hiring experienced developers, this should not be an issue. Your code and your processes are well documented, right?

What more can we do to encourage companies to offer home working? Do we just need higher fuel prices to that employees cannot afford commuting? I hope we can be a bit more proactive. Can more technology help? Do we need more tools?

•           Integrating IDEs with communication tools so that communicating over live code is easier
•           Helping distributed design sessions using things like touch screens and motion capture sensors to better communicate the hand gestures that sometimes help conveying ideas better than word (but on the contrary, I sometimes feel that having to put down an idea in word helps it make clearer, and a trace of that thought can be kept)

Or do we have the tools and just need to get used to them?  Or is it just a shift in employee-employer relationship that may happen slowly and can’t be rushed? In this case, we need to push more stories like 37Signals, about how successful software companies are hiring remote employees and loving it!

EclipseFP 2.3.1 released

It's my pleasure to announce a new release of EclipseFP, the Eclipse plugins for Haskell development.


Main features in that release are command history in GHCi console and Stylish Haskell integration (see my previous post).
We've also enhanced tooltips, that now use HTML rendering and can combine errors/warnings with "thing at point" information. The error markers also show again squiggly lines at their exact location.


I'm pleased to be able to say "we" once again. Martijn Schrage has heavily contributed to this release, both on the Haskell and Eclipse sides. Thanks Martijn for the great work!


There are also releases of buildwrapper and scion-browser, the EclipseFP Haskell helper executables.
Update your plugins via the EclipseFP update site (http://eclipsefp.sf.net/updates), as usual. You can view the release notes here.


Happy Haskell Hacking!

EclipseFP: integrating stylish-haskell

Stylish-haskell is a new code formatter for Haskell, and people have asked for it to be integrated in EclipseFP. This will be part of the 2.3.1 release.

You install stylish-haskell and set its path as with any helper executable (if it's your path you don't even need to specify the path in EclipseFP). Then there's a preference page to set preferences for all projects in your workspace, and a property page on each project if you want project-specific settings:

Then it's only a matter of right-clicking on the source, and choose Source-> Format. Or of course Ctrl+Shift+F which is the standard formatting shortcut in Eclipse. EclipseFP handles writing the YAML configuration file for you!

Don't know yet when EclipseFP 2.3.1 will be released, pretty soon I think!

Command history in EclipseFP GHCi console

It seems to be a very basic functionality, but when you launch a GHCi session in EclipseFP, you don't have command history. You see the GHCi prompt, you can type all GHCi commands, but if you want to use the REPL repeatedly, you have the retype the same command every time. A few users have of course asked for command history.
Funny thing is, Eclipse doesn't give that out of the box, and on the web you can see a few frustrated people asking for it in their console views. But there seem to be no obvious solution. I was rapidly becoming one of these frustrated people, so I had a look.
The most obvious is to create our own view to handle GHCi interaction, and manage the GHCi process ourselves. Doable, but that means rewriting the whole console UI (multiple console, potentially different colors for input/output, etc). So instead I had a look at contributing to the console Eclipse opens for us.
Oh boy. As usual with Eclipse, the console API is quite daunting at first. You can create your own console, or you can just add contributors to the console UI, but since we're using the Eclipse standard classes for displaying a process, we're quite limited, and a lot of classes are created for us without any possibility of injecting our own. And nowhere did I find an easy way to just listen to what the user is sending to the process input stream!
So I didn't give up, and I managed to register key listeners and verify listeners on the actual console StyledText control, so I can listen to key presses and such. I then maintain the history of commands. There is a drop down action to see the list of commands, you can choose one from there, or you can use Control-Up and Control-Down to navigate the list. It also works when you paste in text from elsewhere. This will account for 95% of the cases, I hope. There are definitely things that don't work well: if you edit a command in the middle it won't record the proper one, for example, but these bugs I think I can live with for the moment.
For the moment it's enabled on interactive consoles (mainly for GHCi), but probably that code could be used outside of EclipseFP if anybody is interested. And of course if somebody has a better solution to the problem, let me know!

Command history will be part of the next minor release of EclipseFP, 2.3.1.

EclipseFP 2.3.0 released!

Hello, this is my pleasure to announce a new release of EclipseFP, 2.3.0, and of its companion Haskell packages, Buildwrapper (0.6.0) and Scion-Browser (0.2.9).
EclipseFP is a set of Eclipse plugins for Haskell development.

You can see the release notes for full details, but these are the main points:

• you can do search and replace in a Haskell aware manner. Basically we keep a database of where everything is defined and used in all the modules of your workspace, which lets you open a module, search for definitions or usage, rename things everywhere they're used.
• better integration with Yesod: when you have a yesod project, you can easily launch "yesod devel" on it.
• Preferences to specify the paths of all the Haskell tools we use
• More quick fixes: add OverloadedString, install missing package, etc.

Hopefully you'll find this release and its new features helpful. Of course, post any issue to the sourceforge forum. As usual, use the Eclipse install feature to update/install your plugins by pointing it to http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!