Dynamic-cabal to solve dependency hell in BuildWrapper

As I've explained in several places, we have an issue in BuildWrapper (the underlying Haskell program for the EclipseFP plugins): BuildWrapper depends on both the GHC API and the Cabal API. Since GHC itself has a dependency on Cabal, once you've installed GHC, BuildWrapper can only use the same version of the Cabal API that was used by that GHC. Moreover, some Cabal files (setup-config in the dist directory for example) are cabal version dependent (since they only serialize Haskell structures, this is for safety). The net result is that if you install a newer version of Cabal and cabal-install, BuildWrapper still uses the old version, and hence the Cabal library BuildWrapper uses and the cabal-install executable have different versions, and hell breaks loose.

I have looked at solving the issue in GHC itself: GHC does not need to have a dependency to the Cabal API. I had started investigating with help from GHC devs, but it was end of October, and I had understood that GHC 7.8 was to be released in November, so there was no time to do such a change. So I thought I would wait for the next release. But it's now end of December, and GHC 7.8 still hasn't been released. So should we have to wait another year to solve that issue? Gasp!

But then, out of the blue, help came! Benno Fünfstück released dynamic-cabal, with the promise that it would solve exactly this issue! I actually should have had that idea myself, but hey... Basically, dynamic-cabal provides simpler structures than the full Cabal API, and extracts them from the Cabal API via dynamic code generation. The code is generated and ran dynamically using the GHC API. Since the running code itself doesn't depend on GHC, it can use the latest version of the Cabal library, and hence read the setup-config file properly!

I have now rewritten parts of BuildWrapper to use dynamic-cabal, and I only had to add minor enhancements to dynamic-cabal. I have upgraded my own cabal-install to 1.18 and everything works (my GHC still uses Cabal 1.16)!! I'm not releasing that version yet because it will provide some new features for EclipseFP 2.6, but if really you want to give it a try just build it from Github.

So my thanks go to Benno! A big hurdle has been cleared on EclipseFP's path to world domination!

EclipseFP 2.5.6 released

It's my pleasure to announce a new version of the Eclipse plugins for Haskell development, version 2.5.6! This is a minor, bug fixing release. The release notes can be found here.

To install, just point Eclipse to the update site: http://eclipsefp.sf.net/updates.

I expect that the next version will be a 2.6 with more features, as hopefully I'll have more free time.

Happy Haskell Hacking!

Reactive-banana-SDL on Hackage!

I've uploaded to Hackage the reactive-banana-sdl library. This library was developed by R. Kyle Murphy(orclev), and I've just added a couple of functions and some documentation in the code. Orclev didn't want to maintain it anymore so I've taken the maintainer role, but I'm certainly not a FRP expert (in fact I started to use the library to learn about FRP), so be gentle...

I have a little typing game called TypeClass, which uses the library. You can have a look at the code, it may help you get started.

There's not that much there, but hopefully this can be of use to some people, and let me know what goodies we could add!

Happy Reactive SDL Hacking!

Acme editor for Haskell development?

 A post on reddit made me discover the Oberon system and the Acme editor, that drew inspiration from it. There is a cool video on YouTube of the Acme editor in action. The concept that "everything is Text" and "every bit of text can be an executable action" looks really powerful, even if it looks like it would take a bit of practice to get used to the mouse chording and the intensive usage of the three mouse buttons.

Working with an IDE that's quite heavy (understatement of the year), I'm certainly drawn to that simplicity and power much more than to other editors like Emacs and Vi, where it seems you have to remember an near-infinite list of arcane key combinations to be productive.

So I'm wondering, are there people out there using Acme or Oberon or Plan 9 for Haskell development? How do you find it? What good ideas could be applicable to other environments? I'm always looking out for ways to improve EclipseFP and Haskell development tools in general, is this a path worth exploring?
I've seen that blog entry, from somebody apparently using Erlang, and I noticed the following: Other sacrifices are syntax highlighting, automatic indentation, specific language support. Is that true? Isn't there a way to add syntax highlighting in Acme? And I suppose the specific language support you get by exposing your tool via the Acme system for text commands...

Happy Haskell Hacking, on Acme or not!

EclipseFP 2.5.5 released!

It is my pleasure to announce a new release of EclipseFP, version 2.5.5. EclipseFP are Eclipse plugins for Haskell development. This is mainly a bug fix release with only small enhancements, so I recommend everybody upgrade (from the update site http://eclipsefp.sf.net/updates).
The release notes can be found here.

I'm hoping that the next release will be a major one, with big enhancements. So I'll gladly have contributors if people want to join the fun!

Happy Haskell Hacking!

EclipseFP: laundry list for 2.6

I seem to have a bit more free time these days to work on EclipseFP, so I was thinking about what we should try to add in the next major release. In no particular order:

• Sandboxing via Cabal 1.18 support and not only cabal-dev. Maybe also using cabal repl could replace some code in the GHCi launch configuration management.
• HSpec 
• Cabal bench
• An Haskell worksheet similar to the Scala worksheet. This would allow a better integration with the UI than running a GHCi launch, and would be able to persist test expressions over reloads and even Eclipse restarts. This would help people to play around with their code and see the results straight away. We could add things like time of execution, history of timings so you could check if you're improving the performance, pluggable graphical representations, save as unit test (make a unit test that checks that the given expression always give the current result)... Note that buildwrapper already has code to perform evaluations of expressions using the GHC API, but this is not used yet by the Eclipse front-end.
• Integrate HaRe (into buildwrapper since it now uses the GHC API?) to provide more refactorings.
• See if new versions of tools we depend on, like haskell-src-exts, can do more things for us or allow us to get rid of some of our code to rely on theirs (yeah!)

Let me know if you have some great idea that's not on the list, or if any of these are of special interest to you. I know there some other requests that people have made that I haven't implemented yet, so you can also check the issue list and comment on those you'd like to see treated with higher priority.

If you feel the urge to fork the github repository and start working on something, please don't refrain, I'll be happy to get more contributors!

EclipseFP: package version bounds in dependencies

Coming in EclipseFP 2.5.5, the cabal editor lets you specify what policy you want to use in respect to versin bounds when adding a dependency to a package, instead of just referencing the package without any constraints, or having to type them by hand:

You have four options:

• None: as before, doesn't generate any bound
• Current major version: will restrict to the major version of the package you currently have
• Current major version from current minor: the minor version of the package you currently have is going to be the lowest version allowed, up to the next major version. This is the default, since it assumes the current version you have is the one you're testing against
• Current minor version: only use the minor version of the package you currently have

The text field shows you the actual values that get generated. This field is now read-only since you can first select a package then change the version restriction options. If you want to enter manually the version bounds, you can still do that on the main page where all the current dependencies are listed.

I hope it will let you conform to the Package Versioning Policy and provide reasonable dependency bounds on your libraries.

No date on the 2.5.5 release yet, there's a couple of things I'd like to do first. The Github version should be usable if any of you want to live on the edge (and perform some valuable testing!).

EclipseFP 2.5.4 released

Hello folks, a new bug-fixing release of EclipseFP. Nothing major, mainly fixed a big bug that caused the plugin to randomly not start properly in Kepler, and few little enhancements to make life easier. The full release notes have all the details.

I'm also pleased to announce a new contributor, Maik Riechert, who contributed a hyperlink detector so that hyperlinks in source files can be followed.

As usual, just update from within Eclipse using update site http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!

EclipseFP 2.5.3 released

Hello, I've just released a new version of EclipseFP, 2.5.3. This is a minor release for bug fixes, general stability and hopefully better performance.

You can find the release notes here: https://raw.github.com/JPMoresmau/eclipsefp/master/docs/releasenotes/net.sf.eclipsefp.haskell_2.5.3.txt.

I don't have a lot of time for EclipseFP at the moment, being busy on other projects, but I'm well aware that there are a few enhancements that people have asked for in the queue. I'll try to address these later on, and of course I'll happily accept pull request on https://github.com/JPMoresmau/eclipsefp.

As usual install or update by pointing your Eclipse to http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!

Eclipse Hate and Haskell IDEs

A few weeks ago I was reading the comments about the release of Android Studio on reddit. I was a bit shocked by all the Eclipse hate. But hey, I suppose I use Eclipse and not IntelliJ, so I don't know what I'm missing and how great it would be to work on IDEA. But then I was surprised to see that there has been no release of ideah, the Haskell plug-in for IDEA, for a year and a half. How come there isn't more momentum if IntelliJ is such a great IDE to build on and work with?

Maybe we're just suffering from too much dispersion in the Haskell IDE space. We have plug-ins for the major IDEs, but none of them can probably called great (I know, some people positively hate EclipseFP, but for my defence I'll say I get much more bug reports and feature requests than pull requests, I probably need a course on "building a passionate programming community around your open source project"). They however have the massive advantage of being able to reuse a wealth of existing code (I can use the Eclipse Git plugin to work with Github, I don't need somebody to write the Haskell version). An IDE in Haskell like Leksah or something built on top of Yi would be amazing to showcase that you can do real applications in Haskell (since you still get people saying that Haskell is only for toying with), but then you need to build everything, which requires people (see note above on building a community). Then we have web based editors like the offering from FPComplete, but for developers to use them on real projects we need to think on how to package a web based interface and what it means for a development work-flow: I'm not sure developers would embrace developing using a browser. Of course we have plug-ins for the Unixy editors, emacs and vim, but if we want to open up Haskell to non hackers, maybe we need something more...

So can we continue like that and hope to have a few decent environments? Or shall we all agree on a direction and unite to provide the one true development environment for Haskell? Sometimes people say "Haskell is so different and advanced as a programming language, it needs a new type of editor/IDE". I don't disagree with it, but who has the vision of what the Haskell IDE should be?

Moving from Windows 8 to Ubuntu (I'm a Haskell on Windows developer no more!)

I bought myself a new laptop the other day, and while I'm happy with it, it came with Windows 8. This wasn't an issue per se, once I looked up on the internet how to find the "shut down" button... Sure, the gestures are a bit annoying when you don't have a touch screen, but I could live with that.

But then I started running into familiar issues again, as I tried to set all the Haskell libraries I needed for a project: libraries not working on Windows, MinGW/MSYS/Cygwin hell, etc... I just couldn't take it anymore. So I guess this is an admission of defeat that when you need work done quickly using a significant number of Haskell libraries, you need a Unix based OS.

So I decided to dual boot the computer with Ubuntu. I don' really know why I chose Ubuntu, from looking around it looked like it was both stable and developer friendly.

I was first pleased to see that Windows would give me tools to resize my main partition. I remember a time where you had to use third party software to to do that!

I burned a DVD with the 12.10 image (I see 13.04 is out, I'll have to upgrade some day I suppose), and went with the Linux Secure install. It didn't recognize that I had a Windows install, so I configured my partitions and tried to understand what the messages about the boot loader meant. But when I restarted, I went straight to Windows as if Ubuntu didn't exist. So I restarted using the DVD, and launched the Boot Repair utility. At the end it told me that I had to change something in the UEFI options of the computer. I went in there and sure enough there was an entry for Ubuntu. Once I selected that, I could now see the dual boot screen, giving me the choice between Ubuntu and Windows, and both work! Success!

After that, not much sweat to report. I quickly appreciated apt-get, as opposed to the various downloads+install procedures you get for windows software. Sure, at the time I installed it, Eclipse was still 3.8 and not 4.2, but I'm not going to complain.

Ho, and the library I needed that I couldn't get to work on Windows? Installed like a charm.

I'm now a Ubuntu Haskell developer, and I don't regret it.

EclipseFP 2.5.2 released

I'm happy to announce a new bug-fixing release of EclipseFP. You can see the release notes here.
A few issues around the use of cabal-dev were found, as expected, and fixed, and a couple of little feature requests have been put in.
Hope this helps! As usual, just upgrade Eclipse using the update site http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!

EclipseFP 2.5.1 released

Quickly on the heels of 2.5.0 I'm releasing 2.5.1. This is mainly to fix an issue that caused the Editor->Syntax preference page to not open.
This release also add the possibility to change the font and colors of the tooltip in the editor (under Preferences->General->Appearance->Colors and Fonts, Haskell). It also searches for executables under $HOME/Library/Haskell/bin, which seems to be a path for executables on MacOS.
Also, some UI enhancements for Eclipse 4.2. The full release notes are here.

To upgrade just point your Eclipse to http://eclipsefp.sf.net/updates. Report any issue to the SourceForge forum.

Happy Haskell Hacking

EclipseFP 2.5.0 released

It's my pleasure to announce another release of EclipseFP, the Eclipse plugins for Haskell development.

In this feature, I've added support for cabal-dev, as demonstrated in this video.
I also have added a "organize imports" function on an opened source file. This cleans the import lists by only importing what's really used and formatting it in a manner similar to what stylish-haskell does.

I have also tried to improve the performance of the editor, and fixed all bugs we were made aware of. Syntax highlighting can now highlight more tokens in different colors (non alphanumeric symbols, different types of comment).

Full release notes can be seen here.

As usual, to upgrade just point your Eclipse to http://eclipsefp.sf.net/updates. Report any issue to the SourceForge forum.

Happy Haskell Hacking!

Porting my typeclass game to reactive-banana-sdl

Some time ago I wrote a tiny game called TypeClass, just to experiment with the SDL bindings for Haskell. I was always curious to learn more about FRP, but never really knew where to start. So I just decided to change TypeClass to use reactive-banana, since it looked like a nice library with a dedicated author. And there are SDL bindings too!
It took a bit efforts since I didn't find any sample in reactive-banana-sdl, but I finally got it working. The game is exactly the same from the outside, so it still not that exciting. But I feel I've made progress in understanding events and behaviors and how it all fits together.

I've put the game on github, so anybody else interested in using reactive-banana with SDL can at least have a sample. It requires my own version of reactive-banana.

Happy Haskell Hacking!

EclipseFP + cabal-dev

Some of EclipseFP users had a dream: use cabal-dev sandboxing capabilities to manage their Haskell projects in EclipseFP.  Well, we're getting there. I've uploaded a video of the current state of EclipseFP and cabal-dev integration. The workflow is as follows:
- Fill in the cabal-dev location in the preferences page. Once this is filled, all your Haskell packages will be sandboxed
- Create projects as you would normally
- In the Cabal editor, local projects will now be available as dependencies. When you select a project as a dependency, a project reference is created by Eclipse. The dependent project is then installed in the current project sandbox
- Building, autocomplete, GHCi sessions, etc. will use the project sandbox

Under the scenes, we're only using cabal-dev install --sandbox=.... Thanks Rogan for the help!

This will be part of EclipseFP 2.5.0. I urge adventurous hackers to get the current version from github and start testing!

Discovering the value of QuickCheck

I like tests. I like the confidence that you get when you have an extensive test suite so you know you'll be notified when things break. But so far I've used mainly frameworks like HUnit to explicitly state my assertions. I know what the code should produce given the inputs, and I encode that information in tests. So I didn't really see what QuickCheck brought to that. I'm still not that comfortable with the random aspect of it, but I understand that it can find corner cases for you, whereas with assertions the corner cases you need to think of beforehand, or add them when your users encounter them (ouch). But I'm starting to see the value of QuickCheck.

I was working inside the HTF source code, and the goal was to produce a textual representation of a diff between strings similar to what the diff utility does on Unix. I just wanted to have a fallback for machines that don't have diff installed. And Stefan had written a QuickCheck property to verify that the outputs of the pure Haskell code and the utility matched. I used the property to fall back to my comfortable way of working: fire QuickCheck, let it find a failing case, add it to a HUnit TestCase, and try to understand and fix the issue without breaking the rest. So QuickCheck was just a test case generator.

But then Sterling, the maintainer of Diff, convinced me that 100% matches between the code and the utility was an utopian goal, since diff does some tradeoffs for speed and compactness, and rewriting diff with all its quirks wasn't really worthwhile. And that's when I started to like QuickCheck. I thought: we want exact matches most of the time, but it's fine to have differences as long as what we generate is not too big compared to what the diff utility outputs. This is what QuickCheck classify function can do: I first verify if the size of the output is ok (less than 10% bigger that the utility output), and then I classify the test case as an exact match if the two outputs match. And then I use the cover function (not, ahem, covered in the manual ) to ensure that I get at least  90% exact match:


cover (haskDiff == utilDiff) 90 "exact match" $ 
  classify (haskDiff == utilDiff) "exact match"    
    (div ((length haskDiff)*100) (length utilDiff) < 110)


So the tests are random, yes, but I've pretty good confidence that the output function does what's it's supposed to do. 90% of the time :-).

EclipseFP and performance

OK, ok, performance in EclipseFP could be better. I'll try to present a bit the challenges and why things are the way they are.

You see, EclipseFP used scion for a while, and scion was a long running process, which ended up eating loads of memory, and had a few issues due to the GHC API maintaining state and not releasing memory (I've written about this elsewhere). So I rewrote the whole Haskell backend with BuildWrapper, and I took opposite approach: BuildWrapper is an short-lived executable. You start BuildWrapper with some parameters, it does its stuff and outputs some JSON answer. And that's fine for a lot of operations. But people have started to complain that when they edit a source file, the feedback loop is quite slow, and they keep seeing that the synchronization job keeps on running all the time. That's because every time Eclipse says that the editor has changed (not every keystroke, luckily), EclipseFP goes back to BuildWrapper to analyze the source. So it fires the BuildWrapper executable (and only this can be slow if you have an over zealous anti virus running), which in turn builds up a GHC session, analyses the AST, writes it to JSON, and tears everything down. For small projects and fast machines, this is acceptable. But I eat my own dogfood, of course, and when working on BuildWrapper itself, I noticed things could be too slow for comfort. Sometimes it would take 4 seconds to analyze completely a file.

So I went the middle route. I create another command in BuildWrapper that makes the executable stay around and listen for more commands. More specifically, there is now a long running build command, that can repeatedly analyze a file while staying in the GHC session. It won't have the same issues as the long running scion because it's tied to the file you're editing. When you close the file in Eclipse, the executable dies. When you change something in the Cabal file, the executable restarts to it can take into account new modules, flags or dependencies. The synchronization job now runs in something like a quarter of the time, and even faster.

I believe we have a happy medium: most operations are short-lived and we don't need to worry about memory usage or stray processes, but where performance matters (when you're in the flow and writing Haskell and don't want to wait for the editor to catch up) we have a long running process that's efficient, but not too persistent.

This will be part of the upcoming 2.5.0 release of EclipseFP. Of course people are welcome to get the source and test it for themselves before the official release!