Usage queries in EclipseFP 2.3.0

This post is to present a new functionality of EclipseFP 2.3.0: the ability to search where a specific item (function, type, constructor, module) is being used in a project or in the whole workspace.
The way it's done under the hood can be visualized by the following diagram:

• The EclipseFP code invokes the BuildWrapper executable when a file in the project has changed. BuildWrapper uses the GHC API to load the Haskell module from files, and get the AST. We actually use both the RenamedSource and the TypecheckedSource to combine information from both. From the AST we get a list of all the usages in the file: all the functions, for example, used in the file, and where. All this information is stored in a hidden bwinfo file,  in JSON format.
• EclipseFP then picks up the generated file and parses it. The usage data is then written in a SQLite database, whose file is stored in the plugin folder.
• All Haskell searches are then done against the SQLite database.

For the user, this gives the following possibilities:
- open a Haskell module from a dialog showing all known local modules in the workspace
- perform contextual searches for usages of a given object
- perform a search by typing a name in a dialog and maybe specifying additional criteria (search references and/or declarations, search constructors and/or types, etc).
- perform contextual rename of a given object. You can rename a function and everywhere where that function is invoked will be changed, using the Eclipse refactoring plugin.

I'm planning on releasing EclipseFP 2.3.0 pretty soon, so that people can start testing!

Autocomplete enhancements in EclipseFP 2.3.0

Hello,
Following improvements in the core of EclipseFP and BuildWrapper, autocomplete in EclipseFP 2.3.0 manages to give you proposals coming from other modules from your project, and adding the relevant imports on the fly if needed. However, there were already concerns about autocomplete giving back a lot of proposals and feeling cluttered. The preference of restricting the proposals to only those coming from existing imports or only to the packages referenced in the project cabal file applied all the time, so it wasn't easy to balance between the two concerns of power and clarity.
So I've made the following change: the preference is gone. Instead you can choose what you want at runtime, in the fashion of Java completion proposals. So let me show you how it works now.
First of all, if you trigger "content assist" without a prefix, you only get the list of symbols the current module knows about: functions defined in the module and in imported modules, because it would be too much to bring back everything. In this case, you get a simple list:

If you have a prefix, you get first the list of known symbols:

But pressing Ctrl-Space again shows you symbols from other modules in your project or packages you have referenced in your cabal file (it shows you the module name after the function name):

And you can press Ctrl-Space again to get all matching symbols in packages you haven't indicated in your cabal file (you can see the package name before the module name):

Of course, pressing Ctrl-Space again brings you back to the initial list.

Hope this answers the concerns people had, and will make EclipseFP more pleasant to use.

Oh, and I don't have a date for the 2.3.0 release, but hopefully sometime this summer. I still need to build the rename functionality (now I have search working, so rename shouldn't be too difficult to implement).

Happy Haskell Hacking

EclipseFP progress report April 2012

There's not going to be a new EclipseFP release at the end of April, I'm not going to keep the monthly release rhythm I had over the past couple of months. In fact, I have a few simple things I could have done that warranted a release (thanks tobbebex for all the suggestions), but I had already started the next big step, so I want to finish what I'm on before moving to other things.
The goal now is to have a database of usages over all projects in the workspace, so you can see where each module is imported, where each function is used, and then potentially do things like workspace-wide renames and such.
The way it's implemented at the moment:
- on the buildwrapper side, I use the GHC API to load all the modules for a given cabal component in a project, and extract usage information from the RenamedSource.
- the usage information is serialized as JSON to a file in the buildwrapper temporary folder
- these files will be read by EclipseFP after it's called buildwrapper generateUsage call
- the information will then be stored in a sqlite database stored in the workspace plugin folder.

So it will use sqlite as scion-browser does, but on the Eclipse side for performance reasons.

Most of the Haskell side of things is done, and I've started the Eclipse side. So far, this has allowed me the humble but useful "open a module" dialog, similar to "Open Type" in the JDT: it lists all the Haskell modules defined in the workspace and let you select one or several for opening the corresponding file in the editor. Selecting a module in the list shows in which project the file belongs. Wildcards are supported of course.

Hopefully all of this will prove useful.

Happy Haskell Hacking!

EclipseFP 2.2.4 released

I can't believe I've managed to keep the rhythm again, but a month after 2.2.3, here's 2.2.4! It's another minor release featuring:
- task tags: TODO, FIXME, etc in source code translate to task tags. This is configurable.
- GTK wizard: creates a new executable with the default sample GTK code and GTK dependency.
- Documentation generation goes through cabal haddock: less hassle, more power.
- You can now configure extra cabal parameters for your projects. For example, you can add more include and library directories that are specific to your install without modifying the cabal file.
- Hopefully better content assist: using the power of scion-browser to be able to provide more and better auto complete proposals.
- Better handling of related projects (for declaration resolution, breakpoint resolution, etc).

There are also a few bug fixes in there of course. Thanks to everybody for their feedback! Buildwrapper and scion-browser have had minor releases too.

I haven't yet started the big addition that would be a workspace wide embedded database to be able to do Haskell-sensitive searches and replace, but I've though about it, so I have a good idea on how I'm going to attack that. Hopefully for a 2.3.0!

The full release notes can be found here. As usual, to install, just point your Eclipse update UI to http://eclipsefp.sf.net/updates. 


New to eclipseFP? Check out the web site, and there's documentation inside Eclipse too...


Happy Haskell Hacking with EclipseFP!

Heads-up: EclipseFP moving to Java 1.6

Up to now EclipseFP kept Java 1.5 compatibility. Java 6 does not bring that much in terms of API so I was happy to keep it that way. However, I moved a few months back to Eclipse Indigo and since then I've had problem generating the plugin code. I have the feeling that exporting plugins compiled with a 1.6 JDK in 1.5 compatibility mode didn't work well (ending up in errors like "class java.lang.Object could not be resolved", which is scary enough in Java). So I bit the bullet and changed the compatibility level on all plugins. They do now require 1.6.
This change will come in EclipseFP 2.2.4 which I think will be released pretty soon (next week maybe) with a few minor enhancements. So if you were still stuck with Java 1.5, get ready to move... I suppose it will not affect many people anyway, but I'd rather warn in advance to avoid nasty surprises...

Can Frege (Haskell on Java) deliver?

As you know, I'm the main (and more or less, at the moment the sole) developer for EclipseFP. It's going reasonably well, I'm putting a few enhancements here and there, and there are a few big things that I'm planning to do that would be exciting to do and hopefully to use.

However, I can't shake off a nagging feeling that maybe my energy is not 100% useful to the community. The compelling argument of EclipseFP is that it allows developers that are familiar with Eclipse to develop in Haskell as easily as possible: they can concentrate on writing their Haskell code and not have to learn another IDE to do it. But I'm wondering if really there is an audience (apart for myself).
- Unix Hackers will use Vi/Emacs and will scoff at the very notion of using an IDE.
- People really into Haskell would probably prefer an IDE written in Haskell like Leksah, especially since such an IDE actually proves that developing GUI programs in Haskell is possible!
- People that are in the Eclipse/Java world and want to go into functional programming can do so in Scala, Clojure and of course now Frege. Frege now boasts an Eclipse plug-in as well, so really the incentive is great: you can use a Haskell-like language in Eclipse, with probably better integration that what EclipseFP can achieve since everything in Frege boils down to Java. I suppose you could even write the Eclipse plugin code in Frege.

I'm even tempted to make the jump myself. Maybe with Frege I'll get a ecosystem that's doesn't consider Windows as a second grade platform. But can it deliver, in terms of power, performance, reliability?
One thing is clear: Frege is moving forward, which is more that can be said, as far as I know, of others Haskell on the JVM efforts (even the Wiki page says that it'd be a hard job to do, obviously the Frege developers didn't let themselves be scared).

I'll be interested in hearing feedback on people's experience with Frege, and I think I'll take it for a spin myself soon. I'll post my experiences!

Revamping Haddock generation in EclipseFP

My attention was attracted to Haddock documentation generation when Serguei Trofimovich tried to build the documentation for buildwrapper. Not only did he fix some issues with it in my code (thanks Serguei!), but I then realized that I couldn't build the docs from EclipseFP. EclipseFP does provide a Haddock generation wizard, but it forgets to pass the proper GHC options, so if you reference hidden packages for example it won't work. The wizard seemed also very complex.
So I did what I do best, I hacked and slashed. I've removed the full haddock plugin and replaced it with a one page export wizard that uses cabal haddock, and only exposes the options cabal haddock supports. Works well.
So if ever you're reading this and you were using the old Haddock wizard in EclipseFP and really you need some features from it to be kept, give me a shout! Otherwise, you can have a look at the new wizard by getting the source from github.

Hopefully I'll release version 2.2.4 of EclipseFP soon enough, anyway.

Happy Haskell Hacking (and slashing :-))!

EclipseFP 2.2.3 released!

I didn't mean to make it exactly one month since last release, but EclipseFP 2.2.3 is out! There is also a new release of buildwrapper to go with it (0.5.0) and Alejandro released scion-browser 0.2.7 yesterday.
This comes with both bug fixes and minor enhancements that I hope you like. The readme has all the details, but basically:
- Michael Jones has tested EclipseFP thoroughly on Mac OSX, so it should run fine on that platform now. Please remember that UI applications may not use the same PATH as your Terminal, which may explain why things are not found, etc. Thanks Michael for the work!
- We have tried to improve autocomplete and text hovers by adding documentation and types when possible, autocompletion of pragmas, etc.
- Hopefully everything should build and work fine under GHC 7.4.1.

Hopefully the next step will be a big one: I'd like to be able to put together the GHC analysis of all modules of the project in some kind of DB to do things like project and workspace-wide type-aware searches, safe renaming, etc.

Thanks to the supportive users that not only report issues but are happy to investigate, you know who you are!

Happy Haskell hacking in EclipseFP!

Samples of NXT (Lego Mindstorms) programming in Haskell

I've just created a new repository on GitHub called nxt-samples. It's only got one program now, a Bumper program that has the robot going straight till it hits something, then reversing and turning a bit, and going again (inspired by this Bumper Car, even though mine has tracks and no wheel). I've put this up because I didn't find it easy to get started programming a NXT robot, even with the excellent NXT library. Samples on the web were pretty scarce.

A few things bit me:
- not waiting for the final orders to be sent to the robot before exiting the program. This got me wondering a long time why the robot wasn't performing in a consistent manner. Now I'm resetting the motors at the start and end of the program.
- I found it's more reliable to check how much the motors have moved by repeatedly and wait till you reach the amount you wanted in the first place
- I added a simple command: when you press space the robot halts and the program stops. However, this doesn't work on Windows: you need to press space AND enter. Look here to see the woeful story of a Windows only, 4 year old, GHC bug that was fixed, but then reverted as it caused regressions in Cygwin and such. Every few months, that poor bug gets pushed to next release...  I know I'm so uncool to use Windows, but really... Can Haskell be really successful without working well on the OS that a lot of corporate developers still use?

Hopefully I'll add to these samples as I go on playing with NXT... Any feedback appreciated!

EclipseFP 2.2.2 released!

A quick message to say a new version of EclipseFP has been released. It's a bit of a rushed released, because a series of cascading updates meant that scion-browser and EclipseFP got out of sync. A while back I had made changes to the scion-browser API to improve performance, and then persistent got upgraded and a new compatible version of scion-browser got released on sunday. Unfortunately my API changes got released too, which meant a lot of headaches for the poor users that upgraded. So I've released 2.2.2 which is compatible with scion-browser 0.2.5.
It does bring a few enhancements, though:
- outline, tooltips and jump to definition should be much faster
- HLint suggestions can be applied via the quick fix action on the suggestion marker
- clean project does what it advertises: deletes the whole .dist-buildwrapper directory

As usual, just use the update site from inside Eclipse to upgrade. Please report all bugs on the Source forge forum or tracker.

Happy Haskell hacking!!

Lego Mindstorms, Haskell, Windows: all set!

A new version of the NXT library has been released, and it supports Windows!! I lamented a few weeks ago that it only ran on Unix/MacOs (requiring the unix package and using Posix file descriptors), but that's a thing of the past. It was really easy to port Mitar's code to use the serialport package instead of file descriptors. Now Mitar has merged back my changes, we've tested the code on Linux and Windows, and it's been released on Hackage!

So now I can control my Lego Mindstorm robot from Haskell on my Windows machine! World domination is only a couple more steps away!

I wish to point out that Mitar's code is extremely well commented and a pleasure to work with, which of course made the adaptation to Windows a lot easier. Thank you!

EclipseFP 2.2.1 released

It is my pleasure to announce a new version of EclipseFP. Release 2.2.1 does not bring huge changes in the UI, but comes with important bug fixes and small enhancements. More importantly, it integrates with the latest versions of buildwrapper and scion-browser. You can see the release notes here.

We tried to enhance the performance and stability of EclipseFP, while making it easier to work with. The "open definition" action for example should be able to resolve a lot more symbols than before, including in type annotations. Quick fixes and auto-completion we've also worked on.

I wish to thank all users of EclipseFP, most notably all the people that submit bug reports or questions to the forum and thus help make the product move forward!

The instructions to install are the same as always: point your Eclipse to the update site http://eclipsefp.sourceforge.net/updates/. Once you restart EclipseFP should install or update buildwrapper and scion-browser.

Happy Haskell Hacking!

EclipseFP and "the future of programming"

Paul Chiusano's essay "The Future Of Programming" has sparkled a lot of debate, for example on Reddit. His points on IDEs remind me somewhat on what Luke Palmer has to say on them. And he's strongly in favor of lazy, statically-typed, functional languages.

I suddenly find myself in a strange (good?) position: as a maintainer of an IDE (ok, plugins) for Haskell, I can somehow participate towards fulfilling that dream. I know, I know, we're far from it, and EclipseFP has not yet reached the ease of use and power of the JDT in Eclipse. But maybe we can achieve success by doing things differently, not by trying to mimic what's already there and anyway applies to a different language.

We've already started to take some steps in the right direction: for example, Alejandro is working on a version of scion-browser that stores its data in a database. It's not such a big leap to envision that a lot of information about the developer's projects and modules could be kept in that database, to give accurate and fast searching and refactoring operations. We'll also need to think about providing an editor that is a lot more structured than the standard text-with-foldable-sections editor we have now, but it's not clear now if

• the benefits of a structured editor justify developing an interface programmers are not used to
• people would take the plunge even if we could prove tangible benefits

I can already see a clear advantage in having an editor that always force you to have valid code: EclipseFP loses a lot of functionality (thingAtPoint for example doesn't work) if the module you're working on has errors, because the GHC API can provide a lot more info once the module can be analysed correctly. With things like "undefined" and type-directed autocomplete support we could have modules that are always correct. I'm not sure myself that having a compiling module with loads of undefined that fails miserably at run-time is such a great proposition, but hey... In Java, Eclipse let you run a file with errors and only throws a run-time error when you access the incorrect code.

So we'll have to overcome usability and performance issues, deal with GHC limitations and programmers' expectations, but we're in the position to at least try, and see if we can move forward a little bit!

Lego Mindstorms and Haskell?

Santa brought me a Lego Mindstorms kit! Suddenly the time I'm going to have available for EclipseFP is greatly reduced (I joke). I'm having fun building my first little models and fiddling with the graphical interface for programming the little beasts, but I was wondering if I could use Haskell to control the robots. Things look bleak:
- the NXT library on Hackage looks good, but requires unix... Maybe getting it to work on Windows would be easy, but I wonder...
- the fantom development kit (fantom is the low level communication API for Mindstorms, if I understand correctly) comes with a Windows version that will only work with Visual C++. It uses C++ lib and dll files whose name mangling is not compatible with MinGW.

So as a Windows and MinGW user, I'm in a bit of a ditch. I'm wondering now about the Urbi platform, that seems to be open source and to have a module for Mindstorms. It seems to provide a C++, Java and scripting interfaces, so it may be possible to get a Haskell wrapper on top of it. I wonder has anybody worked on Haskell bindings for Urbi? Might be interesting...

Buildwrapper 0.2.2 released

Hello all, I've just released a new version of buildwrapper, which is one of the backend Haskell executables used in EclipseFP. This is a bug fixing release, addressing some of the issues reported by people. Just install the new version via cabal install or via the EclipseFP cabal packages views.
There is ongoing work on EclipseFP and scion-browser, so a new version of both should hit the shelves pretty soon, in a matter of weeks!

Happy Haskell Hacking!

EclipseFP helper executables for Windows

Since people had sometimes problems building Haskell executables we use in EclipseFP, I thought I'd help by putting versions of the current Windows executables on sourceforge. So in you run Windows and have trouble building scion-browser or buildwrapper, you can grab them from here.

I know this isn't really in the spirit of "build from sources so you can check them yaddi yadda", but since even Cabal does it...

As usual, use at your own risk, etc.

Happy Haskell Hacking!

EclipseFP 2.2.0 released!

I'm glad to announce that EclipseFP 2.2.0 has been released. EclipseFP is a set of Eclipse plugins for Haskell development.

The minor version increase is justified by the replacement of the Scion library for the backend by the BuildWrapper executable. I appreciate that lots of work went and is still going into Scion, but I decided that building my own Haskell backend was better for EclipseFP. It calls cabal directly for building, which allows EclipseFP to support things like new style component dependencies (the executable in a project depends on its own library), c files (now my HJVM project, which call Java from Haskell can build, and even report errors in the C code), etc. Note the the haskell code is not bundled with EclipseFP anymore but gets downloaded and installed from Hackage.

This version also comes with a new view, allowing the installation of cabal packages from Hackage. There are also bug fixes and small enhancements all round.

To install, just use the built-in Eclipse install/update functionality. The update site URL is http://eclipsefp.sourceforge.net/updates. The release notes can be found here.

Happy Haskell Hacking with EclipseFP!

GUI Testing using OCR: project Sikuli

I had a great idea this morning... and then realized that there was an open source project already doing it.
I thought: instead of using specific testing frameworks for each UI technology I have (a tool for the web, a tool for SWT UIs, etc) what about a tool that could recognize the text on screen and click or type in the proper places. The same technology for everything type of UI, and something that doesn't rely on pixel perfect positioning to work! Turns out (surprise!) I'm not the first one that had the idea, and looking around I stumbled on Project Sikuli. This looks great: it uses images to find "interesting" parts of your UI and there's also support for text recognition. So potentially you could say "click on the button that says 'Submit'"! It integrates with Java code so you can easily have JUnit tests using it.
Unfortunately, for my own purposes I couldn't get very far easily because of lack of support for transparent images (or, more precisely, it takes into account transparent areas for images, so it doesn't recognize an image if the background changes, which happens if your desktop theme changes, if you use the same icon in different contexts in your UI, etc). But it still looks like a promising tool!

Manage Cabal packages from inside EclipseFP

I'm thinking the new version of EclipseFP will not bundle the Haskell code we use, but will download it from Hackage, which would make sense: we would be able to update the Haskell side without requiring a EclipseFP upgrade, and vice versa. I would expect Haskell developers would be quite used to install packages from Hackage using cabal install. However, let's try to minimize the number of things that a user would need to go outside of EclipseFP to do on a normal Haskell project. So I've added another view:

This view lets you see the list of packages you've installed, with their version, or the full list from Hackage, that you can filter by typing the prefix of the name you're looking for. From there you can install the packages as either Global or User, see the package information, or click on the "More Information" link to open the Hackage page directly.
You can also update the list. This GUI is only a wrapper above the cabal list and cabal info commands.
Hopefully this will be useful to people! Should I add a text field somewhere to be able to specify more options to cabal install, or is that overkill for a GUI?

New backend for EclipseFP

A while ago I was wondering how I could enhance EclipseFP, regarding things like memory usage, support for multi components projects, etc. I now have a working version for adventurous testers!

The main thing is the removal of Scion. EclipseFP now uses another backend component called "buildwrapper". There is no server, buildwrapper is only an executable that can be launched with flags, and writes a JSON result on the standard output. This means no more huge memory usage, and less synchronization headaches.

Buildwrapper uses a copy of the project inside a hidden folder. This means that when we want to get say build errors on a dirty editor content, we can write the dirty content to the file inside the hidden folder, and launch a build process on these files, thus avoiding the limitations that the GHC API has on building from a StringBuffer (doesn't work for files requiring preprocessing, etc).

We only use the GHC API to get the types of things in a source file, but we use Cabal to build the project. Buildwrapper actually launches the cabal executable and parses the results. I know, sounds crude compare to an API only approach, but I have the feeling it just works better. Basically the IDE will now behave a lot close to what you get if you were building your project from the command line. And then you get additional goodies, like multi component support (so you can have the executable of your project reference the library in the same project, and it works!).

Buildwrapper uses haskell-src-exts for outline. I've actually seen cases where it needed language pragmas to parse correctly that GHC didn't need, but such is life.

Performance seems to be quite good, except for things that use the GHC API, where there's a little lag. Hopefully the fact that we don't use gigs of memory any more will make up for it.
I'm also thinking about distribution. EclipseFP came with Scion and Scion-browser bundled, if only because we used a different version of scion than the one on Hackage. I think that moving forward we'll rely on cabal install and Haskage. I'm going to add a simple interface to run cabal install inside EclipseFP, so you shouldn't have to go outside to install a package. So the EclipseFP preferences will just ask the user for the path to the buildwrapper and scion-browser executables, with an option to install them from hackage.

If people want to start testing, just go on github, and grab the BuildWrapper repository (main branch) and the EclipseFP project (buildwrapper branch). For the moment, BuildWrapper probably only builds on GHC 7.0. Once I have the feeling it's robust enough and there's no big flaw in the new architecture, I can work on enhancing portability.