Monday, December 30, 2013

Dynamic-cabal to solve dependency hell in BuildWrapper

As I've explained in several places, we have an issue in BuildWrapper (the underlying Haskell program for the EclipseFP plugins): BuildWrapper depends on both the GHC API and the Cabal API. Since GHC itself has a dependency on Cabal, once you've installed GHC, BuildWrapper can only use the same version of the Cabal API that was used by that GHC. Moreover, some Cabal files (setup-config in the dist directory for example) are cabal version dependent (since they only serialize Haskell structures, this is for safety). The net result is that if you install a newer version of Cabal and cabal-install, BuildWrapper still uses the old version, and hence the Cabal library BuildWrapper uses and the cabal-install executable have different versions, and hell breaks loose.

I have looked at solving the issue in GHC itself: GHC does not need to have a dependency to the Cabal API. I had started investigating with help from GHC devs, but it was end of October, and I had understood that GHC 7.8 was to be released in November, so there was no time to do such a change. So I thought I would wait for the next release. But it's now end of December, and GHC 7.8 still hasn't been released. So should we have to wait another year to solve that issue? Gasp!

But then, out of the blue, help came! Benno Fünfstück released dynamic-cabal, with the promise that it would solve exactly this issue! I actually should have had that idea myself, but hey... Basically, dynamic-cabal provides simpler structures than the full Cabal API, and extracts them from the Cabal API via dynamic code generation. The code is generated and ran dynamically using the GHC API. Since the running code itself doesn't depend on GHC, it can use the latest version of the Cabal library, and hence read the setup-config file properly!

I have now rewritten parts of BuildWrapper to use dynamic-cabal, and I only had to add minor enhancements to dynamic-cabal. I have upgraded my own cabal-install to 1.18 and everything works (my GHC still uses Cabal 1.16)!! I'm not releasing that version yet because it will provide some new features for EclipseFP 2.6, but if really you want to give it a try just build it from Github.

So my thanks go to Benno! A big hurdle has been cleared on EclipseFP's path to world domination!


lee woo said...
This comment has been removed by a blog administrator.
jitendra said...

I am getting following error when i am trying to install buildwrapper using cabal install. I am using windows 7

Resolving dependencies...
Configuring buildwrapper-0.7.7...
Building buildwrapper-0.7.7...
Preprocessing library buildwrapper-0.7.7...
[1 of 7] Compiling Language.Haskell.BuildWrapper.Base ( src\Language\Haskell\BuildWrapper\Base.hs, dist\build\Language\Haskell\BuildWrapper\Base.o )
[2 of 7] Compiling Language.Haskell.BuildWrapper.GHCStorage ( src\Language\Haskell\BuildWrapper\GHCStorage.hs, dist\build\Language\Haskell\BuildWrapper\GHCStorage.o )

Couldn't match expected type `scientific-'
with actual type `Number'
In the pattern: I l
In the pattern: Number (I l)
In the pattern: Just (Number (I l))
cabal.exe: Error: some packages failed to install:
buildwrapper-0.7.7 failed during the building phase. The exception was:
ExitFailure 1

Please help me !

JP Moresmau said...

Known issue, see:

jitendra said...


Thanks for your immediate response.There you have mentioned

" Use a constraint to use attoparsec 0.10 instead."

Can you please tell me how and where to give this constraint.

Thanks once again.


JP Moresmau said...


Either you install attoparsec first, forcing the version:
cabal install attoparsec-0.10.0

Or you install buildwrapper with

jitendra said...

Now the error that I am getting is

Could not find module `Data.DList'
There are files missing in the `dlist-' package,
try running 'ghc-pkg check'.
Use -v to see a list of the files searched for.
Failed to install aeson-
cabal: Error: some packages failed to install:
aeson- failed during the building phase. The exception was:
ExitFailure 1
buildwrapper-0.7.7 depends on aeson- which failed to install.

Can you please help me.

jitendra said...

Forgot in the previous comment. Now the command that I have given is

cabal install buildwrapper --constraint=attoparsec==

But the error that I am getting is what I have mentioned in the above comment. Thanks

JP Moresmau said...

Nothing to do with buildwrapper. Read the message and the instructions it gives you. Reinstall DList if it wasn't installed properly before.

Cindy Dy said...
This comment has been removed by a blog administrator.