Unison File Synchronizer

The excellent MSYS2 (mingw64 and mingw32) subsytem makes compiling native Windows compilations "as easy as compilation can be". However, as with everything in life, sometimes when trying to do one thing (compile a program), you end up chasing other vaguely related issues (one exotic compile error after another).

For synchronizing files between servers and workstations I use the open source GPLv3 licensed Unison File Synchronizer [1]. Although the text interface version of Unison compiles straight-out-of-the box on mingw64, the GTK2 interface proved to be a bit more cumbersome.

To compile Unison with the GTK2 interface, lablgtk [2] is needed, an OCaml interface to GTK.

So, the journey began with firing up a shell in a fresh mingw64 environment, and installing the build prerequisites:

pacman -Sy --noconfirm base-devel git \
mingw-w64-x86_64-{glib2,gtk2,ocaml,pango,toolchain}

After downloading the latest source (2.18.5 [3]) and trying to compile it (using make ) after running

./configure --prefix=/mingw64 --disable-gtktest

the first error message is shown:

mingw64/include/gtk-2.0/gdk/gdkwin32.h:40:36: fatal error: gdk/win32/gdkwin32keys.h: No such file or directory

It seems that GTK2 version 2.24.31 contains an error [4], and incorrectly still references the file gdkwin32keys.h. This could be solved by removing the reference:

 sed -i "s/#include <gdk\/win32\/gdkwin32keys.h>/\/\/#include <gdk\/win32\/gdkwin32keys.h>/" \
/mingw64/include/gtk-2.0/gdk/gdkwin32.h

After retrying the build process for lablgtk using make , the next error message pops up during the build process:

Error on dynamically loaded library: .\dlllablgtk2.dll: %1 is not a valid Win32 application.

Apparently flexlink 0.34 (which is installed as dependency of OCaml) has an issue [5] with building 64 bit applications on MSYS2 / mingw64. This could be solved by stripping the incorrect library of its debug symbols:

strip src/dlllablgtk2.dll

And indeed, make && make opt works after this. Next up is installing all lablgtk files in the correct locations:

make old-install INSTALLDIR=/mingw64/lib/ocaml/lablgtk2/ BINDIR=/mingw64/bin/ \
DLLDIR=/mingw64/lib/ocaml/stublibs/

After this, the final destination is in sight: cloning, configuring and compiling the git version of Unison for GTK2.

pushd /tmp
git clone --depth=1 https://github.com/bcpierce00/unison
cd unison
make windres
make src OSARCH=win32gnuc

While re-validating these instructions in 2017, I ran into another build issue:

error: 'caml__frame' undeclared (first use in this function)

In file included from ./lwt/lwt_unix_stubs.c:8:0:
./lwt/lwt_unix_stubs.c: In function 'invoke_completion_callback':
mingw64\lib\ocaml/caml/memory.h:236:12: error: 'caml__frame' undeclared (first use in this function)
  (void) caml__frame, \
         ^
mingw64\lib\ocaml/caml/memory.h:236:12: note: in definition of macro 'CAMLxparam2'
  (void) caml__frame, \
         ^~~~~~~~~~~
./lwt/lwt_unix_stubs.c:82:3: note: in expansion of macro 'CAMLlocal2'
CAMLlocal2 (err, name);
^
mingw64\lib\ocaml/caml/memory.h:236:12: note: each undeclared identifier is reported only once for each function it appears in
  (void) caml__frame, \
         ^
mingw64\lib\ocaml/caml/memory.h:236:12: note: in definition of macro 'CAMLxparam2'
  (void) caml__frame, \
         ^~~~~~~~~~~
./lwt/lwt_unix_stubs.c:82:3: note: in expansion of macro 'CAMLlocal2'
CAMLlocal2 (err, name);
^
make[1]: *** [Makefile.OCaml:434: lwt/lwt_unix_stubs.o] Error 2
make[1]: Leaving directory 'unison/src'
make: *** [Makefile:14: src] Error 2

It seems that the current Unison code does not properly follow OCaml's guidelines with regards to interfacing. From the OCaml documentation [6]

Rule 1
    A function that has parameters or local variables of type value must
    begin with a call to one of the CAMLparam macros and return with
    CAMLreturn, CAMLreturn0, or CAMLreturnT. In particular, CAMLlocal and
    CAMLxparam can only be called after CAMLparam.

A simple call using CAMLparam0() fixes that problem.

--- a/src/lwt/lwt_unix_stubs.c       2017-06-30 04:46:47.061185900 +0000
+++ b/src/lwt/lwt_unix_stubs.c       2017-06-30 04:45:42.023470300 +0000
@@ -79,6 +79,7 @@

static void invoke_completion_callback
(long id, long len, long errCode, long action) {
+  CAMLparam0();
   CAMLlocal2 (err, name);
   value args[4];
   err = Val_long(0);

A pull request was sent upstream (https://github.com/bcpierce00/unison/pull/82) which has been merged in October 2017.

And after applying the latest patch...hooray, the binary gets built successfully, and when starting the unison.exe executable under the mingw64 subsytem, the GTK2 interface is shown. Note that the windres command is to make sure that the resource files are 64-bit compatible.

If you want to run the unison.exe executable standalone, on another system or without the MSYS2 / mingw64 subsytem, you'll need the 64-bit GTK2 library files as well as the two MSYS2 libraries libgcc_s_seh-1.dll and libstdc++-6.dll (as flexlink currently doesn't allow static builds). These can found and copied using the following commands:

YOURTARGETDIR=/your/target/search/path
cp
/mingw64/bin/{lib{atk-*,bz2-*,cairo-*2,expat-*,ffi-*,fontconfig-*,freetype-*,gailutil-18,gcc_s_dw2-*,gcc_s_seh-*,gdk*-2.*,gio-2*,glib-2*,gmodule-2*,gobject-2*,gthread-2*,gtk-win32-2*,harfbuzz-*,graphite2,iconv-*,intl-*,pixman-*,pango*-1*,pcre-1,png*-*,stdc++-*,winpthread-*},zlib1}.dll $YOURTARGETDIR
mkdir -p $YOURTARGETDIR/lib/gdk-pixbuf-2.0/2.10.0
cp /mingw64/lib/gdk-pixbuf-2.0/2.10.0/loaders $YOURTARGETDIR/lib/gdk-pixbuf-2.0/2.10.0/

However, after the first try of running unison.exe in a windows shell, with the GTK2 files in a search path, Unison greeted me with the following message:

Fatal error - Uncaught exception Gdk.Error ("Gdk.Pixmap.create_from_xpm_data")

After some searching it became apparent that image (xpm) support is modular, and therefore the Gdk Pixbuf cache (which loads the image modules) needs to be rebuilt. This can be done by copying the file gdk-pixbuf-query-loaders.exe and executing it once on/in the standalone system/path:

gdk-pixbuf-query-loaders.exe --update-cache

This might show some warnings, and will create the file lib/gdk-pixbuf-2.0/2.10.0/loaders.cache. This file contains necessary information for Gdk Pixbuf, so that xpm files can be loaded.

Indeed, this was the final step in running the latest version of Unison standalone on a 64-bit system, with the latest GTK2 64-bit library files.

This was by far a linear path and took me more time than anticipated. However, the satisfaction at the end and the accumulated knowledge makes it worth the journey. Every time.

For debugging purposes I can heartily recommend the Windows version of ldd, ntldd [7]

Summary

To summarize, here are all steps to compile the latest master branch of Unison from scratch on a vanilla mingw64 (MSYS2) installation, using flexdll version 0.34, gcc version 6.2.0, GTK2 version 2.24.31, lablgtk version 2.18.5 and OCaml version 4.02.3:

pacman -Sy --noconfirm base-devel git mingw-w64-x86_64-{glib2,gtk2,ocaml,toolchain}
sed -i "s/#include <gdk\/win32\/gdkwin32keys.h>/\/\/#include
<gdk\/win32\/gdkwin32keys.h>/" /mingw64/include/gtk-2.0/gdk/gdkwin32.h

VERSION=2.18.5
pushd /tmp
wget -c https://forge.ocamlcore.org/frs/download.php/1627/$VERSION.tar.gz
tar -xzvf $VERSION.tar.gz
cd lablgtk-$VERSION
./configure --prefix=/mingw64 --disable-gtktest
make ; strip src/dlllablgtk2.dll
make
make opt
make old-install INSTALLDIR=/mingw64/lib/ocaml/lablgtk2/ \
BINDIR=/mingw64/bin/ DLLDIR=/mingw64/lib/ocaml/stublibs/

pushd /tmp
git clone --depth=1 https://github.com/bcpierce00/unison
cd unison
# Apply patch https://patch-diff.githubusercontent.com/raw/bcpierce00/unison/pull/82.diff
make windres
make src OSARCH=win32gnuc

Pango errors

If you see the message Error: Unbound module Pango , then the OCAMLFIND_CONF path might be incorrect. This can be fixed by setting and exporting the path to the correct value, where findlib.conf lives on your file system. Then, (re)run the make commands.

Example:

export OCAMLFIND_CONF=/mingw64/etc/findlib.conf

lablgtk errors

If you see the message Error: Unbound value Gpointer.region_of_bytes , then the lablgtk version that you're using is older than 2.8.16. You can either upgrade lablgtk, or revert the change from commit 2e7ea9 [8] on 2018-02-26:

git revert --no-commit 2e7ea9

Changelog

  • 2018-05-14: Added lablgtk error message and solution
  • 2017-11-05: Added Pango error message and solution
[1]https://github.com/bcpierce00/unison
[2]http://lablgtk.forge.ocamlcore.org/
[3]https://forge.ocamlcore.org/frs/download.php/1627/2.18.5.tar.gz
[4]http://osdir.com/ml/commits.gnome/2016-11/msg06962.html
[5]https://github.com/alainfrisch/flexdll/issues/6
[6]http://caml.inria.fr/pub/docs/manual-ocaml-4.04/intfc.html
[7]https://github.com/LRN/ntldd
[8]https://github.com/bcpierce00/unison/commit/2e7ea9481c

Comments

comments powered by Disqus