It's about the journey: Compiling 64-bit Unison / GTK2 on Windows

Unison File Synchronizer

The excellent MSYS2 (mingw64 and mingw32) subsytem makes compiling native Windows compilations "as easy as compilation can be". However, as with everything in life, sometimes when trying to do one thing (compile a program), you end up chasing other vaguely related issues (one exotic compile error after another).

For synchronizing files between servers and workstations I use the open source GPLv3 licensed Unison File Synchronizer [1]. Although the text interface version of Unison compiles straight-out-of-the box on mingw64, the GTK2 interface proved to be a bit more cumbersome.

To compile Unison with the GTK2 interface, lablgtk [2] is needed, an OCaml interface to GTK.

So, the journey began with firing up a shell in a fresh mingw64 environment, and installing the build prerequisites:

pacman -Sy --noconfirm base-devel git \
mingw-w64-x86_64-{glib2,gtk2,ocaml,pango,toolchain}

After downloading the latest source (2.18.5 [3]) and trying to compile it (using make ) after running

./configure --prefix=/mingw64 --disable-gtktest

the first error message is shown:

mingw64/include/gtk-2.0/gdk/gdkwin32.h:40:36: fatal error: gdk/win32/gdkwin32keys.h: No such file or directory

It seems that GTK2 version 2.24.31 contains an error [4], and incorrectly still references the file …

more ...

Compile Emacs for Windows using MSYS2 and mingw64

Emacs 25.1

Emacs 25.1 was officially released on September 17th, 2016. The excellent MSYS2 subsystem and the open source gcc compiler make it super-easy to build binaries on/for Windows (7, 8, 10). In three easy steps from source to binaries:

1: Install and prepare the MSYS2 subsytem

Download and run the installer at http://repo.msys2.org/distrib/msys2-x86_64-latest.exe After installing, run MSYS2 64bit which drops you in a Bash shell. Update all packages using the following command:

pacman -Syuu

Sometimes updates of the runtime/filesystem can cause update errors. This is no cause for panic - kill and restart the terminal. For building 64-bit Windows binaries, always use mingw64.exe to start the terminal.

Install all packages necessary for building:

pacman -Sy --noconfirm base-devel git \
mingw-w64-x86_64-{giflib,gnutls,jbigkit,lib{jpeg-turbo,png,rsvg,tiff,xml2},toolchain,xpm-nox}

2: Clone the Emacs source

To simplify building, you can define the environment variables BUILDDIR (where the binaries are built), INSTALLDIR (where the binaries will be installed to), and SOURCEDIR (where the source lives, the git repository). Note that since you're in the MSYS2 subsystem, paths are Unix-style, using forward slashes. This command creates SOURCEDIR if it doesn't exist yet, clones the …

more ...

MSYS2 - successful successor of MSYS ?

MSYS2

MSYS can best be described as a Bash shell and some GNU tools, which facilitate compiling sources under and for a Windows environment. It's the environment to use when compiling for instance OpenSSL or Emacs on Windows.

MSYS2 is the 'new and improved' version of MSYS.

One of its biggest advantages is better package management. MSYS2 has pacman, a package manager from Arch Linux. Upgrading can finally be done from within the shell session itself, with only a few basic commands.

# download package descriptions from the remote repositories
pacman -Sy
# upgrade MSYS2 core components and the shell itself
pacman --needed -S bash pacman msys2-runtime
# restart MSYS2 if any package needed updating, then update the rest
pacman -Su

This was 'somewhat more difficult' under MSYS.

Another advantage is the Bash version - currently at 4.3.30 versus 3.1.17 on MSYS. Bash 4 means support for functions like associative arrays and fancier redirections:

# redirect stdout and stderr at the same time
command &> output

# same command in Bash 3 syntax
command > output 2>&1

# pipe stdout and stderr at the same time
command |& someothercommand

# same command in Bash 3 syntax
command 2>&1 \| someothercommand

The third big plus is that it's …

more ...

git on Windows - location of configuration files

Git is used as distributed version control system for the majority of projects I work on. On Windows I use the official Git for Windows version, as well as the 'native' mingw/MSYS2 git binary when using the MSYS2 shell.

The location of the system and global gitconfig configuration files varies, depending on which environment (native Windows command, Windows shell or MSYS2 shell) you're using, and depending on which binary (Git for Windows versus native git). There's a logic to it, but it can be hard to figure out...

Git version 2 introduced a much easier method of finding where the git configuration files are stored, the --show-origin flag. This parameter tells you exactly where each of the configuration files can be found.

Retrieve the locations (and name value pairs) of all git configuration files:

git config --list --show-origin

Retrieve the location (and name value pairs) of the system git configuration file:

git config --list --system --show-origin

Retrieve the unique locations of all git configuration files:

git config --list --show-origin | awk '{print $1}' | uniq

Local

Regardless from where you use git on Windows, the repository (local) configuration always resides at the same location, in the root directory of your repository …

more ...

OpenSSL on 64 bit Windows with ChaCha and Poly1305 support

Note: see https://www.onwebsecurity.com/cryptography/openssl for the latest binary. The version below is obsoleted by newer builds

The main development branch of OpenSSL doesn't have support yet for the (relatively new) ChaCha 20 and Poly1305 ciphers. These can be found however on the 1.0.2-aead branch.

By slightly modifying some makefiles the source can be compiled for 64-bit Windows using mingw64 and msys.

Please find a binary build from a 27-05-2014 snapshot of the source code (1.0.2-aead branch) with assembly code enabled (imported from the 1.0.2 stable branch), and a lot of insecure, new and experimental ciphers enabled. I added the GOST engine gosteay32.dll as well.

The source code for this build can be found at https://github.com/PeterMosmans/openssl

Build commands:

  • Configure mingw64 shared experimental-jpake enable-md2 enable-rc5 enable-rfc3779 enable-ssl-trace enable-ec_nistp_64_gcc_128 enable-static-engine --openssldir=c:/tools
  • make depend
  • make util/libeay.num
  • make util/ssleay.num
  • make
  • make test

Enabled ciphers:

  • ECDHE-RSA-AES256-GCM-SHA384
  • ECDHE-ECDSA-AES256-GCM-SHA384
  • ECDHE-RSA-AES256-SHA384
  • ECDHE-ECDSA-AES256-SHA384
  • ECDHE-RSA-AES256-SHA
  • ECDHE-ECDSA-AES256-SHA
  • SRP-DSS-AES-256-CBC-SHA
  • SRP-RSA-AES-256-CBC-SHA
  • DH-DSS-AES256-GCM-SHA384
  • DHE-DSS-AES256-GCM-SHA384
  • DH-RSA-AES256-GCM-SHA384
  • DHE-RSA-AES256-GCM-SHA384
  • DHE-RSA-AES256-SHA256
  • DHE-DSS-AES256-SHA256
  • DH-RSA-AES256-SHA256
  • DH-DSS-AES256-SHA256
  • DHE-RSA-AES256-SHA
  • DHE-DSS-AES256-SHA
  • DH-RSA-AES256-SHA
  • DH-DSS-AES256-SHA
  • ECDHE-ECDSA-CHACHA20-POLY1305
  • ECDHE-RSA-CHACHA20-POLY1305
  • DHE-RSA-CHACHA20-POLY1305
  • DHE-RSA-CAMELLIA256-SHA
  • DHE-DSS-CAMELLIA256-SHA
  • DH-RSA-CAMELLIA256-SHA
  • DH-DSS-CAMELLIA256-SHA
  • GOST2001-GOST89-GOST89
  • GOST94-GOST89-GOST89
  • AECDH-AES256-SHA
  • SRP-AES-256-CBC-SHA
  • ADH-AES256-GCM-SHA384
  • ADH-AES256-SHA256
  • ADH-AES256-SHA
  • ADH-CAMELLIA256-SHA
  • ECDH-RSA-AES256-GCM-SHA384
  • ECDH-ECDSA-AES256-GCM-SHA384
  • ECDH-RSA-AES256-SHA384
  • ECDH-ECDSA-AES256-SHA384
  • ECDH-RSA-AES256-SHA …
more ...

test_bn fails for OpenSSL on Windows

Compiling OpenSSL on Windows using MSYS and mingw64 is pretty straightforward. However, one of the tests (test_bn) to verify OpenSSL fails: The temporary file that test_bncreates contains Windows newline characters (\r\n) instead of the Unix type newline charater (\n).

The original regular expression checks for a zero (0) at the beginning of a line, and a newline character (\n).

(!/^0$$/)

A change to the regular expression that test_bn uses fixes this problem, and can be used on Unix as well as Windows environments. This makes the Makefile more cross-platform friendly. The modified regular expression checks for a zero (0) at the beginning of a line, an optional Windows newline character (\r) and a newline character (\n).

(!/^0\r?$$/)

md5sum: 1032dff7f957c4d1cdfa96af305c152b
Here's the patchfile (can be applied in the source directory using patch -Np1)
--- openssl-1.0.1g/test/Makefile 2014-04-07 16:55:44 +0000 +++ patched/test/Makefile 2014-05-06 00:07:20 +0000 @@ -227,7 +227,7 @@ @../util/shlib_wrap.sh ./$(BNTEST) >tmp.bntest @echo quit >>tmp.bntest @echo "running bc" - @) {if (/^test (.*)/) {print STDERR "\nverify $$1";} elsif (!/^0$$/) {die "\nFailed! bc: $$_";} else {print STDERR "."; $$i++;}} print STDERR "\n$$i tests passed\n"' + @) {if (/^test (.*)/) {print STDERR "\nverify …
more ...