Subscribed unsubscribe Subscribe Subscribe

Installing Torch (OpenBLAS) on Ubuntu 15.04 with Skylake CPUs

As I encountered a problem installing Torch in Ubuntu 15.04 probably because of using Skylake CPUs (Intel Core i7 6600), I would like to share how I have managed installing it on my system. The main reasons were installation errors of OpenBLAS and luacrypto.

My system setup is as follows:
MB: z170 Extreme 6 (with z170 chipset)
CPU: Intel Core i7 6600
OS: Ubuntu 15.04 Japanese Remix

I was installing Torch as said on its official website:
Torch | Getting started with Torch
Installation failed on the following command:

curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps | bash

With the following error indicating OpenBLAS was not compiled:

Cloning into 'OpenBLAS'...
remote: Counting objects: 17300, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 17300 (delta 2), reused 0 (delta 0), pack-reused 17292
Receiving objects: 100% (17300/17300), 14.80 MiB | 3.03 MiB/s, done.
Resolving deltas: 100% (11998/11998), done.
Checking connectivity... done.
getarch_2nd.c: In function ‘main’:
getarch_2nd.c:12:35: error: ‘SGEMM_DEFAULT_UNROLL_M’ undeclared (first use in this function)
printf("SGEMM_UNROLL_M=%d\n", SGEMM_DEFAULT_UNROLL_M);
^
getarch_2nd.c:12:35: note: each undeclared identifier is reported only once for each function it appears in
getarch_2nd.c:13:35: error: ‘SGEMM_DEFAULT_UNROLL_N’ undeclared (first use in this function)
printf("SGEMM_UNROLL_N=%d\n", SGEMM_DEFAULT_UNROLL_N);
^
getarch_2nd.c:14:35: error: ‘DGEMM_DEFAULT_UNROLL_M’ undeclared (first use in this function)
printf("DGEMM_UNROLL_M=%d\n", DGEMM_DEFAULT_UNROLL_M);
^
getarch_2nd.c:15:35: error: ‘DGEMM_DEFAULT_UNROLL_N’ undeclared (first use in this function)
printf("DGEMM_UNROLL_N=%d\n", DGEMM_DEFAULT_UNROLL_N);
^
getarch_2nd.c:19:35: error: ‘CGEMM_DEFAULT_UNROLL_M’ undeclared (first use in this function)
printf("CGEMM_UNROLL_M=%d\n", CGEMM_DEFAULT_UNROLL_M);
^
getarch_2nd.c:20:35: error: ‘CGEMM_DEFAULT_UNROLL_N’ undeclared (first use in this function)
printf("CGEMM_UNROLL_N=%d\n", CGEMM_DEFAULT_UNROLL_N);
^
getarch_2nd.c:21:35: error: ‘ZGEMM_DEFAULT_UNROLL_M’ undeclared (first use in this function)
printf("ZGEMM_UNROLL_M=%d\n", ZGEMM_DEFAULT_UNROLL_M);
^
getarch_2nd.c:22:35: error: ‘ZGEMM_DEFAULT_UNROLL_N’ undeclared (first use in this function)
printf("ZGEMM_UNROLL_N=%d\n", ZGEMM_DEFAULT_UNROLL_N);
^
getarch_2nd.c:67:50: error: ‘SGEMM_DEFAULT_Q’ undeclared (first use in this function)
printf("#define SLOCAL_BUFFER_SIZE\t%ld\n", (SGEMM_DEFAULT_Q * SGEMM_DEFAULT_UNROLL_N * 4 * 1 * sizeof(float)));
^
getarch_2nd.c:68:50: error: ‘DGEMM_DEFAULT_Q’ undeclared (first use in this function)
printf("#define DLOCAL_BUFFER_SIZE\t%ld\n", (DGEMM_DEFAULT_Q * DGEMM_DEFAULT_UNROLL_N * 2 * 1 * sizeof(double)));
^
getarch_2nd.c:69:50: error: ‘CGEMM_DEFAULT_Q’ undeclared (first use in this function)
printf("#define CLOCAL_BUFFER_SIZE\t%ld\n", (CGEMM_DEFAULT_Q * CGEMM_DEFAULT_UNROLL_N * 4 * 2 * sizeof(float)));
^
getarch_2nd.c:70:50: error: ‘ZGEMM_DEFAULT_Q’ undeclared (first use in this function)
printf("#define ZLOCAL_BUFFER_SIZE\t%ld\n", (ZGEMM_DEFAULT_Q * ZGEMM_DEFAULT_UNROLL_N * 2 * 2 * sizeof(double)));
^
make: *** [getarch_2nd] Error 1
Makefile:131: *** OpenBLAS: Detecting CPU failed. Please set TARGET explicitly, e.g. make TARGET=your_cpu_target. Please read README for the detail.. 中止.
Error. OpenBLAS could not be compiled

I found the following issue at the OpenBLAS repo:
The build system doesn't know about SkyLake · Issue #632 · xianyi/OpenBLAS · GitHub

martin-frbg commented 25 days ago

"TARGET=BROADWELL" should work on the "develop" branch (which you will probably want to try rather than the already somewhat dated 0.2.14 release)
(Edited to add: judging from cpuid_x86.c, Broadwell cpu is simply mapped to Haswell even in the develop branch, but you will probably still want all the fixes that went in since the 0.2.14 release)

xianyi commented 23 days ago

@yuyichao , I didn't optimize skylake yet. Therefore, please use haswell kernels. make TARGET=HASWELL

So I did

curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps > installtorch.sh

and edited it to create a custom installation shellscript: (edit: deleted lines that specified to use the `develop` branch)

diff installtorch.sh installtorch-new.sh 
14,16c14,16
<         make NO_AFFINITY=1 USE_OPENMP=0 USE_THREAD=0
<     else
<         make NO_AFFINITY=1 USE_OPENMP=1
---
>         make NO_AFFINITY=1 USE_OPENMP=0 USE_THREAD=0 TARGET=HASWELL
>     else 
>         make NO_AFFINITY=1 USE_OPENMP=1 TARGET=HASWELL

This shellscript let OpenBLAS compile correctly, letting the command that failed to pass.

Installing luacrypto

I encountered another failure on the third command,

cd ~/torch; ./install.sh

and got the following error:

Missing dependencies for itorch:
lbase64
luacrypto
uuid
lzmq >= 0.4.2

Using https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/lbase64-20120820-1.src.rock... switching to 'build' mode
Do not use 'module' as a build type. Use 'builtin' instead.
gcc -O2 -fPIC -I/home/woodrush/torch/install/include -c lbase64.c -o lbase64.o
gcc -shared -o base64.so -L/home/woodrush/torch/install/lib lbase64.o
Updating manifest for /home/woodrush/torch/install/lib/luarocks/rocks
lbase64 20120820-1 is now built and installed in /home/woodrush/torch/install/ (license: Public domain)

Using https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luacrypto-0.3.2-1.src.rock... switching to 'build' mode

Error: Failed installing dependency: https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luacrypto-0.3.2-1.src.rock - Could not find expected file openssl/evp.h, or openssl/evp.h for OPENSSL -- you may have to install OPENSSL in your system and/or pass OPENSSL_DIR or OPENSSL_INCDIR to the luarocks command. Example: luarocks install luacrypto OPENSSL_DIR=/usr/local

I found the following issue on a different OpenSSL problem:
cmake not able to find openssl - Stack Overflow

so I did

    sudo apt-get install libssl-dev

to make sure I installed luacrypto I also did (may not be necessary)

    sudo apt-get install libssl-dev
    sudo apt-get install luarocks
    source .bashrc       # This was after I failed installing torch with the third command: this seems to have the effect of enabling luarocks
    luarocks install luacrypto

and the rest of the Torch installation finished successfully.