Build rocksdb as a portable build

[silviucpp]

No description provided.

[silviucpp]

Most of the time you don't have the guarantee that build machines supports same SIMD instructions like the machines where the app will run.

In case the build is not portable and on build machine you have some instructions that are not supported on the servers where the app will run you will have crashes.

PORTABLE=1 fix this.

[Licenser]

I ran into similar issues before and when I did my research back then I found that the dependency is AVX/SSE4.2 both were introduced around 2011 in both AMD and Intel. I think however that was with leveldb. Not sure if they are the same with rocksdb but given it is a fork of leveldb it might very well be.

I'm not sure if penalizing the performance on modern systems to allow rocksdb to work/build on 8+-year-old machines without re-compilation is the right step.

[silviucpp]

I'm not sure if only hardware older than 8+ will not have this features. In my case I'm trying to run a build compiled on a physical machine in a VM which runs on a newer hardware than the physical.

And is not working. The app crashes. Looking to official rocksdb build scripts for docker they are using PORTABLE=1

[Licenser]

That’s quite different from what I saw and clearly changes the trade offs

[silviucpp]

Usually a better way is to detect at run-time what features the CPU supports and enable/disable them. This is what lot of libs are doing like libvpx, libyuv etc. But unfortunately rocksdb was not designed in this way

[mocchira]

WIP

[mocchira]

@silviucpp Confirmed that it works on several environments however I think the default value for PORTABLE should be kept (=0) for the (performance) backward compatibility for existing users.

so how about the below approach?

### Comment out the below line when your app crashes on the machine where your app will run
### For more information, please visit https://github.com/facebook/rocksdb/blob/master/INSTALL.md#compilation explaining when you should use "PORTABLE=1"

#PORTABLE=1

[silviucpp]

I'm thinking if there is any better way to pass to rebar3 before compiling an env variable.. Otherwise users that needs to use PORTABLE=1 needs to fork the project, commit the change and use their own forks.

[mocchira]

@silviucpp https://github.com/rebar/rebar/wiki/Dynamic-configuration seems to work as you intend.

1. Check the environment variable PORTABLE in rebar.config.script
2. If it exists then replace prehooks with the below one

{pre_hooks, [{'get-deps', "c_src/build_deps.sh get-deps"},
             {compile, "PORTABLE=1 c_src/build_deps.sh"}
            ]}.

3. set $PORTABLE passed from rebar export PORTABLE=$PORTABLE in c_src/build_deps.sh

I will try in my spare time.

[silviucpp]

Ohh thanks ! this is nice, I'll give a try.

I think you should put in readme this for every one else that might encounter this problem.

Silviu

[mocchira]

I think you should put in readme this for every one else that might encounter this problem.

Yes I will later. Thanks for pointing out that.