Installing NetProphet on Linux Cluster

Introduction

Our NetProphet implementation for distributed computing uses the SLURM workload manager for job scheduling. If you don't have the SLURM in your HPC, you may install the version for serial execution here.

We use Conda to manage the list of dependencies required for NetProphet, as well as specific versions of R and Python needed for execution. This provides an equivalent of virtual environment that can be loaded or unloaded on demand in order to prevent clashing with your other projects, or with other environments on a shared cluster.

If you already have either Anaconda or Miniconda installed, you can skip the first step.

1. Install Conda

Install miniconda3 (64bit) for Linux from https://conda.io.
After installation, do:

$ source <your_path_to_miniconda3>/bin/activate 
$ conda init

To verify installation, run: $ conda info
You should see conda output its version, etc.

2. Add Conda channels

$ conda config --add channels bioconda  
$ conda config --add channels conda-forge

3. Create NetProphet2 environment

$ conda create -n np2 r=3.2.1 readline=6.2 pcre snakemake=3.8.2 numpy=1.15 scipy 
$ conda activate np2

4. Setup Environment variables:

Open ~/.profile and add the following:

NETPROPHET2_DIR=<your_path_to_NetProphet_2.0_dir>    
MEME_BIN=$NETPROPHET2_DIR/SRC/meme/bin  
MEME_LIB=$NETPROPHET2_DIR/SRC/meme/libexec/meme-5.1.0  
FIREDIR=$NETPROPHET2_DIR/SRC/FIRE-1.1a/  
export PATH=${NETPROPHET2_DIR}:${MEME_BIN}:${MEME_LIB}:${FIREDIR}:${PATH}

Save it.
Then update your bash profile by $ source ~/.profile.

5. Install dependencies

a. Install FIRE library:

$ cd $NETPROPHET2_DIR/SRC
$ unzip -q FIRE-1.1a.zip
$ cd FIRE-1.1a/ 
$ chmod 775 configure 
$ ./configure 
$ make 

b. Install MEME library:

$ cd $NETPROPHET2_DIR/SRC/
$ mkdir -p meme/
$ wget http://meme-suite.org/meme-software/5.1.0/meme-5.1.0.tar.gz
$ tar zxf meme-5.1.0.tar.gz
$ cd meme-5.1.0
$ ./configure --prefix=${NETPROPHET2_DIR}/SRC/meme --with-url=http://meme-suite.org/ --enable-build-libxml2 --enable-build-libxslt 
$ make
$ make test
$ make install

c. If openmpi is available in your SLURM, check the version by executing $ module load openmpi/1.8.8.

If not available, install openmpi library:

$ wget https://download.open-mpi.org/release/open-mpi/v1.8/openmpi-1.8.8.tar.gz
$ tar -zxvf openmpi-1.8.8.tar.gz
$ cp configure.sh openmpi-1.8.8
$ cd openmpi-1.8.8
$ chmod 755 configure.sh
$ apt-get install g++ libc6-dev make
$ ./configure.sh
$ make 
$ make install

6. Install custom R packages:

$ cd $NETPROPHET2_DIR/SRC/R_pkgs 
$ R CMD INSTALL BayesTree_0.3-1.3.tar.gz  
$ R CMD INSTALL lars_0.9-8.tar.gz
$ R CMD INSTALL Rmpi_0.6-3.tar.gz --lib=<your_path_to_R_lib> --configure-args="  \\
  --with‌​-Rmpi-type=OPENMPI \\
  --with-Rmpi-include=<your_path_to_mpi_lib>" \\ # For example, use --with-Rmpi-include=/opt/apps/openmpi/1.8.8/include/