Deployment Guide

###Dependencies:

• Java
• Maven 3
• Spark 1.3.1
• Etcd

Deploy Velox Locally

# Download and package velox
git clone [email protected]:amplab/velox-modelserver.git
cd velox-modelserver; mvn package

# Install etcd (instructions can be found on release page as well)
curl -L  https://github.com/coreos/etcd/releases/download/v2.0.10/etcd-v2.0.10-linux-amd64.tar.gz -o etcd-v2.0.10-linux-amd64.tar.gz
tar xzvf etcd-v2.0.10-linux-amd64.tar.gz
cd etcd-v2.0.10-linux-amd64
# Start etcd
./etcd

Edit configuration

Velox reads the cluster-wide configuration from etcd on startup (Note that currently Velox assumes that there is an etcd instance running on every Velox node - each Velox partition tries to connect to etcd using its own hostname). You can use bin/etcd_utils/velox_config.py to define a python dict object containing your config, and the script will automatically write it to the right place in etcd. The script contains a few sample config objects. Edit one to match your setup, and then call add_settings with the your settings dict. You will definitely need to edit the sparkMaster and sparkDataLocation fields, as well as the partition object.

sparkMaster should be set to the IP address of a running Spark cluster. This can be local or remote. If the Spark cluster is remote, make sure that port 7077 is accessible from the Velox machines.

sparkDataLocation can be anywhere that Spark can read from and write to using sc.textFile(...) and rdd.saveAsTextFile(...). For example, we commonly use the local filesystem or HDFS.

Once you have created the configuration, you can upload it to etcd by running:

python velox_config.py
# To check that the config worked, you can query the cluster_config key in etcd
curl http://localhost:4001/v2/keys/cluster_config

Running Velox Locally

Finally, once the Spark and etcd are running, and you have uploaded your config to etcd, you can start Velox:

java -Ddw.hostname=$HOSTNAME -cp veloxms-core/target/veloxms-core-0.0.1-SNAPSHOT.jar edu.berkeley.veloxms.VeloxEntry server

Querying Velox

• Predictions: curl -H "Content-Type: application/json" -d '{"context": 4, "uid":6}' http://localhost:8080/predict/matrixfact
• Observations: curl -H "Content-Type: application/json" -d '{"context": 2, "uid":5, "score":8}' http://localhost:8080/observe/matrixfact
• Batch Train: curl http://localhost:8080/retrain/matrixfact

##TODO: Deploying and Running a Velox cluster