Milestone 1 Plan

1. Implement splits for sharded mongo setup (assigned to Joseph)

   • Modify previous code as per Brendan's comments (complete)
   • Clean up my existing test case and push to repository.
     • I wanted to push it in a state where it could be run with hadoop so the CLASSPATH would not have to be set in set-env.sh but I have not solved that problem yet, so I pushed as is.
     • UPDATE: problem solved. I pushed a run-test.sh script that runs with the hadoop command.
   • Coding: have driver work with replica set (complete)
     • Handle replica set uris in shards table
     • Set slaveok from config property
   • Coding: work on case where there is a query on the shard key passed in (complete)
     • Come up with test case
     • Do actual coding should work with current code
     • Push to repository

2. Need to come up with test harness. Test harness will run tests with a variety of options (assigned to Priya and Rushin)

   • Test that the output is the same as reference output
     • Store the results in a separate database.
     • Figure out how to fetch the values and test parameters into the database for the final output.
   • Test the speed of the jobs run at given different settings
     • Backend: runs tests, write results to mongo database
       • Need to fix the code which stores the results in a single collection
     • Frontend: web page that connects via javascript to the backend to display the results
       • Explore the possibilities of Highcharts and Flot api for user interface

3. Test programs and datasets (assigned to Sumin)

   • Sumin will import his project from his cloud computing class to be a test program.
     • I have completed one test case and run it successfully. Later, I will write 3 - 4 test cases based on network log data sets.

4. Implement splits for unsharded mongo setup (mongodb replica set) (canceled by Brendan)

   • Need to figure out best way to split up input set.
     • I haven't been able to come up with anything so far. Waiting for Brendan to get back to me.

5. Write documentation for all of this

6. Have code reviewed before rollout