Benchmark

This is a variant of the CityBench benchmark framework that is supposed to be ran in a distributed environment on Amazon. More information on how to setup this environment: https://github.com/LinkedDataFragments/CityBench-Amazon

CityBench is a java-based benchmarking toolset for RSP engines, currently CQELS, C-SPARQL and TPQ Query Streamer are supported.

##Prerequisite

• JVM 1.7
• Java IDE (for debugging and extensions)
• TPF Query Streamer (https://github.com/LinkedDataFragments/QueryStreamer.js)

##Folders & Files

1. cqels_query: sample queries in CQELS syntax;
2. csparql_query: sample queries in C-SPARQL syntax;
3. dataset: background knowledge base, mostly sensor service repositories;
4. ontology: ontologies used;
5. result_log: output files generated by CityBench, e.g., query latency, result count and memory consumption;
6. src:source code;
7. lib: libraries used;
8. streams: sensor observation raw data in .csv formats, used to generate RDF streams；
9. EC-log: logger file output;
10. citybench.properties: configuration file loaded by CityBench.
11. querystreamer_query: sample queries in SPARQL syntax to be handled by TPF Query Streamer;

##To run

1. Download all resources and source code
2. Import to your Java IDE and run CityBench.java Or
3. Use "java -jar" command to execute the CityBench.jar

##Configuration file

• dataset = dataset/[your_sensor_repository_file] // tell CityBench where to look for static background knowledge.
• ontology = [your_ontology_folder] // tell CityBench where to look for ontologies used.
• streams = [your_streams_folder] // tell CityBench where to look for raw data to simulate sensor streams.
• cqels_query = [your_cqels_queries_folder] // tell CityBench where to look for cqels queries.
• csparql_query = [your_csparql_queries_folder] // tell CityBench where to look for csparql queries.
• querystreamer_query = [your_querysreamer_queries_folder] // Tell CityBench where to look for querstreamer queries

// All paths are relative path to the project root

// Normally if you use the downloaded resources as they are, you would not need to change the default content in the configuration file. The exception is the dataset property: you might want to change the default sensor repositories and use a different repository file (e.g., dataset/simrepo-5-10.n3) in order to test the performance of the cqels engine with different KB sizes loaded

##Program Parameters Acceptable params:

• rates = (double)x, // sensor stream acceleration rate (based on real world sensor observation intervals)
• queryDuplicates = (int)y, // number of duplicates to run concurrently
• duration = (long)z, // duration of the test in milliseconds
• startDate = (date in the format of "yyyy-MM-dd'T'HH:mm:ss")a, // start time of the sensor observations used
• endDate = b, // ending time of the sensor observations used
• frequency = (double)c. // fixed frequency for sensors, only has effects when rate=1.0
• engine = "cqels", "csparql" or "querystreamer" // engine to test
• query = (String) q // file names of the queries to run (under cqels_query, csparql_query or querystreamer_query), separate with ","

engine, start and end dates are mandatory.

In order to manually run the TPF Query Streamer, the file querystreamer.properties_template needs to be copied to querystreamer.properties and filled in.