-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathrelatedtestbed.html
298 lines (250 loc) · 10.2 KB
/
relatedtestbed.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" type="text/css" href="testbed.css">
</head>
<body>
<h2 align="center">Existing testbeds mapped into the Tsumiki model</h2>
<!--
<img src="tsumiki-related-work-table.jpg" alt="related work"
style="width:912px;height:570px">
//-->
<p><font size="5">This is a <i>work-in-progress</i> based on our paper under
submission (title and abstract listed below). Please <a
us</a> if you find an error. </font></p>
<h3>Tsumiki: A Meta-Platform for Building Your Own Testbed</h3>
<h4>Abstract</h4>
<p>Network testbeds are an essential research tool that have been
responsible for major advances in distributed systems and network
measurements. However, no single testbed can satisfy the requirements
of every research project, prompting continual efforts to develop
new testbeds.<br>
We present Tsumiki, a set of reusable components for building network
testbeds. These components are repurposeable and interchangeable in
deriving new kinds of testbeds. We demonstrate that Tsumiki components
are well-suited to implementing new, diverse, and useful types of
testbeds. We describe three testbeds that were built using Tsumiki
components and demonstrate their utility. Across these deployments,
Tsumiki components have been installed on tens of thousands of devices
and the testbeds had been used by thousands of researchers. We also
detail a user-study with seven graduate students in which we evaluated
the effort involved in using Tsumiki to build and deploy custom
testbeds. Our results indi- cate that students with no prior
experience with networked testbeds are able to use Tsumiki to create a
testbed with new functionality and run an experiment in under an
hour. <br></p>
<p><b>Note:</b> The following testbeds are ordered alphabetically. </p>
<table border="1" style="width:100%">
<TR>
<TH colspan="2" align="center">Testbed</TH>
<TH>Sandbox</TH>
<TH>Resource manager</TH>
<TH>Device manager</TH>
<TH>Experiment management tool</TH>
<TH>Custom installer builder</TH>
<TH>Clearinghouse</TH>
<TH>Lookup service</TH>
<TH>Reference publication</TH>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://projectbismark.net/">BISmark</a></TD>
<TD>BISmark routers <sup><a href="#fn1" id="ref1">1</a></sup></TD>
<TD>Measurement servers</TD>
<TD>Custom package management utilities on top of OpenWrt's
built-in <font face="verdana">opkg</font></TD>
<TD>Data collection servers, <font
face="verdana">bismark-data-transmit</font> </TD>
<TD>Package repository servers</TD>
<TD>User registration system</TD>
<TD>Monitoring servers</TD>
<TD><a
href="http://dl.acm.org/citation.cfm?id=2018452">SIGCOMM'11</a>,
<a href="https://www.usenix.org/conference/atc14/technical-sessions/presentation/sundaresan">ATC'14</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a
href="http://boinc.berkeley.edu/">BOINC</a></TD>
<TD>BOINC client, general preferences</TD>
<TD>Scheduling servers, data servers</TD>
<TD>BOINC installer tool</TD>
<TD>Client GUI</TD>
<TD>anonymous platform mechanism</TD>
<TD>Project master URL</TD>
<TD>NA</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=1033223">GC'04</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a
href="http://www.aqualab.cs.northwestern.edu/projects/115-dash-dual-purpose-platform">
Dasu</a></TD>
<TD>Dasu client's experiment sandbox</TD>
<TD>Configuration Service</TD>
<TD>NA</TD>
<TD>Data Service, Secondary Experiment Administration Service</TD>
<TD>NA</TD>
<TD>Primary Experiment Administration Service</TD>
<TD>Coordination Service</TD>
<TD><a href="https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/sanchez">NSDI'13</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://deter-project.org/">DETER</a></TD>
<TD>Container</TD>
<TD>Container, Risky Experiment Management</TD>
<TD>Same as Emulab (?)</TD>
<TD>SEER, Experiment Lifecycle Manager</TD>
<TD colspan="3" align="center">Same as Emulab (?)</TD>
<TD><a href="https://www.acsac.org/2011/program/keynotes/benzel.pdf">ACSAC'11</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://www.netdimes.org/new/">DIMES</a></TD>
<TD>DIMES agent</TD>
<TD>Experiment planning system</TD>
<TD>Auto update mechanism</TD>
<TD>Management system</TD>
<TD>NA</TD>
<TD>Experiment planning system</TD>
<TD>Discovery</TD>
<TD><a href="http://arxiv.org/pdf/cs/0506099.pdf">CCR'05</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://www.emulab.net/">Emulab</a></TD>
<TD>Virtual nodes, OpenVZ linux containers, Xen <sup><a
href="#fn2" id="ref2">2</a></sup> </TD>
<TD>Global resource allocation process <font face="verdana">assign</font></TD>
<TD>Control infrastructure</TD>
<TD>Emulab web interface</TD>
<TD colspan="3" align="center">masterhost</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=844152">OSDI'02</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://fathom.icsi.berkeley.edu/">Fathom</a></TD>
<TD>JavaScript extension for Firefox</TD>
<TD>Fathom object, API, socket worker</TD>
<TD>NA</TD>
<TD>Firefox browser tab</TD>
<TD>Fathom script</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=2398786">IMC'12</a></TD>
</TR>
<TR>
<TD colspan="2" align="center">Flexlab</TD>
<TD>Emulab container</TD>
<TD>Emulab's control infrastructure</TD>
<TD>Application monitor</TD>
<TD>Experimentation workbench <sup><a href="#fn3"
id="ref3">3</a></sup></TD>
<TD>NA</TD>
<TD>Emulab's masterhost</TD>
<TD>NA</TD>
<TD><a href="https://www.cs.utah.edu/flux/papers/flexlab-nsdi07.pdf">NSDI'07</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://modelnet.ucsd.edu/">ModelNet</a></TD>
<TD>Virtual edge node, core node</TD>
<TD>Binding, Run phases (?)</TD>
<TD>Assignment phase (?)</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD><a
href="https://www.usenix.org/legacy/event/osdi02/tech/full_papers/vahdat/vahdat.pdf">
OSDI'02</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="https://www.planet-lab.org/">PlanetLab</a></TD>
<TD>VM, slice</TD>
<TD>Node manager</TD>
<TD>Node manager, boot CD</TD>
<TD>Slice creation service, ssh</TD>
<TD colspan="3" align="center">PlanetLab Central</TD>
<TD><a
href="http://www.cs.princeton.edu/courses/archive/fall03/cs597B/handouts/pdn02-001.pdf">CCR'03</a>,
<a href="http://dl.acm.org/citation.cfm?id=1298489">OSDI'06</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://nms.csail.mit.edu/ron/">RON</a></TD>
<TD>RON client, <font face="verdana">conduit</font> API <sup><a href="#fn4" id="ref4">4</a></sup></TD>
<TD>RON router, forwarder (?)</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD>NA</TD>
<TD>RON membership manager, flooder</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=502048">SOSP'01</a></TD>
</TR>
<TR>
<TD rowspan="2"><a
href="http://satellitelab.mpi-sws.org/">SatelliteLab</a></TD>
<TD>Planet</TD>
<TD colspan="7" align="center">Same as PlanetLab</TD>
<TD rowspan="2"><a href="http://dl.acm.org/citation.cfm?id=1402994">SIGCOMM'08</a></TD>
</TR>
<TD>Satellite</TD>
<TD>JVM (does not permit experimenter code execution)</TD>
<TD>None</TD>
<TD>None</TD>
<TD>Manual</TD>
<TD>OS-specific installers</TD>
<TD>Manual allocation</TD>
<TD>Heartbeat mechanism <sup><a href="#fn5" id="ref5">5</a></sup></TD>
<TR>
<TD colspan="2" align="center"><a href="https://seattle.poly.edu/">Seattle</a></TD>
<TD>Vessel</TD>
<TD>Node manager</TD>
<TD>Software updater</TD>
<TD><font face="verdana">seash</font>, etc.</TD>
<TD>Seattle custom installer builder</TD>
<TD>Seattle clearinghouse</TD>
<TD>Adtertising service</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=1508905">SIGCSE'09</a></TD>
</TR>
<TR>
<TD colspan="2" align="center"><a href="http://www.splay-project.org/">SPLAY</a></TD>
<TD>Sandboxed <sup><a href="#fn6" id="ref6">6</a></sup> process
forked by <font face="verdana">splayd</font>
and libraries</TD>
<TD>Daemon <font face="verdana">splayd</font></TD>
<TD>NA</TD>
<TD>Command line, Web interface, churn management system</TD>
<TD>NA</TD>
<TD>Controller <font face="verdana">splayctl</font></TD>
<TD>Controller (keeps track of all daemons and processes)</TD>
<TD><a href="https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/leonini/leonini.pdf">NSDI'09</a></TD>
</TR>
<TR>
<TD colspan="2" align="center">VINI</TD>
<TD>PlanetLab slice, extended CPU scheduling, network device isolation</TD>
<TD colspan="6" align="center">Same as PlanetLab</TD>
<TD><a href="http://dl.acm.org/citation.cfm?id=1159916">SIGCOMM'06</a></TD>
</TR>
</table>
<br>
<h3>Other testbeds with no publication</h3>
<h4>1. Speed testing tools</h4>
<h4>2. Special purpose testbeds</h4>
<br>
<hr></hr>
<sup id="fn1">1. Netgear WNDR 3800 routers running a custom version of
OpenWrt distribution. <a href="#ref1" title="Jump back to footnote 1
in the text.">back</a></sup><br>
<sup id="fn2">2. <a href="https://wiki.emulab.net/wiki/vnodes">Emulab
wiki</a>. <a href="#ref2" title="Jump back to footnote 2 in the
text.">back</a></sup><br>
<sup id="fn3">3. "An Experimentation Workbench for Replayable
Networking Research", NSDI'07, <a
href="https://www.usenix.org/legacy/event/nsdi07/tech/full_papers/eide/eide_html/">link</a>. <a
href="#ref3" title="Jump back to footnote 3 in the
text.">back</a></sup><br>
<sup id="fn4">4. The API that a RON client uses to send and receive
packets. <a href="#ref4" title="Jump back to footnote 4 in the
text.">back</a></sup><br>
<sup id="fn5">5. The satellite helper periodically sends a small
status message to its planet proxy. <a href="#ref5" title="Jump back
to footnote 5 in the text.">back</a></sup><br>
<sup id="fn6">6. A Lua-based sandbox. <a href="#ref6" title="Jump back
to footnote 6 in the text.">back</a></sup><br>
</body>
</html>