Skip to content

Commit

Permalink
renamed kollaps and thunderstorm
Browse files Browse the repository at this point in the history
  • Loading branch information
paulojlgouveia committed Oct 11, 2019
1 parent a961ca9 commit 3084565
Show file tree
Hide file tree
Showing 654 changed files with 155,618 additions and 904 deletions.
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "need/TCAL/libutc"]
path = need/TCAL/libutc
[submodule "kollaps/TCAL/libutc"]
path = kollaps/TCAL/libutc
url = https://github.com/joaoneves792/libutc.git
26 changes: 13 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,34 +33,34 @@ RUN pacman -Sy --noconfirm \



ADD ./ /NEED/
ADD ./ /Kollaps/


# RUN sysctl net.core.rmem_max=2097152 && \
# sysctl net.core.wmem_max=2097152

RUN tar -C /NEED/ -zxvf NEED/Aeron.tar.gz && \
cp -r /NEED/Aeron/binaries /usr/bin/Aeron && \
RUN tar -C /Kollaps/ -zxvf Kollaps/Aeron.tar.gz && \
cp -r /Kollaps/Aeron/binaries /usr/bin/Aeron && \
mkdir -p /home/daedalus/Documents/aeron4need/cppbuild/Release/ && \
cp -r /NEED/Aeron/lib /home/daedalus/Documents/aeron4need/cppbuild/Release/lib && \
cp /NEED/Aeron/usr/lib/libbsd.so.0.9.1 /usr/lib/libbsd.so.0.9.1 && \
cp /NEED/Aeron/usr/lib/libbsd.so.0 /usr/lib/libbsd.so.0 && \
cp -r /Kollaps/Aeron/lib /home/daedalus/Documents/aeron4need/cppbuild/Release/lib && \
cp /Kollaps/Aeron/usr/lib/libbsd.so.0.9.1 /usr/lib/libbsd.so.0.9.1 && \
cp /Kollaps/Aeron/usr/lib/libbsd.so.0 /usr/lib/libbsd.so.0 && \
rm -f Aeron.tar.gz

# LL: only added kubernetes in l.33
RUN make -C /NEED/pid1 && \
cp /NEED/pid1/pid1 /usr/bin/pid1 && \
make -C /NEED/need/TCAL -j8 && \
RUN make -C /Kollaps/pid1 && \
cp /Kollaps/pid1/pid1 /usr/bin/pid1 && \
make -C /Kollaps/kollaps/TCAL -j8 && \
pip3 --no-cache-dir install wheel dnspython flask docker kubernetes netifaces && \
pip3 --no-cache-dir wheel --no-deps -w /NEED /NEED && \
pip3 --no-cache-dir install /NEED/need-2.0-py3-none-any.whl && \
rm -rf /NEED && \
pip3 --no-cache-dir wheel --no-deps -w /Kollaps /Kollaps && \
pip3 --no-cache-dir install /Kollaps/kollaps-1.0-py3-none-any.whl && \
rm -rf /Kollaps && \
pip3 --no-cache-dir uninstall -y setuptools wheel pip && \
pacman -R --noconfirm make gcc flex bison pkgconf && \
pacman -Scc --noconfirm


ENTRYPOINT ["/usr/bin/pid1", "/usr/bin/python3", "-m", "need.bootstrapper"]
ENTRYPOINT ["/usr/bin/pid1", "/usr/bin/python3", "-m", "kollaps.bootstrapper"]


#RUN git clone --branch master --depth 1 --recurse-submodules git@NEED:joaoneves792/NEED.git ;\
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Execute in this folder:
$pip wheel --no-deps . .
$pip install need-2.0-py3-none-any.whl
```
Installing the python package will give you access to the `NEEDdeploymentGenerator` command to translate NEED topology descriptions into Docker Swarm Compose files or Kubernetes Manifest files on your local machine. It will also give you access to the `NDLTranslator` command, which lets you declare an experiment in a language with higher-level concepts; these are then translated into XML topology descriptions.
Installing the python package will give you access to the `NEEDdeploymentGenerator` command to translate NEED topology descriptions into Docker Swarm Compose files or Kubernetes Manifest files on your local machine. It will also give you access to the `ThunderstormTranslator` command, which lets you declare an experiment in a language with higher-level concepts; these are then translated into XML topology descriptions.

### Building the NEED image
You also need to build the NEED Docker image. To do so, execute in this folder:
Expand Down
2 changes: 1 addition & 1 deletion examples/1000nodes_m1_scalefree.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="n666" image="warpenguin.no-ip.org/pingclient:vm" command="['n173', '600', '158']" />
Expand Down
2 changes: 1 addition & 1 deletion examples/2000nodes_m1_scalefree.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:vm" supervisor="true" port="8088"/>
<service name="n1344" image="warpenguin.no-ip.org/pingclient:vm" command="['n738', '600', '86']" />
Expand Down
2 changes: 1 addition & 1 deletion examples/500nodes_m1_scalefree.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="n342" image="warpenguin.no-ip.org/pingclient:vm" command="['n132', '600', '150']" />
Expand Down
2 changes: 1 addition & 1 deletion examples/simple_dynamic.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology100.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology200.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology400.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology5.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology50.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology5_losses.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_dynamic.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_dynamic_links.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="client1" image="warpenguin.no-ip.org/alpineclient:1.0" command="['server', '0', '0']"/>
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_dynamic_replicas.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_ping.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="client1" image="alpine:edge" command="['ping', 'client3-$$NEED_UUID']"/>
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_ring128.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_ring16.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_ring32.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_ring64.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
2 changes: 1 addition & 1 deletion examples/topology_simple.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" ?>
<experiment boot="need:2.0">
<experiment boot="kollaps:1.0">
<services>
<service name="dashboard" image="warpenguin.no-ip.org/dashboard:1.0" supervisor="true" port="8088"/>
<service name="logger" image="warpenguin.no-ip.org/logger:1.0" supervisor="true" />
Expand Down
10 changes: 5 additions & 5 deletions need/Dashboard.py → kollaps/Dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@

from flask import Flask, render_template, request, flash, redirect, url_for, jsonify, json

from need.NEEDlib.CommunicationsManager import CommunicationsManager
from need.NEEDlib.NetGraph import NetGraph
from need.NEEDlib.XMLGraphParser import XMLGraphParser
from need.NEEDlib.utils import int2ip, ip2int
from need.NEEDlib.utils import print_message, print_error, print_and_fail, print_named
from kollaps.Kollapslib.CommunicationsManager import CommunicationsManager
from kollaps.Kollapslib.NetGraph import NetGraph
from kollaps.Kollapslib.XMLGraphParser import XMLGraphParser
from kollaps.Kollapslib.utils import int2ip, ip2int
from kollaps.Kollapslib.utils import print_message, print_error, print_and_fail, print_named

import sys
if sys.version_info >= (3, 0):
Expand Down
10 changes: 5 additions & 5 deletions need/EmulationManager.py → kollaps/EmulationManager.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#! /usr/bin/python

from need.NEEDlib.NetGraph import NetGraph
from need.NEEDlib.XMLGraphParser import XMLGraphParser
from need.NEEDlib.EmulationCore import EmulationCore
from need.NEEDlib.utils import ENVIRONMENT, int2ip, ip2int, setup_container
from need.NEEDlib.utils import print_and_fail, print_message, print_identified
from kollaps.Kollapslib.NetGraph import NetGraph
from kollaps.Kollapslib.XMLGraphParser import XMLGraphParser
from kollaps.Kollapslib.EmulationCore import EmulationCore
from kollaps.Kollapslib.utils import ENVIRONMENT, int2ip, ip2int, setup_container
from kollaps.Kollapslib.utils import print_and_fail, print_message, print_identified

from signal import signal, SIGTERM
import socket
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@

from need.NEEDlib.NetGraph import NetGraph
import need.NEEDlib.PathEmulation as PathEmulation
from need.NEEDlib.EventScheduler import EventScheduler
from need.NEEDlib.utils import start_experiment, stop_experiment, BYTE_LIMIT, SHORT_LIMIT
from need.NEEDlib.utils import LOCAL_IPS_FILE, REMOTE_IPS_FILE, AERON_LIB_PATH
from need.NEEDlib.utils import int2ip, ip2int, print_identified, print_error, print_and_fail, print_message, print_error_named
from kollaps.Kollapslib.NetGraph import NetGraph
import kollaps.Kollapslib.PathEmulation as PathEmulation
from kollaps.Kollapslib.EventScheduler import EventScheduler
from kollaps.Kollapslib.utils import start_experiment, stop_experiment, BYTE_LIMIT, SHORT_LIMIT
from kollaps.Kollapslib.utils import LOCAL_IPS_FILE, REMOTE_IPS_FILE, AERON_LIB_PATH
from kollaps.Kollapslib.utils import int2ip, ip2int, print_identified, print_error, print_and_fail, print_message, print_error_named

from threading import Thread, Lock
from multiprocessing import Pool
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
from threading import Lock
from os import environ, getenv

from need.NEEDlib.NetGraph import NetGraph
from need.NEEDlib.EventScheduler import EventScheduler
import need.NEEDlib.PathEmulation as PathEmulation
from need.NEEDlib.CommunicationsManager import CommunicationsManager
from need.NEEDlib.utils import ENVIRONMENT
from need.NEEDlib.utils import print_named, print_message, print_identified
from kollaps.Kollapslib.NetGraph import NetGraph
from kollaps.Kollapslib.EventScheduler import EventScheduler
import kollaps.Kollapslib.PathEmulation as PathEmulation
from kollaps.Kollapslib.CommunicationsManager import CommunicationsManager
from kollaps.Kollapslib.utils import ENVIRONMENT
from kollaps.Kollapslib.utils import print_named, print_message, print_identified

import sys
if sys.version_info >= (3, 0):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

from need.NEEDlib.utils import start_experiment, stop_experiment, crash_experiment, print_message
from need.NEEDlib.PathEmulation import disconnect, reconnect, change_latency, change_loss
from need.NEEDlib.NetGraph import NetGraph
from kollaps.Kollapslib.utils import start_experiment, stop_experiment, crash_experiment, print_message
from kollaps.Kollapslib.PathEmulation import disconnect, reconnect, change_latency, change_loss
from kollaps.Kollapslib.NetGraph import NetGraph

from threading import Timer
from copy import copy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import os
import dns.resolver

from need.NEEDlib.utils import print_and_fail, ip2int
from kollaps.Kollapslib.utils import print_and_fail, ip2int

import sys
if sys.version_info >= (3, 0):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from need.NEEDlib.NetGraph import NetGraph
from kollaps.Kollapslib.NetGraph import NetGraph
from threading import Lock
from ctypes import CDLL, c_float, CFUNCTYPE, c_voidp, c_int, c_ulong, c_uint
from os import path
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@

import sys
import ast
import random
import datetime
from xml.etree import ElementTree as ET
from xml.dom import minidom
from need.NEEDlib.NDLParser import BootstrapperDeclaration, NodeDeclaration, BridgeDeclaration, LinkDeclaration, EventDeclaration
from kollaps.Kollapslib.Thunderstorm.Parser import BootstrapperDeclaration, NodeDeclaration, BridgeDeclaration, LinkDeclaration, EventDeclaration

if sys.version_info >= (3, 0):
from typing import Dict, List, Tuple
Expand Down Expand Up @@ -455,7 +456,7 @@ def makeXML():
def addInfo(seed):
now = datetime.datetime.now()
e = ET.SubElement(topology, "informations")
e.attrib["source"] = "Topology compiled with NDLTranslator on " + now.strftime("%d.%m.%Y %H:%M")
e.attrib["source"] = "Topology compiled with ThunderstormTranslator on " + now.strftime("%d.%m.%Y %H:%M")
e.attrib["seed"] = str(seed)

def ndl_generate(declarations, seed=12345):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

import re
import ply.lex as lex #pip install ply
import ply.yacc as yacc
Expand Down Expand Up @@ -141,22 +142,22 @@ def t_error(t):
#Build the lexer
lex.lex()

class NDLDeclaration:
class ThunderstormDeclaration:
pass

class BootstrapperDeclaration(NDLDeclaration):
class BootstrapperDeclaration(ThunderstormDeclaration):
pass

class NodeDeclaration(NDLDeclaration):
class NodeDeclaration(ThunderstormDeclaration):
pass

class BridgeDeclaration(NDLDeclaration):
class BridgeDeclaration(ThunderstormDeclaration):
pass

class LinkDeclaration(NDLDeclaration):
class LinkDeclaration(ThunderstormDeclaration):
pass

class EventDeclaration(NDLDeclaration):
class EventDeclaration(ThunderstormDeclaration):
pass

class Time:
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
from random import choice, randint, seed, randrange
from string import ascii_letters

from need.NEEDlib.utils import print_and_fail, print_message
from need.NEEDlib.NetGraph import NetGraph
from need.NEEDlib.EventScheduler import EventScheduler
from kollaps.Kollapslib.utils import print_and_fail, print_message
from kollaps.Kollapslib.NetGraph import NetGraph
from kollaps.Kollapslib.EventScheduler import EventScheduler

import sys
if sys.version_info >= (3, 0):
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
from multiprocessing import Process
from time import sleep

from need.NEEDlib.utils import int2ip, ip2int
from need.NEEDlib.utils import print_error, print_and_fail, print_named, print_error_named
from need.NEEDlib.utils import LOCAL_IPS_FILE, REMOTE_IPS_FILE, GOD_IPS_SHARE_PORT
from kollaps.Kollapslib.utils import int2ip, ip2int
from kollaps.Kollapslib.utils import print_error, print_and_fail, print_named, print_error_named
from kollaps.Kollapslib.utils import LOCAL_IPS_FILE, REMOTE_IPS_FILE, GOD_IPS_SHARE_PORT


class Bootstrapper(object):
Expand Down
Loading

0 comments on commit 3084565

Please sign in to comment.