-
Notifications
You must be signed in to change notification settings - Fork 1
Simian army and ha documentation
We ported the Chaos Monkey in order to test and demonstrate the HA resilience of the rest of our porting work in the SoftLayer environment.
This was more complex than originally expected, because the Amazon EC2 concepts were sprinkled across more of the code than the abstraction model first implies.
The base Monkey class has been slightly restructured. The restructuring included the use Java Generics to allow for different implementation of CloudClients be used along with reusing as much of the basic implementation classes as possible. The basic abstraction of cloudclient was there, but incomplete as many places still relied on the AWSClient implementation. We then created an AbstractSimianArmyContext, which can be extended by both AWS and SoftLayer implementations, with the most reuse of code.
The ChaosMonkey provides implementations of the Email Notifier, a simple in-memory recorder, and client for controlling instances.
The SoftLayer chaos crawler is similar to the AWS one. We implemented the client, to retrieve running instances from Asgard (rather than SoftLayer or RightScale) because it provided the greatest portability. This collects and organizes instances in the same way as the original implementation.
The recorder is a copy of the in-memory recorder, which is used for testing. It would be a trivial next step to swap to using the DB instance provided for SoftLayer.
We restructured the email notifier to allow other email providers to be utilized. Due to timing, we didn't complete the restructure, so the chaos notifier was just a cut and paste of the original one. This proved portability and functionality, but not the complete abstraction. It would be straight forward to finish this restructuring. That would have the additional benefit of allowing reuse of a common chaos notifier.
We reimplemented the client to make use of the SoftLayer REST API. We also did some [investigation] (https://github.com/EmergingTechnologyInstitute/SoftLayer-NetflixOSS/wiki/Portability-layer#apache-jclouds-as-an-api) in using the jclouds API for controlling the instance, but ran into a known issue with the jclouds library and SoftLayer.
The following are properties added to configure the chaos monkey for SoftLayer.
- simianarmy.client.context.class=com.netflix.simianarmy.softlayerSoftLayerSimianArmyContext
- simianarmy.client.softlayer.account SoftLayer account API user
- simianarmy.client.softlayer.apiSecret SoftLayer secret for accessing API
- simianarmy.client.asgard.userid Userid for asgard
- simianarmy.client.asgard.password Password for asgard
- simianarmy.client.asgard.server Server hostname of asgard
- simianarmy.client.softlayer.region Asgard region to crawl for instances
The following are properties added to configure the simian army email notifier for SoftLayer.
- simianarmy.emailnotifier.sendgrid.username SendGrid userid for SoftLayer notifier
- simianarmy.emailnotifier.sendgrid.password SendGrid password for SoftLayer notifier