New testing option for ec2 (reuse existing server)

[JackUrb]

Overview

For some reason, redeploying servers to EC2 has become increasingly painful due to delays downloading the required resources. This option allows reuse of an existing server, with the following workflow:

Deploy

$ python run_task.py mephisto.architect._deploy_type=retain mephisto.architect.subdomain=retain-test +profile=sandbox

Reuse

$ python run_task.py mephisto.architect._deploy_type=use_existing mephisto.architect.subdomain=retain-test +profile=sandbox

Cleanup

$ python ../../mephisto/abstractions/architects/ec2/cleanup_ec2_server_by_name.py 
Please enter server name you want to clean up (existing servers: ['retain-test'])
>> retain-test

note: No new files are pushed to the server, so no actual changes to tasks will be reflected with this option currently. @meta-paul this may be something worth expanding on.

note 2: Only one Mephisto task can be actively serviced by a routing server, so this should not be used to connect to servers that are in-use.

[meta-paul]

For the sample commands (e.g. python ../../mephisto/abstractions/architects/ec2/cleanup_ec2_server_by_name.py) I'd make the paths such that they can be run from the root of the code repo.

[JackUrb]

For the sample commands (e.g. python ../../mephisto/abstractions/architects/ec2/cleanup_ec2_server_by_name.py) I'd make the paths such that they can be run from the root of the code repo.

In this case I ran the above from a sample folder. True proper documentation should note the frame of reference, and when the feature is documented that would be good.

[meta-paul]

note: No new files are pushed to the server - this narrows the scope of reuse, during debugging usually something is changed (code files or YAML config).

Since the main bottleneck is assembly of NodeJS part (while Python part sails through quickly), in the ideal world we'd cache the NodeJS build, while allowing to alter Python and/or YAML files. Not sure if this piecemeal caching is possible here.

[meta-paul]

if deploy type is different, I suggest to output an INFO message here saying that for deploy type X the server continue to live, and needs to be shut down manually via cleanup command.

[JackUrb]

The warning message appears on launch for the retain server currently. Feel free to add additional logging elsewhere as appropriate.

[meta-paul]

Could include here what deploy type this is

[JackUrb]

Since the main bottleneck is assembly of NodeJS part (while Python part sails through quickly), in the ideal world we'd cache the NodeJS build, while allowing to alter Python and/or YAML files. Not sure if this piecemeal caching is possible here.

Indeed - this I'd consider to be future work, requiring splitting the setup part of the server from the package/deploy part, and only running the setup part on initial launch, but instead running a content wipe for the latter.