Setup the tripaware system for a live demo

[shankari]

Buildsys is coming up and we would like to have the tripaware system running with a live demo.
This bug tracks the changes required to recreate the tripaware system with a server running on my home server.

[shankari]

First step was to get the server running. The tripaware project included server-side changes, so I pulled in their version of the code into a branch (https://github.com/e-mission/e-mission-server/tree/urap-2017) using PR e-mission/e-mission-server#723

There were merge conflicts, but they were not significant.

[shankari]

I then created a Docker container from this branch. I did so by using the standard docker-compose file for the e-mission server (e.g. https://github.com/e-mission/e-mission-docker/blob/master/examples/juypter-notebook/docker-compose.yml) but with a Dockerfile that pulls from this branch. I also copy the appropriate config files over to ensure google auth. I then set up a reverse proxy from my apache http instance to this new docker container.

The custom Dockerfile is

FROM emission/e-mission-server:latest

RUN git fetch origin urap-2017
RUN git checkout -f urap-2017

COPY conf/net/ext_service/push.json /usr/src/app/conf/net/ext_service/push.json
COPY conf/net/ext_service/nominatim.json /usr/src/app/conf/net/ext_service/nominatim.json
COPY conf/net/auth/google_auth.json /usr/src/app/conf/net/auth/google_auth.json
COPY conf/net/api/webserver.conf /usr/src/app/conf/net/api/webserver.conf.sample

CMD ["/bin/bash", "/usr/src/app/start_script.sh"]

Note that for the webserver only, we copy the conf file to conf/net/api/webserver.conf.sample. This is because the start_script.sh from the emission/e-mission-server:latest image overwrites the conf/net/api/webserver.conf with the WEB_SERVER_HOST variable from the docker-compose.yml

[shankari]

I then updated the phone connection config for the emotion app only to the new server
https://tripaware.damajash.org/

There have been multiple changes to the phone app on master since then and there appear to be some non-trivial merge conflicts.
e-mission/e-mission-phone#612

I didn't want to resolve them without sufficient testing time.

[shankari]

@jackcsullivan @jesbu1 I have now switched to google auth for you to retest. @jesbu1 make sure that you are testing the emotion UI since that is the only channel that I updated

[shankari]

Also, I haven't turned on any of the cronjobs yet. I can run the existing pipeline in a cronjob, but is there anything else that needs to run periodically?

[shankari]

ok so I tried to test in the emulator and with creating fake trips. I got the error

web-server_1  | Traceback (most recent call last):
web-server_1  |   File "/usr/src/app/emission/net/api/bottle.py", line 997, in _handle
web-server_1  |     out = route.call(**args)
web-server_1  |   File "/usr/src/app/emission/net/api/bottle.py", line 1998, in wrapper
web-server_1  |     rv = callback(*a, **ka)
web-server_1  |   File "emission/net/api/cfc_webapp.py", line 390, in createUserProfile
web-server_1  |     TierSys.addUser(user.uuid)
web-server_1  |   File "/usr/src/app/emission/core/wrapper/tiersys.py", line 113, in addUser
web-server_1  |     newTier = TierSys.getNewUserTier()[0]
web-server_1  |   File "/opt/conda/envs/emission/lib/python3.6/site-packages/pymongo/cursor.py", line 586, in __getitem__
web-server_1  |     raise IndexError("no such item for Cursor instance")
web-server_1  | IndexError: no such item for Cursor instance

It looks like the problem is that TierSys.getNewUserTier() returns nothing.

>>> list(edb.get_new_tier_db().find().sort('created_at', -1).limit(1))
[]

So it looks like we need to add tiers somehow. It looks like this is called from bin/make_tiersys.py.
Let's try running that first.

./e-mission-py.bash bin/make_tiersys.py
DEBUG:root:[{'rank': 1, 'users': []}, {'rank': 2, 'users': []}, {'rank': 3, 'users': []}]
[{'rank': 1, 'users': []}, {'rank': 2, 'users': []}, {'rank': 3, 'users': []}]

>>> import emission.core.get_database as edb
Connecting to database URL db
>>> edb.get_new_tier_db().find().sort('created_at', -1).limit(1)[0]
{'_id': ObjectId('5dc4f09c1e81fb0035a22f09'), 'users': [], 'created_at': datetime.datetime(2019, 11, 8, 4, 35, 40, 956000)}

Ok so now we have a user tier...

[shankari]

So it looks like we may need to manually run scripts in bin. The full list of scripts in bin (from e-mission/e-mission-server#723) is

	new file:   bin/check_for_notifications.py
	new file:   bin/make_tiersys.py
	new file:   bin/update_polarbear.py

And I don't see them being run from the intake pipeline (https://github.com/e-mission/e-mission-server/pull/723/files#diff-3978c499fd0cc4554325f74d219117d1) So I suspect we need to run all of them in a separate cronjob. But let's first try running the pipeline without them to confirm.

[shankari]

ok so I ran the pipeline for this user, and I can confirm that at least one trip showed up. But there are no polar bears and no entries in the leaderboard. Let's try manually running the three scripts above.

[shankari]

After running the scripts, I can see a polar bear and one user in the leaderboard. So it looks like we need to run the three scripts separately from the regular pipeline. When we clean this up, we should really add the scripts as additional pipeline steps.

[shankari]

also checked in a fix to the client to comment out the survey step since the survey is not accepting any responses.

[shankari]

@PatGendre you might want to try this out to see if it is useful for the Nantes deployment. It will be running at least until the end of Buildsys, so the end of next week. Again, instructions are:

• go to https://tripaware.damajash.org
• refresh until you see polar bears
• follow the instructions

Alternatively, if you have emTripLog already installed, you can just use this link
https://e-mission.eecs.berkeley.edu/#/client_setup?new_client=urap2017emotion&clear_usercache=true&clear_local_storage=true

[PatGendre]

@shankari thanks for having deployed the app, I installed it, it worked fine :-) the Nantes team has a very tight delay/budget constraint but at least I will show this UI to them.

[shankari]

@PatGendre It is because the Nantes team has tight delay/budget constraints that I thought this may be useful. They don't have to develop something new, they can use this as the base, merge from master, and translate the text.

Also, the students did a careful behavioral design - e.g. instead of a single leaderboard, they created tiers so that people wouldn't feel disappointed that the top was too far away for them to reach. Instead, they could compete with people from their own tier, etc. The behavioral background is in the tech report https://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-2.html