Canvas API may return duplicate courses

[ssciolla]

Today, the two jobs in OpenShift failed because of duplicate course primary keys, for different courses. This appears to be unexpected API behavior from the account/{account_id}/courses endpoint, as I don't believe this has happened before. Still, we should likely modify the existing code to drop_duplicates and note how many courses were dropped, so we can avoid this error while also monitoring API anomalies.

[ssciolla]

I discussed the problem with @jonespm, as this may also occur with his use of canvasapi in the CANVAS_ZOOM_MEETINGS job. Ultimately we'll want to avoid fetching the same data multiple times (see issue #162), but we'll need to handle this there as part of resolving this issue in the short-term.

[jonespm]

It was mentioned on #176 that we might want to move the code from get_lti_tabs out one more level and only call this once.

[jonespm]

This one is not yet finished. We could probably bump it with #162 to the next release but I think it could be finished by itself tomorrow. I've never seen any case where there were duplicate courses returned as part of this process.

[jonespm]

I believe because of chances of this happening are low and the entire process should be changed in #162 I'm going to close this for now. Doing this would require:

• Retrieving all courses in a term
• Removing Duplicates (if any)
• Add in ADD_COURSE_IDS courses that aren't already in the list
• Process all these courses

This should all be refactored into one place to populate the courses table so it can be used by future classes.