Parse snow sensor data from CDEC

[christophertull]

The first step in this project is to pull snow sensor data from the California Data Exchange Center.

• Pull historical data for all sensors
• Create a parser than can be scheduled to pull daily snow readings

After pulling the historical data, there should be some room to divide-and-conquer with some working on creating an automated data parser and others working on machine learning from the historical data.

[christophertull]

Here is the California Data Exchange Center we will be sourcing data from: https://cdec.water.ca.gov/

[christophertull]

Outstanding items here from the meeting on 1/22 include:

• Polish datacollection.py script to fix possible copy-paste issue and incorporate reading from previous data file so that only data from last available date up to today are parsed.

• Coordinate between @tanyani and @vvt221 to avoid duplication in data parsing and writing to S3

[vvt221]

Hi Chris, Can I get the fresh keys to access the s3? Regards Vaidehi

…

On Wed, Jan 23, 2019, 2:37 PM Christopher Tull ***@***.*** wrote: Outstanding items here from the meeting on 1/22 include: - Polish datacollection.py script to fix possible copy-paste issue and incorporate reading from previous data file so that only data from last available date up to today are parsed. - Coordinate between @tanyani <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tanyani&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=aGZd1mV_HIyCfzKUSc0yVg&m=odnf_l410etcLnBiPaX9D3vZhWfFGmT8eIxaENmN0Rg&s=m3y9FmhlsB5CK_cb-voAFADgnwH8AGvtYj-cGMO7kwY&e=> and @vvt221 <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_vvt221&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=aGZd1mV_HIyCfzKUSc0yVg&m=odnf_l410etcLnBiPaX9D3vZhWfFGmT8eIxaENmN0Rg&s=fN91hW2pQzNGMMDkghuE1FsijheCCVjIhG5YPYMD18g&e=> to avoid duplication in data parsing and writing to S3 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_California-2DData-2DCollaborative_snowbot_issues_2-23issuecomment-2D456937595&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=aGZd1mV_HIyCfzKUSc0yVg&m=odnf_l410etcLnBiPaX9D3vZhWfFGmT8eIxaENmN0Rg&s=RUMLYC0vmu2xRuDwKG9Qhb4DzVD8BOZnK8gm87gK0_0&e=>, or mute the thread <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_Ao4vzyZ0qiMUNoE8bUAsEq3PPTypYonnks5vGLoUgaJpZM4Ytif9&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=aGZd1mV_HIyCfzKUSc0yVg&m=odnf_l410etcLnBiPaX9D3vZhWfFGmT8eIxaENmN0Rg&s=nYzXdZ8j4x5voXMl9a4EsbVJVYXy-aTEuZdkg67SluM&e=> .

[christophertull]

Sent! Thanks for the reminder