Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporate reproducible data-preparation documentation #39

Open
carlhiggs opened this issue Oct 17, 2024 · 1 comment
Open

Incorporate reproducible data-preparation documentation #39

carlhiggs opened this issue Oct 17, 2024 · 1 comment

Comments

@carlhiggs
Copy link
Contributor

carlhiggs commented Oct 17, 2024

To date, I have been documenting data preparation and other development aspects in a One Note notebook. Additionally, I have been conducting these somewhat ad hoc while experimenting --- e.g. using QGIS interactively for data filtering/joining and tippecanoe at command line to generate pmtiles using the resulting data.

However, this of course is not ideal and means that some aspects --- like using tippecanoe to generate vector tile data referenced in data stories --- are not adequately documented publicly, and should processes need to be reproduced they will need to be done manually.

Moving forwards, I will be progressively documenting the process in a Quarto markdown document (.qmd file) in the data-preparation folder. R will be used to read/transform/save data, and use of additional command line utilities like felt/tippecanoe will be either described or ideally automated in that process too. The .qmd file will be rendered as a GitHub-flavoured markdown (.md) document, so that the processes used for preparing data are documented in this repository too.

The plan is to eventually transfer/consolidate the core notes from the private OneNote document related to data-prearation that I have used while developing this platform into the data-preparation document.

See the current state of the rendered data-preparation markdown document here:
https://github.com/jibeproject/jibe-vis/blob/main/data-preparation/JIBE-vis-data-preparation-R.md

This issue may be closed when historical data preparation activities (eg using QGIS and tippecanoe) have been fully documented and adapted as a reproducible workflow.

@carlhiggs
Copy link
Contributor Author

carlhiggs commented Oct 17, 2024

see:
7aaa81e
bb7a2c8
6260603
etc

In future commits related to this, I'll aim to tag this issue

carlhiggs added a commit that referenced this issue Oct 17, 2024
…on for Manchester areas, population and network as per #39
carlhiggs added a commit that referenced this issue Oct 17, 2024
carlhiggs added a commit that referenced this issue Oct 17, 2024
carlhiggs added a commit that referenced this issue Oct 18, 2024
carlhiggs added a commit that referenced this issue Oct 18, 2024
…ation using tippecanoe (finalised and checked outputs; good to go)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant