Refactor Freesound to use ProviderDataIngester (#746)

* Initial refactor steps * Move functions * Refactor other functions * Update tests * Add class to workflow list * Remove TODO note * Update DAGs.md * Fixes identified in PR review Co-authored-by: Staci Cooper <[email protected]> * Simplify some logic * Remove per-license selection logic * Simplify URL acquisition & tests Co-authored-by: Staci Cooper <[email protected]>
WordPress · Oct 12, 2022 · 8f92318 · 8f92318
1 parent 46c2c16
commit 8f92318
Show file tree

Hide file tree

Showing 4 changed files with 304 additions and 370 deletions.
diff --git a/DAGs.md b/DAGs.md
@@ -282,6 +282,8 @@ Output:                 TSV file containing the image, the respective
 
 Notes:                  https://freesound.org/apiv2/search/text'
                         No rate limit specified.
+                        This script can be run either to ingest the full dataset or
+                        as a dated DAG.
 
 
 ## `image_data_refresh`