refactor: shared dependencies unification #1163

mgorsk1 · 2021-05-27T16:03:19Z

Summary of Changes

Following merging amundsen-io/rfcs#38 this PR unifies dependencies across amundsen services.

Assumptions

global unification (& upgrade to more recent vrersions) of libraries has been conducted (every repo now uses same version for pytest, flake8 and so on). if there were mulitple versions of library X across different projects (for example - we had 3 different versions of mypy - 0.660, 0.782 and 0.812 across frontend,metadata and search) - the most recent one was selected for unification (in case of mypy - 0.812)
any linting issues that has arisen because of the previous point are solved by excluding certain types of errors (fixing linting in whole repo would make this PR a major pain to review and resolve)
databuilder is excluded from requirements-common.txt unification

Tests

all existing linter rules passing
all existing unit tests passing

TODO

verify correctness of building docker images

Documentation

updated appropriate README.md and installation.md files

CheckList

Make sure you have checked all steps below to ensure a timely review.

PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
- In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
PR includes a summary of changes.
PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain docstrings that explain what it does

frontend/Makefile

sewardgw

Thanks for the work here! clearly a big effort for a much needed change. A few comments / questions inline

Dockerfile.frontend.local

search/Makefile

metadata/requirements-common.txt

sewardgw · 2021-06-01T16:46:29Z

One last thought - worth squashing some of the commits?

mgorsk1 · 2021-06-01T16:47:13Z

One last thought - worth squashing some of the commits?

I always Squash and Merge so main stays decluttered but good point, it's a messy one!

mgorsk1 · 2021-06-01T16:50:24Z

I think this requires approval of codeowner of all repos to merge, are you up for review @feng-tao ?

Signed-off-by: mgorsk1 <[email protected]>

* feat: Apache Superset Extractor (amundsen-io#1076) Adds support for syncing dashboard metadata from Apache Superset. * boring_cyborg | 🎉 Initial commit. (amundsen-io#1101) Signed-off-by: mgorsk1 <[email protected]> * fix: Try to use the app context (amundsen-io#1102) Signed-off-by: verdan <[email protected]> * feat: Apache Superset UI Icons (amundsen-io#1100) adds Apache Superset icons to frontend * feat: Pre-work needed for the lineage graph visualization (amundsen-io#1084) * parent f6b21b3 author verdan <[email protected]> 1620122095 +0200 committer verdan <[email protected]> 1620776994 +0200 initial commit Fixes the lineage query, updates the common model more updates updates the look and feel removes commented code Signed-off-by: verdan <[email protected]> passes the paramters to backend adds the zooming feature code refactor, addresses review comments Signed-off-by: verdan <[email protected]> docs: update faq.md (amundsen-io#1038) Signed-off-by: mgorsk1 <[email protected]> Modified methods to throw NOTFOUNDEXCEPTION instead RuntimeEror in neo4j_proxy (metadata_service) (amundsen-io#1028) Signed-off-by: aparna-cimpress <[email protected]> perf: Update ModeDashboardLastModifiedTimestampExtractor to use Mode discovery api (amundsen-io#1040) * update ModeDashboardLastModifiedTimestampExtractor to use Mode discovery api Signed-off-by: Junda Yang <[email protected]> * add empty line at the end of the file Signed-off-by: Junda Yang <[email protected]> * update comment Signed-off-by: Junda Yang <[email protected]> * fix lint Signed-off-by: Junda Yang <[email protected]> * fix import Signed-off-by: Junda Yang <[email protected]> ci: fix metadata ci config (amundsen-io#1039) Signed-off-by: Dorian Johnson <[email protected]> perf: update ModeDashboardLastSuccessfulExecutionExtractor to discovery api (amundsen-io#1042) * update ModeDashboardLastSuccessfulExecutionExtractor to discovery api Signed-off-by: Junda Yang <[email protected]> * lint Signed-off-by: Junda Yang <[email protected]> * remove unused import Signed-off-by: Junda Yang <[email protected]> perf: Update ModeDashboardOwner extractor to use Mode discovery api (amundsen-io#1043) * update ModeDashboardOwnerExtractor to discovery api Signed-off-by: Junda Yang <[email protected]> * update import Signed-off-by: Junda Yang <[email protected]> update ModeDashboardQueryExtractor to discovery api (amundsen-io#1044) Signed-off-by: Junda Yang <[email protected]> chore: fix databuilder dep typo (amundsen-io#1046) Signed-off-by: feng-tao <[email protected]> build(deps): bump py from 1.5.3 to 1.10.0 in /databuilder (amundsen-io#1049) chore: Update ModeDashboardLastModifiedTimestampExtractor typing (amundsen-io#1048) update Mode dashboard extractor tests (amundsen-io#1047) fix: Temporarily fix potential 404 in mode dashboard executions extractor (amundsen-io#1051) * fix ModeDashboardExecutionsExtractor Signed-off-by: Junda Yang <[email protected]> * nit Signed-off-by: Junda Yang <[email protected]> test: refactor: update test and directory of ModeDashboardChartsBatchExtractor (amundsen-io#1052) Temporarily fix potential 404 in ModeDashboardUsageExtractor (amundsen-io#1053) feat: Add dbt extractor for table lineage + metadata (amundsen-io#1015) * converted dbt extractor to mono repo Signed-off-by: Grant Seward <[email protected]> * Flake8 formatting Signed-off-by: Grant Seward <[email protected]> * Updates per feedback from Convoy Signed-off-by: Grant Seward <[email protected]> * Commented sample logging Signed-off-by: Grant Seward <[email protected]> * Added dbt to main amundsen README Signed-off-by: Grant Seward <[email protected]> fix bug in rest_api_query (amundsen-io#1056) Signed-off-by: Junda Yang <[email protected]> feat: Add merge query results functionality to rest_api_query (amundsen-io#1058) * Add merge query results functionality Signed-off-by: Junda Yang <[email protected]> * fix mypy Signed-off-by: Junda Yang <[email protected]> * nit Signed-off-by: Junda Yang <[email protected]> chore: fix gh page build workflow (amundsen-io#1062) Signed-off-by: feng-tao <[email protected]> chore: add mailmap to associate the right user (amundsen-io#1061) Signed-off-by: feng-tao <[email protected]> fix: Update ModeDashboardExtractor to Mode discovery api (amundsen-io#1063) * Update ModeDashboardExtractor to Mode discovery api Signed-off-by: Junda Yang <[email protected]> * nit Signed-off-by: Junda Yang <[email protected]> chore: delete legacy ModeDashboardChartsExtractor (amundsen-io#1064) * remove-legacy-charts-extractor Signed-off-by: Junda Yang <[email protected]> * nit Signed-off-by: Junda Yang <[email protected]> bump up databuilder version (amundsen-io#1065) Signed-off-by: Junda Yang <[email protected]> fix in another PR Signed-off-by: verdan <[email protected]> makes Verdan and Dorian the code owners (amundsen-io#1081) Signed-off-by: verdan <[email protected]> Adds a new "parent" field in lineage model (amundsen-io#1080) * adds 'parent' field to help build the graph Signed-off-by: verdan <[email protected]> * make the default to None Signed-off-by: verdan <[email protected]> deploy_release: on release, publish to pypi/docker (amundsen-io#1079) Given a release with a title of `<module_name> <version_number>`, this will push that release to PyPi and docker. Note that this does not attempt to create the changelog or any of that stuff. That will need to be done by a separate action. This creates a nice modularity boundary, in that this action doesn't care where the release came from (manual or another automated step) Signed-off-by: Dorian Johnson <[email protected]> Fixes the test cases Signed-off-by: verdan <[email protected]> Removes extra functionality Signed-off-by: verdan <[email protected]> removes redundant flag Signed-off-by: verdan <[email protected]> * FIxes the betterer issues Signed-off-by: verdan <[email protected]> * code review changes Signed-off-by: verdan <[email protected]> * updates the common version Signed-off-by: verdan <[email protected]> * Fixes lint errors for metadata Signed-off-by: verdan <[email protected]> * feat: Atlas Dashboard Support (amundsen-io#1066) - metadata: extended atlas proxy with dashboard support - databuilder: extended atlas search extractor with dashboard support * feat: ML Features common schema and metadata API definition (amundsen-io#1059) * feature model Signed-off-by: Allison Suarez Miranda <[email protected]> * feature model lint Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * added query schema Signed-off-by: Allison Suarez Miranda <[email protected]> * reorder Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * metadata swagger template Signed-off-by: Allison Suarez Miranda <[email protected]> * stubbing out all classes and methods Signed-off-by: Allison Suarez Miranda <[email protected]> * Created models for tabs, stubbed out all API methods Signed-off-by: Allison Suarez Miranda <[email protected]> * defined feature API routes Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * bumped common version Signed-off-by: Allison Suarez Miranda <[email protected]> * more specific type Signed-off-by: Allison Suarez Miranda <[email protected]> * workflow change Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * isort Signed-off-by: Allison Suarez Miranda <[email protected]> * badge import and tag import chnages Signed-off-by: Allison Suarez Miranda <[email protected]> * fixed imports Signed-off-by: Allison Suarez Miranda <[email protected]> * small fixes Signed-off-by: Allison Suarez Miranda <[email protected]> * added standalone query file to common Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * fix: dashboard in atlas metadata proxy (amundsen-io#1110) Signed-off-by: DOMINIK CHOMA <[email protected]> * Fix: Update Amundsen Common to 0.11.0 (amundsen-io#1088) * Update amundsen-common version to '0.10.0' Signed-off-by: Kevin Dukelow <[email protected]> * Update search and metadata services Signed-off-by: Kevin Dukelow <[email protected]> * Update to 0.11.0 Signed-off-by: Kevin Dukelow <[email protected]> * feat: Implement ML Feature Details API (amundsen-io#1091) * feature model Signed-off-by: Allison Suarez Miranda <[email protected]> * feature model lint Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * added query schema Signed-off-by: Allison Suarez Miranda <[email protected]> * reorder Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * metadata swagger template Signed-off-by: Allison Suarez Miranda <[email protected]> * stubbing out all classes and methods Signed-off-by: Allison Suarez Miranda <[email protected]> * Created models for tabs, stubbed out all API methods Signed-off-by: Allison Suarez Miranda <[email protected]> * defined feature API routes Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * bumped common version Signed-off-by: Allison Suarez Miranda <[email protected]> * more specific type Signed-off-by: Allison Suarez Miranda <[email protected]> * Started implwementation Signed-off-by: Allison Suarez Miranda <[email protected]> * workflow change Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * isort Signed-off-by: Allison Suarez Miranda <[email protected]> * badge import and tag import chnages Signed-off-by: Allison Suarez Miranda <[email protected]> * fixed imports Signed-off-by: Allison Suarez Miranda <[email protected]> * small fixes Signed-off-by: Allison Suarez Miranda <[email protected]> * functional, need to verify structure, finish up some queries, and write tests Signed-off-by: Allison Suarez Miranda <[email protected]> * looks fully functional Signed-off-by: Allison Suarez Miranda <[email protected]> * fully functional details Signed-off-by: Allison Suarez Miranda <[email protected]> * cleaned up a bit Signed-off-by: Allison Suarez Miranda <[email protected]> * lint and start testing Signed-off-by: Allison Suarez Miranda <[email protected]> * added standalone query file to common Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * tests and some small changes Signed-off-by: Allison Suarez Miranda <[email protected]> * made _exec_feature_query less complex Signed-off-by: Allison Suarez Miranda <[email protected]> * added type annotation to classify tags arg Signed-off-by: Allison Suarez Miranda <[email protected]> * fixed import order Signed-off-by: Allison Suarez Miranda <[email protected]> * more unit tests Signed-off-by: Allison Suarez Miranda <[email protected]> * added final except and fixed query order Signed-off-by: Allison Suarez Miranda <[email protected]> * created methods to extract objects from neo4j query Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * pinned common version in FE to latest Signed-off-by: Allison Suarez Miranda <[email protected]> * Update search PR workflow to use search directory (amundsen-io#1112) Signed-off-by: Grant Seward <[email protected]> * perf: Update ModeDashboardUsageExtractor to discovery endpoint (amundsen-io#1115) * move mode dashboard usage extractor to discovery api Signed-off-by: Junda Yang <[email protected]> * update Signed-off-by: Junda Yang <[email protected]> * mock post process Signed-off-by: Junda Yang <[email protected]> * fix test Signed-off-by: Junda Yang <[email protected]> * solve lint error Signed-off-by: Junda Yang <[email protected]> * fix isort Signed-off-by: Junda Yang <[email protected]> * perf: Update mode dashboard executions extractor to discovery api (amundsen-io#1116) * update ModeDashboardExecutionsExtractor to discovery api Signed-off-by: Junda Yang <[email protected]> * update README Signed-off-by: Junda Yang <[email protected]> * feat: Implement Feature Tag, Badge, Description and Owner APIs (amundsen-io#1108) * feature model Signed-off-by: Allison Suarez Miranda <[email protected]> * feature model lint Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * added query schema Signed-off-by: Allison Suarez Miranda <[email protected]> * reorder Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * metadata swagger template Signed-off-by: Allison Suarez Miranda <[email protected]> * stubbing out all classes and methods Signed-off-by: Allison Suarez Miranda <[email protected]> * Created models for tabs, stubbed out all API methods Signed-off-by: Allison Suarez Miranda <[email protected]> * defined feature API routes Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * bumped common version Signed-off-by: Allison Suarez Miranda <[email protected]> * more specific type Signed-off-by: Allison Suarez Miranda <[email protected]> * Started implwementation Signed-off-by: Allison Suarez Miranda <[email protected]> * workflow change Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * isort Signed-off-by: Allison Suarez Miranda <[email protected]> * badge import and tag import chnages Signed-off-by: Allison Suarez Miranda <[email protected]> * fixed imports Signed-off-by: Allison Suarez Miranda <[email protected]> * small fixes Signed-off-by: Allison Suarez Miranda <[email protected]> * functional, need to verify structure, finish up some queries, and write tests Signed-off-by: Allison Suarez Miranda <[email protected]> * looks fully functional Signed-off-by: Allison Suarez Miranda <[email protected]> * fully functional details Signed-off-by: Allison Suarez Miranda <[email protected]> * cleaned up a bit Signed-off-by: Allison Suarez Miranda <[email protected]> * lint and start testing Signed-off-by: Allison Suarez Miranda <[email protected]> * added standalone query file to common Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * tests and some small changes Signed-off-by: Allison Suarez Miranda <[email protected]> * made _exec_feature_query less complex Signed-off-by: Allison Suarez Miranda <[email protected]> * added type annotation to classify tags arg Signed-off-by: Allison Suarez Miranda <[email protected]> * fixed import order Signed-off-by: Allison Suarez Miranda <[email protected]> * more unit tests Signed-off-by: Allison Suarez Miranda <[email protected]> * implemented feature badge API Signed-off-by: Allison Suarez Miranda <[email protected]> * implementing more feature APIs Signed-off-by: Allison Suarez Miranda <[email protected]> * manual testing and fixed bug Signed-off-by: Allison Suarez Miranda <[email protected]> * added final except and fixed query order Signed-off-by: Allison Suarez Miranda <[email protected]> * created methods to extract objects from neo4j query Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * pinned common version in FE to latest Signed-off-by: Allison Suarez Miranda <[email protected]> * description API works Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * functional owners API Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * reverted and added separate method Signed-off-by: Allison Suarez Miranda <[email protected]> * cleaning up Signed-off-by: Allison Suarez Miranda <[email protected]> * added from base proxy Signed-off-by: Allison Suarez Miranda <[email protected]> * more to base proxy and other proxies Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * reuse code Signed-off-by: Allison Suarez Miranda <[email protected]> * uodated test Signed-off-by: Allison Suarez Miranda <[email protected]> * added unit tests Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * cleanup Signed-off-by: Allison Suarez Miranda <[email protected]> * oops Signed-off-by: Allison Suarez Miranda <[email protected]> * added logging Signed-off-by: Allison Suarez Miranda <[email protected]> * updated comment Signed-off-by: Allison Suarez Miranda <[email protected]> * fix: handle columns in manifest and not in catalog (amundsen-io#1095) Signed-off-by: Adam Boscarino <[email protected]> * boring-cyborg: make it a bit friendlier (amundsen-io#1123) Given that this doesn't actually check if people filled out the issue template, I think it's a bit unfriendly. I also don't think it will change behavior for people who don't follow the template Signed-off-by: Dorian Johnson <[email protected]> * feat: ML Feature Generation Code API (amundsen-io#1122) * WIP not functional Signed-off-by: Allison Suarez Miranda <[email protected]> * still wip Signed-off-by: Allison Suarez Miranda <[email protected]> * Functional generation code API Signed-off-by: Allison Suarez Miranda <[email protected]> * added method to base proxy and others Signed-off-by: Allison Suarez Miranda <[email protected]> * space Signed-off-by: Allison Suarez Miranda <[email protected]> * whitespace Signed-off-by: Allison Suarez Miranda <[email protected]> * eliminated weirdness Signed-off-by: Allison Suarez Miranda <[email protected]> * unit tests!' Signed-off-by: Allison Suarez Miranda <[email protected]> * lint Signed-off-by: Allison Suarez Miranda <[email protected]> * isort Signed-off-by: Allison Suarez Miranda <[email protected]> * fix: Database filter to return string, not object (amundsen-io#1041) * Database filter returns string not object Signed-off-by: Grant Seward <[email protected]> * Added comma to fix syntax text Signed-off-by: Grant Seward <[email protected]> * docs: update databuilder directory name after deprecating databuilder submodule (amundsen-io#1106) Signed-off-by: Marcin Mazurek <[email protected]> * docs: fix some broken links (amundsen-io#1125) Signed-off-by: Dorian Johnson <[email protected]> * update ModePaginatedRestApiQuery default parameterrs (amundsen-io#1126) Signed-off-by: Junda Yang <[email protected]> * Fix: Make BigQuery extractors support Relational Metadata store (amundsen-io#1109) * Modify Watermark extractor to support relational metadata store Signed-off-by: Sahithi Velma <[email protected]> * Modify BigQuery Usage extractor to support relational metadata store Signed-off-by: Sahithi Velma <[email protected]> * Modify BigQuery Usage extractor to only count usage of tables present in the project_id_key Signed-off-by: Sahithi Velma <[email protected]> Co-authored-by: Sahithi Velma <[email protected]> * feat: Pandas-Profiling TableColumnStats Extractor (amundsen-io#1105) introduces pandas-profiling based extractor for table column stats * feat: support skipping configured Mode spaces (amundsen-io#1129) * bump up databuilder version to 4.4.1 (amundsen-io#1130) * fix typo aws_secret_access_key in neptune csv publisher (amundsen-io#1134) Signed-off-by: iamtodor <[email protected]> * fix: search_table API swagger file bug fix (amundsen-io#1120) * search table swagger file bug fix Signed-off-by: nishant_cimpress <[email protected]> * removed comment Signed-off-by: nishant_cimpress <[email protected]> Co-authored-by: nishant_cimpress <[email protected]> * feat: Atlas Lineage support (amundsen-io#1103) introduces support for lineage in atlas metadata proxy * fix: atlas proxy fix (amundsen-io#1139) Signed-off-by: dechoma <[email protected]> * feat: use chart_names in dashboard basic search (amundsen-io#1140) Signed-off-by: mgorsk1 <[email protected]> * fix: pytest setup.cfg section fix (amundsen-io#1144) Signed-off-by: Jakub Nowacki <[email protected]> * chore: remove a comment that conflicts with implementation (amundsen-io#1154) Signed-off-by: Tero Paananen <[email protected]> * feat: Common query models (amundsen-io#1155) * Added join and where models for how tables are joined / filtered with sql Signed-off-by: Grant Seward <[email protected]> * removed addl space Signed-off-by: Grant Seward <[email protected]> * version bump Signed-off-by: Grant Seward <[email protected]> * feat: Configuration option to init custom routes (amundsen-io#1160) * configuration option to init custom routes Signed-off-by: verdan <[email protected]> * Fixes isort Signed-off-by: verdan <[email protected]> * :sigh: really fixes isort Signed-off-by: verdan <[email protected]> * feat: Search service adding suport for AWS Elasticsearch (amundsen-io#1141) Signed-off-by: Jakub Nowacki <[email protected]> * Fix installation link (amundsen-io#1159) The link was broken Signed-off-by: Fokko Driesprong <[email protected]> * fix: 1157 issude - metadata sql alchemy for iodc component (amundsen-io#1169) Signed-off-by: iamtodor <[email protected]> * Helm service versions moved up to latest ones (amundsen-io#1165) Signed-off-by: Jakub Nowacki <[email protected]> * Fix doc links in README.md (amundsen-io#1164) Just a quick driveby to fix a few doc links that still point to old locations. Signed-off-by: James Davidheiser <[email protected]> * fix: aws_config (amundsen-io#1167) Signed-off-by: mgorsk1 <[email protected]> * fix: add optional list to atlas_search_extractor (amundsen-io#1180) Signed-off-by: dechoma <[email protected]> * feat: introduce atlas serializable for dashboards (amundsen-io#1138) Signed-off-by: dechoma <[email protected]> * build(deps): bump amundsenrds from `589be11` to `4509bb0` (amundsen-io#1174) Bumps [amundsenrds](https://github.com/amundsen-io/amundsenrds) from `589be11` to `4509bb0`. - [Release notes](https://github.com/amundsen-io/amundsenrds/releases) - [Commits](amundsen-io/amundsenrds@589be11...4509bb0) Signed-off-by: dependabot-preview[bot] <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * chore: bump up amundsen_common to 0.13.0 (amundsen-io#1183) Signed-off-by: dechoma <[email protected]> * fix: add amundsen-common requirement (amundsen-io#1184) Signed-off-by: dechoma <[email protected]> * feat: support mysql in metadata service (amundsen-io#1182) * support metadata service for mysql Signed-off-by: xuans <[email protected]> * update doc Signed-off-by: xuans <[email protected]> * fixed some typo in doc Signed-off-by: xuans <[email protected]> * Add configurable cli registration Signed-off-by: xuans <[email protected]> * test: Increase test coverage for atlas proxy (amundsen-io#1173) Signed-off-by: sinkuladis <[email protected]> * Fix broken anchor link in readme (amundsen-io#1193) the section anchor for troubleshoot section is broken, I added the correct anchor. Signed-off-by: qiman-wang <[email protected]> * chore: update requirement file in metadata (amundsen-io#1194) * chore: update requirement file in metadata Signed-off-by: feng-tao <[email protected]> * chore: update CI Signed-off-by: feng-tao <[email protected]> * update version Signed-off-by: feng-tao <[email protected]> * Docs: add AWS Neptune to metadata*s Readme (amundsen-io#1197) * add AWS Neptune to metadata*s Readme Signed-off-by: iamtodor <[email protected]> * Update README.md Co-authored-by: Tao Feng <[email protected]> * feat: Add databuilder models for Feature (amundsen-io#1201) * [feat] Add databuilder models for Feature Signed-off-by: Dmitriy Kunitskiy <[email protected]> * add feature model docs Signed-off-by: Dmitriy Kunitskiy <[email protected]> * fix dashboard owner test Signed-off-by: Dmitriy Kunitskiy <[email protected]> * lint failures Signed-off-by: Dmitriy Kunitskiy <[email protected]> * docstring improvements Signed-off-by: Dmitriy Kunitskiy <[email protected]> * bump version Signed-off-by: Dmitriy Kunitskiy <[email protected]> * feat: Generic Usage Extractor (amundsen-io#1199) Signed-off-by: Allison Doami <[email protected]> * add information about default ES indexes to search Readme (amundsen-io#1198) Signed-off-by: iamtodor <[email protected]> * chore: add myself as codeowner for other services (amundsen-io#1204) Signed-off-by: Dorian Johnson <[email protected]> * Add backwards compatible support for Jira cloud/hosted. (amundsen-io#1207) Signed-off-by: BB <[email protected]> * refactor: shared dependencies unification (amundsen-io#1163) * unify_deps | 🎉 Initial commit. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ✅ Adding tests. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ⬆️ Upgrading dependencies. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ✨ Introducing new features. (run tests when reqs change) Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💡 Documenting source code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 📝 Writing docs. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💄 Updating the UI and style files. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ⬆️ Upgrading dependencies. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💡 Documenting source code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 👌 Updating code due to code review changes. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🎉 Initial commit. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> Co-authored-by: Dorian Johnson <[email protected]> * fix: update sharded_table filter in BigQuery extractor (amundsen-io#1209) * updated shared-table validator Signed-off-by: xuans <[email protected]> * renaming Signed-off-by: xuans <[email protected]> * fix mypy Signed-off-by: xuans <[email protected]> * fix lint Signed-off-by: xuans <[email protected]> * fix: Add forgotten init file to databuilder/models/feature module (amundsen-io#1211) * add init file to feature module Signed-off-by: Dmitriy Kunitskiy <[email protected]> * loosen mypy annotation Signed-off-by: Dmitriy Kunitskiy <[email protected]> * fix: bug amundsen-io#1213 relates to AWS creds Neptune (amundsen-io#1214) * fix bug amundsen-io#1213 relates to AWS creds Neptune Signed-off-by: iamtodor <[email protected]> * code clean up Signed-off-by: iamtodor <[email protected]> * fix - Update README to GenericUsageExtractor (amundsen-io#1217) Signed-off-by: Allison Doami <[email protected]> * fix db key constraints (amundsen-io#1216) Signed-off-by: Junda Yang <[email protected]> * install stemma reqs Signed-off-by: Grant Seward <[email protected]> * requirement version pin Signed-off-by: Grant Seward <[email protected]> * req update again Signed-off-by: Grant Seward <[email protected]> * frontend reqs Signed-off-by: Grant Seward <[email protected]> * Applied to metadata and search Signed-off-by: Grant Seward <[email protected]> * added responses Signed-off-by: Grant Seward <[email protected]> * fstring fix Signed-off-by: Grant Seward <[email protected]> Co-authored-by: mgorsk1 <[email protected]> Co-authored-by: Verdan Mahmood <[email protected]> Co-authored-by: Allison Suarez Miranda <[email protected]> Co-authored-by: dechoma <[email protected]> Co-authored-by: KevinDukelow5 <[email protected]> Co-authored-by: Junda Yang <[email protected]> Co-authored-by: Adam Boscarino <[email protected]> Co-authored-by: Dorian Johnson <[email protected]> Co-authored-by: Marcin Mazurek <[email protected]> Co-authored-by: Sahithi Reddy Velma <[email protected]> Co-authored-by: Sahithi Velma <[email protected]> Co-authored-by: iamtodor <[email protected]> Co-authored-by: Nishant Sharma <[email protected]> Co-authored-by: nishant_cimpress <[email protected]> Co-authored-by: Jakub Nowacki <[email protected]> Co-authored-by: gorbachev <[email protected]> Co-authored-by: Fokko Driesprong <[email protected]> Co-authored-by: James Davidheiser <[email protected]> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Xuan <[email protected]> Co-authored-by: sinkuladis <[email protected]> Co-authored-by: QimanWang <[email protected]> Co-authored-by: Tao Feng <[email protected]> Co-authored-by: Tao Feng <[email protected]> Co-authored-by: Dmitriy Kunitskiy <[email protected]> Co-authored-by: Allison Doami <[email protected]> Co-authored-by: Boyan Bonev <[email protected]> Co-authored-by: Junda Yang <[email protected]>

* unify_deps | 🎉 Initial commit. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ✅ Adding tests. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ⬆️ Upgrading dependencies. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ✨ Introducing new features. (run tests when reqs change) Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💡 Documenting source code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 📝 Writing docs. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💄 Updating the UI and style files. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ⬆️ Upgrading dependencies. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | ♻️ Refactoring code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 💡 Documenting source code. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🚨 Removing linter warnings. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 👌 Updating code due to code review changes. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🎉 Initial commit. Signed-off-by: mgorsk1 <[email protected]> * unify_deps | 🐛 Fixing a bug. Signed-off-by: mgorsk1 <[email protected]> Co-authored-by: Dorian Johnson <[email protected]>

boring-cyborg bot added area:common From common folder area:databuilder From databuilder folder area:all Related to all the project area:frontend From the Frontend folder labels May 27, 2021

mgorsk1 mentioned this pull request May 27, 2021

other: Increased pytest and related packages version in search service #1162

Closed

4 tasks

mgorsk1 added the area:search From the search folder label May 27, 2021

mgorsk1 mentioned this pull request May 28, 2021

Unify shared dependencies amundsen-io/rfcs#38

Merged

mgorsk1 force-pushed the feat/unify_deps branch from 97da9ef to 28a1656 Compare May 28, 2021 09:51

mgorsk1 added the area:metadata From the metadata folder label May 28, 2021

mgorsk1 force-pushed the feat/unify_deps branch from 060c080 to 0a17887 Compare May 29, 2021 05:11

boring-cyborg bot added the area:docs label May 29, 2021

mgorsk1 changed the title ~~WIP: refactor: shared dependencies unification~~ refactor: shared dependencies unification May 29, 2021

mgorsk1 marked this pull request as ready for review May 29, 2021 08:59

mgorsk1 requested review from allisonsuarez, bolkedebruin, dikshathakur3119, dorianj, feng-tao, jinhyukchang, ttannis, verdan and a team as code owners May 29, 2021 08:59

bbsbb reviewed May 31, 2021

View reviewed changes

frontend/Makefile Outdated Show resolved Hide resolved

sewardgw approved these changes Jun 1, 2021

View reviewed changes

Dockerfile.frontend.local Show resolved Hide resolved

search/Makefile Show resolved Hide resolved

metadata/requirements-common.txt Show resolved Hide resolved

mgorsk1 force-pushed the feat/unify_deps branch from 7bfad41 to b58ba94 Compare June 1, 2021 15:15

sewardgw approved these changes Jun 1, 2021

View reviewed changes

mgorsk1 force-pushed the feat/unify_deps branch from b58ba94 to 29de152 Compare June 2, 2021 10:25

mgorsk1 added 11 commits June 9, 2021 07:51

unify_deps | 🐛 Fixing a bug.

36945aa

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

197407c

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | ♻️ Refactoring code.

ee68d29

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 💡 Documenting source code.

177c2eb

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 📝 Writing docs.

6c80419

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 💄 Updating the UI and style files.

65e280e

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

6ff974a

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | ⬆️ Upgrading dependencies.

44f1f29

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | ♻️ Refactoring code.

1531c26

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

b63a177

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

a63c914

Signed-off-by: mgorsk1 <[email protected]>

mgorsk1 force-pushed the feat/unify_deps branch from 30942aa to a63c914 Compare June 9, 2021 05:51

unify_deps | 💡 Documenting source code.

c816223

Signed-off-by: mgorsk1 <[email protected]>

boring-cyborg bot added the category:models label Jun 9, 2021

unify_deps | 🚨 Removing linter warnings.

b58b0f4

Signed-off-by: mgorsk1 <[email protected]>

mgorsk1 force-pushed the feat/unify_deps branch from c341c98 to b58b0f4 Compare June 9, 2021 06:51

mgorsk1 and others added 8 commits June 9, 2021 09:01

unify_deps | 👌 Updating code due to code review changes.

d5aefa6

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

34f28b7

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

cd3e8c0

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

c25a1ce

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

1cb0930

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🎉 Initial commit.

3a7fc15

Signed-off-by: mgorsk1 <[email protected]>

unify_deps | 🐛 Fixing a bug.

6da191e

Signed-off-by: mgorsk1 <[email protected]>

Merge branch 'main' into feat/unify_deps

3ce2806

dorianj merged commit 11b0325 into main Jun 9, 2021

dorianj deleted the feat/unify_deps branch June 9, 2021 20:06

dkunitsk mentioned this pull request Jun 24, 2021

refactor: undo the great unpinning #1267

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: shared dependencies unification #1163

refactor: shared dependencies unification #1163

mgorsk1 commented May 27, 2021 •

edited

Loading

sewardgw left a comment

sewardgw commented Jun 1, 2021

mgorsk1 commented Jun 1, 2021

mgorsk1 commented Jun 1, 2021

refactor: shared dependencies unification #1163

refactor: shared dependencies unification #1163

Conversation

mgorsk1 commented May 27, 2021 • edited Loading

Summary of Changes

Assumptions

Tests

TODO

Documentation

CheckList

sewardgw left a comment

Choose a reason for hiding this comment

sewardgw commented Jun 1, 2021

mgorsk1 commented Jun 1, 2021

mgorsk1 commented Jun 1, 2021

mgorsk1 commented May 27, 2021 •

edited

Loading