Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENG-6185][ENG-6186][ENG-6187][ENG-6188] additional searchable metadata for institutional dashboard #10779

Conversation

aaxelb
Copy link
Contributor

@aaxelb aaxelb commented Oct 15, 2024

Purpose

add searchable metadata that will be used by the new™️ institutional dashboard

Changes

(recommend reading commit-by-commit)

  • update metadata test helpers for more helpful feedback when adding metadata properties
  • add osf:hasOsfAddon property (supplemental)
    • identify addons with urn:osf.example:addons:...
  • add osf:storageRegion property (supplemental)
    • identify regions with https://api.osf.example/v2/regions/.../
  • add prov:qualifiedAttribution (non-supplemental)
    • qualifies contributorship with a role: osf:admin-contributor, osf:write-contributor, or osf:readonly-contributor (and could add more nuanced/helpful contributor roles in the future)
  • add osf:storageByteCount property (supplemental)
    • small refactor of update_storage_usage_cache:
      • add compute_storage_usage_total for the actual query
      • update query to use named params
      • replace a join with a separate ContentType.objects.get_for_model() call (which also allows for non-node file targets)
    • add get_storage_usage_total function (alternative to Node.storage_usage that waits for it to compute)

QA Notes

Please make verification statements inspired by your code and what your code touches.

  • Verify changes in the described properties are reflected in the search api (or the institutional dashboard) shortly enough
  • Verify existing search indexing does not break

What are the areas of risk?

Any concerns/considerations/questions that development raised?

Documentation

Side Effects

Ticket

@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch from 43a8475 to 45a1629 Compare October 18, 2024 14:48
@aaxelb aaxelb changed the title Feature/insti dash metadata adds [ENG-6188] addl searchable metadata for institutional dashboard Oct 18, 2024
@aaxelb aaxelb changed the title [ENG-6188] addl searchable metadata for institutional dashboard [wip][ENG-6188] addl searchable metadata for institutional dashboard Oct 18, 2024
@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch 8 times, most recently from 68742fb to 33bb4f3 Compare October 21, 2024 15:53
@aaxelb aaxelb changed the title [wip][ENG-6188] addl searchable metadata for institutional dashboard [wip][ENG-6185][ENG-6186][ENG-6187][ENG-6188] addl searchable metadata for institutional dashboard Oct 21, 2024
@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch 3 times, most recently from 2c15ccd to b4720f6 Compare October 21, 2024 20:49
@aaxelb aaxelb changed the title [wip][ENG-6185][ENG-6186][ENG-6187][ENG-6188] addl searchable metadata for institutional dashboard [wip][ENG-6185][ENG-6186][ENG-6187][ENG-6188] additional searchable metadata for institutional dashboard Oct 22, 2024
@aaxelb aaxelb changed the title [wip][ENG-6185][ENG-6186][ENG-6187][ENG-6188] additional searchable metadata for institutional dashboard [ENG-6185][ENG-6186][ENG-6187][ENG-6188] additional searchable metadata for institutional dashboard Oct 24, 2024
@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch 2 times, most recently from 0bba13d to 44427ba Compare October 24, 2024 14:52
@aaxelb aaxelb marked this pull request as ready for review October 24, 2024 16:55
@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch from 44427ba to 462fae4 Compare October 24, 2024 18:44
Copy link
Contributor Author

@aaxelb aaxelb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(comments from group review)

osf/metadata/osf_gathering.py Outdated Show resolved Hide resolved
osf/metadata/osf_gathering.py Outdated Show resolved Hide resolved
@felliott felliott self-assigned this Oct 25, 2024
@felliott
Copy link
Member

LGTM, Abram found a few minor things that he will address, but no major blockers.

@cslzchen cslzchen self-requested a review October 25, 2024 16:06
Copy link
Collaborator

@cslzchen cslzchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎆

osf_tests/metadata/test_serialized_metadata.py Outdated Show resolved Hide resolved
@aaxelb aaxelb force-pushed the feature/insti-dash-metadata-adds branch from 462fae4 to 19be71b Compare October 25, 2024 17:23
@aaxelb aaxelb merged commit ddb46d8 into CenterForOpenScience:feature/insti-dash-improv Oct 25, 2024
6 checks passed
@aaxelb aaxelb deleted the feature/insti-dash-metadata-adds branch October 25, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants