Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate dimension when instantiating datamodel from shape #395

Merged
merged 4 commits into from
Feb 11, 2025

Conversation

emolter
Copy link
Contributor

@emolter emolter commented Feb 10, 2025

Closes #307

This PR addresses a bug when instantiating datamodels from an array shape. The problem is discussed in detail on the issue, but in short, the input shape was not checked against the schema ndim attribute when creating a default array

Tasks

  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
    • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
    • run jwst regression tests with this branch installed ("git+https://github.com/<fork>/stdatamodels@<branch>")
news fragment change types...
  • changes/<PR#>.feature.rst: new feature
  • changes/<PR#>.bugfix.rst: fixes an issue
  • changes/<PR#>.doc.rst: documentation change
  • changes/<PR#>.removal.rst: deprecation or removal of public API
  • changes/<PR#>.misc.rst: infrastructure or miscellaneous change

@emolter
Copy link
Contributor Author

emolter commented Feb 10, 2025

Copy link

codecov bot commented Feb 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.21%. Comparing base (4e83608) to head (13e138f).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
+ Coverage   78.16%   78.21%   +0.04%     
==========================================
  Files         115      115              
  Lines        5144     5155      +11     
==========================================
+ Hits         4021     4032      +11     
  Misses       1123     1123              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -142,7 +142,7 @@ class _NonstandardPrimaryArrayModel(JwstDataModel):
def get_primary_array_name(self):
return "wavelength"

m = _NonstandardPrimaryArrayModel((10,))
m = _NonstandardPrimaryArrayModel((10, 10))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appeared to be an instance of an array passing validation that we would expect to fail, and indeed it does fail as of the bugfix in this PR. It doesn't look like this change makes much difference for what the test intends to cover, but someone can correct me if that's wrong.

@emolter
Copy link
Contributor Author

emolter commented Feb 11, 2025

@emolter emolter marked this pull request as ready for review February 11, 2025 14:29
@emolter emolter requested a review from a team as a code owner February 11, 2025 14:29
@emolter
Copy link
Contributor Author

emolter commented Feb 11, 2025

@jdavies-st Not sure why I can't request you as a reviewer, so pinging you in the comments to take a look

@@ -100,6 +100,22 @@ def test_init_with_array2():
dm.data # noqa: B018


def test_init_invalid_shape():
"""Requested some number of dimensions unequal to ndim"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want just mention in the docstring here what the expected dimension of BasicModel.data should be.

Copy link
Contributor

@jdavies-st jdavies-st left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I would also run the jwst regtests against this (or this combined with the other changes) to make sure there's no misuse of this bug in the pipeline code itself. Same with romancal.

@emolter
Copy link
Contributor Author

emolter commented Feb 11, 2025

@jdavies-st I already ran the jwst regtests here: https://github.com/spacetelescope/RegressionTests/actions/runs/13245788321

Romancal no longer uses stdatamodels.

Copy link
Contributor

@jdavies-st jdavies-st left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh! Sorry, missed that. I can't see those results as I'm no longer internal to ST. But everything looks good from my side.

@jdavies-st
Copy link
Contributor

@jdavies-st Not sure why I can't request you as a reviewer, so pinging you in the comments to take a look

It seems this repo only accepts reviews from people with write access to the repo. I'm no longer internal at STScI, so I don't have write access here. So my review above looks like a normal comment, not a review.

@emolter
Copy link
Contributor Author

emolter commented Feb 11, 2025

It seems this repo only accepts reviews from people with write access to the repo. I'm no longer internal at STScI, so I don't have write access here. So my review above looks like a normal comment, not a review.

Gotcha, well thanks for looking anyway. I can see your approval, but it shows up as a gray checkmark instead of a green checkmark

Copy link
Collaborator

@braingram braingram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@emolter emolter enabled auto-merge (squash) February 11, 2025 18:26
@emolter emolter merged commit 14005e8 into spacetelescope:main Feb 11, 2025
19 checks passed
@emolter emolter deleted the issue-307 branch February 12, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

datamodels doesn't validate arrays in some cases
3 participants