Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare like pols in test_gain #885

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

james-smith-za
Copy link
Contributor

The qualification/tied_array_channelised_voltage/test_gain test occasionally shows a difference one greater expected for a single value in the test every now and then. It's very intermittent. To a first approximation this shouldn't be possible because the tolerance for each test should be taken into account and there's no way for dithering to cause a bigger difference assuming that the incoming signal is the same.

We think that this is caused by separate dithering for each pol in the F-engine which means that despite having the same d-sim input, the F-engine output for pol0 and pol1 may be ever so slightly different from each other, just different to cause the error.

In this commit I adjust the test to using the same pol in three different beams for comparison, rather than three half-beams which would have been pol0, pol1, pol0.

This appears to solve the problem.

Closes NGC-1473.

Checklist (if not applicable, edit to add (N/A) and mark as done):

  • (N/A) If dependencies are added/removed: update setup.cfg and .pre-commit-config.yaml
  • (N/A) If modules are added/removed: use sphinx-apidoc -efo doc/ src/ to update files in doc/
  • Ensure copyright notices are present and up-to-date
  • If qualification tests are changed: attach a sample qualification report No significant change in report.
  • (N/A) If design has changed: ensure documentation is up to date
  • (N/A) If ICD-defined sensors have been added: update fake_servers.py in katsdpcontroller to match

The qualification/tied_array_channelised_voltage/test_gain test
occasionally shows a difference one greater expected for a single
value in the test every now and then. It's very intermittent. To a first
approximation this shouldn't be possible because the tolerance for each
test should be taken into account and there's no way for dithering to
cause a bigger difference assuming that the incoming signal is the same.

We think that this is caused by separate dithering for each pol in the
F-engine which means that despite having the same d-sim input, the
F-engine output for pol0 and pol1 may be ever so slightly different from
each other, just different to cause the error.

In this commit I adjust the test to using the same pol in three
different beams for comparison, rather than three half-beams which would
have been pol0, pol1, pol0.

This appears to solve the problem.

Closes NGC-1473.
Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you're capturing 6 half-beams, but then ignoring 3 of them. I'd suggest either
(a) set stream_names to grab only the 3 half-beams you're interested in; or
(b) update the comparison code to test the odd-indexed polarisations against each other.

I'd lean towards option (b) because it's a stronger test (ensures we haven't mucked up gain-setting such that it only applies to one polarisation).

It's a stronger test than previously, and ensures that we haven't mucked
up gain setting such that it only applies to one polarisation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants