Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New regression in python #69

Open
wants to merge 27 commits into
base: master
Choose a base branch
from

Conversation

epernod
Copy link
Contributor

@epernod epernod commented Aug 19, 2024

This is a new version of Regression using SofaPython3. Feel free to improve it.
TODO:

  • Remove unnecessary test code on numpy export, as soon as we agree on the formart
  • Move all code of file/folder into a several files
    • For folder and scenes parsing
    • for code on numpy array comparison
  • Hide SOFA logs
  • Improve end report format
  • Add CSV format
  • Create branch with new reference files
    • Test it in a SOFA pr with a ci-depends-on
    • Compare reference to the old ones

Style based on .regression-test file but the last argument is the number of keyframe to test:
for example:
demos/mydemo.py 4000 1e-3 1 100
100 step will be tested with a threshold of 1e-3.
key frame will be: [0 40 80 ... 4000]*dt so 101 values.


Right now, at each key frame, full distance between the ref dofs and simulated dofs is computer.
totalError = sum(fullDist) accumulated over all key frames.
errorByDos = sum(fullDist)/nbrDofs accumulated over all key frames.

Computation is done using numpy array. Maybe better metrics could be used.


example of output:

Compare All sets: 100%|##########| 1/1 [06:30<00:00, 390.56s/it]
Scenes.regression-tests: 100%|##########| 6/6 [06:30<00:00, 46.54s/it].44it/s]
### Number of sets Done:  1
### Number of scenes Done:  6
### Number of scenes failed:  2
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_Judkins_Left.py | Number of key frames compared without error: 101
### Failed: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_Judkins_Right.py
    ### Total Error per MechanicalObject: [0.0010049364875422493, 0.0009830053353054394]
    ### Error by Dofs: [1.4074740721880243e-07, 3.991089465308321e-07]
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_Heart_SpireCatheter.py | Number of key frames compared without error: 101
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_Judkins_Left.py | Number of key frames compared without error: 101
### Success: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_Judkins_Right.py | Number of key frames compared without error: 101
### Failed: ../SofaLnRobotics/scenes\benchmarks/Bench_inHeart_01_SpireCatheter.py
    ### Total Error per MechanicalObject: [0.021162743293277383, 0.019589862858611747]
    ### Error by Dofs: [3.023249041896769e-06, 7.953659301100991e-06]

Comment on lines +38 to +41
#write_file.write(json.dumps(numpyData, cls=NumpyArrayEncoder, indent=4))
#res = json.dumps(numpyData, cls=NumpyArrayEncoder, indent=4)
#print(res)
#json.dump(numpyData, zipfile, cls=NumpyArrayEncoder)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be removed?

for step in range(0, self.steps + 1):
# export rest position, final position + modulo steps:
if (step == 0 or counterStep >= moduloStep or step == self.steps):
#print("step: " + str(step) + " | counterStep: " + str(counterStep) + " | moduloStep: " + str(moduloStep) + " | dt: " + str(self.rootNode.dt.value*(step)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be removed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants