Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] ExportSWC: exceptions are wrong #330

Open
sanjayankur31 opened this issue Apr 9, 2024 · 2 comments
Open

[Bug] ExportSWC: exceptions are wrong #330

sanjayankur31 opened this issue Apr 9, 2024 · 2 comments
Labels
T: bug Type: bug

Comments

@sanjayankur31
Copy link
Member

Using the ExportSWC class to export NeuroML2 to SWC works, but it has some checks that aren't necessarily correct for all cell models.

For example, on valid cells like the HL23PYR cell, it throws this exception:

$ pynml -swc HL23PYR.cell.nml 
pyNeuroML >>> INFO - Trying to convert HL23PYR.cell.nml to swc format...
pyNeuroML >>> INFO - Loading NeuroML2 file: HL23PYR.cell.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/CaDynamics_E2_NML2.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/pas.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Ih.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/SK.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/K_T.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/K_P.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Kv3_1.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Im.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/NaTg/NaTg.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Ca_HVA.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Ca_LVA.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/L23-example/channels/Nap.channel.nml
Traceback (most recent call last):
  File "/home/asinha/.local/share/virtualenvs/neuroml-311-dev/bin/pynml", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/pynml.py", line 971, in main
    _evaluate_arguments(args)
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/pynml.py", line 866, in _evaluate_arguments
    convert_count += 1 if convert_to_swc(f) else 0
                          ^^^^^^^^^^^^^^^^^
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/swc/ExportSWC.py", line 174, in convert_to_swc
    raise Exception(
Exception: The numbers of the segments in groups: soma_group+dendrite_group+axon_group (5818), is not the same as total number of segments (5840)! All bets are off!

I need to check why these numbers don't add up---where are the 22 segments? But I also see it with other cells, like this one from the Allen cells: https://github.com/OpenSourceBrain/AllenInstituteNeuroML/blob/master/CellTypesDatabase/models/NeuroML2/Cell_497232312.cell.nml

$ pynml -swc Cell_497232312.cell.nml 
pyNeuroML >>> INFO - Trying to convert Cell_497232312.cell.nml to swc format...
pyNeuroML >>> INFO - Loading NeuroML2 file: Cell_497232312.cell.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/pas.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/NaV.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/K_T.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Kd.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Kv2like.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Kv3_1.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/SK.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Ih.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Im_v2.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Ca_HVA.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/Ca_LVA.channel.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/CaDynamics_all.nml
pyNeuroML >>> INFO - Loading NeuroML2 file: /home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/tests/plot/CaDynamics.nml
Traceback (most recent call last):
  File "/home/asinha/.local/share/virtualenvs/neuroml-311-dev/bin/pynml", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/pynml.py", line 971, in main
    _evaluate_arguments(args)
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/pynml.py", line 866, in _evaluate_arguments
    convert_count += 1 if convert_to_swc(f) else 0
                          ^^^^^^^^^^^^^^^^^
  File "/home/asinha/Documents/02_Code/00_mine/NeuroML/software/pyNeuroML/pyneuroml/swc/ExportSWC.py", line 174, in convert_to_swc
    raise Exception(
Exception: The numbers of the segments in groups: soma_group+dendrite_group+axon_group (1161), is not the same as total number of segments (3489)! All bets are off!
@sanjayankur31 sanjayankur31 added the T: bug Type: bug label Apr 9, 2024
@sanjayankur31
Copy link
Member Author

In the allen inst cell, the apical dendrites aren't included in the "dendrite_group":

a = cell.get_ordered_segments_in_groups(["all", "soma_group", "dendrite_group", "axon_group", "dend", "apic"])

Checking:

In [39]: len(a["apic"]) + len(a["dendrite_group"]) + len(a["soma_group"]) + len(a["axon_group"])
Out[39]: 3489

In [40]: len(a["all"])
Out[40]: 3489

In [41]: len(a["dendrite_group"]) + len(a["soma_group"]) + len(a["axon_group"])
Out[41]: 1161

@pgleeson : I guess the apical dendrites segment group apic should also be included in the dendrite_group here?

In the HL23PYR cell, there are 22 segments in the myelin_group that aren't in the conventional groups:

a = cell.get_ordered_segments_in_groups(["all", "soma_group", "dendrite_group", "axon_group", "basal_dendrite_group", "apical_dendrite_group", "myelin_group"])

here's verification:

(a["myelin_group"])
Out[12]: 
[<Segment|5819|Seg0_myelin_0>,
 <Segment|5820|Seg1_myelin_0>,
 <Segment|5821|Seg2_myelin_0>,
 <Segment|5822|Seg3_myelin_0>,
 <Segment|5823|Seg4_myelin_0>,
 <Segment|5824|Seg5_myelin_0>,
 <Segment|5825|Seg6_myelin_0>,
 <Segment|5826|Seg7_myelin_0>,
 <Segment|5827|Seg8_myelin_0>,
 <Segment|5828|Seg9_myelin_0>,
 <Segment|5829|Seg10_myelin_0>,
 <Segment|5830|Seg11_myelin_0>,
 <Segment|5831|Seg12_myelin_0>,
 <Segment|5832|Seg13_myelin_0>,
 <Segment|5833|Seg14_myelin_0>,
 <Segment|5834|Seg15_myelin_0>,
 <Segment|5835|Seg16_myelin_0>,
 <Segment|5836|Seg17_myelin_0>,
 <Segment|5837|Seg18_myelin_0>,
 <Segment|5838|Seg19_myelin_0>,
 <Segment|5839|Seg20_myelin_0>,
 <Segment|5840|Seg21_myelin_0>]

In [7]: len(a["dendrite_group"]) + len(a["soma_group"]) + len(a["axon_group"])
Out[7]: 5818

In [8]: len(a["myelin_group"])
Out[8]: 22

In [9]: len(a["all"])
Out[9]: 5840

I don't think the myelin segments are meant to be in any of the standard groups, so this seems correct?

@sanjayankur31 sanjayankur31 moved this to 🔖 Ready in NeuroML backlog Apr 23, 2024
@sanjayankur31
Copy link
Member Author

This is not actually an exception bug. The cells we tested here are valid but not completely. We just don't have a test to check that the sum of dendrite_group + axon_group + soma_group = all. For correctness, this should hold, so we'll add another validation test for this: if one is using the default segment groups, this check should hold.

@sanjayankur31 sanjayankur31 moved this from 🔖 Ready to 🏗 In progress in NeuroML backlog Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
T: bug Type: bug
Projects
NeuroML backlog
Status: 🏗 In progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sanjayankur31