Discussion: Why use many single-row tables for TaskEpoch?

[pauladkisson]

I am working on a project to adapt NWB files so that they are spyglass-compatible, and I noticed that TaskEpoch make() function expects many single-row DynamicTable objects to populate the task information:

spyglass/src/spyglass/common/common_task.py

Lines 145 to 192 in dbae127

	for task in tasks_mod.data_interfaces.values():
	if self.check_task_table(task):
	# check if the task is in the Task table and if not, add it
	Task.insert_from_task_table(task)
	key["task_name"] = task.task_name[0]
	# get the CameraDevice used for this task (primary key is
	# camera name so we need to map from ID to name)
	camera_ids = task.camera_id[0]
	valid_camera_ids = [
	camera_id
	for camera_id in camera_ids
	if camera_id in camera_names.keys()
	]
	if valid_camera_ids:
	key["camera_names"] = [
	{"camera_name": camera_names[camera_id]}
	for camera_id in valid_camera_ids
	]
	else:
	logger.warning(
	f"No camera device found with ID {camera_ids} in NWB "
	+ f"file {nwbf}\n"
	)
	# Add task environment
	if hasattr(task, "task_environment"):
	key["task_environment"] = task.task_environment[0]
	# get the interval list for this task, which corresponds to the
	# matching epoch for the raw data. Users should define more
	# restrictive intervals as required for analyses
	session_intervals = (
	IntervalList() & {"nwb_file_name": nwb_file_name}
	).fetch("interval_list_name")
	for epoch in task.task_epochs[0]:
	# TODO in beans file, task_epochs[0] is 1x2 dset of ints,
	# so epoch would be an int
	key["epoch"] = epoch
	target_interval = self.get_epoch_interval_name(
	epoch, session_intervals
	)
	if target_interval is None:
	logger.warning("Skipping epoch.")
	continue
	key["interval_list_name"] = target_interval
	task_inserts.append(key.copy())

Why not just use one table with many rows (one per task)? Is there a general principle in spyglass that I'm missing? Is this just a bug? Is there documentation somewhere that explains this?

Curiously, the Task table propagates all the rows of each DynamicTable, so if the tables are multi-row, you can get a task_name in the Task table that doesn't show up in the TaskEpoch table.

[samuelbray32]

No overarching principle here, just a legacy piece from how the frank lab nwb files were organized when this began (enhancement to do in our nwb conversion package). Agree that we should be able to read in multiple entries from a nwb table here.