When do we need media.csv
?
#275
Labels
camtrapdp/camtraptor
To be decided if this is related to camtrapdp or camtraptor
enhancement
New feature or request
The
media.csv
is often the largest file to read, which is whyread_camtrap_dp()
initially had an option to skip it while reading data. We plan to remove that parameter to avoid complexity in further functions and not put the burden on the user to make this choice.Alternative options to speed up reading:
col_select
)write_
functions)For documentation, here's when I think we need media columns:
So a potential solution could be to:
read_camtrap_dp()
usescol_select
and only readsmediaID
,deploymentID
,captureMethod
andtimestamp
. This will speed up this function.filter_
functions applied to deployments andobservations also filter the media (ondeploymentID
,mediaID
or atimestamp
that falls betweeneventStart
andeventEnd
)write_
functions read the fullmedia.csv
(can still be found using the$directory
) and join with (potentially filtered) media already in memory. This will slow down those functions.I don't know what the performance gain would be and if users are more likely to be patient when using
read_camtrap_dp()
orwrite_
functions. What is certain is that far more users will useread_camtrap_dp()
than thewrite_
functions, so any speed gain benefits more users. I would wait and see if we hear about performance issues before considering the approach described above.The text was updated successfully, but these errors were encountered: