You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading a TDMS file with channels sampled at different rates, synchronization between channels is lost if _channels_to_dataframe is called with absolute_time=False. Each channel is read with a time series relative to itself, as opposed to relative to the first data point timestamp in all specified channels.
A proposed fix would be to read all channels with index = channel.time_track(absolute_time=True) if time_index else None, then calculate a time relative to the group if the argument provided to _channels_to_dataframe is absolute_time=False.
For example, a TDMS file that has several channels sampled at three different data rates and six different starting times:
_channels_to_dataframe with default arguments left-justifies all the data on a channel-by-channel basis. This results in an event that is sampled at an absolute time of 01:30:33.058532 on one channel to be reported at the same relative time (time = 0) as an event that is sampled at time 01:30:33.268115.
def_channels_to_dataframe(channels_to_export, time_index=False, absolute_time=False, scaled_data=True):
importpandasaspddataframe_dict=OrderedDict()
forcolumn_name, channelinchannels_to_export.items():
index=channel.time_track(absolute_time) iftime_indexelseNoneifscaled_data:
dataframe_dict[column_name] =pd.Series(data=_array_for_pd(channel[:]), index=index)
elifchannel.scaler_data_types:
# Channel has DAQmx raw dataraw_data=channel.read_data(scaled=False)
forscale_id, scaler_datainraw_data.items():
scaler_column_name=column_name+"[{0:d}]".format(scale_id)
dataframe_dict[scaler_column_name] =pd.Series(data=scaler_data, index=index)
else:
# Raw data for normal TDMS fileraw_data=channel.read_data(scaled=False)
dataframe_dict[column_name] =pd.Series(data=_array_for_pd(raw_data), index=index)
returnpd.DataFrame.from_dict(dataframe_dict)
Proposed implementation:
def_channels_to_dataframe(channels_to_export, time_index=False, absolute_time=False, scaled_data=True):
importpandasaspddataframe_dict=OrderedDict()
forcolumn_name, channelinchannels_to_export.items():
# This try/except block deals with a particular group of TDMS files # I encountered that don't play nicely. Maybe it's better to allow the # code to throw an exception, rather than silently discarding the channels.# If that's the case, only keep the `index = channel.....` linetry:
index=channel.time_track(absolute_time=True) iftime_indexelseNoneexceptKeyErrorase:
iftime_index==True:
continueelse:
index=Noneifscaled_data:
dataframe_dict[column_name] =pd.Series(data=_array_for_pd(channel[:]), index=index)
elifchannel.scaler_data_types:
# Channel has DAQmx raw dataraw_data=channel.read_data(scaled=False)
forscale_id, scaler_datainraw_data.items():
scaler_column_name=column_name+"[{0:d}]".format(scale_id)
dataframe_dict[scaler_column_name] =pd.Series(data=scaler_data, index=index)
else:
# Raw data for normal TDMS fileraw_data=channel.read_data(scaled=False)
dataframe_dict[column_name] =pd.Series(data=_array_for_pd(raw_data), index=index)
df=pd.DataFrame.from_dict(dataframe_dict)
if (time_index==Trueandabsolute_time==False):
df.index-=df.index[0]
returndf
The text was updated successfully, but these errors were encountered:
Hi, thanks for the bug report. Yes I think your proposed behaviour makes more sense, although it would be a breaking change as other people may be relying on the existing behaviour. So rather than changing the behaviour, it would be more appropriate to add a new parameter to control this behaviour and make the new approach opt-in. Then for a future 2.0 release we could consider making the new behaviour the default.
When reading a TDMS file with channels sampled at different rates, synchronization between channels is lost if
_channels_to_dataframe
is called withabsolute_time=False
. Each channel is read with a time series relative to itself, as opposed to relative to the first data point timestamp in all specified channels.A proposed fix would be to read all channels with
index = channel.time_track(absolute_time=True) if time_index else None
, then calculate a time relative to the group if the argument provided to_channels_to_dataframe
isabsolute_time=False
.For example, a TDMS file that has several channels sampled at three different data rates and six different starting times:
Which results in
_channels_to_dataframe
with default arguments left-justifies all the data on a channel-by-channel basis. This results in an event that is sampled at an absolute time of01:30:33.058532
on one channel to be reported at the same relative time (time = 0
) as an event that is sampled at time01:30:33.268115
.Present implementation:
Proposed implementation:
The text was updated successfully, but these errors were encountered: