Analyzing the custom format file

[varsh2829]

Hi! I am trying to start analyzing my SMPS data with PY-SMPS. It is a long term dataset and hence been stored in a format different than what we get from AIM software. An example file is also attached here. I am looking for some information regarding what changes should I make to this file before it could be used with py-smps functions. Do I need to change some names of the columns or do I need to specify somewhere what kind of file it is. Just a general tutorial on this will be helpful. Please find a small chunk of data attached.
Test SMPS data.xlsx
In the file the first row contains headings in bold. Some are clear but some headings contain numbers like 10.9, 11.5 etc. These are actually midpoints of the size bins which go all the way upto about 450nm.

[dhhagan]

Hi @varsh2829 thanks for the question! You would follow the instructions for "working with data from a csv" found here.

I've written up a quick example for you using your sample data (thanks!). You should be able to follow this general format and plug in the numbers that don't exist in the data you sent.

import smps
import numpy as np
import pandas as pd

# Load the dataframe
df = pd.read_excel("Test.SMPS.data.xlsx")

# Let's grab just the meta data columns
meta = df.iloc[:, :12]

# Let's grab just the data columns
data = df.iloc[:, 12:-1]

# Take the list of columns from data and turn those into the midpoint diameters
midpoints = [float(x) for x in data.columns]

# Rename the columns in data to bin{x}
bin_labels = [f"bin{x}" for x in range(len(midpoints))]
data.columns = bin_labels

# Merge the data back together
merged = pd.merge(meta, data, left_index=True, right_index=True)

# Let's convert the timestamp into a timestamp object
merged["Datomaerke"] = merged["Datomaerke"].map(pd.to_datetime)

# Turn it all into an SMPS object
obj = smps.SMPS(
    data=merged, 
    bins=smps.utils.make_bins(midpoints=np.asarray(midpoints), lb=9.5, ub=485.)
)

[varsh2829]

Hi David!
Thank you for taking time to look at it. I just copy pasted your code in Spyder and it seems to work perfectly. I am wondering what did you mean when you said "plug in the numbers that don't exist in the data you sent.".
Also just for my own understanding: why are we doing this "bin_labels = [f"bin{x}" for x in range(len(midpoints))]
data.columns = bin_labels".
I mean when we create obj we simply give the data and bin midpoints that we extracted. Then why do we replace data columns with bin0, bin1 etc.?