dealing with missing data in stream.iter_csv

[cdeterman]

I am trying to use the stream.iter_csv on a file that I have saved previously. This initially had some None values in it that pd.to_csv will convert to empty strings. It appears that even when I try to set the argument drop_nones=True it will still error out when I am trying to use the converters dictionary expecting the filed to be an int. Instead of dropping the row with the missing value it still tries to process it given that it looks like a string. Ultimately, I want to be able to skip the row when this missing value is encountered. My initial thought would be to remove the column from the converter dictionary, evaluate after the row is loaded and if it is an empty string then to skip and proceed to the next loop iteration. But this seems rather clunky and I'm not sure if there is another way.

Any recommendations on how to handle situations like this?

[MaxHalford]

Dropping a row because it has a missing value sounds like something specific to what you're doing. My suggestion would be to use Python's built-in DictReader from the csv module and apply this logic yourself.