Add a decimals int = 2 parameter in describe

[stevenlis]

Problem description

import polars as pl

df = pl.DataFrame(
    {"A": [1.0, 2.0, 3.0, 4.0, 6.0, 8.0],
     "B": [1.4, 2.0, 3.3, 4.6, 6.0, 8.0],
     "C": ['a', 'b', 'c', 'c', 'a', 'd']}
)
df.describe()

The mean and standard deviation results always have many decimal places.

shape: (9, 4)
┌────────────┬──────────┬──────────┬──────┐
│ describe   ┆ A        ┆ B        ┆ C    │
│ ---        ┆ ---      ┆ ---      ┆ ---  │
│ str        ┆ f64      ┆ f64      ┆ str  │
╞════════════╪══════════╪══════════╪══════╡
│ count      ┆ 6.0      ┆ 6.0      ┆ 6    │
│ null_count ┆ 0.0      ┆ 0.0      ┆ 0    │
│ mean       ┆ 4.0      ┆ 4.216667 ┆ null │
│ std        ┆ 2.607681 ┆ 2.503131 ┆ null │
│ min        ┆ 1.0      ┆ 1.4      ┆ a    │
│ max        ┆ 8.0      ┆ 8.0      ┆ d    │
│ median     ┆ 3.5      ┆ 3.95     ┆ null │
│ 25%        ┆ 2.0      ┆ 2.0      ┆ null │
│ 75%        ┆ 6.0      ┆ 6.0      ┆ null │
└────────────┴──────────┴──────────┴──────┘

This could be fixed with the existing Expr.round, so I don't think there will be any performance drawbacks.

df.describe().with_columns(
    pl.col(pl.Float64).round(2)
)

shape: (9, 4)
┌────────────┬──────┬──────┬──────┐
│ describe   ┆ A    ┆ B    ┆ C    │
│ ---        ┆ ---  ┆ ---  ┆ ---  │
│ str        ┆ f64  ┆ f64  ┆ str  │
╞════════════╪══════╪══════╪══════╡
│ count      ┆ 6.0  ┆ 6.0  ┆ 6    │
│ null_count ┆ 0.0  ┆ 0.0  ┆ 0    │
│ mean       ┆ 4.0  ┆ 4.22 ┆ null │
│ std        ┆ 2.61 ┆ 2.5  ┆ null │
│ min        ┆ 1.0  ┆ 1.4  ┆ a    │
│ max        ┆ 8.0  ┆ 8.0  ┆ d    │
│ median     ┆ 3.5  ┆ 3.95 ┆ null │
│ 25%        ┆ 2.0  ┆ 2.0  ┆ null │
│ 75%        ┆ 6.0  ┆ 6.0  ┆ null │
└────────────┴──────┴──────┴──────┘

We could have a default value like decimals: int = 2 or 3 so that users don't have to do expr.round themselves anymore.

[ritchie46]

You shouldn't round floats for formatting purposes. This is a formatting issue, the underlying data should not change.

I think you should discuss on a thread regarding floating point formatting, but I think this should not be in describe. Rather in the config settings.

[stevenlis]

@ritchie46 The issue is that a Config option would have a global impact. But I want to be able to see the actual decimal places in my main dataframe so that I know if I need to round those float columns or not. So it would make sense to control results in describe separately. I don't think it makes much sense to use a Config context manager to format the describe results every time you run it.

#7475

[stevenlis]

@ritchie46
Maybe we can bring the float_precision in describe somehow #7475 (comment)

[ritchie46]

Config context manager to format the describe results every time you run it.

I think it does. Formatting shouldn't be in keyword arguments.

[stevenlis]

@ritchie46 Sure, let me rephrase that. A describe returns a summary of calculation results. We should have a means to control the precision of the calculations, right? So, it's not formatting but precision.

[ritchie46]

Rounding floating point values decreases the precision.