Tooltips "@value" no longer works

[yt87]

ALL software version info

Software Version Info

Include version information here

python                    3.13.0          h9ebbce0_100_cp313    conda-forge
hvplot                    0.10.0             pyhd8ed1ab_0    conda-forge
holoviews                 1.18.3             pyhd8ed1ab_0    conda-forge
bokeh                     3.6.0              pyhd8ed1ab_0    conda-forge

Description of expected behavior and the observed behavior

hvplot 1.19 broke bokeh tooltips. The ("value", "@value") entry in tooltips does not make hover tool to display dataframe column values.

Complete, minimal, self-contained example code that reproduces the issue

This is a slightly modified example from https://discourse.holoviz.org/t/how-to-set-up-hovertool-for-multiple-columns-with-hvplot/4294/4. I run it as

panel serve --dev Untitled.ipynb

It works fine with holoviews up to 1.18.3, fails with higher versions, see the screen capture below..

# code goes here between backticks
import hvplot.pandas
import numpy as np
import pandas as pd
import panel as pn

pn.extension()

index = pd.date_range("2020-07-01", "2021-07-01", freq="D")
data = np.random.random((index.size, 4)) + 10 * np.arange(4)[np.newaxis, :]
df = pd.DataFrame(data, index=index, columns=list("ABCD"))

from bokeh.models import DatetimeTickFormatter, HoverTool

tickfmt = DatetimeTickFormatter(years="%m-%d", months="%m-%d")

tooltips = [
    ("Month-Day", "@index{%m-%d}"),
    ("value", "@value"),
    ("name", "@Variable")
]
hover = HoverTool(tooltips=tooltips, formatters={"@index": "datetime"})

plot = df.hvplot(xformatter=tickfmt, tools=[hover])
pn.Row(plot).servable()

#### Stack traceback and/or browser JavaScript console output

#### Screenshots or screencasts of the bug in action
![2024-10-15-064047_726x360_scrot](https://github.com/user-attachments/assets/0311ee0f-4568-4871-9aba-e32161b83ca1)

- [ ] I may be interested in making a pull request to address this

[yt87]

Trying again...

[yt87]

The new features hover_tooltips and hover_formatters do work, so this is not a big issue, just a backward incompatible change. I guess this issue can be closed.

[ahuang11]

Can you share how you used the hover_tooltips/hover_formatters?

[ahuang11]

The behavior was changed here:
#1350

[ahuang11]

Found it: https://github.com/holoviz/hvplot/pull/1350/files#diff-994c24099e9daf2dc458d1c4a0cad7381cccb075c5f438509b92fbb3bc3e8021L2132

Previously:

chart = element(data, kdims, vdims).redim(**{c: self.value_label})

Now:

chart = element(data, kdims, vdims)

And self._redim is {}. Not sure if this intentional; will defer to @maximlt

[yt87]

This is the code that works:

import hvplot.pandas
import numpy as np
import pandas as pd
import panel as pn

pn.extension()

index = pd.date_range("2020-07-01", "2021-07-01", freq="D")
data = np.random.random((index.size, 4)) + 10 * np.arange(4)[np.newaxis, :]
df = pd.DataFrame(data, index=index, columns=list("ABCD"))

from bokeh.models import DatetimeTickFormatter

tickfmt = DatetimeTickFormatter(years="%m-%d", months="%m-%d")

tooltips = [
    ("Month-Day", "@index{%m-%d}"),
    ("value", "@value"),
    ("name", "@Variable")
]
formatters={"@index": "datetime"}

plot = df.hvplot(xformatter=tickfmt, hover_tooltips=tooltips, hover_formatters=formatters)
pn.Row(plot).servable()

[maximlt]

The behavior was changed here: #1350

Just to confirm, did you do a git bisect?

Found it: https://github.com/holoviz/hvplot/pull/1350/files#diff-994c24099e9daf2dc458d1c4a0cad7381cccb075c5f438509b92fbb3bc3e8021L2132

Previously:

chart = element(data, kdims, vdims).redim(**{c: self.value_label})

Now:

chart = element(data, kdims, vdims)

And self._redim is {}. Not sure if this intentional; will defer to @maximlt

I can't look at this today but just want to note that the whole code now is:

chart = element(data, kdims, vdims)
chart = relabel_redim(chart, self._relabel, self._redim)
charts.append((c, chart))

[ahuang11]

Manually bisected :)

I initially thought my hover_tooltips PR broke it, but was surprised to find it was not the culprit, so I checked out relevant sounding ones, re-ran the code, and found it.

[maximlt]

Manually bisected :)

I initially thought my hover_tooltips PR broke it, but was surprised to find it was not the culprit, so I checked out relevant sounding ones, re-ran the code, and found it.

Yes I confirmed that the PR you identified broke caused this issue. Need to see if there's a way around this without losing the optimization.

[maximlt]

Finally managed to spend some time looking into this issue.

---

From Bokeh's docs:

Field names that begin with @ are associated with columns in a ColumnDataSource.

So @value refers to a column called value in the cds, that is no longer present.

Before the optimization (1e61065) recently-ish merged, hvPlot was calling .redim(**{c: self.value_label}) on each element of the overlay => the y Dimension had a name and label both equal to 'value' for all elements. After the optimization, the y Dimension has name equal to the column name and label to 'value'.

HoloViews uses the dimension name to create the dataset that ends up being passed to instantiate the ColumnDataSource. So after the change, the column in the cds is no longer called 'value' but 'A' (and 'B', 'C', in each their respective glyph/cds). Explaining why @value stopped working.

https://github.com/holoviz/holoviews/blob/2928aa20a0ffa0947e7263c3ba069cee6fe24e5c/holoviews/plotting/bokeh/chart.py#L366-L374

But then, how does this code from above work as it still references @value?

tooltips = [
    ("Month-Day", "@index{%m-%d}"),
    ("value", "@value"),
    ("name", "@Variable")
]
formatters={"@index": "datetime"}
plot = df.hvplot(xformatter=tickfmt, hover_tooltips=tooltips, hover_formatters=formatters)

This method in HoloViews ElementPlot._prepare_hover_kwargs() has some logic to "replace hover value aliases" (that's the name of the method _replace_hover_value_aliases()), that ends up replacing @value with @{A} (and @{B} and @{C} ...). With the code above, it's important to observe that, contrary to before, there's now a hover tool per curve (check the toolbar).

---

My feeling is that using @value in a custom HoverTool was more a workaround than anything else and that there are too many layers between hvPlot's code and how @value is used to ensure this can be maintained (obviously, we don't test that). However, the current "solution" isn't super satisfying either with a hover tool per curve.

@ahuang11 do you have a feeling for what we could do to improve the situation? Something in HoloViews?

[ahuang11]

My first thought is doing exactly:

This method in HoloViews ElementPlot._prepare_hover_kwargs() has some logic to "replace hover value aliases" (that's the name of the method _replace_hover_value_aliases()), that ends up replacing @value with @{A} (and @{B} and @{C} ...). With the code above, it's important to observe that, contrary to before, there's now a hover tool per curve (check the toolbar).

Does bokeh have a mechanism to combine those hover tools?