Enhance documentation of param namespace #997

MarcSkovMadsen · 2024-12-28T07:47:37Z

Continues from #992. Please review and merge that one first.

Focus is on updating the docstrings of Parameters class and its methods, i.e. the .param namespace.
I've added type annotations too.

codecov · 2024-12-28T07:49:12Z

Codecov Report

Attention: Patch coverage is 96.42857% with 1 line in your changes missing coverage. Please review.

Project coverage is 87.24%. Comparing base (210419d) to head (2b708e0).

Files with missing lines	Patch %	Lines
param/parameterized.py	96.42%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
- Coverage   87.25%   87.24%   -0.02%     
==========================================
  Files           9        9              
  Lines        4928     4931       +3     
==========================================
+ Hits         4300     4302       +2     
- Misses        628      629       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

MarcSkovMadsen · 2024-12-29T06:13:16Z

param/parameterized.py

+    # Please update the docstring with better description and examples
+    # I've (MarcSkovMadsen) not been able to understand this. Its probably because I lack context.
+    # Its not mentioned in the documentation.
+    # The pytests do not make sense to me.
    def set_dynamic_time_fn(self_,time_fn,sublistattr=None):


I hope someone else than me will update the docstring. I've not been able to understand this method. Its probably because I lack context. Its not mentioned in the documentation. The pytests are not valuable to me.

MarcSkovMadsen · 2024-12-29T06:15:28Z

param/parameterized.py

@@ -2815,6 +3178,9 @@ def values(self_, onlychanged=False):
        vals.sort(key=itemgetter(0))
        return dict(vals)

+    # Please update the docstring with better description and examples
+    # I've (MarcSkovMadsen) not been able to understand this. Its probably because I lack context.
+    # Its not mentioned in the documentation or pytests
    def force_new_dynamic_value(self_, name): # pylint: disable-msg=E0213


I hope someone else than me will update this docstring. I lack context and cannot find it in the documentation or pytests.

MarcSkovMadsen · 2024-12-29T18:50:43Z

I don't know why this fails on windows + python 3.12. Because this downstream PR #998 succeeds.

maximlt · 2025-01-02T10:51:56Z

I don't know why this fails on windows + python 3.12

These are unrelated test failures.

@MarcSkovMadsen can you resolve the conflicts?

philippjfr · 2025-01-02T10:56:10Z

Your efforts on these PRs is heroic, but please make PRs like this self-contained in future the merge conflicts are a nightmare.

maximlt · 2025-01-02T11:04:30Z

Thanks @philippjfr !!! 🙏 I didn't feel brave enough to deal with these crazy conflicts on my first day back from holidays :D

MarcSkovMadsen · 2025-01-02T14:56:58Z

Your efforts on these PRs is heroic, but please make PRs like this self-contained in future the merge conflicts are a nightmare.

I would also like to avoid merge conflicts. But technically I don't know how to do this if one PR has to build on another PR. I was asked not to make one big PR.

How can I do this differently?
At work this almost always work. Why does it not work here?
As far as I can see the merge conflicts have been resolved (thanks) and I don't have to do anything.

philippjfr · 2025-01-02T15:15:32Z

I was asked not to make one big PR.

I think this is quite context dependent, I would certainly have suggested that improving docstrings should have been a single PR, though separate PRs would have been fine if they didn't build on each other. For code I get the fact that each PR has to build on the other but I'm a little perplexed why that would be needed for docstrings. Certainly I would never suggest creating more than two separate PRs that depend on each other because you end up with this awful daisy chain of merge conflicts. This was probably the worst case scenario of that because the main issues were in the base PR, i.e. the Ruff PR with all the wrongly formatted docstrings. In any case, we're mostly done now.

maximlt · 2025-01-02T16:03:56Z

Yes I suggested multiple PRs as I'm pretty sure that documenting some parts isn't going to be straightforward (e.g. documenting parameters.py without duplicating all the docstrings and making the module much longer to import) and I was concerned this would block merging any improvements at all! I also think we should give @jbednar a chance to chime in before merging all these docstring changes (even if it's just to say "go ahead!").

As far as I can see the merge conflicts have been resolved

FYI not in #998

Oh and I've just noticed you added type hints. It's nice but also more work for reviewers (shouldn't we set up mypy before adding type hints?). From a quick look, I saw for example that the docstring of watch shows that what is optional but the type hint doesn't. Which one is correct?

    def watch(
        self_, fn, parameter_names: list[str], what: str='value', onlychanged: bool=True,
        queued: bool=False, precedence: int=0
    ) -> Watcher:
        ...
        what : str, optional
            The type of change to watch for. By default, this is 'value', but it
            can be set to other slots such as 'constant'. Default is 'value'.

MarcSkovMadsen · 2025-01-02T22:26:17Z

I saw for example that the docstring of watch shows that what is optional but the type hint doesn't. Which one is correct?

    def watch(

        self_, fn, parameter_names: list[str], what: str='value', onlychanged: bool=True,

        queued: bool=False, precedence: int=0

    ) -> Watcher:

        ...

        what : str, optional

            The type of change to watch for. By default, this is 'value', but it

            can be set to other slots such as 'constant'. Default is 'value'.

Thx.

I think the type annotation and docstring is correct and consistent:

You dont type annotate that an argument has a default value and its optional to provide a value
the docstring "optional" does not refer to a type annotation. It means a default value has been set and that its optional to provide a value.

?

maximlt · 2025-01-02T23:13:24Z

Ah true! I got confused with typing.Optional.

philippjfr · 2025-01-03T13:21:53Z

As for the general typing question. I certainly would like to fully type param in the near future. I'm personally okay with adding a few types here or there until we start on that effort but if we want to add some basic mypy validation to the test suite first I'm also okay with that.

maximlt

Note that there are some comments I haven't repeated in all the places where they were relevant (e.g. when Returns is specified with None).

param/parameterized.py

maximlt · 2025-01-03T13:54:25Z

param/parameterized.py

@@ -1933,7 +1938,29 @@ def watchers(self_, value):
        self_.self._param__private.watchers = value

    @property
-    def self_or_cls(self_):
+    def self_or_cls(self_) -> Union['Parameterized', Type['Parameterized']]:


self_or_cls is not documented on the website. It feels more like a helper for Param's internals, as far as I can see it's not used by other HoloViz libraries. Do you intend to make it more public? If not, I think we should make it private. Not sure it's worth the long docstring in that case, the code is straightforward.

Its not the intent of these docstring PRs to change the api.

I would suggest that when this is made private the docstring is removed. As long as its public it should have a proper docstring.

I don't think it's public.

For me its public because it can be accessed via param.parameterized.self_or_class. There is no leading _ in the path.

For me as a consumer of the library that would mean its public and something i can use.

I guess it's fine for this one. But please do not make the mistake of assuming that an API that doesn't start with _ is necessarily public:

This is just a convention, packages can adopt other ones (https://pandas.pydata.org/docs/reference/index.html)

HoloViz packages don't do a great job at separating their private vs public API

Param just recently started to make that distinction clearer (Deprecate private or unused functions in __init__.py #768)

For Param, I'd say the real public API is anything that is documented on the website.

param/parameterized.py

maximlt · 2025-01-03T13:58:51Z

param/parameterized.py

-                'constructor before trying to access instance Parameter objects, or '
-                'looking up the class Parameter objects with `.param.objects(instance=False)` '
-                'may be enough for your use case.',
+                'Cannot access instance parameters before the Parameterized instance '


Why was this changed?

In my judgement the existing string was very hard to understand and very long. To me this string is much easier to comprehend.

Please do not make unrelated code changes in a PR dedicated to touching the docstrings only.

I changed it because I got the error while testing the examples. I did not understand the error message. So I spent time trying to improve it. I do believe its an improvement.

Would it be ok to include anyways? Thx.

The most important message imo is that super().__init__(**params) has to be called before self.param.objects(). I don't think the updated message conveys that. I still prefer the original message overall.

Besides that, I know I'm annoying with asking not to mix different changes in a single PR, but as a release manager this is the only sane way to produce an accurate changelog. This PR title for example isn't correct, it is really "Enhance docstrings of the .param namespace and Parameterized class + Add type hints + Enhance error message of .param.objects()" 🙃

param/parameterized.py

maximlt · 2025-01-03T15:29:00Z

param/parameterized.py

@@ -4383,51 +5081,125 @@ def __setstate__(self, state):

 class Parameterized(metaclass=ParameterizedMetaclass):
    """


The Numpy docstring guide seems to say that you should document __init__ in the class docstring (here), see https://numpydoc.readthedocs.io/en/latest/format.html#class-docstring. That's what Pandas does for example https://github.com/pandas-dev/pandas/blob/b8a4691647a8850d681409c5dd35a12726cd94a1/pandas/core/frame.py#L505.
So the Parameters section. you added to __init__ should be moved to the class docstring. Apparently you could also add an Attributes section to document name and param.

Thx. I'm a little bit in doubt what to do here because

the numpy recommendation was created long time before for example VS Code was a thing. In VS Code I believe I have seen the class docstring and the __init__ docstring being used at different times by pylance tooltips. Depending on whether you have not yet typed ( or you have.

In param docs the __init__ method is documented. See for example https://param.holoviz.org/reference/param/generated/param.Action.html#param.Action.__init__. If I remove the docstring from __init__ the param docs will be empty there. Might not be a problem.

What would you recommend me to do?

Someone will need to look more into that, without forgetting Jupyter as it's a major target for HoloViz packages.

In param docs the init method is documented

Yes I already mentioned that documenting the Parameters is tricky.

Co-authored-by: Maxime Liquet <[email protected]>

…/holoviz/param into enhancement/docs-param-namespace

MarcSkovMadsen · 2025-01-04T16:10:22Z

Note that there are some comments I haven't repeated in all the places where they were relevant (e.g. when Returns is specified with None).

Sorry. I think I missed the the "Returns is specified with None" comment. What does this mean?

maximlt · 2025-01-04T18:28:01Z

Sorry. I think I missed the the "Returns is specified with None" comment. What does this mean?

This means that I'm not sure it's useful to document that?

Returns
--------
None

MarcSkovMadsen · 2025-01-04T18:38:28Z

Sorry. I think I missed the the "Returns is specified with None" comment. What does this mean?

This means that I'm not sure it's useful to document that?
Returns
--------
None

I sort of agree. The problem is that if its not include then the user does not know if its intended or forgotten.

We have started using ruff at work. And with the rules that are standard at work its actually required to type annotate -> None for functions that do not return anything. Again because you don't know if the type annotation was forgotten if you don't include it.

maximlt · 2025-01-06T22:27:41Z

param/parameterized.py

@@ -29,6 +29,7 @@
 from itertools import chain
 from operator import itemgetter, attrgetter
 from types import FunctionType, MethodType
+from typing import Any, Union, Literal, Iterable, Callable # When python 3.9 support is dropped replace Union with |


I learned that typing.Callable and typing.Iterable are sort of deprecated, instead one should import them from collections.abc. See https://docs.python.org/3/library/typing.html#typing.Iterable.

maximlt · 2025-01-06T22:34:48Z

I sort of agree. The problem is that if its not include then the user does not know if its intended or forgotten.

Ok yes.

MarcSkovMadsen added 6 commits December 25, 2024 05:00

ruff D400 implementation

f7d679d

Enable ruff D401

7843a0a

add package docstrings

bf9c27e

improve package docstring

fcd87c0

remove script file

d1ca89e

docs Parameterized

b66a846

MarcSkovMadsen added 3 commits December 28, 2024 08:02

doc paramterized

2aa6547

.param docstrings

b8f8ec7

.param docstrings

8aa7385

MarcSkovMadsen commented Dec 29, 2024

View reviewed changes

MarcSkovMadsen marked this pull request as ready for review December 29, 2024 06:17

MarcSkovMadsen requested review from hoxbro and maximlt December 29, 2024 06:17

MarcSkovMadsen added the ready for review label Dec 29, 2024

use param logging

b33c49b

MarcSkovMadsen mentioned this pull request Dec 29, 2024

Other Docstring improvements #998

Open

Merge branch 'main' into enhancement/docs-param-namespace

d4fd0fc

maximlt requested changes Jan 3, 2025

View reviewed changes

MarcSkovMadsen and others added 5 commits January 4, 2025 16:45

Update param/parameterized.py

d7f47fe

Co-authored-by: Maxime Liquet <[email protected]>

Update param/parameterized.py

9a48c8f

Co-authored-by: Maxime Liquet <[email protected]>

Update param/parameterized.py

b9529ab

Co-authored-by: Maxime Liquet <[email protected]>

review feedback

2e554d4

Merge branch 'enhancement/docs-param-namespace' of https://github.com…

189c19d

…/holoviz/param into enhancement/docs-param-namespace

MarcSkovMadsen requested a review from maximlt January 4, 2025 16:07

pre-commit

2b708e0

maximlt reviewed Jan 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance documentation of param namespace #997

Enhance documentation of param namespace #997

MarcSkovMadsen commented Dec 28, 2024 •

edited

Loading

codecov bot commented Dec 28, 2024 •

edited

Loading

MarcSkovMadsen Dec 29, 2024

MarcSkovMadsen Dec 29, 2024

MarcSkovMadsen commented Dec 29, 2024

maximlt commented Jan 2, 2025

philippjfr commented Jan 2, 2025

maximlt commented Jan 2, 2025

MarcSkovMadsen commented Jan 2, 2025 •

edited

Loading

philippjfr commented Jan 2, 2025

maximlt commented Jan 2, 2025

MarcSkovMadsen commented Jan 2, 2025 •

edited

Loading

maximlt commented Jan 2, 2025

philippjfr commented Jan 3, 2025

maximlt left a comment

maximlt Jan 3, 2025

MarcSkovMadsen Jan 4, 2025

maximlt Jan 4, 2025

MarcSkovMadsen Jan 5, 2025

maximlt Jan 6, 2025

maximlt Jan 3, 2025

MarcSkovMadsen Jan 4, 2025

maximlt Jan 4, 2025

MarcSkovMadsen Jan 4, 2025

maximlt Jan 6, 2025

maximlt Jan 6, 2025 •

edited

Loading

maximlt Jan 3, 2025

MarcSkovMadsen Jan 4, 2025 •

edited

Loading

maximlt Jan 4, 2025

MarcSkovMadsen commented Jan 4, 2025

maximlt commented Jan 4, 2025

MarcSkovMadsen commented Jan 4, 2025 •

edited

Loading

maximlt Jan 6, 2025

maximlt commented Jan 6, 2025

		@@ -4383,51 +5081,125 @@ def __setstate__(self, state):

		class Parameterized(metaclass=ParameterizedMetaclass):
		"""

Enhance documentation of param namespace #997

Are you sure you want to change the base?

Enhance documentation of param namespace #997

Conversation

MarcSkovMadsen commented Dec 28, 2024 • edited Loading

codecov bot commented Dec 28, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcSkovMadsen commented Dec 29, 2024

maximlt commented Jan 2, 2025

philippjfr commented Jan 2, 2025

maximlt commented Jan 2, 2025

MarcSkovMadsen commented Jan 2, 2025 • edited Loading

philippjfr commented Jan 2, 2025

maximlt commented Jan 2, 2025

MarcSkovMadsen commented Jan 2, 2025 • edited Loading

maximlt commented Jan 2, 2025

philippjfr commented Jan 3, 2025

maximlt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maximlt Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcSkovMadsen Jan 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcSkovMadsen commented Jan 4, 2025

maximlt commented Jan 4, 2025

MarcSkovMadsen commented Jan 4, 2025 • edited Loading

Choose a reason for hiding this comment

maximlt commented Jan 6, 2025

MarcSkovMadsen commented Dec 28, 2024 •

edited

Loading

codecov bot commented Dec 28, 2024 •

edited

Loading

MarcSkovMadsen commented Jan 2, 2025 •

edited

Loading

MarcSkovMadsen commented Jan 2, 2025 •

edited

Loading

maximlt Jan 6, 2025 •

edited

Loading

MarcSkovMadsen Jan 4, 2025 •

edited

Loading

MarcSkovMadsen commented Jan 4, 2025 •

edited

Loading