"Undefined template variable" During pytest Runs

[Dresdn]

I'm trying to integrate django-cotton into a mature project, and I'm running into issues when default vars not being used during pytest runs.

Templates

cotton/button.html:

<c-vars type="button" classes="" />
<button type="{{ type }}" class="c-button {{ classes }}" {{ attrs }}>
  {{ slot }}
</button>

index.html:

<c-button classes="foo">Cotton Button</c-button>

Issue

When running the app, the component renders with type="button" as expected. However, running pytest, I get:

Failed: Undefined template variable 'type' in '/path/to/app/server/templates/cotton/button.html'

I've tried this on a simpler project (cloned the Mozilla Library Tutorial), and it works as expected, so it's my project. The challenge is that I'm not strong with the templating system, so any guidance on where to look would be great.

I appreciate any help and/or guidance!

Extra Info

Settings

Happy to provide more if helpful.

INSTALLED_APPS = [..., 'django_cotton']

TEMPLATES = [
    {
        'APP_DIRS': True,
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            BASE_DIR.joinpath('server', 'templates'),
        ],
        'OPTIONS': {
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.template.context_processors.debug',
                'django.template.context_processors.i18n',
                'django.template.context_processors.media',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.request',
                'server.apps.base.context_processors.export_vars',
                'server.apps.base.context_processors.meta',
            ],
        },
    },
]

pytest Stack Track

.venv/lib/python3.11/site-packages/django/template/base.py:880: in _resolve_lookup
    current = current[bit]
.venv/lib/python3.11/site-packages/django/template/context.py:83: in __getitem__
    raise KeyError(key)
E   KeyError: 'type'

During handling of the above exception, another exception occurred:
.venv/lib/python3.11/site-packages/django/template/base.py:886: in _resolve_lookup
    if isinstance(current, BaseContext) and getattr(
E   AttributeError: type object 'Context' has no attribute 'type'

During handling of the above exception, another exception occurred:
.venv/lib/python3.11/site-packages/django/template/base.py:896: in _resolve_lookup
    current = current[int(bit)]
E   ValueError: invalid literal for int() with base 10: 'type'

During handling of the above exception, another exception occurred:
.venv/lib/python3.11/site-packages/django/template/base.py:715: in resolve
    obj = self.var.resolve(context)
.venv/lib/python3.11/site-packages/django/template/base.py:847: in resolve
    value = self._resolve_lookup(context)
.venv/lib/python3.11/site-packages/django/template/base.py:903: in _resolve_lookup
    raise VariableDoesNotExist(
E   django.template.base.VariableDoesNotExist: Failed lookup for key [type] in [{'True': True, 'False': False, 
'None': None}, {'True': True, 'False': False, 'None': None, 'csrf_token': <SimpleLazyObject: <function csrf.
<locals>._get_val at 0x129357e20>>, 'user': <Mock id='4984067536'>, 'perms': PermWrapper(<Mock id
='4984067536'>), 'LANGUAGES': (('en', 'English'),), 'LANGUAGE_CODE': 'en-us', 'LANGUAGE_BIDI': False, 
'MEDIA_URL': '/media/', 'messages': [], 'DEFAULT_MESSAGE_LEVELS': {'DEBUG': 10, 'INFO': 20, 'SUCCESS': 
25, 'WARNING': 30, 'ERROR': 40}, 'request': <WSGIRequest: GET '/'>, 'DEVELOPMENT': True, 'DOMAIN_NAME': 
'domain.test', 'tenants_count': 0, 'first_additional_tenant': None, 'project_meta': {'NAME': 'Company', 'URL': 
'http://domain.test', 'DESCRIPTION': 'Some Text', 'IMAGE': 'https://www.domain.com/image.jpg', 'KEYWORDS': 
'neat, thing', 'CONTACT_EMAIL': '[email protected]', 'TITLE': 'Page Title'}, 'page_url': 'https:////', 'page_title': 
'', 'page_description': '', 'page_image': '', 'slot': 'Cotton Button', 'ctn_unprocessable_dynamic_attrs': set(), 
'attrs': 'classes="foo"', 'attrs_dict': {'classes': 'foo'}, 'classes': 'foo'}]

During handling of the above exception, another exception occurred:
tests/test_apps/test_index_view.py:24: in test_root_page
    response = index(request)
server/apps/main/views.py:24: in index
    return render(request, 'index.html')
.venv/lib/python3.11/site-packages/django/shortcuts.py:24: in render
    content = loader.render_to_string(template_name, context, request, using=using)
.venv/lib/python3.11/site-packages/django/template/loader.py:62: in render_to_string
    return template.render(context, request)
.venv/lib/python3.11/site-packages/django/template/backends/django.py:61: in render
    return self.template.render(context)
.venv/lib/python3.11/site-packages/django/template/base.py:175: in render
    return self._render(context)
.venv/lib/python3.11/site-packages/django/test/utils.py:112: in instrumented_test_render
    return self.nodelist.render(context)
.venv/lib/python3.11/site-packages/django/template/base.py:1005: in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
.venv/lib/python3.11/site-packages/django/template/base.py:1005: in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
.venv/lib/python3.11/site-packages/django/template/base.py:966: in render_annotated
    return self.render(context)
.venv/lib/python3.11/site-packages/django_cotton/templatetags/_component.py:97: in render
    return tpl.render(ctx)
.venv/lib/python3.11/site-packages/django/template/backends/django.py:61: in render
    return self.template.render(context)
.venv/lib/python3.11/site-packages/django/template/base.py:175: in render
    return self._render(context)
.venv/lib/python3.11/site-packages/django/test/utils.py:112: in instrumented_test_render
    return self.nodelist.render(context)
.venv/lib/python3.11/site-packages/django/template/base.py:1005: in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
.venv/lib/python3.11/site-packages/django/template/base.py:1005: in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
.venv/lib/python3.11/site-packages/django/template/base.py:966: in render_annotated
    return self.render(context)
.venv/lib/python3.11/site-packages/django_cotton/templatetags/_vars_frame.py:43: in render
    c_vars[key] = value.resolve(context)
.venv/lib/python3.11/site-packages/django/template/base.py:723: in resolve
    return string_if_invalid % self.var
.venv/lib/python3.11/site-packages/pytest_django/plugin.py:703: in __mod__
    pytest.fail(msg)
E   Failed: Undefined template variable 'type' in '/path/to/app/server/templates/cotton/button.html'

[wrabit]

@Dresdn which version of cotton are you on?

[Dresdn]

The original post was using 0.9.38. I just updated it to 0.9.40 and have the same results.

As an FYI, I was thinking that the way pytest loads Django, maybe something was being loaded/read too soon, so I tried using the implicit registration. That had the same results.

Editing to add other versions:

django = "4.2.11"
pytest = "8.3.3"
pytest-django = "4.9.0"

[Dresdn]

Since I'm in it, here's what the settings.TEMPLATES is during pytest runs. Everything looks to be loaded correctly.

[
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            PosixPath(
                '/path/to/app/server/templates'
            )
        ],
        'OPTIONS': {
            'builtins': ['django_cotton.templatetags.cotton'],
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.template.context_processors.debug',
                'django.template.context_processors.i18n',
                'django.template.context_processors.media',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.request',
            ],
            'debug': True,
            'loaders': [
                (
                    'django.template.loaders.cached.Loader',
                    [
                        'django_cotton.cotton_loader.Loader',
                        'django.template.loaders.filesystem.Loader',
                        'django.template.loaders.app_directories.Loader',
                    ],
                )
            ],
            'string_if_invalid': <pytest_django.plugin._fail_for_invalid_template_variable.<locals>.InvalidVarException object at 0x10c8b9650>'
        },
    }
]

[wrabit]

@Dresdn I wasn't able to replicate with

@pytest.mark.django_db
def test_index_view(client):
    response = client.get('/')
    assert "Cotton Button" in response.content.decode('utf-8')

python 3.11
same versions as you, index file and button component.

I recommend trying to create in a minimal reproducible..

[Dresdn]

Thanks for taking a look. I wasn't able to reproduce in a fresh project either.

I hoped something was glaringly missing, so I'll keep stripping down the project to see if I can pinpoint anything.

Feel free to close or leave this open, but it'll take me a few days to really get into this.

[Dresdn]

@wrabit I couldn't let it go, but I figured out what is causing the failure. In our pyteset options, we use pytest-django's --fail-on-template-vars. Removing this allows the tests to pass.

The challenge is, looking at what they're doing for this option, I'm not sure if there's a good way to solve this on the Cotton end.

I'm thinking I'll submit an MR there to allow ignoring template directories or something.

[wrabit]

@Dresdn I did have a play with this and even when I tried to catch the exception in cotton, it still seemed to trigger the fail on pytest..

[Dresdn]

This looks to be pretty core to Django since the django-pytest team is setting the string_if_invalid when the flag is enabled. I did a quick search in the project, and seems it's pretty challenging issue. This thread is a good read and gives the sentiment of the maintainers: pytest-dev/pytest-django#922

AdamChainz wrote an interesting blog about the perils of doing so, and it seems we're hitting the default filter challenge.

[Dresdn]

Actually, if anything, maybe just a note about pytest-django users and the flag when using <c-vars> with defaults? Could save someone some headache.

[wrabit]

it seems we're hitting the default filter challenge.

Ah makes sense, that's what's behind the defaults for cvars.

I'll make a note to add this to the docs/readme