enterprise/rac: Add e2e tests

[BeryJu]

Details

REPLACE ME

---

Checklist

• Local tests pass (ak test authentik/)
• The code has been formatted (make lint-fix)

If an API change has been made

• The API schema has been updated (make gen-build)

If changes to the frontend have been made

• The code has been formatted (make web)

If applicable

• The documentation has been updated
• The documentation has been formatted (make website)

[netlify]

✅ Deploy Preview for authentik-storybook canceled.

Name	Link
🔨 Latest commit	6000081
🔍 Latest deploy log	https://app.netlify.com/sites/authentik-storybook/deploys/678a7c08c0a9a800082313e4

[netlify]

✅ Deploy Preview for authentik-docs canceled.

Name	Link
🔨 Latest commit	6000081
🔍 Latest deploy log	https://app.netlify.com/sites/authentik-docs/deploys/678a7c088d55c100082a2e4e

[codecov]

❌ 4 Tests Failed:

Tests completed	Failed	Passed	Skipped
1655	4	1651	2

View the top 2 failed tests by shortest run time

tests.e2e.test_provider_proxy.TestProviderProxy::test_proxy_simple

Stack Traces | 0.001s run time

self = <docker.api.client.APIClient object at 0x7f15ae088080>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/client.py:275: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker:.../localhost/v1.45/networks/4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../site-packages/requests/models.py:1024: HTTPError

The above exception was the direct cause of the following exception:

self = <unittest.case._Outcome object at 0x7f15ae0887d0>
test_case = <tests.e2e.test_provider_proxy.TestProviderProxy testMethod=test_proxy_simple>
subTest = False

    @contextlib.contextmanager
    def testPartExecutor(self, test_case, subTest=False):
        old_success = self.success
        self.success = True
        try:
>           yield

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy.TestProviderProxy testMethod=test_proxy_simple>
result = <TestCaseFunction test_proxy_simple>

    def run(self, result=None):
        if result is None:
            result = self.defaultTestResult()
            startTestRun = getattr(result, 'startTestRun', None)
            stopTestRun = getattr(result, 'stopTestRun', None)
            if startTestRun is not None:
                startTestRun()
        else:
            stopTestRun = None
    
        result.startTest(self)
        try:
            testMethod = getattr(self, self._testMethodName)
            if (getattr(self.__class__, "__unittest_skip__", False) or
                getattr(testMethod, "__unittest_skip__", False)):
                # If the class or method was skipped.
                skip_why = (getattr(self.__class__, '__unittest_skip_why__', '')
                            or getattr(testMethod, '__unittest_skip_why__', ''))
                _addSkip(result, self, skip_why)
                return result
    
            expecting_failure = (
                getattr(self, "__unittest_expecting_failure__", False) or
                getattr(testMethod, "__unittest_expecting_failure__", False)
            )
            outcome = _Outcome(result)
            start_time = time.perf_counter()
            try:
                self._outcome = outcome
    
                with outcome.testPartExecutor(self):
                    self._callSetUp()
                if outcome.success:
                    outcome.expecting_failure = expecting_failure
                    with outcome.testPartExecutor(self):
                        self._callTestMethod(testMethod)
                    outcome.expecting_failure = False
                    with outcome.testPartExecutor(self):
>                       self._callTearDown()

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:637: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy.TestProviderProxy testMethod=test_proxy_simple>

    def _callTearDown(self):
>       self.tearDown()

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:594: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy.TestProviderProxy testMethod=test_proxy_simple>

    def tearDown(self):
        if IS_CI:
            print("::endgroup::", file=stderr)
>       super().tearDown()

tests/e2e/utils.py:206: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy.TestProviderProxy testMethod=test_proxy_simple>

    def tearDown(self):
        containers: list[Container] = self.docker_client.containers.list(
            filters={"label": ",".join(f"{x}={y}" for x, y in self.docker_labels.items())}
        )
        for container in containers:
            self.output_container_logs(container)
            try:
                container.kill()
            except DockerException:
                pass
            try:
                container.remove(force=True)
            except DockerException:
                pass
>       self.__network.remove()

tests/e2e/utils.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Network: 4f8f3d56e186>

    def remove(self):
        """
        Remove this network.
    
        Raises:
            :py:class:`docker.errors.APIError`
                If the server returns an error.
        """
>       return self.client.api.remove_network(self.id)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/models/networks.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae088080>
resource_id = '4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6'
args = (), kwargs = {}

    @functools.wraps(f)
    def wrapped(self, resource_id=None, *args, **kwargs):
        if resource_id is None and kwargs.get(resource_name):
            resource_id = kwargs.pop(resource_name)
        if isinstance(resource_id, dict):
            resource_id = resource_id.get('Id', resource_id.get('ID'))
        if not resource_id:
            raise errors.NullResource(
                'Resource ID was not provided'
            )
>       return f(self, resource_id, *args, **kwargs)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/utils/decorators.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae088080>
net_id = '4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6'

    @check_resource('net_id')
    def remove_network(self, net_id):
        """
        Remove a network. Similar to the ``docker network rm`` command.
    
        Args:
            net_id (str): The network's id
        """
        url = self._url("/networks/{0}", net_id)
        res = self._delete(url)
>       self._raise_for_status(res)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/network.py:186: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae088080>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
            response.raise_for_status()
        except requests.exceptions.HTTPError as e:
>           raise create_api_error_from_http_exception(e) from e

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/client.py:277: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('404 Client Error: Not Found for url: http+docker:.../localhost/v1.45/networks/4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.text or '').strip()
        cls = APIError
        if response.status_code == 404:
            explanation_msg = (explanation or '').lower()
            if any(fragment in explanation_msg
                   for fragment in _image_not_found_explanation_fragments):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation) from e
E       docker.errors.NotFound: 404 Client Error for http+docker:.../localhost/v1.45/networks/4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6: Not Found ("network 4f8f3d56e1866b850c3667a2c0c7874159feb1b6ed5ffdbdc5822c79d4442ad6 not found")

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../site-packages/docker/errors.py:39: NotFound

tests.e2e.test_provider_proxy_forward.TestProviderProxyForward::test_caddy

Stack Traces | 0.001s run time

self = <docker.api.client.APIClient object at 0x7f15ae3905f0>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/client.py:275: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker:.../localhost/v1.45/networks/c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../site-packages/requests/models.py:1024: HTTPError

The above exception was the direct cause of the following exception:

self = <unittest.case._Outcome object at 0x7f15adb9ef60>
test_case = <tests.e2e.test_provider_proxy_forward.TestProviderProxyForward testMethod=test_caddy>
subTest = False

    @contextlib.contextmanager
    def testPartExecutor(self, test_case, subTest=False):
        old_success = self.success
        self.success = True
        try:
>           yield

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy_forward.TestProviderProxyForward testMethod=test_caddy>
result = <TestCaseFunction test_caddy>

    def run(self, result=None):
        if result is None:
            result = self.defaultTestResult()
            startTestRun = getattr(result, 'startTestRun', None)
            stopTestRun = getattr(result, 'stopTestRun', None)
            if startTestRun is not None:
                startTestRun()
        else:
            stopTestRun = None
    
        result.startTest(self)
        try:
            testMethod = getattr(self, self._testMethodName)
            if (getattr(self.__class__, "__unittest_skip__", False) or
                getattr(testMethod, "__unittest_skip__", False)):
                # If the class or method was skipped.
                skip_why = (getattr(self.__class__, '__unittest_skip_why__', '')
                            or getattr(testMethod, '__unittest_skip_why__', ''))
                _addSkip(result, self, skip_why)
                return result
    
            expecting_failure = (
                getattr(self, "__unittest_expecting_failure__", False) or
                getattr(testMethod, "__unittest_expecting_failure__", False)
            )
            outcome = _Outcome(result)
            start_time = time.perf_counter()
            try:
                self._outcome = outcome
    
                with outcome.testPartExecutor(self):
                    self._callSetUp()
                if outcome.success:
                    outcome.expecting_failure = expecting_failure
                    with outcome.testPartExecutor(self):
                        self._callTestMethod(testMethod)
                    outcome.expecting_failure = False
                    with outcome.testPartExecutor(self):
>                       self._callTearDown()

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:637: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy_forward.TestProviderProxyForward testMethod=test_caddy>

    def _callTearDown(self):
>       self.tearDown()

.../hostedtoolcache/Python/3.12.8........./x64/lib/python3.12/unittest/case.py:594: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy_forward.TestProviderProxyForward testMethod=test_caddy>

    def tearDown(self):
        if IS_CI:
            print("::endgroup::", file=stderr)
>       super().tearDown()

tests/e2e/utils.py:206: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.e2e.test_provider_proxy_forward.TestProviderProxyForward testMethod=test_caddy>

    def tearDown(self):
        containers: list[Container] = self.docker_client.containers.list(
            filters={"label": ",".join(f"{x}={y}" for x, y in self.docker_labels.items())}
        )
        for container in containers:
            self.output_container_logs(container)
            try:
                container.kill()
            except DockerException:
                pass
            try:
                container.remove(force=True)
            except DockerException:
                pass
>       self.__network.remove()

tests/e2e/utils.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Network: c975ee0fc3b0>

    def remove(self):
        """
        Remove this network.
    
        Raises:
            :py:class:`docker.errors.APIError`
                If the server returns an error.
        """
>       return self.client.api.remove_network(self.id)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/models/networks.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae3905f0>
resource_id = 'c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3'
args = (), kwargs = {}

    @functools.wraps(f)
    def wrapped(self, resource_id=None, *args, **kwargs):
        if resource_id is None and kwargs.get(resource_name):
            resource_id = kwargs.pop(resource_name)
        if isinstance(resource_id, dict):
            resource_id = resource_id.get('Id', resource_id.get('ID'))
        if not resource_id:
            raise errors.NullResource(
                'Resource ID was not provided'
            )
>       return f(self, resource_id, *args, **kwargs)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/utils/decorators.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae3905f0>
net_id = 'c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3'

    @check_resource('net_id')
    def remove_network(self, net_id):
        """
        Remove a network. Similar to the ``docker network rm`` command.
    
        Args:
            net_id (str): The network's id
        """
        url = self._url("/networks/{0}", net_id)
        res = self._delete(url)
>       self._raise_for_status(res)

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/network.py:186: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f15ae3905f0>
response = <Response [404]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
            response.raise_for_status()
        except requests.exceptions.HTTPError as e:
>           raise create_api_error_from_http_exception(e) from e

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../docker/api/client.py:277: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('404 Client Error: Not Found for url: http+docker:.../localhost/v1.45/networks/c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.text or '').strip()
        cls = APIError
        if response.status_code == 404:
            explanation_msg = (explanation or '').lower()
            if any(fragment in explanation_msg
                   for fragment in _image_not_found_explanation_fragments):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation) from e
E       docker.errors.NotFound: 404 Client Error for http+docker:.../localhost/v1.45/networks/c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3: Not Found ("network c975ee0fc3b059d61b87332204c224d4f2f649d9639d8b27a9c384f4b11f81f3 not found")

../../../..../pypoetry/virtualenvs/authentik-xvtLQ9eE-py3.12/lib/python3.12.../site-packages/docker/errors.py:39: NotFound

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

[github-actions]

authentik PR Installation instructions

Instructions for docker-compose

Add the following block to your .env file:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-67626f77b7da572deb59b935e1a9b175740e7d51
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

For arm64, use these values:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-67626f77b7da572deb59b935e1a9b175740e7d51-arm64
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

Afterwards, run the upgrade commands from the latest release notes.

Instructions for Kubernetes

Add the following block to your values.yml file:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
global:
    image:
        repository: ghcr.io/goauthentik/dev-server
        tag: gh-67626f77b7da572deb59b935e1a9b175740e7d51

For arm64, use these values:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
global:
    image:
        repository: ghcr.io/goauthentik/dev-server
        tag: gh-67626f77b7da572deb59b935e1a9b175740e7d51-arm64

Afterwards, run the upgrade commands from the latest release notes.