How can I configure log persistence using an OCI PVC ?

[leoliveroso]

Official Helm Chart version
1.15.0 (latest released)

Apache Airflow version
2.9.3

Kubernetes Version
v1.31.1

Helm Chart configuration
All the initial configurations provided by the production guide. Except:

executor: "KubernetesExecutor"

Docker image custom

FROM apache/airflow:2.9.3-python3.11

# Cambiar a usuario root para instalar paquetes del sistema
USER root

# # Crear directorios necesarios con los permisos correctos
# RUN umask 0002; \
#     mkdir -p /opt/airflow/logs/scheduler && \
#     chown -R airflow:root /opt/airflow/logs && \
#     chmod -R 775 /opt/airflow/

# Instalar paquetes del sistema necesarios
COPY packages.txt .

# Copiar DAGs
COPY --chown=airflow:root ./dags/ /opt/airflow/dags/

RUN if [[ -s packages.txt ]]; then \
    apt-get update && \
    cat packages.txt | xargs apt-get install -y --no-install-recommends && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*; \
  fi

# Cambiar a usuario airflow para instalar paquetes de Python
USER airflow

# Copiar y instalar dependencias de Python
COPY requirements.txt .
RUN pip install --no-cache-dir -q -r requirements.txt

CREATE PV

CREATE PVC

Status pod

logs pod

ERROR! Maximum number of retries (20) reached.

Last check result:
$ airflow db check
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/opt/airflow/logs/scheduler/2025-01-13'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/logging/config.py", line 573, in configure
    handler = self.configure_handler(handlers[name])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/config.py", line 757, in configure_handler
    result = factory(**kwargs)
             ^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/log/file_processor_handler.py", line 53, in __init__
    Path(self._get_log_directory()).mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1120, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/opt/airflow/logs/scheduler'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/__init__.py", line 74, in <module>
    settings.initialize()
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/settings.py", line 531, in initialize
    LOGGING_CLASS_PATH = configure_logging()
                         ^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/logging_config.py", line 74, in configure_logging
    raise e
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/logging_config.py", line 69, in configure_logging
    dictConfig(logging_config)
  File "/usr/local/lib/python3.11/logging/config.py", line 823, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/local/lib/python3.11/logging/config.py", line 580, in configure
    raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'processor'

Describe pvc

$kubectl describe pvc airflow-cl-fss-pvc
Name:          airflow-cl-fss-pvc
Namespace:     default
StorageClass:
Status:        Bound
Volume:        airflow-cl-fss-pv
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      100Gi
Access Modes:  RWX
VolumeMode:    Filesystem
Used By:       airflow-cl-scheduler-55549f8795-kzpzd
               airflow-cl-triggerer-0
               airflow-cl-webserver-7f47d45fc9-25mkm
Events:        <none>

"I've already tried granting permissions to the paths from the Dockerfile, and the error persists. Therefore, I have commented it out from the Dockerfile. I have also read similar discussions both in this repository and on Stack Overflow without success. I would greatly appreciate your help in resolving this stopper."