Q/A: ENV questions

[IngwiePhoenix]

Hello!

I am trying to put together a Kubernetes deployment and I am a little stuck with assigning the proper values for the environment.

Here's the full document so far:

Deployment

apiVersion: v1
kind: Namespace
metadata:
  name: solidtime
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: solidtime-env
  namespace: solidtime
data:
  APP_DOMAIN: solidtime.birb.it
  APP_URL: "https://solidtime.domain.tld"
  APP_NAME: "SolidTime"
  VITE_APP_NAME: "SolidTime"
  APP_ENV: "production"
  APP_DEBUG: "false"
  APP_FORCE_HTTPS: "true"
  TRUSTED_PROXIES: "0.0.0.0/0,2000:0:0:0:0:0:0:0/3"

  # Authentication
  APP_KEY: ""
  PASSPORT_PRIVATE_KEY: ""
  PASSPORT_PUBLIC_KEY: ""
  SUPER_ADMINS: ""

  # Logging
  LOG_CHANNEL: "stderr_daily"
  LOG_LEVEL: "info"

  # Database
  DB_CONNECTION: "pgsql"
  #DB_SSLMODE: "require"

  # Mail
  MAIL_MAILER: "smtp"
  MAIL_HOST: ""
  MAIL_PORT: ""
  MAIL_ENCRYPTION: "tls"
  MAIL_FROM_ADDRESS: "[email protected]"
  MAIL_FROM_NAME: "SolidTime"
  MAIL_USERNAME: ""
  MAIL_PASSWORD: ""

  # Queue
  QUEUE_CONNECTION: "database"

  # File storage
  FILESYSTEM_DISK: "local"
  PUBLIC_FILESYSTEM_DISK: "public"
---
apiVersion: postgresql.easymile.com/v1alpha1
kind: PostgresqlDatabase
metadata:
  name: solidtime-db
  namespace: solidtime
spec:
  # Engine configuration link
  engineConfiguration:
    # Resource name
    name: default-cluster-instance
    namespace: postgres
  # Database name
  database: solidtime
  # Master role name
  # Master role name will be used to create top group role.
  # Database owner and users will be in this group role.
  # Default is ""
  masterRole: "solidtime-role"
  # Should drop on delete ?
  # Default set to false
  dropOnDelete: true
  # Wait for linked resource deletion to accept deletion of the current resource
  # See documentation for more information
  # Default set to false
  waitLinkedResourcesDeletion: true
---
apiVersion: postgresql.easymile.com/v1alpha1
kind: PostgresqlUserRole
metadata:
  name: solidtime-db-user
  namespace: solidtime
spec:
  # Mode
  mode: MANAGED
  # Role prefix to be used for user created in database engine
  rolePrefix: "solidtime"
  # User password rotation duration in order to roll user/password in secret
  userPasswordRotationDuration: 720h
  # Privileges list
  privileges:
    - # Privilege for the selected database
      privilege: OWNER
      # Database link
      database:
        name: solidtime-db
      # Generated secret name with information for the selected database
      generatedSecretName: solidtime-db-creds
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: solidtime-app
  namespace: solidtime
  labels:
    app: solidtime
spec:
  replicas: 1
  selector:
    matchLabels:
      app: solidtime
  template:
    metadata:
      labels:
        app: solidtime
    spec:
      volume:
        - name: generic-storage-vol
        - name: logs-storage-vol
        - name: app-storage-vol
      containers:
        - name: http
          image: solidtime/solidtime:latest # FIXME
          ports:
            - name: http
              containerPort: 8000
          envFrom:
            - configMapRef:
                name: solidtime-env
          env:
            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: HOST
            - name: DB_DATABASE
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: DATABASE
            - name: DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: LOGIN
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: PASSWORD
            - name: AUTO_DB_MIGRATE
              value: "true"
            - name: CONTAINER_MODE
              value: http
            - name: OCTANE_SERVER
              value: frankenphp
          volumeMounts:
            - name: generic-storage-vol
              mountPath: /var/www/html/storage"
            - name: logs-storage-vol
              mountPath: /var/www/html/storage/logs"
            - name: app-storage-vol
              mountPath: /var/www/html/storage/app"
        - name: scheduler
          image: solidtime/solidtime:latest # FIXME
          envFrom:
            - configMapRef:
                name: solidtime-env
          env:
            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: HOST
            - name: DB_DATABASE
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: DATABASE
            - name: DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: LOGIN
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: PASSWORD
            - name: CONTAINER_MODE
              value: "scheduler"
          volumeMounts:
            - name: generic-storage-vol
              mountPath: /var/www/html/storage"
            - name: logs-storage-vol
              mountPath: /var/www/html/storage/logs"
            - name: app-storage-vol
              mountPath: /var/www/html/storage/app"
        - name: worker
          image: solidtime/solidtime:latest # FIXME
          envFrom:
            - configMapRef:
                name: solidtime-env
          env:
            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: HOST
            - name: DB_DATABASE
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: DATABASE
            - name: DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: LOGIN
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: PASSWORD
            - name: CONTAINER_MODE
              value: "worker"
          volumeMounts:
            - name: generic-storage-vol
              mountPath: /var/www/html/storage"
            - name: logs-storage-vol
              mountPath: /var/www/html/storage/logs"
            - name: app-storage-vol
              mountPath: /var/www/html/storage/app"
---
# todo: service, traefik ingressRoute

So far, so good. Here's what I am left with:

• Do I need the horizon mode?
• What do I need exactly for the PASSPORT_ variables? Depending on what, I could use an initContainer.
• Sadly I have no way of "statically" defining my Mailgun credentials (I wish I could use external-secrets with the Webhook feature to do that...). Are there other ways/places to store it? If possible, it'd be nice to not have to put them into the manifest.
• What exactly is the difference between APP_DOMAIN and APP_URL as well as APP_NAME and VITE_APP_NAME?
• Are there more logging options? Long term, I want to put some Prometheus exporters and log aggregators ontop - so it'd be helpful in advance to see what I can do with the log settings.

This'll run inside k3s and might make a neat Helm chart down the line. :)

Kind regards,
Ingwie

[korridor]

Hi @IngwiePhoenix, glad to see that you want to try to run solidtime on K8S. Our solidtime cloud also runs on Kubernetes and we do have plans to create a official Helm chart for self-hosting. Currently I'm building a generic Helm chart for Laravel applications called Larakube. The official solidtime Helm chart will most likely be based on that in the future.

You can find the code to this here: https://github.com/solidtime-io/larakube

This is currently not perfect and it's missing documentation, but maybe it helps you.
Regarding your questions:

• Do I need the horizon mode?
  • No
• What do I need exactly for the PASSPORT_ variables? Depending on what, I could use an initContainer.
  • This is the private and public key for that oauth server. This should not change. See docs here: https://docs.solidtime.io/self-hosting/configuration#authentication
• Sadly I have no way of "statically" defining my Mailgun credentials
  • Sorry I don't understand the question. You can store the mailgun credentials in a secret and load it in the deployment like you did for the DB_PASSWORD
• What exactly is the difference between APP_DOMAIN and APP_URL as well as APP_NAME and VITE_APP_NAME?
  • APP_DOMAIN is for the example with the reverse proxy, you can ignore that if you build your own setup. APP_URL is the url that soldtime run under. You can ignore APP_NAME and VITE_APP_NAME.
• Are there more logging options? Long term, I want to put some Prometheus exporters and log aggregators ontop - so it'd be helpful in advance to see what I can do with the log settings.
  • Since solidtime is a Laravel application you can check the logging options in the Laravel docs here: https://laravel.com/docs/11.x/logging#main-content

[korridor]

@IngwiePhoenix I'll close this issue for now, If you have further questions I can reopen the issue.

[IngwiePhoenix]

Hello there!

I finally had some time to sink into this - and I am at ~450 lines of plain YAML...help. x)

Anyway! I got most of the things configured, except:

# kubectl logs -f -n solidtime deployments/solidtime-app -c worker
Container mode: worker

   INFO  Clearing cached bootstrap files.

  cache .......................................................... 2.15ms DONE
  compiled ....................................................... 6.83ms DONE
  config ......................................................... 0.68ms DONE
  events ......................................................... 0.54ms DONE
  routes ......................................................... 0.46ms DONE
  views .......................................................... 1.58ms DONE
  blade-icons .................................................... 1.69ms DONE
  support ........................................................ 1.04ms DONE


   INFO  Events cached successfully.


   INFO  Configuration cached successfully.


   INFO  Routes cached successfully.

Error: Format string '%(ENV_WORKER_COMMAND)s' for 'program:worker.command' contains names ('ENV_WORKER_COMMAND') which cannot be expanded. Available names: ENV_APP_DEBUG, ENV_APP_DOMAIN, ENV_APP_ENV, ENV_APP_FORCE_HTTPS, ENV_APP_KEY, ENV_APP_URL, ENV_COMPOSER_FUND, ENV_COMPOSER_MAX_PARALLEL_HTTP, ENV_CONTAINER_MODE, ENV_DB_CONNECTION, ENV_DB_DATABASE, ENV_DB_HOST, ENV_DB_PASSWORD, ENV_DB_USERNAME, ENV_DEBIAN_FRONTEND, ENV_FILESYSTEM_DISK, ENV_GODEBUG, ENV_GPG_KEYS, ENV_HOME, ENV_HOSTNAME, ENV_KUBERNETES_PORT, ENV_KUBERNETES_PORT_443_TCP, ENV_KUBERNETES_PORT_443_TCP_ADDR, ENV_KUBERNETES_PORT_443_TCP_PORT, ENV_KUBERNETES_PORT_443_TCP_PROTO, ENV_KUBERNETES_SERVICE_HOST, ENV_KUBERNETES_SERVICE_PORT, ENV_KUBERNETES_SERVICE_PORT_HTTPS, ENV_LC_CTYPE, ENV_LOG_CHANNEL, ENV_LOG_LEVEL, ENV_MAIL_ENCRYPTION, ENV_MAIL_FROM_ADDRESS, ENV_MAIL_FROM_NAME, ENV_MAIL_HOST, ENV_MAIL_MAILER, ENV_MAIL_PASSWORD, ENV_MAIL_PORT, ENV_MAIL_USERNAME, ENV_OCTANE_SERVER, ENV_PASSPORT_PRIVATE_KEY, ENV_PASSPORT_PUBLIC_KEY, ENV_PATH, ENV_PHPIZE_DEPS, ENV_PHP_ASC_URL, ENV_PHP_CFLAGS, ENV_PHP_CPPFLAGS, ENV_PHP_INI_DIR, ENV_PHP_LDFLAGS, ENV_PHP_SHA256, ENV_PHP_URL, ENV_PHP_VERSION, ENV_PUBLIC_FILESYSTEM_DISK, ENV_PWD, ENV_QUEUE_CONNECTION, ENV_ROOT, ENV_SOLIDTIME_SVC_PORT, ENV_SOLIDTIME_SVC_PORT_80_TCP, ENV_SOLIDTIME_SVC_PORT_80_TCP_ADDR, ENV_SOLIDTIME_SVC_PORT_80_TCP_PORT, ENV_SOLIDTIME_SVC_PORT_80_TCP_PROTO, ENV_SOLIDTIME_SVC_SERVICE_HOST, ENV_SOLIDTIME_SVC_SERVICE_PORT, ENV_SOLIDTIME_SVC_SERVICE_PORT_HTTP, ENV_SUPER_ADMINS, ENV_TERM, ENV_TRUSTED_PROXIES, ENV_USER, ENV_WITH_HORIZON, ENV_WITH_SCHEDULER, ENV_XDG_CONFIG_HOME, ENV_XDG_DATA_HOME, group_name, here, host_node_name, numprocs, process_num, program_name in section 'program:worker' (file: '/etc/supervisor/conf.d/supervisord.worker.conf')
For help, use /usr/bin/supervisord -h

Here is the snippet that configures this particular container...perhaps I overlooked something?

        - name: worker
          image: solidtime/solidtime:main # FIXME
          envFrom:
            - configMapRef:
                name: solidtime-env
          env:
            - name: APP_KEY
              valueFrom:
                secretKeyRef:
                  name: app-key-secret
                  key: APP_KEY
            - name: PASSPORT_PUBLIC_KEY
              valueFrom:
                secretKeyRef:
                  name: app-key-secret
                  key: PASSPORT_PUBLIC_KEY
            - name: PASSPORT_PRIVATE_KEY
              valueFrom:
                secretKeyRef:
                  name: app-key-secret
                  key: PASSPORT_PRIVATE_KEY
            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: HOST
            - name: DB_DATABASE
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: DATABASE
            - name: DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: LOGIN
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: solidtime-db-creds
                  key: PASSWORD
            - name: CONTAINER_MODE
              value: "worker"
          volumeMounts:
            - name: generic-storage-vol
              mountPath: /var/www/html/storage"
            - name: logs-storage-vol
              mountPath: /var/www/html/storage/logs"
            - name: app-storage-vol
              mountPath: /var/www/html/storage/app"

The referenced environment config:

apiVersion: v1
kind: ConfigMap
metadata:
  name: solidtime-env
  namespace: solidtime
data:
  APP_DOMAIN: solidtime.birb.it
  APP_URL: "https://solidtime.birb.it"
  #APP_NAME: "SolidTime"
  #VITE_APP_NAME: "SolidTime"
  APP_ENV: "production"
  APP_DEBUG: "false"
  APP_FORCE_HTTPS: "true"
  TRUSTED_PROXIES: "0.0.0.0/0,2000:0:0:0:0:0:0:0/3"

  # Authentication
  #APP_KEY: ""
  #PASSPORT_PRIVATE_KEY: ""
  #PASSPORT_PUBLIC_KEY: ""
  SUPER_ADMINS: ""

  # Logging
  LOG_CHANNEL: "stderr_daily"
  LOG_LEVEL: "info"

  # Database
  DB_CONNECTION: "pgsql"
  #DB_SSLMODE: "require"

  # Mail
  #MAIL_MAILER: "smtp"
  #MAIL_HOST: ""
  #MAIL_PORT: ""
  #MAIL_ENCRYPTION: "tls"
  #MAIL_FROM_ADDRESS: ""
  #MAIL_FROM_NAME: "SolidTime"
  #MAIL_USERNAME: ""
  #MAIL_PASSWORD: ""

  # Queue
  QUEUE_CONNECTION: "database"

  # File storage
  FILESYSTEM_DISK: "local"
  PUBLIC_FILESYSTEM_DISK: "public"

Got any idea what it is complaining about here, exactly? Or perhaps I am using the wrong image...?

[IngwiePhoenix]

I nicked the missing value from here: #311 (comment)

Curious why this isn't prefilled? Oh well - the containers are now booted and working ... except, I can't make an account, because I have no E-Mail server configured... oh well, I'll do that some other time.