Skip to content

Configuration

Jump to bottom
Akase Cho edited this page Jun 1, 2023 · 1 revision

Server Configuration

  • bind: This is the IP address and port that the OpenAI Hub server will bind to. By default, it's set to "0.0.0.0:8080", which means the server will listen on all available network interfaces on port 8080.

OpenAI API Configuration

  • api_keys: This is a list of API keys for OpenAI. You can add as many keys as you need for load balancing purposes.

  • organization: This is the organization ID for OpenAI. This is optional and should be uncommented and filled in if applicable.

  • api_base: This is the base URL for the OpenAI API. By default, it's set to OpenAI's official API. However, you can uncomment and replace it with your own endpoint if needed.

  • api_type: This specifies the type of the API. By default, it's set to OpenAI's official API. However, you can uncomment and replace it as needed for other API types, such as Azure or Azure AD.

  • api_version: This is the version of the API. It's optional and should be uncommented and filled in if you're using Azure.

JWT Authentication

  • jwt-auth: This section is for enabling JWT authentication. If you uncomment this section, you need to provide a secret for JWT token generation and verification.

Access Log Configuration

The [audit] section specifies where and how access logs should be stored. You can choose from several backends, including file, SQLite, MySQL, or PostgreSQL.

  • backend: Specifies the backend for storing access logs.

  • filters.access: Specifies what information about the requests and responses should be logged.

  • filters.tokens: Specifies whether token consumption should be logged and on which endpoints.

Each backend has its own configuration section:

  • audit.backends.file: For the file backend, specify the file path for the access log.

  • audit.backends.sqlite: For the SQLite backend, specify the SQLite database file path.

  • audit.backends.mysql: For the MySQL backend, specify the host, port, and optional socket for a MySQL database, as well as the username, password, and database name for access log storage.

  • audit.backends.postgres: For the PostgreSQL backend, specify the host, port, and socket for a PostgreSQL database, as well as the username, password, and database name for access log storage.

Note: Remember to uncomment the relevant sections and fill in the necessary details based on your preferred configuration.

Global Acl Configuration

Global Configuration

  • whitelist: This is a boolean setting that determines the default behavior for requests. If set to true, all unlisted requests are rejected by default unless explicitly allowed. If set to false, all requests are allowed by default unless explicitly rejected.

  • allow_deployments: This is a list that allows requests with specified Azure deployment IDs.

Method Configuration

  • global.methods: This section allows you to set default behaviors for specific HTTP methods. For example, you can choose to allow all POST requests and disallow all GET and DELETE requests by default.

Endpoint Configuration

  • endpoint.<METHOD>: This section allows you to set behaviors for specific endpoints with a certain HTTP method. You can choose to allow or disallow requests to specific endpoints.

Model Configuration

  • model.<METHOD>.<ENDPOINT>: This section allows you to set behaviors for specific models in specific endpoints with a certain HTTP method. You can choose to allow or disallow requests for specific models.

  • path: This is a boolean setting. If set to true, it means that the model name in the request path will be filtered.

  • allows: This is a list of models that are allowed to make requests. You can use "*" to indicate that any model is allowed.

  • disallows: This is a list of models that are disallowed to make requests.

  • allow_omitted: This is a boolean setting. If set to true, it means that if the model name is omitted in the request, the request is allowed.

In general, the ACL configuration file gives you fine-grained control over which types of requests are allowed or disallowed. You can set defaults for all requests, or specify rules for particular HTTP methods, endpoints, and models. This allows you to ensure that the right people and services have access to the right resources.

Clone this wiki locally