Skip to content

Commit

Permalink
Add authentication instructions to tutorial
Browse files Browse the repository at this point in the history
  • Loading branch information
dyastremsky committed Nov 7, 2024
1 parent a793b8b commit a9493cb
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 0 deletions.
22 changes: 22 additions & 0 deletions genai-perf/docs/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,17 @@ Example output:
└───────────────────────────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┘
```

### Running with Authentication
GenAI-Perf can benchmark secure endpoints such as OpenAI, which requires API
key authentication. To do so, you can add your API key directly in the command.
At the end of your command, append the below flags. Replace the key with your
API key. The `--` flag allows arguments to pass directly into Perf Analyzer in
superuser mode. The `-H` flag is used to add HTTP headers.

```bash
-- -H "Authorization: Bearer ${OPENAI_API_KEY}" -H "Accept: text/event-stream"
```

## Profile GPT-2 running on OpenAI Completions API-Compatible Server <a id="openai-completions"></a>

Serve the model on the vLLM server with [OpenAI Completions API](https://platform.openai.com/docs/api-reference/completions) endpoint:
Expand Down Expand Up @@ -200,3 +211,14 @@ Example output:
│ Request throughput (per sec) │ 2.28 │ N/A │ N/A │ N/A │ N/A │ N/A │
└───────────────────────────────────┴────────┴────────┴────────┴────────┴────────┴────────┘
```

### Running with Authentication
GenAI-Perf can benchmark secure endpoints such as OpenAI, which requires API
key authentication. To do so, you can add your API key directly in the command.
At the end of your command, append the below flags. Replace the key with your
API key. The `--` flag allows arguments to pass directly into Perf Analyzer in
superuser mode. The `-H` flag is used to add HTTP headers.

```bash
-- -H "Authorization: Bearer ${OPENAI_API_KEY}" -H "Accept: text/event-stream"
```
21 changes: 21 additions & 0 deletions templates/genai-perf-templates/tutorial_template
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,17 @@ Example output:
└───────────────────────────────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┘
```

### Running with Authentication
GenAI-Perf can benchmark secure endpoints such as OpenAI, which requires API
key authentication. To do so, you can add your API key directly in the command.
At the end of your command, append the below flags. Replace the key with your
API key. The `--` flag allows arguments to pass directly into Perf Analyzer in
superuser mode. The `-H` flag is used to add HTTP headers.

```bash
-- -H "Authorization: Bearer ${OPENAI_API_KEY}" -H "Accept: text/event-stream"
```

## Profile GPT-2 running on OpenAI Completions API-Compatible Server <a id="openai-completions"></a>

Serve the model on the vLLM server with [OpenAI Completions API](https://platform.openai.com/docs/api-reference/completions) endpoint:
Expand Down Expand Up @@ -201,3 +212,13 @@ Example output:
└───────────────────────────────────┴────────┴────────┴────────┴────────┴────────┴────────┘
```

### Running with Authentication
GenAI-Perf can benchmark secure endpoints such as OpenAI, which requires API
key authentication. To do so, you can add your API key directly in the command.
At the end of your command, append the below flags. Replace the key with your
API key. The `--` flag allows arguments to pass directly into Perf Analyzer in
superuser mode. The `-H` flag is used to add HTTP headers.

```bash
-- -H "Authorization: Bearer ${OPENAI_API_KEY}" -H "Accept: text/event-stream"
```

0 comments on commit a9493cb

Please sign in to comment.