Rhapsode - a virtual dungeon master

This is a virtual dungeon master I created hastily for a hackathon project at Daily. It's quite messy, but achieved my goal of creating an interactive dungeon master for solo questing. It handles character creation, portrait and setting generation, as well as inventory management.

The UI was mostly an after thought to facilitate the gameplay - I wanted to test function calling, particularly in allowing the "dungeon master" craft items, rolls, spells, the setting, etc.

Run it locally

Grab a Daily Bots API key

Sign-up here: https://bots.daily.co

Configure your local environment

cp env.example .env.local

DAILY_BOTS_URL URL of the Daily Bots start endpoint (https://api.daily.co/v1/bots/start)

DAILY_BOTS_API_KEY your Daily API key obtained by registering at https://bots.daily.co.

OPEN_API_KEY your open ai key, which is used to generate character portraits

Install dependencies

yarn 

Run the project

yarn run dev

How does this work?

Daily Bots is built on two open-source technologies:

• Pipecat - Python library for building real-time agent
• RTVI - Open-standard for Real-Time Voice [and Video] Inference

Learn more about the RTVI web client libraries on the docs.

The application itself guides a player through character creation, and loads that into a beefy prompt. This prompt has a set of functions that control the state of the game. All of this logic is located in App.tsx. The game is "saved" by encoding a big json blob into a base64 text file, which is really just a transcript of the entire play session last time.

This works pretty well, but is very expensive in terms of tokens. We could port this to use Llama 8b from claude, but we'd have to change the function calling pieces.

Configuration

The majority of the RTVI configuration can be found in App.tsx. Here, we set up our prompt, which is built upon the bits of information we gathered during character creation.

API routes

This project exposes two server-side routes:

• api/route.ts
• api/generate/route.ts

The routes project a secure way to pass any required secrets or configuration directly to the Daily Bots API. Your NEXT_PUBLIC_BASE_URL must point to your /api route and passed to the VoiceClient.

The routes are passed a config array and services map, which can be passed to the Daily Bots REST API, or modified securely.

Daily Bots https://api.daily.co/v1/bots/start has some required properties, which you can read more about here. You must set:

• bot_profile
• max_duration
• config
• services
• Optional, if using OpenAI: api_keys