From 25c2967de852c2aa025bf1880ec5ec4129c33fbc Mon Sep 17 00:00:00 2001 From: Corneliu Croitoru Date: Thu, 7 Nov 2024 12:20:01 +0100 Subject: [PATCH] add text 2 structured output sample code --- .gitignore | 1 + README.md | 2 + examples/text-2-structured-output/README.md | 177 ++++++++++++++ .../multi_agent_query_analyzer.py | 164 +++++++++++++ .../product_search_agent.py | 101 ++++++++ examples/text-2-structured-output/prompts.py | 218 ++++++++++++++++++ .../text-2-structured-output/requirements.txt | 2 + 7 files changed, 665 insertions(+) create mode 100644 examples/text-2-structured-output/README.md create mode 100644 examples/text-2-structured-output/multi_agent_query_analyzer.py create mode 100644 examples/text-2-structured-output/product_search_agent.py create mode 100644 examples/text-2-structured-output/prompts.py create mode 100644 examples/text-2-structured-output/requirements.txt diff --git a/.gitignore b/.gitignore index 16ff2c6..29ade7a 100644 --- a/.gitignore +++ b/.gitignore @@ -12,6 +12,7 @@ examples/chat-demo-app/bin/*.js !examples/lambda/url_rewrite/*.js examples/resources/ui/public/aws-exports.json examples/resources/ui/dist +examples/text-2-structured-output/venv .DS_Store diff --git a/README.md b/README.md index 3fa0ff4..cd54c11 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,8 @@ Get hands-on experience with the Multi-Agent Orchestrator through our diverse se - [`ecommerce-support-simulator`](https://github.com/awslabs/multi-agent-orchestrator/tree/main/examples/ecommerce-support-simulator): AI-powered customer support system - [`chat-chainlit-app`](https://github.com/awslabs/multi-agent-orchestrator/tree/main/examples/chat-chainlit-app): Chat application built with Chainlit - [`fast-api-streaming`](https://github.com/awslabs/multi-agent-orchestrator/tree/main/examples/fast-api-streaming): FastAPI implementation with streaming support + - [`text-2-structured-output`](https://github.com/awslabs/multi-agent-orchestrator/tree/main/examples/text-2-structured-output): Natural Language to Structured Data + All examples are available in both Python and TypeScript implementations. Check out our [documentation](https://awslabs.github.io/multi-agent-orchestrator/) for comprehensive guides on setting up and using the Multi-Agent Orchestrator! diff --git a/examples/text-2-structured-output/README.md b/examples/text-2-structured-output/README.md new file mode 100644 index 0000000..29d0690 --- /dev/null +++ b/examples/text-2-structured-output/README.md @@ -0,0 +1,177 @@ +# Natural Language to Structured Data + +A demonstration of how to transform free-text queries into structured, actionable data using a multi-agent architecture. + +## Overview + +This project implements a proof-of-concept system that: +1. Takes natural language input from users +2. Routes queries to specialized agents using an orchestrator +3. Transforms free text into structured formats (JSON for product searches, contextual responses for returns) + +Perfect for teams looking to build systems that need to: +- Convert customer queries into structured database searches +- Transform natural language into API-ready parameters +- Handle multiple types of queries with different output structures +- Maintain context and provide real-time responses + +## Overview + +This project implements a multi-agent orchestration system designed to handle various e-commerce related queries. It features: + +- **Product Search Agent**: Processes natural language queries about products and converts them into structured search parameters +- **Returns & Terms Assistant**: Handles inquiries about return policies, refunds, and terms & conditions +- **Greeting Agent**: Welcomes users and helps them navigate the available services + +The system uses AWS Bedrock with Claude 3 Sonnet as the underlying language model and implements streaming responses for real-time interaction. + +## Prerequisites + +- Python 3.12 +- AWS Account with Bedrock access + +## Setup + +Install dependencies: + ```bash + pip install -r requirements.txt + ``` + +## Usage + +Run the script using: + +```bash +python main.py +``` + +The system will start in interactive mode, allowing you to input queries and receive responses in real-time. + +## Sample Interactions + +### 1. Product Search + +**Input:** +``` +Show me Prime-eligible headphones under $100 with good reviews +``` + +**Output:** +```json +{ + "department": "Electronics", + "categories": ["Headphones"], + "priceRange": { + "max": 100, + "currency": "USD" + }, + "customerReview": { + "stars": 4, + "operator": "gte" + }, + "shippingOptions": { + "prime": true + }, + "condition": "New" +} +``` + +### 2. Returns Policy + +**Input:** +``` +How do I return a damaged item? +``` + +**Output:** +``` +For damaged items, here's our return process: + +1. Document the damage with photos +2. Contact customer service within 48 hours of receipt +3. You'll receive a prepaid return label +4. Package the item securely with all original materials +5. Ship within 14 days of receiving the return label + +The refund will be processed to your original payment method within 3-5 business days after we receive the item. Shipping costs are covered for damaged items. +``` + +### 3. General Greeting + +**Input:** +``` +hello +``` + +**Output:** +```markdown +## Welcome! 👋 + +I'm the greeting agent, here to help you navigate our services. We have several specialized agents available: + +- **Product Search Agent**: Find products, compare prices, and discover deals +- **Returns and Terms Assistant**: Get help with returns, refunds, and policies + +How can we assist you today? Feel free to ask about: +- Product searches and recommendations +- Return policies and procedures +- General assistance and guidance +``` + +## Agents + +The system is built on three main components: + +1. **MultiAgentOrchestrator**: Routes queries to appropriate agents +2. **Agents**: Specialized handlers for different types of queries +3. **Streaming Handler**: Manages real-time response generation + + +### Product Search Agent +The current implementation demonstrates the agent's capability to convert natural language queries into structured JSON output. This is just the first step - in a production environment, you would: + +1. Implement the TODO section in the `process_request` method +2. Add calls to your internal APIs, databases, or search engines +3. Use the structured JSON to query your product catalog +4. Return actual product results instead of just the parsed query + +Example implementation in the TODO section: +```python +# After getting parsed_response: +products = await your_product_service.search( + department=parsed_response['department'], + price_range=parsed_response['priceRange'], + # ... other parameters +) +return ConversationMessage( + role=ParticipantRole.ASSISTANT.value, + content=[{"text": format_product_results(products)}] +) +``` + +### Returns and Terms Assistant +The current implementation uses a static prompt. To make it more powerful and maintenance-friendly: + +1. Integrate with a vector storage solution like [Amazon Bedrock Knowledge Base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) or other vector databases +2. Set up a retrieval system to fetch relevant policy documents +3. Update the agent's prompt with retrieved context + +Example enhancement: +```python +retriever = BedrockKnowledgeBaseRetriever( + kb_id="your-kb-id", + region_name="your-region" +) +# Add to the agent's configuration +``` + +### Greeting Agent +The greeting agent has been implemented as a crucial component for chat-based interfaces. Its primary purposes are: + +1. Providing a friendly entry point to the system +2. Helping users understand available capabilities +3. Guiding users toward the most appropriate agent +4. Reducing user confusion and improving engagement + +This pattern is especially useful in chat interfaces where users might not initially know what kinds of questions they can ask or which agent would best serve their needs. + diff --git a/examples/text-2-structured-output/multi_agent_query_analyzer.py b/examples/text-2-structured-output/multi_agent_query_analyzer.py new file mode 100644 index 0000000..6f9a991 --- /dev/null +++ b/examples/text-2-structured-output/multi_agent_query_analyzer.py @@ -0,0 +1,164 @@ +import uuid +import asyncio +import argparse +from queue import Queue +from threading import Thread + +from multi_agent_orchestrator.orchestrator import MultiAgentOrchestrator, AgentResponse, OrchestratorConfig +from multi_agent_orchestrator.types import ConversationMessage +from multi_agent_orchestrator.classifiers import BedrockClassifier, BedrockClassifierOptions +from multi_agent_orchestrator.storage import DynamoDbChatStorage +from multi_agent_orchestrator.agents import ( + BedrockLLMAgent, + AgentResponse, + AgentCallbacks, + BedrockLLMAgentOptions, +) + +from typing import Dict, List, Any + +from product_search_agent import ProductSearchAgent, ProductSearchAgentOptions +from prompts import RETURNS_PROMPT, GREETING_AGENT_PROMPT + + +class MyCustomHandler(AgentCallbacks): + def __init__(self, queue) -> None: + super().__init__() + self._queue = queue + self._stop_signal = None + + def on_llm_new_token(self, token: str, **kwargs) -> None: + self._queue.put(token) + + def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) -> None: + print("generation started") + + def on_llm_end(self, response: Any, **kwargs: Any) -> None: + print("\n\ngeneration concluded") + self._queue.put(self._stop_signal) + +def setup_orchestrator(streamer_queue): + + classifier = BedrockClassifier(BedrockClassifierOptions( + model_id='anthropic.claude-3-sonnet-20240229-v1:0', + )) + + + orchestrator = MultiAgentOrchestrator(options=OrchestratorConfig( + LOG_AGENT_CHAT=True, + LOG_CLASSIFIER_CHAT=True, + LOG_CLASSIFIER_RAW_OUTPUT=True, + LOG_CLASSIFIER_OUTPUT=True, + LOG_EXECUTION_TIMES=True, + MAX_RETRIES=3, + USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED=False, + NO_SELECTED_AGENT_MESSAGE = """ +I'm not quite sure how to help with that. Could you please: + +- Provide more details, or +- Rephrase your question? + +If you're unsure where to start, try saying **"hello"** to see: + +- A list of available agents +- Their specific roles and capabilities + +This will help you understand the kinds of questions and topics our system can assist you with. +""", + MAX_MESSAGE_PAIRS_PER_AGENT=10 + ), + classifier = classifier + ) + + product_search_agent = ProductSearchAgent(ProductSearchAgentOptions( + name="Product Search Agent", + description="Specializes in e-commerce product searches and listings. Handles queries about finding specific products, product rankings, specifications, price comparisons within an online shopping context. Use this agent for shopping-related queries and product discovery in a retail environment.", + model_id="anthropic.claude-3-sonnet-20240229-v1:0", + save_chat=True, + )) + + my_handler = MyCustomHandler(streamer_queue) + returns_agent = BedrockLLMAgent(BedrockLLMAgentOptions( + name="Returns and Terms Assistant", + streaming=True, + description="Specializes in explaining return policies, refund processes, and terms & conditions. Provides clear guidance on customer rights, warranty claims, and special cases while maintaining up-to-date knowledge of consumer protection regulations and e-commerce best practices.", + model_id="anthropic.claude-3-sonnet-20240229-v1:0", + #TODO SET a retriever to fetch data from a knowledge base + callbacks=my_handler + )) + + returns_agent.set_system_prompt(RETURNS_PROMPT) + + orchestrator.add_agent(product_search_agent) + orchestrator.add_agent(returns_agent) + + agents = orchestrator.get_all_agents() + + greeting_agent = BedrockLLMAgent(BedrockLLMAgentOptions( + name="Greeting agent", + streaming=True, + description="Says hello and lists the available agents", + model_id="anthropic.claude-3-sonnet-20240229-v1:0", + save_chat=False, + callbacks=my_handler + )) + + agent_list = "\n".join([f"{i}-{info['name']}: {info['description']}" for i, (_, info) in enumerate(agents.items(), 1)]) + + greeting_prompt = GREETING_AGENT_PROMPT(agent_list) + greeting_agent.set_system_prompt(greeting_prompt) + + orchestrator.add_agent(greeting_agent) + return orchestrator + +async def start_generation(query, user_id, session_id, streamer_queue): + try: + # Create a new orchestrator for this query + orchestrator = setup_orchestrator(streamer_queue) + + response = await orchestrator.route_request(query, user_id, session_id) + if isinstance(response, AgentResponse) and response.streaming is False: + if isinstance(response.output, str): + streamer_queue.put(response.output) + elif isinstance(response.output, ConversationMessage): + streamer_queue.put(response.output.content[0].get('text')) + except Exception as e: + print(f"Error in start_generation: {e}") + finally: + streamer_queue.put(None) # Signal the end of the response + +async def response_generator(query, user_id, session_id): + streamer_queue = Queue() + + # Start the generation process in a separate thread + Thread(target=lambda: asyncio.run(start_generation(query, user_id, session_id, streamer_queue))).start() + + #print("Waiting for the response...") + while True: + try: + value = await asyncio.get_event_loop().run_in_executor(None, streamer_queue.get) + if value is None: + break + yield value + streamer_queue.task_done() + except Exception as e: + print(f"Error in response_generator: {e}") + break + +async def run_chatbot(): + user_id = str(uuid.uuid4()) + session_id = str(uuid.uuid4()) + + while True: + query = input("\nEnter your query (or 'quit' to exit): ").strip() + if query.lower() == 'quit': + break + try: + async for token in response_generator(query, user_id, session_id): + print(token, end='', flush=True) + print() # New line after response + except Exception as error: + print("Error:", error) + +if __name__ == "__main__": + asyncio.run(run_chatbot()) \ No newline at end of file diff --git a/examples/text-2-structured-output/product_search_agent.py b/examples/text-2-structured-output/product_search_agent.py new file mode 100644 index 0000000..fc0e3c1 --- /dev/null +++ b/examples/text-2-structured-output/product_search_agent.py @@ -0,0 +1,101 @@ +import os +import json +import boto3 +from typing import List, Dict, Any, AsyncIterable, Optional, Union +from dataclasses import dataclass +from enum import Enum +from multi_agent_orchestrator.agents import Agent, AgentOptions +from multi_agent_orchestrator.types import ConversationMessage, ParticipantRole +from multi_agent_orchestrator.utils import conversation_to_dict, Logger +import asyncio +from concurrent.futures import ThreadPoolExecutor + +from prompts import PRODUCT_SEARCH_PROMPT + + +@dataclass +class ProductSearchAgentOptions(AgentOptions): + max_tokens: int = 1000 + temperature: float = 0.0 + top_p: float = 0.9 + client: Optional[Any] = None + +class ProductSearchAgent(Agent): + def __init__(self, options: ProductSearchAgentOptions): + self.name = options.name + self.id = self.generate_key_from_name(options.name) + self.description = options.description + self.save_chat = options.save_chat + self.model_id = options.model_id + self.region = options.region + self.max_tokens = options.max_tokens + self.temperature = options.temperature + self.top_p = options.top_p + + # Use the provided client or create a new one + self.client = options.client if options.client else self._create_client() + + self.system_prompt = PRODUCT_SEARCH_PROMPT + + + @staticmethod + def generate_key_from_name(name: str) -> str: + import re + key = re.sub(r'[^a-zA-Z\s-]', '', name) + key = re.sub(r'\s+', '-', key) + return key.lower() + + def _create_client(self): + #print(f"Creating Bedrock client for region: {self.region}") + return boto3.client('bedrock-runtime', region_name=self.region) + + + + async def process_request( + self, + input_text: str, + user_id: str, + session_id: str, + chat_history: List[ConversationMessage], + additional_params: Optional[Dict[str, str]] = None + ) -> Union[ConversationMessage, AsyncIterable[Any]]: + print(f"Processing request for user: {user_id}, session: {session_id}") + print(f"Input text: {input_text}") + try: + print("Sending request to Bedrock model") + + user_message = ConversationMessage( + role=ParticipantRole.USER.value, + content=[{'text': input_text}] + ) + + conversation = [*chat_history, user_message] + request_body = { + "modelId": self.model_id, + "messages": conversation_to_dict(conversation), + "system": [{"text": self.system_prompt}], + "inferenceConfig": { + "maxTokens": self.max_tokens, + "temperature": self.temperature, + "topP": self.top_p, + "stopSequences": [] + }, + } + + print("Starting Bedrock call...") + + response=self.client.converse(**request_body) + + llm_response = response['output']['message']['content'][0]['text'] + parsed_response = json.loads(llm_response) + + #TODO use the output to call the backend to fetch the data matching the user query + + return ConversationMessage( + role=ParticipantRole.ASSISTANT.value, + content=[{"text": json.dumps({"output": parsed_response, "type": "json"})}] + ) + + except Exception as error: + print(f"Error processing request: {str(error)}") + raise ValueError(f"Error processing request: {str(error)}") \ No newline at end of file diff --git a/examples/text-2-structured-output/prompts.py b/examples/text-2-structured-output/prompts.py new file mode 100644 index 0000000..4bf06d2 --- /dev/null +++ b/examples/text-2-structured-output/prompts.py @@ -0,0 +1,218 @@ + +PRODUCT_SEARCH_PROMPT = """ +# E-commerce Query Processing Assistant + +You are an AI assistant designed to extract structured information from natural language queries about an Amazon-style e-commerce website. Your task is to interpret the user's intent and provide specific field values that can be used to construct a database query. + +## Query Processing Steps + +1. Analyze the user's query for any ambiguous or personalized references (e.g., "my category", "our brand", "their products"). +2. If such references are found, ask for clarification before proceeding with the JSON response. +3. Once all ambiguities are resolved, provide the structured information as a JSON response. + +## Field Specifications + +Given a user's natural language input, provide values for the following fields: + +1. department: Main shopping category (string) +2. categories: Subcategories within the department (array of strings) +3. priceRange: Price range + - min: number + - max: number + - currency: string (e.g., "USD", "EUR", "GBP") +4. customerReview: Average rating filter + - stars: number (1-5) + - operator: string ("gte" for ≥, "eq" for =) +5. brand: Brand names (array of strings) +6. condition: Product condition (string: "New", "Used", "Renewed", "All") +7. features: Special product features (array of strings) + - Examples: ["Climate Pledge Friendly", "Small Business", "Premium Beauty"] +8. dealType: Special offer types (array of strings) + - Examples: ["Today's Deals", "Lightning Deal", "Best Seller", "Prime Early Access"] +9. shippingOptions: Delivery preferences + - prime: boolean + - freeShipping: boolean + - nextDayDelivery: boolean +10. sortBy: Sorting criteria (object) + - field: string (e.g., 'featured', 'price', 'avgCustomerReview', 'newest') + - direction: string ('asc' or 'desc') + +## Rules and Guidelines + +1. Use consistent JSON formatting for all field values. +2. Department names should match Amazon's main categories (e.g., "Electronics", "Clothing", "Home & Kitchen"). +3. When price is mentioned without currency, default to "USD". +4. Interpret common phrases: + - "best rated" → customerReview: {"stars": 4, "operator": "gte"} + - "cheap" or "affordable" → sortBy: {"field": "price", "direction": "asc"} + - "latest" → sortBy: {"field": "newest", "direction": "desc"} + - "Prime eligible" → shippingOptions: {"prime": true} +5. Handle combined demographic and product categories: + - "women's shoes" → department: "Clothing", categories: ["Women's", "Shoes"] +6. Special keywords mapping: + - "bestseller" → dealType: ["Best Seller"] + - "eco-friendly" → features: ["Climate Pledge Friendly"] + - "small business" → features: ["Small Business"] +7. Default values for implicit filters: + - If not specified, assume condition: "New" +8. When "Prime" is mentioned: + - Set shippingOptions.prime = true +9. For sorting: + - "most popular" → sortBy: {"field": "featured", "direction": "desc"} + - "best reviews" → sortBy: {"field": "avgCustomerReview", "direction": "desc"} + - "newest first" → sortBy: {"field": "newest", "direction": "desc"} + +## Clarification Process + +When encountering ambiguous or personalized references: +1. Identify the ambiguous term or phrase. +2. Ask a clear, concise question to get the necessary information. +3. Wait for the user's response before proceeding with the JSON output. + +Example: +User: "Show me Prime-eligible headphones under $100 with good reviews" +Assistant: +{ + "department": "Electronics", + "categories": ["Headphones"], + "priceRange": { + "max": 100, + "currency": "USD" + }, + "customerReview": { + "stars": 4, + "operator": "gte" + }, + "shippingOptions": { + "prime": true + }, + "condition": "New" +} + +## Response Format + +Skip the preamble, provide your response in JSON format, using the structure outlined above. Omit any fields that are not applicable or not mentioned in the user's query. +""" + + +RETURNS_PROMPT = """ +You are the Returns and Terms Assistant, an AI specialized in explaining return policies, terms & conditions, and consumer rights in clear, accessible language. Your goal is to help customers understand their rights and the processes they need to follow. + +Your primary functions include: +1. Explaining return policies and procedures +2. Clarifying terms and conditions +3. Guiding customers through refund processes +4. Addressing warranty questions +5. Explaining consumer rights and protections + +Key points to remember: +- Use clear, simple language avoiding legal jargon +- Provide step-by-step explanations when describing processes +- Consider different scenarios and edge cases +- Be thorough but concise in your explanations +- Maintain a helpful and empathetic tone +- Reference specific timeframes and requirements when applicable + +When responding to queries: +1. Identify the specific policy or process being asked about +2. Provide a clear, direct answer upfront +3. Follow with relevant details and requirements +4. Include important exceptions or limitations +5. Offer helpful tips or best practices when appropriate +6. Suggest related information that might be useful + +Always structure your responses in a user-friendly way: +- Start with the most important information +- Break down complex processes into steps +- Use examples when helpful +- Highlight crucial deadlines or requirements +- Include relevant warnings or cautions +- End with constructive suggestions or next steps + +Example query types you should be prepared to handle: +- "How do I return an item I bought online?" +- "What's your refund policy for damaged items?" +- "Do you accept returns without a receipt?" +- "How long do I have to return something?" +- "What items can't be returned?" +- "Where can I find your terms and conditions?" +- "What are my rights if the product is defective?" +- "How do warranty claims work?" + +Consider these aspects when providing information: +1. Return Windows + - Standard return periods + - Extended holiday periods + - Special item categories + +2. Condition Requirements + - Original packaging + - Tags attached + - Unused condition + - Documentation needed + +3. Refund Process + - Processing timeframes + - Payment methods + - Shipping costs + - Restocking fees + +4. Special Cases + - Damaged items + - Wrong items received + - Sale items + - Customized products + - Digital goods + +5. Consumer Rights + - Statutory rights + - Warranty claims + - Product quality issues + - Service complaints + +Remember to: +- Maintain a professional but friendly tone +- Be precise with information +- Show understanding of customer concerns +- Provide context when necessary +- Suggest alternatives when direct solutions aren't available +- Clarify any ambiguities in the query before providing detailed information + +Your responses should be clear, helpful, and focused on resolving the customer's query while ensuring they understand their rights and responsibilities. If you need any clarification to provide accurate information, don't hesitate to ask for more details.""" + + + +def GREETING_AGENT_PROMPT(agent_list: str) -> str: + return f""" +You are a friendly and helpful Greeting Agent. Your primary roles are to welcome users, respond to greetings, and provide assistance in navigating the available agents. Always maintain a warm and professional tone in your interactions. + +## Core responsibilities: +- Respond warmly to greetings such as "hello", "hi", or similar phrases. +- Provide helpful information when users ask for "help" or guidance. +- Introduce users to the range of specialized agents available to assist them. +- Guide users on how to interact with different agents based on their needs. + +## When greeting or helping users: +1. Start with a warm welcome or acknowledgment of their greeting. +2. Briefly explain your role as the greeting and help agent. +3. Introduce the list of available agents and their specialties. +4. Encourage the user to ask questions or specify their needs for appropriate agent routing. + +## Available Agents: +{agent_list} + +Remember to: +- Be concise yet informative in your responses. +- Tailor your language to be accessible to users of all technical levels. +- Encourage users to be specific about their needs for better assistance. +- Maintain a positive and supportive tone throughout the interaction. +- Always refer to yourself as the "greeting agent or simply "greeting agent", never use a specific name like Claude. + +Always respond in markdown format, using the following guidelines: +- Use ## for main headings and ### for subheadings if needed. +- Use bullet points (-) for lists. +- Use **bold** for emphasis on important points or agent names. +- Use *italic* for subtle emphasis or additional details. + +By following these guidelines, you'll provide a warm, informative, and well-structured greeting that helps users understand and access the various agents available to them . +""" diff --git a/examples/text-2-structured-output/requirements.txt b/examples/text-2-structured-output/requirements.txt new file mode 100644 index 0000000..623fedf --- /dev/null +++ b/examples/text-2-structured-output/requirements.txt @@ -0,0 +1,2 @@ +boto3 +multi-agent-orchestrator \ No newline at end of file