Structured Ouput with Claudette

The claudette library provides a wrapper around Anthropic’s SDK, with features and convenience that make using the API more “pythonic”. claudette-pydantic is an extension to the library, providing structured output with Pydantic.

Below is an example use case, a sale assistant in a pet store and customer.

The interaction will use structured outputs to model the LLMs environment:

Customer pet preference and budget is kept from salesperson, and Pet inventory and price is kept from the customer. Between them they must negotiate an interaction and sale of a pet (or failing, the customer leaves).

I’ll use haiku, which is rather good at structured output tasks.

from claudette.core import *
import claudette_pydantic
from pydantic import BaseModel, Field, create_model
from typing import List, Union, Literal, Type
from textwrap import dedent
import json

First let’s define a model for some pets, primarily with a name and price:

class Pet(BaseModel):
    name: str
    price: float = Field(description="Full price of the Pet")

class Dog(Pet):
    type: Literal["dog"] = "dog"
    breed: str

class Fish(Pet):
    type: Literal["fish"] = "fish"
    species: str
    tank_size: float

dogs = [Dog(name="Buddy", breed="Labrador", price=500), Dog(name="Max", breed="German Shepherd", price=600)]
fish = [Fish(name="Nemo", species="Clownfish", price=110, tank_size=20), Fish(name="Bubbles", species="Betta", price=300, tank_size=5)]

And a rather elaborate salesperson, where the Salesperson has information about the pet shop inventory. They can do actions:

I then set the docstring of the returned function through __doc__. This gives private information to the salesperson about the inventory and prices, without it being revealed in the message history.

class SalesPersonMessage(BaseModel):
    "The salesperson should first talk to the customer to find their budget and negotiate the best deal for a pet"
    type: Literal["message"] = "message"
    content: str

class SalespersonOffer(BaseModel):
    "Send a SellOffer once a pet at a proposed price has informally been agreed upon."
    type: Literal["offer"] = "offer"
    pet: Union[Dog, Fish] = Field(..., discriminator='type')
    price: float

class SalespersonAction(BaseModel):
    action: Union[SalesPersonMessage, SalespersonOffer] = Field(..., discriminator='type')

class SalespersonInfo(BaseModel):
    name: str
    inventory: List[Union[Dog, Fish]] = Field(..., description="List of pets available for sale")


    def create_custom_action(self) -> Type[SalespersonAction]:
        CustomSalespersonAction = create_model(
            "CustomSalespersonAction",
            __base__=SalespersonAction,
        )

        inventory_pretty = '- ' + '\n- '.join(repr(o) for o in self.inventory)

        CustomSalespersonAction.__doc__ = dedent(f"""\
        Represents a salesperson in the pet store simulation.

        The salesperson has access to a list of pets (dogs and fish) available in the store,
        but does not know the customer's budget or preferences. The salesperson's goal is
        to make a sale to maximizes profit.

        ## Discounts

        May offer a 10% discount on all pets
        25% on FISH ONLY

        ## Sales

        You should sell the pet included ONLY. Do not sell accessories, only the items currently in inventory.

        ## Details

        Perform an action based on conversation history and salesperson details.

        Salesperson details:
        Name: {self.name}

        Inventory:
        {inventory_pretty}""")

        return CustomSalespersonAction

salesperson = SalespersonInfo(name="Bob", inventory=dogs + fish)

Customer is similar, with a budget and a pet preference. Actions:

Notably the customers budget is hidden from the salesperson, and the inventory is hidden from the customer, so they must communicate to come to an agreement.


class CustomerOfferResponse(BaseModel):
    """Determine whether to accept the sale.
    Accept only if the offer is consistent to a previously agreed sale.
    this may **only** be given as an immediate response to a SellOffer"""
    type: Literal["offer"] = "offer"
    accept: bool

class CustomerOfferAction(BaseModel):
    action: CustomerOfferResponse

class CustomerMessage(BaseModel):
    type: Literal["message"] = "message"
    content: str

class CustomerLeave(BaseModel):
    "If all other avenues are exhausted, nothing in the store is acceptable, you may leave the store"
    type: Literal["leave"] = "leave"

class CustomerAction(BaseModel):
    action: Union[CustomerMessage, CustomerLeave] = Field(..., discriminator='type')


class CustomerInfo(BaseModel):
    name: str
    pet_preference: str = Field(..., description="The type of pet the customer prefers ('dog' or 'fish')")
    budget: float = Field(..., description="The maximum amount the customer is willing to spend")

    def create_custom_action(self, offer=False) -> Type[CustomerAction]:
        CustomCustomerAction = create_model(
            "CustomCustomerAction",
            __base__=CustomerOfferAction if offer else CustomerAction,
        )

        CustomCustomerAction.__doc__ = dedent(f"""\
        Represents a customer in the pet store simulation.

        The customer has a pet preference and a budget, but does not know what pets
        the store has in stock or their prices. The customer's goal is to negotiate
        the best deal for their preferred type of pet within their budget.

        Perform an action based on conversation history and customer details.

        Customer Information:
        - Name: {self.name}
        - Pet Preference: {self.pet_preference}
        - Budget: ${self.budget}
        """)

        return CustomCustomerAction


pet_preference = f"""\
Alice really wants a dog, but has a hard limit of 100 dollars.
She may also be open to buying a fish"""

customer = CustomerInfo(name="Alice", pet_preference=pet_preference, budget=100)

Then I set up a similar loop to Chat.toolloop but exits on a successful offer or leaving.

sp = """\
You are participating in a role-playing simulation of a pet store negotiation.
If you are the customer, your goal is to purchase your preferred pet within your budget.
If you are the salesperson, your goal is to make a profitable sale while ensuring customer satisfaction.
Be friendly and professional, but also try to get the best deal for yourself.

The negotiation continues until a sale is agreed upon or one party decides to end the conversation.
You should be polite, but do not discuss the specifics of the pets, work out a sale based on price and pet preferences. Each response should be 3 sentences max.
"""


def chat_loop(model, steps): # similar to Chat.toolloop
    chat = Chat(model, sp=sp)
    responses = []
    res = None
    for i in range(steps):
        if i == 0: # prompt required to switch to assistant
            pr = "A customer walks into a store"
        else:
            pr = None

        if i % 2 == 0:
            role = "salesperson"
            s = salesperson.create_custom_action()
        else:
            role = "customer"
            if res is not None and res.action.type == 'offer':
                offer = True
            else:
                offer = False
            s = customer.create_custom_action(offer)

        res = chat.struct(s, treat_as_output=False, pr=pr)
        responses.append(res)
        if res.action.type == 'message':
            print(f"{role}: {res.action.content}\n")
        if i % 2 == 1:
            if res.action.type == 'offer' and res.action.accept:
                return responses[-2].action
            elif res.action.type == 'leave':
                return responses[-1].action
    return None

def sim(model, steps):
    print(f"Model: {model} | Steps: {steps}\n=====\n")
    res = chat_loop(model, steps)
    if res is None:
        print("Timeout")
    elif res.type == 'offer':
        print("Sold:", repr(res))
    elif res.type == 'leave':
        print("Customer left.")
    else:
        raise ValueError("invalid return", res)

Set up so that Alice can not afford a dog, and can only afford a fish with discount. Haiku figures it out.

There’s a shared history between them, but individuals history is kept hidden inside the struct description for each step

sim(models[-1], 20)
Model: claude-3-haiku-20240307 | Steps: 20
=====

salesperson: Hello, welcome to our pet store! How can I assist you today?

customer: Hi, I'm looking to buy a pet. I really want a dog, but my budget is only $100. Do you have any dogs in that price range?

salesperson: Unfortunately, we don't have any dogs in that price range. Our dogs start at $500. However, we do have some nice fish that would fit your budget. Let me show you what we have in stock.

customer: Hmm, I was really hoping for a dog. But I guess I could consider a fish if the price is right. What kind of fish do you have and how much do they cost?

salesperson: Great, let me show you our fish options. We have a Clownfish named Nemo for $110 and a Betta fish named Bubbles for $300. The Clownfish would be a better fit for your $100 budget.

customer: Okay, the Clownfish Nemo sounds good. I think I'll go with that one. Can we negotiate the price a bit to get it closer to my $100 budget?

Sold: SalespersonOffer(type='offer', pet=Fish(name='Nemo', price=110.0, type='fish', species='Clownfish', tank_size=20.0), price=99.0)