The LLM service

By the end of this stage both extractors are proven. Next to the Document Intelligence service sits an Azure OpenAI service, and a playground script gets a model response back through your Foundry deployment.

One difference

The surprise of this stage is how little Azure changes. You use the standard OpenAI Python SDK, the same from openai import OpenAI you would use against OpenAI directly. Dave lands on the point after the first successful call: "there's just literally one difference. It's the endpoint."

Point base_url at your Foundry resource and every request routes through Azure, with the security and data-protection story that made Azure the choice in the first place. Same SDK, same request shape, same pricing.

from openai import OpenAI
from openai.types.responses import Response

from app.config import DEPLOYMENT_NAME, Settings, get_settings


class AzureOpenAIService:
    def __init__(self, settings: Settings | None = None) -> None:
        resolved_settings = settings or get_settings()
        self._client = OpenAI(
            base_url=resolved_settings.azure_openai_endpoint,
            api_key=resolved_settings.azure_openai_api_key,
        )
        self._deployment = resolved_settings.azure_openai_deployment or DEPLOYMENT_NAME

    def create_response(self, input: str) -> Response:
        return self._client.responses.create(
            model=self._deployment,
            input=input,
        )

The deployment name is the model name

Against OpenAI directly you pass a model name. Against Azure you pass your deployment name, the one you created in the Foundry stage. This build uses gpt-5.6-terra, declared once in config:

DEPLOYMENT_NAME = "gpt-5.6-terra"

If the call fails with a model-not-found error, the deployment name in .env does not match what exists in your Foundry project. That is the single most common failure at this stage, and it is why the earlier page insisted a fresh Foundry resource has no deployments until you create one.

Test it from the playground

Same pattern as the extraction service, one throwaway script:

def main() -> None:
    prompt = "What is the capital of France?"
    service = AzureOpenAIService()
    response = service.create_response(prompt)
    print(response.output[0].content[0].text)

cd playground
uv run --project ../backend --locked --no-sync python create_openai_response.py
# The capital of France is Paris.

The one non-obvious part is digging the text out of the Responses API result. The answer lives at response.output[0].content[0].text, and on Azure the full response carries extra metadata such as content-filter results. Dump the whole object once with response.model_dump() to see what you are working with.

Where this leaves the build

This is the first milestone. In Dave's words: "we can take a document, send it to an endpoint and get information back, and we also have an LLM." Two independent services, each proven in isolation from the playground, neither knowing the other exists. Everything from here on is business logic that composes them.

Checkpoint

• create_openai_response.py returns an answer through your Azure endpoint
• You can say what the one difference is between calling OpenAI and calling Azure OpenAI
• You know where the deployment name lives in config and in .env