The extraction service

By the end of this stage the first real code exists. A service class wraps the Document Intelligence SDK, a playground script calls it against a sample invoice, and you have seen the raw extraction output that the rest of the system is built to tame.

The playground pattern

Before writing the service, decide where experiments live. The repo has a playground/ folder at the root for exactly this. Scripts there import the backend code, poke at Azure, and print results, but they never ship. As Dave puts it about the first test run: "this code does not belong in your application layer, it's just for testing."

The split keeps the boundary honest. Proven code gets promoted into backend/app/, throwaway exploration stays in playground/. You will use this pattern at every stage of the build.

Because playground scripts live outside backend/, they need one line of path setup before importing from app. The convention is documented in playground/AGENTS.md:

import sys
from pathlib import Path

sys.path.append(str((REPO_ROOT := Path(__file__).resolve().parents[1]) / "backend"))

from app.services.document_intelligence_service import DocumentIntelligenceService  # noqa: E402

append keeps Ruff calm, the walrus assignment gives you REPO_ROOT for locating sample files, and # noqa: E402 marks the late import as intentional.

The service class

The service itself is small. It reads the endpoint and key from settings, builds the SDK client once, and exposes one method per prebuilt model:

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import AnalyzeDocumentRequest, AnalyzeResult
from azure.core.credentials import AzureKeyCredential

from app.config import Settings, get_settings

PREBUILT_INVOICE_MODEL = "prebuilt-invoice"
PREBUILT_RECEIPT_MODEL = "prebuilt-receipt"


class DocumentIntelligenceService:
    def __init__(self, settings: Settings | None = None) -> None:
        resolved_settings = settings or get_settings()
        self._client = DocumentIntelligenceClient(
            endpoint=resolved_settings.azure_document_intelligence_endpoint,
            credential=AzureKeyCredential(resolved_settings.azure_document_intelligence_key),
        )

    def analyze_invoice(self, document_path: Path) -> AnalyzeResult:
        poller = self._client.begin_analyze_document(
            PREBUILT_INVOICE_MODEL,
            AnalyzeDocumentRequest(bytes_source=document_path.read_bytes()),
        )
        return poller.result()

Two details worth registering. begin_analyze_document returns a poller because analysis is a long-running operation on Azure's side; .result() blocks until it finishes. And the two model names are module constants, which matters later when the pipeline routes invoices and receipts to different models.

First run

The playground script sends Microsoft's sample invoice through the service and dumps the result:

def main() -> None:
    service = DocumentIntelligenceService()
    result = service.analyze_invoice(SAMPLE_INVOICE)
    print(json.dumps(service.to_dict(result), indent=2, default=str))

cd playground
uv run --project ../backend --locked --no-sync python analyze_sample_invoice.py

The call takes a few seconds, then a wall of JSON arrives.

What comes back

The result is enormous, and it pays to scroll through it once. Three things stand out:

• Typed fields. Under documents[0].fields you find semantic names like VendorName, InvoiceTotal, and InvoiceDate, each carrying a value, the raw content string from the page, and a confidence score between 0 and 1.
• Pixel precision. Every field and word comes with bounding polygons, exact coordinates on the page. Dave highlights the vendor name in the video, "this can really do it with pixel precision, that is very useful information." No LLM gives you that.
• Page structure. Top-level keys like content and pages hold the full text and layout, useful when you need more than the prebuilt fields.

This raw shape is powerful and unusable at the same time. It is a deeply nested dictionary with provider-specific naming, and nothing downstream should ever have to know about valueCurrency or valueDate. Turning it into something typed is the next stage.

Checkpoint

• playground/analyze_sample_invoice.py runs and prints a full analysis result
• You can point at the vendor, the total, and their confidence scores in the raw JSON
• You can explain why experiments live in playground/ and not in backend/app/