From DSPy to Promptster: Where Prompt-Optimization Frameworks Fit in Your Stack

There are two distinct activities that both get called "prompt engineering":

1. Optimizing a prompt — given a reference dataset, find the wording that maximizes a target metric.
2. Evaluating a prompt — given a prompt, measure how well it does across models, inputs, and conditions.

DSPy, TextGrad, and MIPRO are the leading frameworks for (1). Promptster is aimed at (2). They solve different problems. Teams that use both end up with better prompts and fewer blind spots than teams that use either alone.

This post walks through the integration pattern.

What DSPy Actually Does

DSPy treats prompt construction as a compilation problem. You write a Python signature (what the module does — inputs and outputs), plus a metric function. DSPy's optimizer then searches over prompt phrasings, few-shot example selection, and chain-of-thought structures to maximize the metric on your training data.

# DSPy sketch
import dspy

class SummarizeInvoice(dspy.Signature):
    """Extract the amount, vendor, and date from an invoice."""
    invoice_text = dspy.InputField()
    amount: float = dspy.OutputField()
    vendor: str = dspy.OutputField()
    date: str = dspy.OutputField()

summarize = dspy.Predict(SummarizeInvoice)

# Compile (= optimize) using a training set and a metric
from dspy.teleprompt import BootstrapFewShot
optimizer = BootstrapFewShot(metric=invoice_accuracy)
compiled = optimizer.compile(summarize, trainset=training_data)

The compiled prompt is the output. You call compiled(invoice_text=...) and it uses the optimized prompt under the hood. No manual prompt engineering.

TextGrad is similar but uses gradient-like feedback signals (hence the name) to iteratively refine prompts. MIPRO does multi-step instruction optimization.

What DSPy Doesn't Do

DSPy optimizes against one model. The optimized prompt is tuned for whatever model you trained against. Swap models and the optimization no longer applies — sometimes the prompt still works; sometimes it's worse than an unoptimized baseline.

DSPy also doesn't:

• Compare quality across providers.
• Detect production drift.
• Maintain a version history of optimization runs.
• Route requests to different models at runtime.

Those are Promptster's scope. The two are compositional.

The Integration Pattern

Here's the end-to-end workflow that uses both tools:

Step 1 — Optimize With DSPy

Run DSPy against your primary model. Get an optimized prompt + few-shot examples.

compiled = optimizer.compile(my_module, trainset=training_data)
optimized_prompt = compiled.predictors()[0].signature.instructions
optimized_examples = compiled.predictors()[0].demos

Serialize the optimized prompt template as a string you can use outside DSPy.

Step 2 — Eval With Promptster Across Providers

Take the optimized prompt from DSPy and run it across 3-5 providers using Promptster's comparison endpoint.

from promptster import compare

results = compare(
    prompt=optimized_prompt_with_examples,
    configurations=[
        {"provider": "openai", "model": "gpt-4o-mini"},
        {"provider": "anthropic", "model": "claude-haiku-4-5"},
        {"provider": "google", "model": "gemini-2.5-flash-lite"},
    ],
)

This tells you which providers the DSPy-optimized prompt ports cleanly to. In our portability analysis, we've seen optimized prompts swing from 95% → 40% accuracy on a provider swap. Promptster's job is to surface that delta before it ships.

Step 3 — Save as a Promptster Test

Save the best-performing (provider × prompt) combination as a Promptster saved test. This becomes your regression baseline.

Step 4 — Schedule Drift Detection

Use Promptster's scheduled tests to re-run the eval weekly. If any provider's score drops, you get an alert. See scheduled drift detection.

Step 5 — Re-Optimize When the Baseline Drifts

When drift alerts fire, re-run DSPy against the new model behavior. The compiled prompt updates; Promptster re-tests; the cycle continues.

Why This Works

DSPy gives you the best prompt for one model. Promptster gives you evidence about that prompt across your real production fleet. Without Promptster, you trust DSPy's claim blindly. Without DSPy, your prompts are hand-tuned and may miss optimizations.

The integration is essentially one function boundary: DSPy emits a prompt template; Promptster tests and monitors it. Both tools operate on the same primitive (a string prompt); they just solve different problems with it.

When You Don't Need Both

A few scenarios where the full integration is overkill:

• Very simple prompts (5-10 words of instruction). Optimization doesn't find much to improve; a direct Promptster comparison is enough.
• Single-provider apps that don't plan to route across providers. DSPy solo is fine.
• One-shot prompts that won't be rerun. Both tools optimize for repeat use.

For mass-volume production workloads, routing tables, or long-lived prompts, both tools pay for themselves.

A Concrete Example

Here's a full script that compiles a prompt with DSPy, then runs the compiled version through Promptster for cross-provider evaluation:

import dspy
from promptster import compare

# 1. DSPy optimization
dspy.settings.configure(lm=dspy.LM("openai/gpt-4o-mini"))

class ExtractInvoice(dspy.Signature):
    """Extract amount, vendor, and due date from invoice text."""
    text = dspy.InputField()
    amount: float = dspy.OutputField()
    vendor: str = dspy.OutputField()
    due_date: str = dspy.OutputField()

module = dspy.Predict(ExtractInvoice)
optimizer = dspy.BootstrapFewShot(metric=my_accuracy_metric)
compiled = optimizer.compile(module, trainset=my_training_data)

# 2. Extract the optimized prompt
prompt_template = format_compiled_for_promptster(compiled)

# 3. Cross-provider eval via Promptster
eval_results = compare(
    prompt=prompt_template.format(text=sample_input),
    configurations=[
        {"provider": "openai", "model": "gpt-4o-mini"},
        {"provider": "anthropic", "model": "claude-haiku-4-5"},
        {"provider": "google", "model": "gemini-2.5-flash-lite"},
    ],
)

# 4. Pick the best provider for the optimized prompt
best = max(eval_results, key=lambda r: score(r, expected_output))
print(f"Optimized prompt performs best on: {best.provider}/{best.model}")

The Trend

DSPy-style frameworks and Promptster-style testing platforms are both maturing toward the same goal: taking the manual guesswork out of prompt engineering. Different layers of the stack; same endpoint. 2026's best prompt pipelines will use both, and the compositional pattern above will become the default.

For the cross-provider testing side, see our 11-provider consensus study and automating prompt testing for production.

DSPy docs: https://dspy.ai. Integration code is illustrative; adapt to your specific DSPy module shape and metric function.