LLMs are built to generate natural language. When you ask a model to “extract the product names and prices,” it returns a conversational paragraph: “Sure! Here are the products I found. The first product is Widget A at $29.99…”

That output is fine for a human reading a chat window. It is useless for any program that needs to parse, store, or process the data. If you are building an AI pipeline, you need structured output—clean JSON, markdown tables, or delimited text that your code can consume without regex gymnastics.

This guide covers how to reliably extract structured data from any LLM.

Why Structured Output Is Hard

LLMs generate text token by token. They have no concept of “valid JSON” as a constraint—they predict what character is most likely to come next. This means:

The model may add conversational padding: “Here is the JSON you requested…”
It may produce syntactically invalid JSON (trailing commas, unescaped quotes, missing brackets).
It may hallucinate extra fields or omit required ones.
It may wrap JSON in markdown code fences (```json ... ```) that your parser does not expect.

Every technique in this guide addresses one or more of these failure modes.

Technique 1: Explicit Schema in the Prompt

The most reliable approach for any model. Define the exact output schema in the prompt and include a “return ONLY” constraint.

Extract every product mentioned in the text below.

For each product, return a JSON object with these exact fields:
- "name": string (the product name)
- "price": number (in USD, no currency symbol)
- "category": string (one of: "electronics", "clothing", "food", "other")
- "in_stock": boolean

Return a JSON array of these objects. If no products are found, return an empty array [].

Rules:
- Return ONLY the JSON array. No explanation, no markdown formatting, no code fences.
- Every object MUST contain all four fields.
- "price" must be a number, not a string. If the price is not mentioned, use 0.
- "in_stock" must be true or false. If not mentioned, use false.

<text>
[Paste source text here]
</text>

Why this works:

The schema is explicit—every field has a type, a default, and a constraint.
The “return ONLY” instruction strips conversational padding.
The enumerated values for category prevent the model from inventing new categories.
Default values for missing data prevent the model from refusing or hallucinating.

Technique 2: One-Shot JSON Example

When the schema is complex or the model is struggling, show it exactly what valid output looks like.

Extract customer feedback from the survey responses below.

Return a JSON array. Each object must match this exact structure:

{
  "customer_id": "C-001",
  "sentiment": "positive",
  "issues": [],
  "verbatim_quote": "I love the new dashboard"
}

Rules:
- "sentiment" is one of: "positive", "negative", "neutral".
- "issues" is an array of strings. Empty array if no issues mentioned.
- "verbatim_quote" is the customer's exact words, max 100 characters. If no clear quote, use empty string.
- Return ONLY the JSON array.

<survey_responses>
[Paste responses here]
</survey_responses>

The one-shot example shows the model the exact shape of the output: field names, value types, empty array syntax, and string formatting. This is more effective than describing the schema in prose because the model can pattern-match directly.

Technique 3: Markdown Tables for Human-Readable Structure

When the output is for a human to read (a report, a comparison, a summary), markdown tables are cleaner than JSON.

Compare the three database options described below. Format the output as a markdown table with these exact columns:

| Database | Best For | Read Speed | Write Speed | Cost | Learning Curve |

Use short phrases (max 5 words per cell). Do not add a summary paragraph after the table.

<database_descriptions>
[Paste descriptions here]
</database_descriptions>

Markdown tables are parseable by most documentation tools, render well in chat interfaces, and are readable by humans without any tooling. They are the right choice when the output destination is a document, a wiki, or a chat response.

Technique 4: Delimited Formats for Spreadsheet Import

When the output needs to land in a spreadsheet or a database, CSV or TSV is the simplest format.

Extract every employee record from the text below.

Return a CSV with these headers:
name,department,title,start_date,salary

Rules:
- Wrap all text fields in double quotes.
- Dates must be in YYYY-MM-DD format.
- Salary must be a number with no commas or currency symbols.
- If a field is missing, use an empty string (two consecutive commas).
- Return ONLY the CSV data. No explanation, no markdown code fences.
- The first line must be the header row.

<employee_text>
[Paste source text here]
</employee_text>

CSV output can be piped directly into a file and opened in any spreadsheet application. The quoting rule prevents commas in text fields from breaking the column structure.

Validating Structured Output

Even with perfect prompting, models occasionally produce malformed output. Always validate before consuming the data.

For JSON:

function parse_llm_json(raw) {
  // Strip markdown code fences if present
  const cleaned = raw.replace(/```json?\n?/g, '').replace(/```$/g, '').trim();
  try {
    const data = JSON.parse(cleaned);
    return { valid: true, data };
  } catch (err) {
    return { valid: false, error: err.message };
  }
}

For tables and CSV:

Count the columns in each row. If any row has a different column count than the header, reject it and re-prompt.

Retry on failure:

When validation fails, send the malformed output back to the model with a correction prompt:

The following JSON is invalid. Fix the syntax errors and return ONLY the corrected JSON.

Invalid JSON:
[paste the broken output]

Error: [paste the parse error message]

This recovery loop catches the majority of formatting failures without manual intervention.

API-Level Structured Output

Several model providers now offer structured output at the API level, which is more reliable than prompt-level instructions:

OpenAI: The response_format parameter with type: "json_object" forces valid JSON. Combined with a JSON schema in the prompt, this produces schema-compliant output.
Anthropic: Tool use (function calling) lets you define a parameter schema, and the model returns arguments matching that schema. This is the most reliable way to get structured output from Claude.
Google Gemini: Supports response_mime_type: "application/json" with a response_schema for strict schema enforcement.

When building production pipelines, prefer API-level structured output over prompt-level instructions. API guarantees are enforced by the model’s inference engine; prompt instructions are suggestions the model usually follows but can ignore.

Choosing the Right Format

Output Destination	Best Format	Why
Another LLM prompt	JSON	Machine-parseable, schema-enforceable
Human reading a report	Markdown table	Readable without tooling
Spreadsheet or database	CSV / TSV	Direct import, no transformation
API response	JSON	Standard web format
Configuration file	YAML or JSON	Depends on the tool consuming it

Match the format to the consumer. Do not generate JSON when the output is a human-readable report, and do not generate markdown tables when the output feeds a database.

Structured Prompts for Structured Output

Getting clean structured output starts with a well-structured prompt. The schema, the constraints, and the “return ONLY” rule all need to be explicit and unambiguous.

PromptCraft on our homepage helps you build these prompts. Paste a rough description of the data you need to extract, and the optimizer returns a prompt with explicit schemas, field constraints, default values, and formatting rules—ready to paste into your pipeline.

Try it on the homepage to turn your data extraction ideas into reliable, parseable prompts.

Structured Output: How to Get JSON, Tables, and Clean Data from LLMs