Showing posts with label react. Show all posts
Showing posts with label react. Show all posts

Friday, April 17, 2026

React Performance in 2026: Server Components, Caching, and Core Web Vitals

Hero image

Introduction

React 19 and Next.js 15 changed the performance model for React applications. Server Components render on the server with zero client JavaScript — not as a pre-render that ships a hydration bundle, but as true server-only components that never touch the client. The bundle size implications are significant: a markdown parser, a syntax highlighter, a data visualization library — all can run on the server and ship only HTML to the client.

But Server Components are one piece of a larger performance picture. This post covers the complete React performance toolkit for 2026: the Server Components mental model and when to use them, React 19's concurrent rendering and the use() hook, caching at multiple layers (component, request, data), bundle optimization with code splitting, and the Core Web Vitals metrics that determine how Google evaluates your app's user experience.

React Server Components: The Mental Model

Server Components execute on the server and return JSX that is serialized and streamed to the client. They can be async, access databases directly, read files, and use server-only secrets — because they never run in the browser.

// app/orders/page.tsx — Server Component (async by default in Next.js 15)
import { db } from '@/lib/db';
import { OrderList } from './order-list';  // Client Component

// This runs on the server — never shipped to the client
export default async function OrdersPage({ searchParams }: { 
  searchParams: { status?: string; cursor?: string } 
}) {
  // Direct database access — no API route needed
  const orders = await db.query(`
    SELECT id, total_cents, status, created_at
    FROM orders
    WHERE user_id = $1
    ${searchParams.status ? 'AND status = $2' : ''}
    ORDER BY created_at DESC
    LIMIT 20
  `, [getCurrentUserId(), searchParams.status].filter(Boolean));

  // The markdown parser runs on the server, ships only HTML
  // import 'marked' → 23KB stays on server, not in client bundle
  const formattedOrders = orders.map(order => ({
    ...order,
    total: formatCurrency(order.total_cents),
  }));

  return (
    <main>
      <h1>Your Orders</h1>
      {/* OrderList is a Client Component — receives serialized props */}
      <OrderList orders={formattedOrders} />
    </main>
  );
}
// app/orders/order-list.tsx — Client Component
'use client';  // directive marks this as a Client Component

import { useState } from 'react';

// This runs in the browser — has access to useState, event handlers, browser APIs
export function OrderList({ orders }: { orders: Order[] }) {
  const [expanded, setExpanded] = useState<string | null>(null);

  return (
    <ul>
      {orders.map(order => (
        <li key={order.id} onClick={() => setExpanded(order.id)}>
          {order.total} — {order.status}
          {expanded === order.id && <OrderDetail id={order.id} />}
        </li>
      ))}
    </ul>
  );
}

The key constraint: Server Components cannot use useState, useEffect, browser APIs, or event handlers. They can import Server-only modules (database clients, file system). Client Components can use all React hooks but cannot import server-only modules.

The correct mental model: push as much as possible up the tree into Server Components. Only the interactive pieces need to be Client Components. A product page with a static description, images, and price can be a Server Component; the "Add to Cart" button is the Client Component.

Architecture diagram

Next.js 15 Caching: Four Layers

Next.js 15 caches at four distinct layers. Understanding which cache applies when determines whether your users see stale data or make unnecessary server round-trips.

// Layer 1: Request memoization — deduplicates identical fetch() calls within one render
// Next.js automatically deduplicates fetch() calls with the same URL + options in one request
async function getUser(id: string) {
  const res = await fetch(`/api/users/${id}`, {
    // No cache option = request memoization only (within one render tree)
  });
  return res.json();
}
// Called in Header component AND in Profile component → one network request

// Layer 2: Data cache — persists between server requests
async function getProductPrice(productId: string) {
  const res = await fetch(`/api/prices/${productId}`, {
    next: { revalidate: 60 }  // cache for 60 seconds, then re-fetch
  });
  return res.json();
}

// Layer 3: Full route cache — HTML + RSC payload cached at the CDN
// export const revalidate = 3600;  // revalidate this page every hour (static)
// export const dynamic = 'force-dynamic';  // never cache (dynamic)

// Layer 4: Router cache — client-side, prefetched on <Link> hover
// Configured via prefetch prop on <Link>

// On-demand cache invalidation — clear specific cached data
import { revalidatePath, revalidateTag } from 'next/cache';

export async function updateProduct(id: string, data: ProductUpdate) {
  await db.products.update(id, data);
  revalidatePath(`/products/${id}`);       // clear page cache
  revalidateTag(`product-${id}`);         // clear all fetches tagged with this
}
// Tagging fetch requests for selective invalidation
async function getProduct(id: string) {
  const res = await fetch(`/api/products/${id}`, {
    next: { 
      revalidate: 3600,
      tags: [`product-${id}`, 'products']  // tag for selective invalidation
    }
  });
  return res.json();
}

// Invalidate all product caches on update
revalidateTag('products');  // clears all fetches tagged 'products'

The most important cache to understand is the data cache. It persists fetch() responses in the Next.js server's data store between requests. If 1,000 users request the same product page, the product data is fetched from the database once and served to all 1,000 users from cache until revalidate expires. Without this, each page request hits the database.

Streaming and Suspense: Progressive Loading

React 19's streaming renders parts of the page as they become ready, rather than waiting for all data before sending HTML. Combined with <Suspense>, this enables progressive loading — the shell arrives immediately, slower data streams in.

// app/dashboard/page.tsx
import { Suspense } from 'react';
import { OrderSummary } from './order-summary';
import { RecentActivity } from './recent-activity';
import { Skeleton } from '@/components/ui/skeleton';

export default function DashboardPage() {
  return (
    <div className="dashboard">
      {/* Fast: user info is cheap to fetch */}
      <UserHeader />

      {/* Slow: order summary requires aggregation query */}
      {/* Suspense boundary: render Skeleton while OrderSummary fetches */}
      <Suspense fallback={<Skeleton className="h-40 w-full" />}>
        <OrderSummary />   {/* async Server Component */}
      </Suspense>

      {/* Slower: activity feed requires multiple joins */}
      <Suspense fallback={<ActivitySkeleton />}>
        <RecentActivity />  {/* async Server Component */}
      </Suspense>
    </div>
  );
}

Without Suspense, the page waits for the slowest data fetch before sending any HTML. The Time to First Byte (TTFB) is bounded by the slowest query. With Suspense boundaries, the shell streams immediately (fast TTFB), and slower sections stream in as their data arrives. The user sees content progressively rather than a blank page.

React 19: use() Hook and Concurrent Features

React 19's use() hook reads a resource (Promise, Context) within a component, triggering Suspense:

'use client';
import { use, Suspense } from 'react';

function UserProfile({ userPromise }: { userPromise: Promise<User> }) {
  // use() reads the promise — suspends the component until resolved
  const user = use(userPromise);
  return <div>{user.name}</div>;
}

// Parent passes a promise (not an awaited value)
function ProfilePage() {
  const userPromise = fetchUser('123');  // starts fetch immediately

  return (
    <Suspense fallback={<Skeleton />}>
      <UserProfile userPromise={userPromise} />
    </Suspense>
  );
}

React 19 transitions with useTransition mark state updates as non-urgent, keeping the UI responsive during heavy re-renders:

'use client';
import { useTransition, useState } from 'react';

function SearchBar() {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState([]);
  const [isPending, startTransition] = useTransition();

  function handleChange(e: React.ChangeEvent<HTMLInputElement>) {
    setQuery(e.target.value);  // urgent: update input immediately

    // Non-urgent: search results update can be interrupted by more typing
    startTransition(async () => {
      const data = await search(e.target.value);
      setResults(data);
    });
  }

  return (
    <>
      <input value={query} onChange={handleChange} />
      {isPending && <Spinner />}  {/* shows while transition is pending */}
      <ResultsList results={results} />
    </>
  );
}

The input stays responsive because setQuery is outside startTransition. The search results update is interruptible — if the user types faster, the pending search is abandoned for the new query.

Bundle Optimization: Code Splitting and Tree Shaking

Every kilobyte in the JavaScript bundle delays interactivity. The strategies:

// Dynamic imports: split rarely-used components out of the main bundle
import dynamic from 'next/dynamic';

// Rich text editor: 180KB — only load when user opens editor
const RichTextEditor = dynamic(
  () => import('@/components/rich-text-editor'),
  { 
    loading: () => <Textarea />,  // show placeholder while loading
    ssr: false                    // don't render on server (uses browser APIs)
  }
);

// Route-level code splitting: automatic in Next.js
// Each page/layout is its own bundle — users only load the code for the route they visit

// Analyzing bundle size:
// npx @next/bundle-analyzer
// Shows each module and its contribution to the bundle
// Tree shaking: import only what you use
// Bad: imports entire lodash (70KB)
import _ from 'lodash';
const unique = _.uniq(arr);

// Good: imports only the uniq function (3KB)
import uniq from 'lodash/uniq';

// Better: use native equivalents when available
const unique = [...new Set(arr)];  // 0KB — no import needed

// Barrel exports can defeat tree shaking:
// import { Button } from '@/components'  — may import all components
// import { Button } from '@/components/button'  — import only Button

Use @next/bundle-analyzer to identify large dependencies. Common culprits: moment.js (use date-fns), lodash (use native equivalents), full icon libraries (import specific icons).

Comparison visual

Core Web Vitals: The Metrics That Matter

Google's Core Web Vitals determine search ranking and user experience quality. Three metrics:

Largest Contentful Paint (LCP): time until the largest element (image or text block) in the viewport is rendered. Target: <2.5 seconds.

// LCP optimization: preload the hero image
// In Next.js: priority prop on the above-the-fold image
import Image from 'next/image';

<Image
  src="/hero.jpg"
  alt="Hero"
  width={1200}
  height={600}
  priority   // adds <link rel="preload"> — loads before other resources
  sizes="100vw"
/>

// Avoid: rendering LCP image from JavaScript (delays LCP)
// The browser can't preload images not in the initial HTML

Cumulative Layout Shift (CLS): total amount of unexpected layout shift. Target: <0.1.

// CLS: always specify width and height on images
// Without dimensions: image loads → page jumps
<Image
  src="/product.jpg"
  width={400}    // always specify
  height={300}   // always specify
  alt="Product"
/>

// CLS: reserve space for dynamic content
// Without min-height: content loads → page jumps
<div style={{ minHeight: '200px' }}>
  {isLoaded ? <DynamicContent /> : <Skeleton />}
</div>

// CLS: font loading — swap causes flash of unstyled text
// font-display: optional prevents layout shift at cost of first-load font miss

Interaction to Next Paint (INP): responsiveness to user interactions — replaced FID in 2024. Target: <200ms. Long tasks (>50ms on main thread) cause high INP.

// INP: move heavy computation off the main thread
// Web Workers run in a separate thread — don't block UI

// app/workers/search.worker.ts
self.onmessage = function(e) {
  const { query, items } = e.data;
  // Heavy fuzzy search — won't block the main thread
  const results = fuzzysearch(query, items);
  self.postMessage(results);
};

// Component
const worker = new Worker(new URL('./workers/search.worker.ts', import.meta.url));

function handleSearch(query: string) {
  worker.postMessage({ query, items: largeItemList });
  worker.onmessage = (e) => setResults(e.data);
}

// React 19: useOptimistic for instant UI feedback
const [optimisticCart, addToOptimisticCart] = useOptimistic(
  cart,
  (currentCart, newItem) => [...currentCart, newItem]
);
// Shows item immediately in cart while server request is in flight

Virtual Lists: Rendering 10,000 Items Without Freezing

Rendering 10,000 list items creates 10,000 DOM nodes. Scrolling through them is janky. Virtualization renders only the visible items — typically 20-50 — regardless of list length.

'use client';
import { useVirtualizer } from '@tanstack/react-virtual';
import { useRef } from 'react';

function OrderHistory({ orders }: { orders: Order[] }) {
  const parentRef = useRef<HTMLDivElement>(null);

  const virtualizer = useVirtualizer({
    count: orders.length,          // total number of items
    getScrollElement: () => parentRef.current,
    estimateSize: () => 64,        // estimated row height in pixels
    overscan: 5,                   // render 5 extra items above/below visible area
  });

  return (
    <div 
      ref={parentRef} 
      style={{ height: '600px', overflow: 'auto' }}
    >
      {/* Total scroll height = estimatedSize * count */}
      <div style={{ height: virtualizer.getTotalSize(), position: 'relative' }}>
        {virtualizer.getVirtualItems().map(virtualItem => (
          <div
            key={virtualItem.key}
            style={{
              position: 'absolute',
              top: 0,
              left: 0,
              width: '100%',
              height: `${virtualItem.size}px`,
              transform: `translateY(${virtualItem.start}px)`,  // position in virtual space
            }}
          >
            <OrderRow order={orders[virtualItem.index]} />
          </div>
        ))}
      </div>
    </div>
  );
}

With virtualization: 10,000 orders → 20 DOM nodes in the viewport. Without: 10,000 DOM nodes, 200MB+ memory, janky scrolling. The virtualizer maintains the full scroll height (so the scrollbar is accurate) but only renders and positions the visible rows.

React Query: Client-Side Data Fetching and Caching

For client-side data fetching (interactive dashboards, real-time updates), React Query (TanStack Query) provides caching, background refetch, optimistic updates, and stale-while-revalidate — without manual state management.

'use client';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';

function OrderDashboard() {
  const queryClient = useQueryClient();

  // Fetch with automatic caching, background refetch, and error state
  const { data: orders, isLoading, error } = useQuery({
    queryKey: ['orders', { status: 'pending' }],
    queryFn: () => api.orders.list({ status: 'pending' }),
    staleTime: 30_000,       // consider data fresh for 30 seconds
    refetchInterval: 60_000, // background refetch every 60 seconds
  });

  // Mutation with optimistic update
  const cancelMutation = useMutation({
    mutationFn: (orderId: string) => api.orders.cancel(orderId),

    // Optimistic update: update UI immediately before server confirms
    onMutate: async (orderId) => {
      await queryClient.cancelQueries({ queryKey: ['orders'] });
      const snapshot = queryClient.getQueryData(['orders', { status: 'pending' }]);

      queryClient.setQueryData(
        ['orders', { status: 'pending' }],
        (old: Order[]) => old.filter(o => o.id !== orderId)  // remove immediately
      );

      return { snapshot };  // for rollback
    },

    onError: (error, orderId, context) => {
      // Rollback on failure
      queryClient.setQueryData(['orders', { status: 'pending' }], context?.snapshot);
    },

    onSettled: () => {
      // Refetch to sync with server state
      queryClient.invalidateQueries({ queryKey: ['orders'] });
    },
  });

  if (isLoading) return <Skeleton />;
  if (error) return <ErrorMessage error={error} />;

  return (
    <ul>
      {orders?.map(order => (
        <OrderItem key={order.id} order={order}
          onCancel={() => cancelMutation.mutate(order.id)} />
      ))}
    </ul>
  );
}

The optimistic update pattern eliminates the perceived latency of server round-trips. The UI responds instantly; the server update happens in the background. If it fails, the UI rolls back. This is the difference between a snappy and a sluggish application at identical network latency.

Performance Measurement and Monitoring

Measuring before optimizing is essential — perceived performance issues often have non-obvious root causes.

// React DevTools Profiler: identifies slow components
// Enable in DevTools → Profiler tab → Record → Interact → Stop

// Measure render time in code
import { Profiler } from 'react';

<Profiler 
  id="OrderList"
  onRender={(id, phase, actualDuration) => {
    if (actualDuration > 16) {  // 16ms = 60fps threshold
      console.warn(`${id} took ${actualDuration}ms to render`);
    }
  }}
>
  <OrderList orders={orders} />
</Profiler>

// Web Vitals measurement in production
import { onLCP, onINP, onCLS } from 'web-vitals';

onLCP((metric) => {
  analytics.track('web_vital', {
    name: metric.name,
    value: metric.value,
    rating: metric.rating,  // 'good', 'needs-improvement', 'poor'
    id: metric.id,
  });
});

memo, useMemo, useCallback: prevent unnecessary re-renders, but each has overhead. Profile before applying — premature memoization adds complexity without benefit.

// memo: skip re-render if props haven't changed (reference equality)
const OrderItem = memo(function OrderItem({ order }: { order: Order }) {
  return <div>{order.total}</div>;
});

// useMemo: memoize expensive computation
const expensiveFilter = useMemo(
  () => orders.filter(o => complexFilter(o, criteria)),
  [orders, criteria]  // only recalculate when these change
);

// useCallback: stable function reference for memo'd children
const handleSelect = useCallback(
  (id: string) => setSelected(id),
  []  // stable forever — no dependencies
);

Rule: use memo on components that render frequently with the same props. Use useMemo for expensive computations that run on every render. Use useCallback for functions passed to memo'd children or used as useEffect dependencies.

Image and Font Optimization

Images and fonts are the two largest contributors to page weight in most React applications. Next.js Image handles most of this automatically.

import Image from 'next/image';

// Next.js Image automatically:
// - Serves WebP/AVIF (30-50% smaller than JPEG/PNG)
// - Generates multiple sizes (srcset for different viewports)
// - Lazy-loads below-the-fold images
// - Prevents layout shift (requires width + height or fill)
// - Caches at CDN edge

<Image
  src="/product-photo.jpg"
  alt="Product"
  width={800}
  height={600}
  sizes="(max-width: 768px) 100vw, (max-width: 1200px) 50vw, 800px"
  // sizes hint: tells browser which source to download at each viewport
/>

// Remote images: allow specific domains
// next.config.ts
const config: NextConfig = {
  images: {
    remotePatterns: [
      { hostname: 'cdn.example.com', protocol: 'https' }
    ],
  },
};

Font optimization with next/font: fonts are downloaded at build time, self-hosted, and loaded with font-display: optional to prevent layout shift. Google Fonts are fetched at build time — no runtime request to Google, no privacy leak, zero CLS from font swap.

// app/layout.tsx
import { Inter } from 'next/font/google';

const inter = Inter({
  subsets: ['latin'],
  display: 'swap',         // or 'optional' for zero CLS
  variable: '--font-inter',
  preload: true,
});

export default function RootLayout({ children }) {
  return (
    <html lang="en" className={inter.variable}>
      {children}
    </html>
  );
}

Self-hosted fonts with next/font/local eliminate the network round-trip to Google's CDN entirely — the font is served from your own CDN alongside your JavaScript.

Conclusion

React performance in 2026 is a layered problem. Server Components eliminate entire categories of client-side JavaScript — the correct architecture pushes UI logic to the server wherever possible. Next.js's four-layer caching model handles the data freshness vs. performance trade-off. Suspense boundaries enable progressive loading that makes slow pages feel fast. Core Web Vitals provide measurable targets (LCP <2.5s, CLS <0.1, INP <200ms) that tie engineering decisions to user experience outcomes.

The highest-impact changes in order: adopt Server Components for data-fetching components (largest bundle reduction), implement Suspense boundaries (fastest perceived loading), add priority to above-the-fold images (LCP improvement), move heavy computations to Web Workers (INP improvement), and use React Query's optimistic update pattern for interactive mutations. Each is a targeted fix for a measurable metric.

Measure before you optimize. Use Lighthouse (LCP, CLS, INP scores), React DevTools Profiler (slow component renders), and @next/bundle-analyzer (bundle contributors). A 20% LCP improvement and a 0.05 CLS reduction are concrete wins that translate directly to search ranking and user retention — not abstract "performance improvements."

Sources

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-06-02 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

Thursday, April 9, 2026

Advanced Prompt Patterns: Tree-of-Thought, ReAct, and Self-Consistency

Hero image: A branching tree of glowing thought paths, some paths lit green (successful reasoning) and others red (dead ends), converging on a golden answer node

Chain-of-Thought prompting was a breakthrough — but it has a fundamental limitation. It follows a single reasoning path. If that path starts with a wrong assumption, every subsequent step is built on a faulty foundation. There's no backtracking, no exploration of alternatives, no way to course-correct.

The advanced prompting patterns we'll cover in this post address exactly this limitation. They were born from a simple question: what if the model could explore multiple reasoning paths, use external tools to verify its assumptions, and check its own work against alternative approaches?

These techniques — Tree-of-Thought, ReAct, Self-Consistency, meta-prompting, and more — represent the current frontier of prompt engineering. They're what separates a clever chatbot from a reliable AI system that can handle complex, multi-step tasks in production.

This is Part 5 of our Prompt Engineering Deep-Dive series. If you haven't read Parts 1-4, the techniques here build directly on system prompts, Chain-of-Thought, few-shot prompting, and structured output.

Tree-of-Thought (ToT): Exploring Multiple Paths

Chain-of-Thought follows one path: Step 1 → Step 2 → Step 3 → Answer. Tree-of-Thought explores a branching tree of possibilities, evaluates each branch, and prunes dead ends before committing to an answer.

How It Works

  1. Generate multiple candidate next-steps at each reasoning point
  2. Evaluate each candidate (is this step promising or a dead end?)
  3. Select the most promising branches to continue
  4. Backtrack from dead ends and explore alternatives
Architecture diagram comparing linear CoT (single path) to Tree-of-Thought (branching paths with evaluation and pruning)
flowchart TB START["Problem"] subgraph COT ["Chain-of-Thought (Linear)"] direction LR C1["Step 1"] --> C2["Step 2"] --> C3["Step 3"] --> CA["Answer"] end subgraph TOT ["Tree-of-Thought (Branching)"] direction TB T1["Step 1a"] T2["Step 1b"] T3["Step 1c"] T1 --> T4["Step 2a"] T1 --> T5["Step 2b"] T2 --> T6["Step 2c ✗"] T3 --> T7["Step 2d"] T4 --> T8["Answer ✓"] T5 --> T9["Dead end ✗"] T7 --> T10["Answer ✓✓"] end START --> COT START --> TOT style C1 fill:#3498db,stroke:#2980b9,color:#fff style C2 fill:#3498db,stroke:#2980b9,color:#fff style C3 fill:#3498db,stroke:#2980b9,color:#fff style CA fill:#3498db,stroke:#2980b9,color:#fff style T1 fill:#2ecc71,stroke:#27ae60,color:#fff style T2 fill:#f39c12,stroke:#e67e22,color:#fff style T3 fill:#2ecc71,stroke:#27ae60,color:#fff style T4 fill:#2ecc71,stroke:#27ae60,color:#fff style T5 fill:#e74c3c,stroke:#c0392b,color:#fff style T6 fill:#e74c3c,stroke:#c0392b,color:#fff style T7 fill:#2ecc71,stroke:#27ae60,color:#fff style T8 fill:#2ecc71,stroke:#27ae60,color:#fff style T9 fill:#e74c3c,stroke:#c0392b,color:#fff style T10 fill:#6C63FF,stroke:#8B83FF,color:#fff style START fill:#6C63FF,stroke:#8B83FF,color:#fff style COT fill:#1a1a2e,stroke:#3498db,color:#fff style TOT fill:#1a1a2e,stroke:#2ecc71,color:#fff

Implementation

def tree_of_thought(problem: str, breadth: int = 3, depth: int = 3) -> str:
    """Explore multiple reasoning paths and select the best."""

    def generate_steps(context: str, n: int) -> list[str]:
        prompt = f"""Given this problem and progress so far:
{context}

Generate {n} different possible next steps. 
For each step, explain your reasoning.
Return as a numbered list."""
        return parse_steps(call_llm(prompt))

    def evaluate_step(context: str, step: str) -> float:
        prompt = f"""Evaluate this reasoning step:
Context: {context}
Step: {step}

Rate from 0.0 to 1.0:
- Is this step logically sound? 
- Does it make progress toward the solution?
- Does it avoid assumptions that could be wrong?

Return ONLY a number between 0.0 and 1.0."""
        return float(call_llm(prompt).strip())

    # BFS through reasoning tree
    candidates = [{"context": problem, "steps": [], "score": 1.0}]

    for level in range(depth):
        next_candidates = []

        for candidate in candidates:
            steps = generate_steps(candidate["context"], breadth)

            for step in steps:
                score = evaluate_step(candidate["context"], step)
                new_context = candidate["context"] + f"\nStep {level+1}: {step}"
                next_candidates.append({
                    "context": new_context,
                    "steps": candidate["steps"] + [step],
                    "score": candidate["score"] * score
                })

        # Keep top candidates (beam search)
        candidates = sorted(
            next_candidates, 
            key=lambda x: x["score"], 
            reverse=True
        )[:breadth]

    # Return the highest-scoring path
    best = candidates[0]
    return synthesize_answer(problem, best["steps"])

When to Use ToT

Use Case CoT Sufficient? ToT Needed?
Simple math Yes No
Code debugging Usually For complex multi-file bugs
Architecture design No Yes — multiple valid approaches
Strategic planning No Yes — tradeoffs require exploration
Game solving (chess, puzzles) No Yes — search required
Creative writing No Yes — exploring different directions

Cost consideration: ToT uses 5-20x more API calls than single CoT. Use it only when the accuracy improvement justifies the cost.

graph TD PROBLEM["Problem"] --> BA["Branch A\nexplore approach 1"] PROBLEM --> BB["Branch B\nexplore approach 2"] PROBLEM --> BC["Branch C\nexplore approach 3"] BA --> EVAL_A{"Evaluate A\npromising?"} BB --> EVAL_B{"Evaluate B\npromising?"} BC --> EVAL_C{"Evaluate C\npromising?"} EVAL_A -->|"Yes"| BEST["Best path\ncontinue exploring"] EVAL_B -->|"Dead end"| PRUNE_B["Prune branch"] EVAL_C -->|"Yes"| BEST BEST --> SOLUTION["Solution"] style PROBLEM fill:#6C63FF,stroke:#8B83FF,color:#fff style BA fill:#2ecc71,stroke:#27ae60,color:#fff style BB fill:#e74c3c,stroke:#c0392b,color:#fff style BC fill:#2ecc71,stroke:#27ae60,color:#fff style BEST fill:#3498db,stroke:#2980b9,color:#fff style SOLUTION fill:#2ecc71,stroke:#27ae60,color:#fff style PRUNE_B fill:#e74c3c,stroke:#c0392b,color:#fff

ReAct: Reasoning + Acting

ReAct (Reasoning + Acting) combines Chain-of-Thought reasoning with tool use. Instead of reasoning in isolation, the model thinks about what information it needs, uses tools to get it, observes the results, and continues reasoning.

The ReAct Loop

Thought: I need to check if the database table exists
Action: query_database("SHOW TABLES LIKE 'users'")
Observation: Table 'users' exists with columns: id, name, email, created_at
Thought: The table exists. Now I need to check if there's an index on email
Action: query_database("SHOW INDEX FROM users WHERE Column_name = 'email'")
Observation: No index found on email column
Thought: Missing email index explains the slow login query. I should recommend adding it.
Answer: Add an index on users.email — this will fix the O(n) scan on every login.

Implementation

def react_agent(
    question: str,
    tools: dict[str, callable],
    max_steps: int = 10
) -> str:
    """ReAct agent: interleave reasoning and tool use."""

    tool_descriptions = "\n".join(
        f"- {name}: {func.__doc__}" for name, func in tools.items()
    )

    system = f"""You are a reasoning agent. For each step:
1. Thought: Reason about what you know and what you need
2. Action: Call a tool if needed (format: tool_name(args))
3. Observation: [Tool result will be inserted here]

Repeat until you have enough information to answer.
When ready, respond with: Answer: [your final answer]

Available tools:
{tool_descriptions}"""

    messages = [
        {"role": "system", "content": system},
        {"role": "user", "content": question}
    ]

    for step in range(max_steps):
        response = call_llm(messages)
        messages.append({"role": "assistant", "content": response})

        # Check if we have a final answer
        if "Answer:" in response:
            return response.split("Answer:")[-1].strip()

        # Parse and execute action
        action_match = re.search(r'Action:\s*(\w+)\((.*?)\)', response)
        if action_match:
            tool_name = action_match.group(1)
            tool_args = action_match.group(2)

            if tool_name in tools:
                result = tools[tool_name](tool_args)
                observation = f"Observation: {result}"
            else:
                observation = f"Observation: Error — tool '{tool_name}' not found"

            messages.append({"role": "user", "content": observation})

    return "Max steps reached without conclusion."
sequenceDiagram participant LLM participant Tool participant User LLM->>LLM: Thought — what information do I need? LLM->>Tool: Action — call tool with query Tool->>LLM: Observation — tool returns result LLM->>LLM: Next thought — does this confirm hypothesis? LLM->>Tool: Action — call another tool if needed Tool->>LLM: Observation — additional result LLM->>User: Final answer based on gathered evidence

ReAct vs. Plain Tool Use

The key difference is that ReAct makes the reasoning explicit. In plain tool use, the model calls tools but doesn't show its reasoning about why it chose that tool or what it expects to find. ReAct forces the model to articulate its hypothesis before acting, which:

  1. Improves tool selection — Thinking first reduces irrelevant tool calls
  2. Enables debugging — You can read the thought trace to understand failures
  3. Supports learning — The reasoning chain becomes training data for improvement

Self-Consistency: Majority Vote on Reasoning

Self-Consistency generates multiple independent reasoning chains for the same problem and takes the majority vote. It's based on the insight that correct reasoning paths are more likely to converge on the same answer.

Implementation

import collections

def self_consistent_answer(
    prompt: str,
    n_samples: int = 5,
    temperature: float = 0.7
) -> dict:
    """Generate multiple reasoning paths and vote on the answer."""

    answers = []
    reasoning_chains = []

    for _ in range(n_samples):
        response = call_llm(
            prompt + "\nThink step by step, then give your final answer on the last line starting with 'ANSWER:'",
            temperature=temperature  # Higher temp for diversity
        )

        # Extract final answer
        lines = response.strip().split('\n')
        answer_line = [l for l in lines if l.startswith('ANSWER:')]
        if answer_line:
            answer = answer_line[-1].replace('ANSWER:', '').strip()
            answers.append(answer)
            reasoning_chains.append(response)

    # Majority vote
    counter = collections.Counter(answers)
    best_answer, vote_count = counter.most_common(1)[0]

    return {
        "answer": best_answer,
        "confidence": vote_count / len(answers),
        "total_votes": len(answers),
        "vote_distribution": dict(counter),
        "reasoning_chains": reasoning_chains
    }

When Self-Consistency Shines

Self-Consistency is most valuable when:
- The problem has a single correct answer (math, classification, yes/no)
- Individual CoT accuracy is in the 60-85% range (high enough to converge, low enough to benefit)
- You can afford N times the API cost (typically N=5 to N=11)

Single CoT Accuracy Self-Consistency (N=5) Improvement
60% ~78% +18%
70% ~87% +17%
80% ~94% +14%
90% ~98% +8%

Diminishing returns above N=11. Research shows that going from 5 to 11 samples provides meaningful improvement, but 11 to 21 provides very little additional benefit.

graph LR PROMPT["Same prompt\n(temperature=0.7)"] --> R1["Response 1\nAnswer: A"] PROMPT --> R2["Response 2\nAnswer: B"] PROMPT --> R3["Response 3\nAnswer: A"] R1 --> VOTE["Majority vote\n(count answers)"] R2 --> VOTE R3 --> VOTE VOTE --> FINAL["Final answer: A\n2/3 = 67% confidence"] style PROMPT fill:#6C63FF,stroke:#8B83FF,color:#fff style R1 fill:#2ecc71,stroke:#27ae60,color:#fff style R2 fill:#f39c12,stroke:#e67e22,color:#fff style R3 fill:#2ecc71,stroke:#27ae60,color:#fff style VOTE fill:#9b59b6,stroke:#8e44ad,color:#fff style FINAL fill:#2ecc71,stroke:#27ae60,color:#fff
flowchart TB PROMPT["Same Problem"] PROMPT --> R1["Chain 1
T=0.7"] PROMPT --> R2["Chain 2
T=0.7"] PROMPT --> R3["Chain 3
T=0.7"] PROMPT --> R4["Chain 4
T=0.7"] PROMPT --> R5["Chain 5
T=0.7"] R1 --> A1["Answer: A"] R2 --> A2["Answer: B"] R3 --> A3["Answer: A"] R4 --> A4["Answer: A"] R5 --> A5["Answer: C"] A1 --> VOTE["Majority Vote"] A2 --> VOTE A3 --> VOTE A4 --> VOTE A5 --> VOTE VOTE --> FINAL["Final: A
3/5 = 60% confidence"] style PROMPT fill:#6C63FF,stroke:#8B83FF,color:#fff style R1 fill:#3498db,stroke:#2980b9,color:#fff style R2 fill:#3498db,stroke:#2980b9,color:#fff style R3 fill:#3498db,stroke:#2980b9,color:#fff style R4 fill:#3498db,stroke:#2980b9,color:#fff style R5 fill:#3498db,stroke:#2980b9,color:#fff style A1 fill:#2ecc71,stroke:#27ae60,color:#fff style A2 fill:#f39c12,stroke:#e67e22,color:#fff style A3 fill:#2ecc71,stroke:#27ae60,color:#fff style A4 fill:#2ecc71,stroke:#27ae60,color:#fff style A5 fill:#e74c3c,stroke:#c0392b,color:#fff style VOTE fill:#9b59b6,stroke:#8e44ad,color:#fff style FINAL fill:#2ecc71,stroke:#27ae60,color:#fff

Meta-Prompting: Prompts That Write Prompts

Meta-prompting uses the LLM itself to generate, refine, and optimize prompts. Instead of manually iterating on prompt wording, you ask the model to help.

Pattern: Automatic Prompt Optimization

def optimize_prompt(
    initial_prompt: str,
    test_cases: list[dict],
    n_iterations: int = 5
) -> str:
    """Use the LLM to iteratively improve a prompt."""

    current_prompt = initial_prompt
    best_score = evaluate_prompt(current_prompt, test_cases)
    best_prompt = current_prompt

    for iteration in range(n_iterations):
        # Ask the model to analyze failures
        failures = get_failures(current_prompt, test_cases)

        improvement_request = f"""Current prompt:
{current_prompt}

This prompt fails on these cases:
{failures}

Analyze why it fails and suggest an improved version of the prompt 
that would handle these cases correctly while maintaining accuracy 
on the cases it already handles well.

Return ONLY the improved prompt, nothing else."""

        new_prompt = call_llm(improvement_request)
        new_score = evaluate_prompt(new_prompt, test_cases)

        if new_score > best_score:
            best_score = new_score
            best_prompt = new_prompt
            current_prompt = new_prompt

    return best_prompt

Pattern: Task Decomposition Prompting

Ask the model to break down a complex task into sub-prompts:

I need to analyze customer support tickets and produce a weekly report.

Break this task into a sequence of focused sub-tasks, where each 
sub-task has:
1. A clear input
2. A specific prompt optimized for that sub-task
3. A defined output format
4. Dependencies on previous sub-tasks

Design the prompts so each one is simple enough to be highly reliable.

Reflexion: Learning from Mistakes

Reflexion extends ReAct by adding a self-reflection step. After completing a task, the model evaluates its own performance and generates feedback that improves future attempts.

def reflexion_agent(
    task: str,
    evaluator: callable,
    max_attempts: int = 3
) -> str:
    """Agent that learns from its own mistakes."""

    reflections = []

    for attempt in range(max_attempts):
        # Include past reflections in the prompt
        reflection_context = ""
        if reflections:
            reflection_context = "\n\nPrevious attempts and reflections:\n"
            for r in reflections:
                reflection_context += f"- Attempt: {r['summary']}\n"
                reflection_context += f"  Reflection: {r['reflection']}\n"
                reflection_context += f"  What to do differently: {r['improvement']}\n"

        prompt = f"""{task}
{reflection_context}
Think step by step. If you've seen reflections above, 
use them to avoid repeating the same mistakes."""

        response = call_llm(prompt)
        score, feedback = evaluator(response)

        if score >= 0.9:  # Good enough
            return response

        # Self-reflect on the failure
        reflection_prompt = f"""You attempted this task:
{task}

Your response:
{response}

Evaluation feedback:
{feedback}

Reflect on what went wrong and what you should do differently 
next time. Be specific and actionable."""

        reflection = call_llm(reflection_prompt)
        reflections.append({
            "summary": response[:200],
            "reflection": reflection,
            "improvement": reflection  # Could parse for action items
        })

    return response  # Return best attempt

Combining Patterns: The Full Stack

In production, these patterns are often combined:

class ProductionReasoningPipeline:
    """Combines multiple advanced patterns for maximum reliability."""

    def __init__(self, tools: dict, schemas: dict):
        self.tools = tools
        self.schemas = schemas

    def solve(self, problem: str, complexity: str = "auto") -> dict:
        if complexity == "auto":
            complexity = self._assess_complexity(problem)

        if complexity == "simple":
            # Direct CoT — cheapest
            return self._solve_cot(problem)

        elif complexity == "medium":
            # Self-Consistency — better accuracy
            return self._solve_self_consistent(problem)

        elif complexity == "complex":
            # ReAct with tools — can gather information
            return self._solve_react(problem)

        elif complexity == "hard":
            # Tree-of-Thought with Reflexion — maximum accuracy
            return self._solve_tot_reflexion(problem)

    def _assess_complexity(self, problem: str) -> str:
        prompt = f"""Rate this problem's complexity: simple, medium, complex, or hard.
Problem: {problem}
Consider: number of steps, need for external info, ambiguity, number of valid approaches.
Return ONLY one word."""
        return call_llm(prompt).strip().lower()
Comparison visual: A decision matrix showing when to use each advanced pattern based on accuracy needs, cost budget, and task type

Performance and Cost Comparison

Pattern API Calls Accuracy Gain Best For
Single CoT 1x Baseline Simple reasoning
Self-Consistency (N=5) 5x +10-18% Classification, math
ReAct 3-10x +15-25% Tasks needing external data
Tree-of-Thought 10-50x +20-35% Complex planning, design
Reflexion 3-9x +10-20% Iterative improvement
Combined (adaptive) 1-50x Optimal per task Production systems

The key insight: match the technique to the task complexity. Using Tree-of-Thought for "What's 2+2?" wastes money. Using single CoT for "Design a distributed database migration strategy" wastes accuracy.

Conclusion

Advanced prompt patterns extend the capabilities of LLMs from simple question-answering to complex reasoning, planning, and problem-solving. The key takeaways:

  1. Tree-of-Thought explores multiple paths — use for design, planning, and ambiguous problems
  2. ReAct combines reasoning with tools — use when the model needs external information
  3. Self-Consistency uses majority voting — use for deterministic problems where individual accuracy is 60-85%
  4. Meta-prompting automates prompt optimization — use to iterate faster on prompt quality
  5. Reflexion learns from mistakes — use for tasks where iterative improvement is possible
  6. Combine adaptively — route tasks to the cheapest technique that achieves acceptable accuracy

In the final post of this series, we'll bring everything together with Production Prompt Engineering — testing, versioning, A/B testing, and optimization at scale.


This is Part 5 of the Prompt Engineering Deep-Dive series. Previous: Structured Output. Next: Production Prompt Engineering — Testing and Optimization at Scale.

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-04-09 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

AI as Infrastructure: Value Moves Up-Stack

For a few years the AI conversation was about who had the biggest model. That is the wrong altitude now. Models still matter, the way CPUs s...