Batch Processing

Generate TTS audio for a large collection of text inputs efficiently. Manage concurrency, handle failures, and persist results to disk.

When to use batch processing

Use batch processing when you have multiple independent text inputs (e.g., product descriptions, notification messages, dataset rows) that each need their own audio file. For a single long text, use Long-Form Audio instead.

How It Works

murmr supports two approaches for batch processing, depending on your throughput needs:

Sync Batch (Recommended)

Submit concurrent POST /v1/audio/speech requests. Each returns 200 with binary audio immediately.

✓ Simplest approach — no polling needed
✓ Bounded by plan concurrency limit
✓ Best for <4096 chars per item

Async Batch (High Volume)

Submit with webhook_url to get job IDs. Poll or receive callbacks. Jobs queue on GPU workers.

✓ Fire-and-forget submission
✓ GPU queue handles backpressure
✓ Best for very large batches

Concurrency Limits

Each plan has a maximum number of in-flight requests per API key. Your batch processor must respect this limit to avoid 429 errors.

Plan	Concurrent Requests	Recommended Batch Concurrency
Free	1	1 (sequential)
Starter ($10)	3	2-3
Pro ($25)	5	4-5
Realtime ($49)	5	4-5
Scale ($99)	10	8-10

Stay below your limit

Set your concurrency pool to your plan's limit minus 1, leaving headroom for other API usage (e.g., dashboard requests). Exceeding the limit returns a 429 with concurrent_limit in the error body.

Sync Batch Processing

Process multiple text inputs concurrently with bounded parallelism, retry on failure, and persist each result to disk. This is the recommended approach for most use cases.

import { MurmrClient, MurmrError } from '@murmr/sdk';
import { writeFile, mkdir } from 'fs/promises';
import { join } from 'path';

const client = new MurmrClient({ apiKey: process.env.MURMR_API_KEY! });

interface BatchItem {
  id: string;
  text: string;
}

interface BatchResult {
  id: string;
  status: 'success' | 'failed';
  filePath?: string;
  error?: string;
}

async function processBatch(
  items: BatchItem[],
  voiceId: string,
  options: {
    concurrency?: number;
    outputDir?: string;
    format?: 'mp3' | 'wav' | 'opus' | 'aac' | 'flac';
    maxRetries?: number;
  } = {},
): Promise<BatchResult[]> {
  const {
    concurrency = 3,
    outputDir = './audio-output',
    format = 'mp3',
    maxRetries = 3,
  } = options;

  await mkdir(outputDir, { recursive: true });

  const results: BatchResult[] = [];
  const queue = [...items];
  const active: Promise<void>[] = [];

  async function processItem(item: BatchItem): Promise<void> {
    for (let attempt = 1; attempt <= maxRetries; attempt++) {
      try {
        const response = await client.speech.create({
          input: item.text,
          voice: voiceId,
          response_format: format,
        });

        const audio = Buffer.from(await response.arrayBuffer());
        const filePath = join(outputDir, `${item.id}.${format}`);
        await writeFile(filePath, audio);

        results.push({ id: item.id, status: 'success', filePath });
        return;
      } catch (error) {
        if (error instanceof MurmrError) {
          // Don't retry 400 (bad input) or 401 (auth) errors
          if (error.status === 400 || error.status === 401) {
            results.push({ id: item.id, status: 'failed', error: error.message });
            return;
          }
          // Retry 429 (rate limit) with backoff
          if (error.status === 429 && attempt < maxRetries) {
            const backoff = Math.pow(2, attempt) * 1000;
            await new Promise((r) => setTimeout(r, backoff));
            continue;
          }
        }
        if (attempt === maxRetries) {
          const msg = error instanceof Error ? error.message : String(error);
          results.push({ id: item.id, status: 'failed', error: msg });
        } else {
          // Exponential backoff for transient errors
          await new Promise((r) => setTimeout(r, Math.pow(2, attempt) * 1000));
        }
      }
    }
  }

  // Process with bounded concurrency
  while (queue.length > 0 || active.length > 0) {
    while (active.length < concurrency && queue.length > 0) {
      const item = queue.shift()!;
      const promise = processItem(item).then(() => {
        active.splice(active.indexOf(promise), 1);
      });
      active.push(promise);
    }
    if (active.length > 0) {
      await Promise.race(active);
    }
  }

  return results;
}

// Usage
const items: BatchItem[] = [
  { id: 'welcome', text: 'Welcome to our platform.' },
  { id: 'goodbye', text: 'Thank you for using our service.' },
  { id: 'error', text: 'An error occurred. Please try again.' },
  { id: 'confirm', text: 'Your order has been confirmed.' },
  // ... hundreds more items
];

const results = await processBatch(items, 'voice_abc123', {
  concurrency: 3,       // Match your plan's limit
  outputDir: './audio',
  format: 'mp3',
  maxRetries: 3,
});

// Report results
const succeeded = results.filter((r) => r.status === 'success');
const failed = results.filter((r) => r.status === 'failed');
console.log(`Batch complete: ${succeeded.length} succeeded, ${failed.length} failed`);

for (const fail of failed) {
  console.error(`  ${fail.id}: ${fail.error}`);
}

Async Batch Processing

For very large batches (hundreds or thousands of items), use async jobs with webhooks. Submit all items quickly, then collect results as they complete.

import { MurmrClient, isAsyncResponse, MurmrError } from '@murmr/sdk';
import { writeFile } from 'fs/promises';

const client = new MurmrClient({ apiKey: process.env.MURMR_API_KEY! });

interface TextItem {
  id: string;
  text: string;
}

// Step 1: Submit all jobs (fast — just queuing)
async function submitBatch(
  items: TextItem[],
  voiceId: string,
  webhookUrl?: string,
): Promise<Map<string, string>> {
  const jobMap = new Map<string, string>(); // jobId → itemId

  for (const item of items) {
    const result = await client.speech.create({
      input: item.text,
      voice: voiceId,
      response_format: 'mp3',
      webhook_url: webhookUrl,
    });

    if (isAsyncResponse(result)) {
      jobMap.set(result.id, item.id);
      console.log(`Submitted ${item.id} → job ${result.id}`);
    }
  }

  return jobMap;
}

// Step 2: Poll all jobs until complete
async function pollBatch(
  jobMap: Map<string, string>,
  outputDir: string,
  pollIntervalMs = 3000,
  timeoutMs = 600_000,
): Promise<void> {
  const pending = new Set(jobMap.keys());
  const start = Date.now();

  while (pending.size > 0) {
    if (Date.now() - start > timeoutMs) {
      console.error(`Timeout: ${pending.size} jobs still pending`);
      break;
    }

    for (const jobId of [...pending]) {
      try {
        const status = await client.jobs.get(jobId);

        if (status.status === 'completed' && status.audio_base64) {
          const itemId = jobMap.get(jobId)!;
          const audio = Buffer.from(status.audio_base64, 'base64');
          await writeFile(`${outputDir}/${itemId}.mp3`, audio);
          console.log(`Completed: ${itemId}`);
          pending.delete(jobId);
        } else if (status.status === 'failed') {
          console.error(`Failed: ${jobMap.get(jobId)} — ${status.error}`);
          pending.delete(jobId);
        }
        // else: still processing, check again next round
      } catch (error) {
        // 410 Gone = audio purged
        if (error instanceof MurmrError && error.status === 410) {
          console.error(`Expired: ${jobMap.get(jobId)} — audio purged`);
          pending.delete(jobId);
        }
      }
    }

    if (pending.size > 0) {
      console.log(`Waiting... ${pending.size} jobs remaining`);
      await new Promise((r) => setTimeout(r, pollIntervalMs));
    }
  }
}

// Usage
const items: TextItem[] = [
  { id: 'welcome', text: 'Welcome to our platform.' },
  { id: 'goodbye', text: 'Thank you for using our service.' },
  // ... many more items
];

const jobMap = await submitBatch(items, 'voice_abc123');
await pollBatch(jobMap, './audio');

Production Patterns

Retry Strategy

Use exponential backoff for transient failures. Never retry client errors (400, 401).

Error Code	Retry?	Strategy
400 Bad Request	No	Fix the input — text too long, missing fields, etc.
401 Unauthorized	No	Check API key. Do not retry.
429 Rate Limit	Yes	Exponential backoff: 2s, 4s, 8s. Check Retry-After header.
429 Concurrent Limit	Yes	Wait for in-flight requests to complete, then retry.
500 Server Error	Yes	Backoff and retry up to 3 times.
503 Unavailable	Yes	Service temporarily down. Retry after 10-30s.

File Persistence

Always write audio to disk immediately after receiving it. For async jobs, retrieve audio promptly — job data expires after 24 hours.

typescript

// Recommended file naming pattern
const filePath = `./audio/${sanitizeFilename(item.id)}.${format}`;

// Atomic writes — write to temp file, then rename
import { writeFile, rename } from 'fs/promises';
import { join } from 'path';

const tmpPath = join(outputDir, `.tmp_${item.id}.${format}`);
await writeFile(tmpPath, audioBuffer);
await rename(tmpPath, join(outputDir, `${item.id}.${format}`));

Progress Tracking

For large batches, track progress with a manifest file so you can resume after interruptions.

typescript

import { readFile, writeFile } from 'fs/promises';

interface Manifest {
  totalItems: number;
  completed: string[];
  failed: Array<{ id: string; error: string }>;
  startedAt: string;
}

// Load or create manifest
async function loadManifest(path: string): Promise<Manifest> {
  try {
    return JSON.parse(await readFile(path, 'utf-8'));
  } catch {
    return { totalItems: 0, completed: [], failed: [], startedAt: new Date().toISOString() };
  }
}

// Skip already-completed items
const manifest = await loadManifest('./audio/manifest.json');
const remaining = items.filter((item) => !manifest.completed.includes(item.id));
console.log(`Resuming: ${remaining.length} of ${items.length} remaining`);

// After each successful generation, update manifest
manifest.completed.push(item.id);
await writeFile('./audio/manifest.json', JSON.stringify(manifest, null, 2));

Rate Limit Awareness

Batch processing consumes characters from your monthly quota. Monitor usage to avoid unexpected overages.

typescript

// Estimate character usage before starting
const totalChars = items.reduce((sum, item) => sum + item.text.length, 0);
console.log(`Batch will use ~${totalChars.toLocaleString()} characters`);

// Check current usage via API
const usage = await fetch('https://api.murmr.dev/v1/usage', {
  headers: { Authorization: `Bearer ${apiKey}` },
}).then((r) => r.json());

console.log(`Used: ${usage.characters_used}/${usage.characters_limit}`);
const remaining = usage.characters_limit - usage.characters_used;

if (totalChars > remaining) {
  console.warn(`Warning: batch needs ${totalChars} chars but only ${remaining} remaining`);
}

Overage billing

Paid plans allow overage beyond the monthly limit. Overage rates: Starter $15/1M, Pro $12/1M, Scale $10/1M characters. Free plans receive a 429 when the limit is reached. See Rate Limits.

Choosing an Approach

Scenario	Approach	Why
10-50 items, need results now	Sync batch	Simple, direct, no polling overhead
100+ items, can wait	Async batch + webhooks	Submit fast, collect results asynchronously
Single long text (article, book)	createLongForm() / create_long_form()	SDK handles chunking, retry, and concatenation
Real-time per-user requests	SSE streaming	Low latency (~450ms TTFC), progressive playback

Batch Processing

How It Works

Sync Batch (Recommended)

Async Batch (High Volume)

Concurrency Limits

Sync Batch Processing

Async Batch Processing

Production Patterns

Retry Strategy

File Persistence

Progress Tracking

Rate Limit Awareness

Choosing an Approach

See Also