Sechno
Backend

Scaling ORM Code Generation: Practical Patterns When Prisma Breaks at Hundreds of Models

Concrete strategies and examples to make ORM code generation (Prisma-style) reliable and fast for large schemas: split schemas, incremental generation, parallel workers, caching, and trade-offs.

SSechno Team 5 min read 84 views
Scaling ORM Code Generation: Practical Patterns When Prisma Breaks at Hundreds of Models

Why this matters

A recent report showed Prisma's generator struggling when a schema contains hundreds of models. Large schemas are common in legacy systems, multi-tenant platforms, and projects that model many domain entities. When code generation becomes the bottleneck it slows developer feedback loops, CI pipelines, and deploys. This post collects practical patterns you can apply today to scale code generation reliably and keep developer ergonomics reasonable.

High-level approaches

  • Divide and conquer: split a monolithic schema into smaller, focused schemas or files.
  • Incremental generation: only regenerate what changed instead of the whole client.
  • Parallel/workers: generate artifacts concurrently for independent units.
  • Caching and CI optimizations: cache generated outputs and skip steps in CI when irrelevant files didn't change.
  • Streaming/AST-aware generators: parse once, emit many files to avoid re-parsing and memory pressure.

When to apply which pattern

  • Split schema when models are logically separable or owned by different teams.
  • Incremental generation when changes are small and frequent.
  • Parallel workers when generation tasks are independent and CPU-bound.
  • Caching for CI speedups and avoiding repeated work across branches/builds.

Actionable example: Split a Prisma schema into per-model files

Splitting your big schema into model-level files lets you run generators per file or compose only affected pieces. The example below extracts models from a schema.prisma into a directory of model files. This is a pragmatic bootstrap for a split strategy.

const fs = require('fs');
const readline = require('readline');
 
async function splitSchema(inputPath, outDir) {
  await fs.promises.mkdir(outDir, { recursive: true });
  const rl = readline.createInterface({ input: fs.createReadStream(inputPath), crlfDelay: Infinity });
  let buffer = '';
  let modelName = null;
 
  for await (const line of rl) {
    const m = line.match(/^\s*model\s+([A-Za-z0-9_]+)\s*\{/);
    if (m) {
      // write previous model
      if (modelName) {
        await fs.promises.writeFile(`${outDir}/${modelName}.prisma`, buffer);
      }
      modelName = m[1];
      buffer = line + '\n';
    } else {
      if (modelName) {
        buffer += line + '\n';
      }
    }
  }
 
  if (modelName) {
    await fs.promises.writeFile(`${outDir}/${modelName}.prisma`, buffer);
  }
}
 
splitSchema('schema.prisma', 'prisma-models').catch(err => { console.error(err); process.exit(1); });

Notes:

  • Keep a central file for generator and datasource blocks, and include/import model files from a build step if your toolchain supports it.
  • For Prisma specifically, you can assemble a temporary schema file in CI by concatenating the central blocks with only changed model files before running prisma generate.

Actionable example: Parallel generation for independent model artifacts

If your generator emits one file per model (TypeScript client types, CRUD services, OpenAPI fragments), you can parallelize across CPU cores. The pattern below spawns worker processes to run a generator command per model in parallel. It keeps the main process simple and avoids single-threaded bottlenecks in long-run tasks.

const { exec } = require('child_process');
const os = require('os');
 
function run(cmd) {
  return new Promise((resolve, reject) => {
    exec(cmd, (err, stdout, stderr) => {
      if (err) return reject({ err, stderr });
      resolve(stdout);
    });
  });
}
 
async function parallelGenerate(models) {
  const concurrency = Math.max(1, Math.floor(os.cpus().length * 0.75));
  const queue = [...models];
  const workers = new Array(concurrency).fill(null).map(async () => {
    while (queue.length) {
      const model = queue.shift();
      // call a script that generates artifacts for a single model
      await run(`node scripts/generate-for-model.js ${model}`);
    }
  });
  await Promise.all(workers);
}
 
parallelGenerate(['User','Post','Comment']).catch(console.error);

Practical tips:

  • Limit concurrency to avoid saturating DB connections or memory.
  • Use dedicated per-worker DB connections if generation queries the DB schema.

Incremental generation: detect changed models and regenerate only those

Integrate simple file-change detection in your build or CI. For Git-based flows, get the list of changed files and only run generation for those models.

  • Use git diff --name-only origin/main...HEAD in CI to find modified model files.
  • Combine with the split-schema approach so changed models correspond to files you can target directly.

CI caching and skipping

  • Cache generated clients/artifacts between runs using your CI cache (circleci, GitHub Actions cache, etc.).
  • On PRs that only change docs or frontend assets, skip backend-generation jobs with path filters.

Streaming AST parsing & memory

If your current generator builds a huge in-memory AST for the whole schema, consider a streaming or incremental AST approach: parse the file once and emit each model's node as you encounter it, or use a lightweight tokenizer that produces model blocks for downstream generators.

Trade-offs

  • Splitting schemas improves generation speed and ownership boundaries but adds complexity: you must assemble the runtime schema or ensure clients are consistent.
  • Incremental generation reduces work but can hide integration issues that only appear when generating everything together.
  • Parallel workers speed up wall time but increase resource usage and may create race conditions if generators write to the same file paths.
  • Streaming parsers reduce memory but can require more complex code than a single AST-based approach.

Quick checklist to implement today

  1. Split your schema into model files and keep generator/datasource blocks centralized.
  2. Add a small utility that builds a temporary composed schema for Prisma generate in CI only for changed models.
  3. Parallelize generation tasks at a safe concurrency level.
  4. Cache generated artifacts in CI and skip jobs when backend files aren't modified.
  5. Measure: add a simple timer around generation steps and record in CI artifacts to track regressions.

Conclusion

Large schemas need pragmatic engineering: split responsibilities, avoid monolithic generation passes, and prefer incremental/parallel workflows. Start with schema splitting and file-based incremental generation — they offer the best ROI with low upfront cost. Measure generation time and resource usage before and after changes. These patterns apply beyond Prisma: any code generator that parses and emits large outputs benefits from splitting, streaming, caching, and safe parallelism.

Was this helpful?

Share this post

Comments (0)

Want to join the conversation?

Log in or sign up to leave a comment and share your thoughts.

Log in to Comment