ExcelJS: FATAL ERROR: Reached heap limit, JavaScript heap out of memory
ExcelJS keeps every row you add as objects until writeFile runs,
and the XML and zip are built from that object graph at the end. The crash is
not a leak. It is the library working as designed, on more rows than the
design allows for, and the fix is a different writer in the same package.
Why it happens
Each addRow creates a Row and one Cell
per column, each holding its value, its address, a style reference and a
pointer back to the worksheet. Nothing is written until writeFile
or writeBuffer, because the default writer walks the whole model
twice: once to build the shared-string and style tables, then again to emit
the sheet XML through the zip stream. All of it has to exist at once.
That works out at roughly 6 KB per row of four plain cells, or about 1.5 KB a
cell. Measured on Node 22.23 with ExcelJS 4.4.0, writing a string, a number,
a date string and a status:
10,000 rows 156 MB peak 0.4s
20,000 rows 218 MB peak 0.6s
50,000 rows 485 MB peak 1.3s
100,000 rows 757 MB peak 2.6s
200,000 rows 1,296 MB peak 5.1s
Node with ExcelJS loaded and nothing else is 71 MB, so the growth is real and
it is linear. What the crash looks like depends on where the heap limit is.
Node sizes its default heap from the machine's memory, which on this box is
2 GB, so a 200,000-row export squeaks through here and dies on a laptop with
less. Pinned to a 512 MB heap, the size of a modest container, 100,000 rows
finished and 200,000 produced the line everyone searches for:
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
Two things are worse than the number suggests. The addRow loop is
synchronous, so for those five seconds the event loop is blocked and every
other request on that process waits. And the crash is fatal: it is not an
exception you can catch and turn into a 500, it is the process gone, with
every other in-flight request on it.
Fix 1: raise the heap
node --max-old-space-size=4096 app.js is the fix everybody tries
first, and it works right up until it doesn't. Memory use grows linearly with
cells, so every fixed limit has a row count that exceeds it, and the limit you
pick becomes the size of export a single user can trigger on a shared process.
It is a reasonable choice for an export whose size you control, such as a
nightly report of a known table. It is the wrong choice for one that grows
with your customers' data, because the day it fails is the day one of them has
the most data, and the failure takes the process with it.
Fix 2: the streaming writer that is already installed
ExcelJS ships a second writer, stream.xlsx.WorkbookWriter, that
writes each row into the zip stream when you commit() it and lets
it go. The API is deliberately close to the one you have:
import ExcelJS from "exceljs";
const workbook = new ExcelJS.stream.xlsx.WorkbookWriter({
filename: "/tmp/invoices.xlsx", // or { stream: res } to write to the response
useSharedStrings: false, // the default; see below
useStyles: false, // true if you set any cell styles
});
const sheet = workbook.addWorksheet("Invoices");
sheet.columns = [
{ header: "Customer", key: "customer" },
{ header: "Amount", key: "amount" },
{ header: "Issued", key: "issued" },
{ header: "Status", key: "status" },
];
for await (const invoice of invoices) { // a cursor, not an array
sheet.addRow([invoice.customer, invoice.amount, invoice.issued, invoice.status]).commit();
}
await workbook.commit();
Same data, same machine:
Workbook 200,000 rows 1,296 MB 5.1s
WorkbookWriter 200,000 rows 158 MB 1.0s
WorkbookWriter 1,000,000 rows 344 MB 4.7s (finishes inside a 64 MB heap)
The 344 MB at a million rows is not heap. Run under
--max-old-space-size=64 it completes at the same peak, so the
growth is in the zip stream's buffers outside V8, and it is a fraction of a
kilobyte per row rather than six. That is the difference between an export
that scales and one that does not.
Leave useSharedStrings off. It defaults to
false, and turning it on because the files are smaller brings the problem
back: every distinct string is kept until the end to build the table. The
same million-row export went from 344 MB to 1,060 MB, and
died in a 128 MB heap where the inline-string version had finished in 64.
The trade is that committed rows are gone. You cannot go back and set a cell,
insert a row above, or compute a total from cells you have already written.
Column widths, merges and freeze panes go on before the first commit. Formulas
and conditional formatting work. Charts are not something ExcelJS produces at
all, in either writer.
Fix 3: move it off the request
Whichever writer you use, a large export should not run on the HTTP process.
The streaming writer's loop is still synchronous between awaits, and a
million-row file is seconds of CPU. Hand it to a worker thread or a queue, and
send the client a link rather than holding the socket open while it builds.
What we would do
If the export is a data table, use the streaming writer. That is the honest
answer and it costs an hour.
If you need charts, or the same export has to come out of two services
identically, or you would rather not own the memory
problem at all, that is what Jarrah is for. Post the rows and get back a
styled .xlsx built outside your process.
Write them as NDJSON: a header line describing the workbook, then one line per
row. Use node:http with a pipeline, not fetch, and
not because we prefer it:
import { request } from "node:http"; // https for the real thing
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
function* ndjson(invoices) {
yield JSON.stringify({ sheets: [{ name: "Invoices", columns: [
{ header: "Customer" }, { header: "Amount", style: "money" },
{ header: "Issued" }, { header: "Status" },
] }] }) + "\n";
for (const invoice of invoices) // a generator, or a cursor
yield JSON.stringify([invoice.customer, invoice.amount, invoice.issued, invoice.status]) + "\n";
}
const req = request("https://api.jarrah.sh/v1/jobs", {
method: "POST",
headers: {
authorization: `Bearer ${process.env.JARRAH_KEY}`,
"content-type": "application/x-ndjson",
},
}, (res) => { /* 202 with a job id: poll /v1/jobs/{id}, or take a webhook */ });
await pipeline(Readable.from(ndjson(invoices)), req);
pipeline stops reading the generator while the socket is full, so
the process never holds more than a few rows: 90 MB at 200,000
rows and 93 MB at a million, of which 71 MB is Node and ExcelJS being
loaded, and it finishes in a 32 MB heap.
The obvious version, fetch with a ReadableStream
body, does not do that. Node's fetch pulls the stream as fast as it yields,
with no backpressure from the socket, and the same million rows ran out of a
64 MB heap. It is the shape every tutorial shows, which is why it is worth
saying.
For small exports there is a synchronous endpoint that hands the file straight
back. See the quickstart. It has a per-plan cell
ceiling, so a 200,000-row sheet takes the streaming path above.
What each option costs you
200,000 rows x 4 columns, Node 22.23 with ExcelJS 4.4.0, a standalone script with no framework. Node with ExcelJS loaded is 71 MB before anything runs, and that is inside every figure. Memory and time are what your own process
spends: the numbers you can reproduce on your machine.
Jarrah's figures are what your process spends writing the rows into the request, measured against a local sink: 90 MB at 200,000 rows and 93 MB at a million, of which 71 MB is Node and ExcelJS sitting there. The workbook is then built on our side, which is a network round trip you are paying for and is not in that column.
Where this is the wrong choice. If your export is a plain data table, the streaming WorkbookWriter wins and it is not close. It is the same library you already have, it finishes 200,000 rows in a second inside your process, and it adds no network dependency to a request that currently has none. Jarrah earns its place when you need charts, which ExcelJS cannot produce, when the same export has to come out of a Node service and a Python one identically, or when you would rather not own the memory problem at all.