Laravel Excel queued exports: why ShouldQueue makes a large export slower, not lighter

Adding ShouldQueue to a Laravel Excel export is what the docs recommend for anything large, and it does move the work off the request. It does not make it smaller or faster. Measured on Laravel Excel 4, a 200,000-row export that takes 23 seconds synchronously takes 41 minutes queued, restarting the worker 174 times on the way. The mechanism is worth understanding before choosing a chunk size, because the chunk size is not the problem.

What a queued export actually does

Excel::queue() dispatches a chain: one job that creates an empty temporary spreadsheet, then one AppendQueryToSheet job per chunk of the query, then jobs to close the sheet and move the file to its disk. Each append job does what its name says, and the first step of appending is reopen(): PhpSpreadsheet's reader loads the whole temporary file so far into memory, the chunk's rows are added to it, and the whole thing is written back.

So chunk two pays for chunk one's rows, chunk two hundred pays for all one hundred and ninety-nine before it, and the total is quadratic in the number of rows. Watching the append job's duration as the file grows, 200,000 rows in 1,000-row chunks on a database queue:

  append job   1   0.10s
  append job 100  11s
  append job 200  24s
  sum of 200 append jobs: 2,263s  (37 minutes of the 41)
  the same export, not queued:   22.6s

Memory follows the same curve, and this is where the worker restarts come from. queue:work exits when its own memory passes --memory, which defaults to 128 MB, and the supervisor starts another. Each append holds a bigger file than the last, so the worker crosses the line every few chunks: 23 restarts to export 50,000 rows, 174 for 200,000, each one a fresh PHP boot and a log line, with the worker's own memory reading peaking at 608 MB. The chain survives the restarts, which is to its credit, so the export does finish. It is just forty-one minutes and a hundred and seventy-four restarts to produce a 5.7 MB file.

Fix 1: a bigger chunk, and what it costs

Fewer chunks means fewer reopens. WithCustomChunkSize at 10,000 rows took the same 200,000-row export from 41 minutes to 4.7 minutes, with 19 restarts instead of 174. It is the first thing to change and it is one method.

Fix 2: one job of your own that streams

The queue is the right place for a large export. Laravel Excel's chunked chain is not the right thing to put on it. A single job that walks the query with cursor() and writes rows through OpenSpout holds one row at a time and never reopens anything:

composer require openspout/openspout

namespace App\Jobs;

use App\Models\Invoice;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use OpenSpout\Common\Entity\Row;
use OpenSpout\Writer\XLSX\{Writer, Options};

class InvoiceExportJob implements ShouldQueue
{
    use Queueable;

    public function handle(): void
    {
        $writer = new Writer(new Options());
        $writer->openToFile(storage_path('app/private/invoices.xlsx'));
        $writer->addRow(Row::fromValues(['Customer', 'Amount', 'Issued', 'Status']));

        foreach (Invoice::query()->cursor() as $invoice) {
            $writer->addRow(Row::fromValues([
                $invoice->customer, (float) $invoice->amount, $invoice->issued_at, $invoice->status,
            ]));
        }

        $writer->close();
    }
}

  Laravel Excel, queued, 1,000-row chunks   200,000 rows   608 MB   41 min   174 restarts
  Laravel Excel, queued, 10,000-row chunks  200,000 rows   618 MB   4.7 min   19 restarts
  one job, OpenSpout + cursor()             200,000 rows    32 MB   10.1s      0 restarts

Thirty-two megabytes is Laravel booted plus one row. It finishes two hundred times faster than the default queued path and never troubles the worker's memory limit. Dispatch it, store the file, notify the user with a link, and the request that started it returned in milliseconds.

The trade is Laravel Excel's conveniences: WithHeadings, WithMapping, styling events, multiple sheets by class. OpenSpout does styles, merges and several sheets, but not formulas, charts or conditional formatting. For a data table you lose nothing. For a formatted report you lose the formatting.

Fix 3: keep the chain, for small exports

Below about 20,000 rows the quadratic cost is seconds, the restarts are few, and the formatting hooks may be worth it. If you keep ShouldQueue, set the chunk to a few thousand, raise --memory on the worker that runs exports so it is not restarting every third job, and put exports on their own queue so a slow one does not sit in front of your password-reset emails.

What we would do

If the export is a data table, write the OpenSpout job. That is the honest answer and it costs an hour.

If the formatting is more than OpenSpout does, or the same export has to come out of two services identically, or you would rather not own the export path at all, that is what Jarrah is for. The job streams the rows and gets back a styled .xlsx built outside your process:

use Illuminate\Support\Facades\Http;
use GuzzleHttp\Psr7\Utils;

$fh = fopen('php://temp/maxmemory:2097152', 'w+');     // 2 MB in memory, the rest on disk

fwrite($fh, json_encode(['sheets' => [[
    'name'    => 'Invoices',
    'columns' => [['header' => 'Customer'], ['header' => 'Amount', 'style' => 'money']],
]]]) . "\n");

foreach (Invoice::query()->cursor() as $invoice) {
    fwrite($fh, json_encode([$invoice->customer, (float) $invoice->amount]) . "\n");
}
rewind($fh);

$job = Http::withToken(config('services.jarrah.key'))
    ->withBody(Utils::streamFor($fh), 'application/x-ndjson')
    ->post('https://api.jarrah.sh/v1/jobs')
    ->json();                                           // poll /v1/jobs/{id}, or take a webhook

Measured from inside a queued job against a local sink: 28 MB at 200,000 rows, 3.3 seconds to hand the rows over, and nothing that grows with the data. The seekable php://temp is deliberate: Guzzle's PumpStream over a generator is the elegant version and it returned 400 on every attempt with the curl handler.

For small exports there is a synchronous endpoint that hands the file straight back. See the quickstart. It has a per-plan cell ceiling, so a 200,000-row sheet takes the streaming path above.