Splitting One Big LLM Call Into Three Broke Everything Downstream, One Bug at a Time

The token limit that only shows up on long input

After fixing the audio-size limit on the transcription step (a separate bug, same episode), the pipeline got one step further and failed again — this time at the LLM call that picks the best moments from the transcript. Groq returned Request too large ... TPM: Limit 12000, Requested 21251. A seventy-minute transcript's raw text, even after merging Whisper's output into clean sentence-boundary chunks, is simply too many tokens for one call under that model's per-minute limit.

The obvious fix: send it in pieces

The transcript-parsing node already grouped Whisper's segments into sentence-boundary chunks. The fix extended that: group those chunks further into batches under a roughly 6000-token budget, and instead of returning one big blob of text, return one item per batch. n8n processes multiple input items through the next node automatically, one at a time — so the single downstream LLM node, completely unchanged, now just runs once per batch instead of once total. A seventy-minute episode turned into about three batches instead of one oversized request. A small delay between calls (65 seconds) was added too, since Groq's rate limit is a rolling account-wide window, not a per-request check — three requests fired back-to-back could still trip the same limit even though each one is individually small enough.

Splitting the input didn't just distribute the risk — it multiplied the assumptions

The very next node in the chain reconstructs each moment's absolute timestamp and other metadata by reaching back to the transcript-parsing node's output for the item it belongs to. That code had always been written as $input.item... — fine, even correct, when there was exactly one input item, because “the item” was never ambiguous. The moment there were three items instead of one, that same code had no way to know which batch a given LLM response actually belonged to.

The fix was to loop over every response explicitly and use n8n's own pairedItem lookup ($('Node').itemMatching(idx)) to trace each result back to its originating batch, instead of assuming there was only ever one. This is the kind of bug that a single-item pipeline can carry silently for months — it was never wrong before, because the situation that would expose it had never occurred.

Bug three: numbering that assumed there was only ever one group

With lineage tracking fixed, the pipeline ran further — and then rendered clips that silently overwrote each other on disk. Every batch's clip-numbering code started counting from 1 independently, with no shared counter across batches, because that code had also been written back when there was only ever one batch to number. Batch one's clip 3 and batch two's clip 3 both rendered to the exact same filename, and the second write quietly clobbered the first with no error from anything in the chain.

The fix: derive each clip's number from its batch index as well as its position within the batch (batchIndex * 100 + i + 1), so clip numbers across the whole episode never collide regardless of how many batches it took. The same reasoning applied to the fallback path used when an LLM response fails to parse — its fallback timestamps also had to be offset by the batch's own start time instead of assuming the video started at zero.

What finally counted as “done”

Each of the first two fixes looked complete in isolation — the pipeline ran further than it had before, with no error, which is exactly what makes a partial fix easy to mistake for a finished one. The episode wasn't confirmed genuinely working until a full re-run produced 15 correctly-numbered clips (three batches of five) with zero filename collisions and every clip's timestamp traceable back to the right source batch. Splitting one call into many isn't just a request-size change — it turns every downstream assumption that quietly depended on “there is only ever one of these” into a bug waiting for the next multi-item run to expose it.

Batching by tokens without a tokenizer

An exact token count needs the model's tokenizer, which isn't available in an n8n Code node. A rough estimate is enough for batching, as long as it errs on the safe side. For English text, about four characters per token is a common rule of thumb, and other languages often need more tokens per character. The batching code used a budget of about 6,000 tokens per group against a 12,000 TPM limit, which leaves room for the prompt, the answer and estimation error:

const BUDGET = 6000;
const est = s => Math.ceil(s.length / 3.5);   // deliberately pessimistic
const groups = [];
let cur = [], tokens = 0;
for (const chunk of $json.chunks) {
  const t = est(chunk.text);
  if (cur.length && tokens + t > BUDGET) { groups.push(cur); cur = []; tokens = 0; }
  cur.push(chunk); tokens += t;
}
if (cur.length) groups.push(cur);
return groups.map((g, i) => ({ json: { batchIndex: i, start: g[0].start, text: g.map(c => c.text).join(' ') } }));

Each batch carries its index and its start time, which the later fixes needed.

A checklist for any one-to-many change

Every bug in this article came from code that silently assumed there was only one item. Before splitting a single item into several, check every node downstream for these assumptions:

• References: $input.item, $json in a Code node set to run once for all items, and .first() anywhere.
• Counters and ids: anything numbered from 1 per run, such as clip numbers or file names.
• Offsets: timestamps, page numbers or positions that assumed the batch starts at zero.
• Aggregation: a step that expects one final result and now gets several, such as one Telegram summary per batch instead of one per run.
• Rate limits: several requests back to back may hit a per-minute limit even if each one is small.

How to test it properly

The first two fixes each looked done because the run got further without an error. What finally counted was checking the output itself: the number of clips, their file names, and that each timestamp pointed into the right part of the source video. For any change like this, decide before the test run what the correct output looks like, and compare against that, not against the absence of errors.

Frequently asked questions

How do I send a long transcript to an LLM in n8n without hitting the TPM limit?

Split it into batches under a token budget well below the limit, emit one item per batch, and let the LLM node run once per item, with a pause between calls if the limit is per minute.

Why did splitting the input break later nodes?

Downstream code assumed exactly one item. References, numbering and offsets that were correct for one item became wrong once there were several.

How do I trace each LLM response back to its batch?

In a Code node, loop over the items and use $('Node').itemMatching(i) to get the source item for each one, or carry a batchIndex field through the chain.

How do I avoid file name collisions between batches?

Derive numbers from both the batch index and the position within the batch, for example batchIndex * 100 + i + 1.