feat(multipart-parser): add streaming support

[Arlen22]

This PR adds streaming support to the multipart-parser.

• It separates the header and content in MultipartPart into two classes.
• It adds two options which easily allows streaming to be implemented.
• It also adds async and await to the relevant sections to allow back pressure.

The base class has the header related fields, and the sub class has the fields related to the content array. Both classes have their own append method, but in the base class it throws by default, so it must be set manually in onCreatePart. If append returns a promise, it is awaited, allowing back pressure.

export interface ParseMultipartOptions {
  useContentPart?: boolean // defaults to true
  onCreatePart?(part: MultipartPart): Promise<void> | void
  onEmitBytes?(part, chunk): Promise<void> | void
}

onCreatePart is called each time a part is created, and useContentPart determines whether to use the base class with just the header components, or the subclass, which includes the contents array and related fields. If useContentPart is false, onEmitBytes must be set, so the default is true.

This should fix

• form-data-parser loads file in memory instead of streaming #10763
• Cannot handle large files with multipart-parser #10728

I have not tested this code on a server (although I did implement it in a project I'm working on), and I have not written any unit tests.

[Copilot]

Pull Request Overview

This PR adds streaming support to the multipart-parser by refactoring the architecture to separate header and content handling, enabling backpressure control and streaming capabilities.

• Converts synchronous generators to async generators with await support for backpressure
• Introduces a two-class hierarchy separating MultipartPart (headers only) and MultipartContentPart (headers + content)
• Adds configuration options useContentPart and onCreatePart to control streaming behavior

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
packages/multipart-parser/src/lib/multipart.ts	Core implementation adding streaming options, async generator support, and class hierarchy refactoring
packages/multipart-parser/src/lib/multipart.node.ts	Updates Node.js wrapper to use async generator interface

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Arlen22]

I felt like setting the append method on the part was a little too hacky, so I changed it to accept a second callback.

const filestreams = new Map<MultipartPart, WritableStream>();
for await (let part of parseNodeMultipartStream(this.reader, {
  boundary,
  useContentPart: false,
  onCreatePart: async (part) => {
    // open a forwarding request to s3 or create a filestream object
    const filename = "some_filename";
    await someAsyncChecks();
    filestreams.set(part, fs.createWriteStream(filename));
  },
  onEmitBytes: async (part, chunk) => {
    await new Promise<void>((resolve) => {
      const filestream = filestreams.get(part);
      // write returns true if we can keep going, false if we need to wait for drain
      filestream!.write(chunk) ? resolve() : filestream!.once("drain", () => resolve());
    });
  }
})) {
  filestreams.get(part).end();
  filestreams.delete(part);
  await somethingElse();
}

[Copilot]

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Arlen22]

Most of the complaints in the last copilot review were junk.

[Nyoxis]

I’m curious why the content property was not changed to a ReadableStream? That might better align with the Web API style goals of this library.
I'm also wondering what the original reasons were for preferring buffered file data instead of streaming with ReadableStream.

[Arlen22]

I’m curious why the content property was not changed to a ReadableStream? That might better align with the Web API style goals of this library. I'm also wondering what the original reasons were for preferring buffered file data instead of streaming with ReadableStream.

I'm guessing it's because the stream has to be parsed as bytes, so that would just be extra boilerplate. If someone wants a ReadableStream, this PR makes that very simple.