Skip to content

feat(uploader): add chunked file upload - #1303

Open
lexasq wants to merge 10 commits into
developmentfrom
feat/chunked-upload
Open

lexasq wants to merge 10 commits into
developmentfrom
feat/chunked-upload

Conversation

@lexasq

@lexasq lexasq commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Adds opt-in chunked uploads through a new ChunkedFileUploader class. This is a fresh implementation on the current codebase, building on the idea and API from #977. Thanks @PauloPeres, who is credited as co-author.

Closes #880, closes #435.

Apps that don't use the new class are not affected. FileItem is unchanged. In FileUploader, form building and request setup move unchanged into two protected helpers (_buildFormData, _openRequest) that ChunkedFileUploader reuses; a new request spec passes on both the old and the new FileUploader.

Usage

const uploader = new ChunkedFileUploader({
  url: '/api/upload',
  chunkSize: 2 * 1024 * 1024, // 2 MB per request
});

// optional: send the remaining chunks to an upload id returned by the server
uploader.onSuccessChunk = (item, chunk, response) => {
  item.url = `/api/upload/${JSON.parse(response).id}`;
};

// optional: the app decides whether and when to retry
const retries = new Map<FileItem, number>();
uploader.onErrorItem = (item, response, status) => {
  const count = retries.get(item) ?? 0;
  if (status >= 500 && count < 3) {
    retries.set(item, count + 1);
    setTimeout(() => uploader.resumeItem(item), 1000);
  }
};

Behaviour

  • Drop-in: it extends FileUploader, so it works with ng2FileSelect / ng2FileDrop and all existing options and callbacks. Without chunkSize it behaves exactly like FileUploader.
  • Requests: multipart requests send each chunk as the file field, keeping the file's MIME type, preceded by chunkIndex and totalChunks fields. Their names can be changed with chunkIndexParam and totalChunksParam. With disableMultipart, raw chunks are sent with a Content-Range header, unless the app sets one itself.
  • Hooks:
    • onBeforeUploadChunk(item, chunk) runs before each chunk request.
    • onSuccessChunk(item, chunk, response, status, headers) runs after each successful chunk. Changing item.url, item.method or item.headers here applies to the next chunk.
    • onErrorChunk(item, chunk, response, status, headers) runs when a chunk fails. The item then fails as usual and no more chunks are sent.
  • Retries are up to the app. The library doesn't retry or delay anything itself; resumeItem(item) uploads a failed or cancelled item again, starting from the chunk that didn't complete. Failed items must stay in the queue, so retrying needs removeAfterUpload off. Each attempt reports the item again (onErrorItem, onCompleteItem, onCompleteAll).
  • Cancel: item.cancel() stops the remaining chunks, whether called during a request or from a hook. It has no effect once the last chunk has completed, and an item whose chunk failed stays failed.
  • Callbacks that throw: if a chunk callback throws, the item fails with status 0 instead of stalling the queue, and the error is rethrown.
  • Callbacks per file and per chunk: onSuccessItem, onErrorItem and onCompleteItem fire once per file. onBuildItemForm and uploader.response fire once per chunk request.
  • Progress is reported across the whole file.
  • Chunk size is read once per upload.

Testing

  • chunked-file-uploader.class.spec.ts (41 tests) and file-uploader-request.spec.ts (7 tests, also passing against the previous FileUploader). The existing FileUploader specs pass unchanged. Lint, build, the demo build and all 76 library tests pass.
  • Manually checked in the demo against a local server: multipart and raw mode, an app-driven retry with resumeItem after a 500, and cancel between chunks.

Alex Umanskiy and others added 2 commits October 1, 2026 17:02
Add opt-in chunked uploads to FileUploader. Setting `chunkSize` sends each
file as a sequence of requests, one per chunk:

- multipart mode sends the chunk as the file field plus `chunkIndex` and
  `totalChunks` fields (names configurable via `chunkIndexParam` /
  `totalChunksParam`)
- with `disableMultipart` the raw chunk is sent with a `Content-Range` header
- `chunkRetries` retries a failed chunk before the item fails
- new `onBeforeUploadChunk` / `onCompleteChunk` hooks; changing `item.url`,
  `item.method` or `item.headers` there targets the next chunk request
- `item.cancel()` stops the remaining chunks, also when called between chunks
- progress is reported across the whole file

Without `chunkSize` uploads behave exactly as before.

Thanks to @PauloPeres for the original implementation and API idea in #977.

Closes #880
Closes #435

Co-authored-by: Paulo <p.peresjr@gmail.com>
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Visit the preview URL for this PR (updated for commit ea46813):

https://ngx-file-upload--pr1303-feat-chunked-upload-vqu3xksn.web.app

(expires Thu, 08 Oct 2026 16:02:28 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: f15ad3fba241d6d58091ac579c27208d04d4562f

Alex Umanskiy added 8 commits October 1, 2026 17:22
…nt failures

- errors thrown while sending a later chunk or retry (e.g. from onBeforeUploadChunk / onCompleteChunk) ran outside FileItem.upload()'s try/catch and left the item and uploader stuck in isUploading; they now fail the item
- chunkRetries now only retries network errors, 408, 429 and 5xx; other 4xx responses fail immediately instead of repeating requests that can't succeed
- cancel from onBeforeUploadChunk now stops before the chunk is sent;
  cancelItem no longer throws when no request exists yet
- cancel from onCompleteChunk on the last chunk reports cancel
- chunks keep the file's MIME type
- retries wait chunkRetryDelay (default 1000ms, doubled per retry) or the
  server's Retry-After, and can be cancelled while waiting
- retries go through onBeforeUploadChunk with chunk.retry increased
- 501 and 505 are not retried
- fractional chunkSize is floored so slices and Content-Range stay valid
- empty files in raw mode send Content-Range: bytes */0
- a throwing hook on the first chunk reports error before complete, and an
  error after a chunk was sent detaches that request so the item is
  reported once
- document per-chunk onBuildItemForm/response, item.chunk and retry delay
- restore the original order for non-chunked uploads: the request is
  created before onBeforeUploadItem again; chunk requests are created
  before onBeforeUploadChunk
- cancel from any hook, including onBuildItemForm, stops before the chunk
  is sent; a failure after cancel is reported as cancel
- rename onCompleteChunk to onSuccessChunk, it only runs for successful
  chunks
- re-uploading an item starts from the url/method/headers it had before
  hooks changed them, unless the app changed them in between
- a throwing app hook while the item is already reported is rethrown
  instead of reporting the item twice
- keep a Content-Range header set by the app
- cap retry delays at 30 seconds
- remove excessive comments, update docs
- keep the original progress rounding for non-chunked uploads
- errors thrown by chunk callbacks fail the item with status 0 and are
  rethrown asynchronously so the app's error handler sees them; an error
  thrown while reporting the item no longer reports it twice
- replace the re-upload url logic with a plain snapshot: url, method and
  headers changed by chunk hooks are reset before onSuccessItem,
  onErrorItem or onCancelItem run, so values the app sets there survive
- read the chunk size once per upload so setOptions mid-upload can't skip
  bytes
- removeFromQueue ignores items no longer in the queue, so remove() from a
  chunk hook with removeAfterUpload doesn't leave the uploader stuck
…urable

- chunk retries no longer read the Retry-After header; they wait chunkRetryDelay, doubled per retry
- the 30 second cap is now the chunkMaxRetryDelay option (default 30000)
FileUploader and FileItem are back to their previous code, so apps that
don't opt in are unaffected. Chunking lives in a ChunkedFileUploader
subclass that works with the existing directives.

The library no longer retries, delays or restores item fields itself;
it reports what happened and lets the app decide:
- onBeforeUploadChunk / onSuccessChunk / onErrorChunk hooks
- resumeItem(item) continues a failed or cancelled upload from the
  chunk that didn't complete, so apps can implement their own retries
- getChunk(item) returns the current chunk

Removed: chunkRetries, chunkRetryDelay, chunkMaxRetryDelay, the
retryable status list, restoring url/method/headers, and the special
handling of errors thrown by callbacks.
- FileUploader's implementation moves to BaseFileUploader, with form
  building and request setup extracted into _buildFormData and
  _openRequest; FileUploader extends it unchanged
- ChunkedFileUploader extends BaseFileUploader and reuses both helpers
  instead of copying them

Fixes from review:
- resumeItem ignores items no longer in the queue (removeAfterUpload)
- cancel from onSuccessChunk resumes at the next chunk; on the last
  chunk it has no effect since the file is already uploaded
- a chunk callback that throws fails the item instead of stalling the
  queue
- docs: per-item retry counter, removeAfterUpload note, cancel and
  error behaviour
- ChunkedFileUploader extends FileUploader again, so it is a real drop-in
  (instanceof FileUploader); FileUploader keeps the two extracted helpers
  _buildFormData and _openRequest and is otherwise unchanged
- chunk fields are sent before the file so streaming servers can read them
- a throwing chunk callback fails the item only if it is still uploading,
  and the error is rethrown instead of swallowed
- tests: FileUploader request regression spec (passes on the previous
  FileUploader too), shared fake XHR, field order, rethrow and
  no-double-report cases
- docs: cancel wording, repeated callbacks per resume attempt, rethrow

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Does this library support chunk file upload feature Does chunked upload supported?

1 participant