gcs-browser-upload-stream

gcs-browser-upload

Chunked, pausable, recoverable uploading to Google Cloud Storage directly from the browser.

How does it work?

1. User selects a file
2. File + a Google Cloud Storage resumable upload URL are given to gcs-browser-upload
3. File is read in chunks
4. A checksum of each chunk is stored in localStorage once succesfully uploaded
5. If the page is closed and re-opened for some reason, the upload can be resumed by passing the same file and URL back to gcs-browser-upload. The file will be validated against the stored chunk checksums to work out if the file is the same and where to resume from.
6. Once the resume index has been found, gcs-browser-upload will continue uploading from where it left off.
7. At any time, the pause method can be called to delay uploading the remaining chunks. The current chunk will be finished. unpause can then be used to continue uploading the remaining chunks.

Example

import Upload from 'gcs-browser-upload'

let input = document.getElementById('fileInput')
let pause = document.getElementbyId('pause')
let unpause = document.getElementbyId('unpause')
let upload = null

input.addEventListener('change', async () => {
  upload = new Upload({
    id: 'foo',
    url: 'https://www.googleapis.com/..../....',
    file: input.files[0],
    onChunkUpload: (info) => {
      console.log('Chunk uploaded', info)
    }
  })

  try {
    await upload.start()
    console.log('Upload complete!')
  } catch (e) {
    console.log('Upload failed!', e)
  } finally () {
    upload = null
  }
})

pause.addEventListener('click', () => {
  if (upload) {
    upload.pause()
  }
})

unpause.addEventListener('click', () => {
  if (upload) {
    upload.unpause()
  }
})

Config

{
  id: null, // required - a unique ID for the upload 
  url: null, // required - GCS resumable URL
  file: null, // required - instance of File
  chunkSize: 262144, // optional - chunk size must be a multiple of 262144
  storage: window.localStorage, // optional - storage mechanism used to persist chunk meta data
  contentType: 'text/plain', // optional - content type of the file being uploaded
  onChunkUpload: () => {} // optional - a function that will be called with progress information
}

Handling errors

Various errors are thrown if something goes wrong during uploading. See src/errors.js for the different types. These are exported as a property on the Upload class.

Developing

make bootstrap     // install dependencies
make test          // run tests
make test-watch    // continuously run tests

Want to work on this for your day job?

This project was created by the Engineering team at Qubit. As we use open source libraries, we make our projects public where possible.

We’re currently looking to grow our team, so if you’re a JavaScript engineer and keen on ES2016 React+Redux applications and Node micro services, why not get in touch? Work with like minded engineers in an environment that has fantastic perks, including an annual ski trip, yoga, a competitive foosball league, and copious amounts of yogurt.

Find more details on our Engineering site. Don’t have an up to date CV? Just link us your Github profile! Better yet, send us a pull request that improves this project.