Uppy - How do you upload to s3 via multipart? Using companion?

Question

https://uppy.io/docs/aws-s3-multipart/

Uppy multipart plugin sounds like exactly what I need but I can't see how to do the backend part of things. The impression I get is that I need to setup a companion to route the upload to S3 but can't find any details on setting up the companion for this.

I can see lots of references about using Companion to fetch external content but none on the multipart S3 uploading.

I neither see anywhere inside Uppy to provide AWS credentials which makes me think Companion even more.

But there are 4 steps to complete a multipart upload and I can't see how providing one companion url will help Uppy.

Thanks in advance to anyone who can help or jog me in the right direction.

Answer 1

Providing Uppy a companion URL makes it so that Uppy will fire off a series of requests to the-passed-url.com/s3/multipart. You then need to configure your server to handle these requests. Your server will be where your credentials are handled for AWS.

In short when you click the upload button in Uppy, this is what happens:

1. Uppy sends a post request to /s3/multipart to create/initiate the multipart upload.
2. Using the data returned from the previous request, Uppy will send a get request to /s3/multipart/{uploadId} to generate AWS S3 pre-signed URLs to use for uploading the parts.
3. Uppy will then upload the parts using the pre-signed URLs from the previous request.
4. Finally, Uppy will send a post request to /s3/multipart/{uploadId}/complete to complete the multipart upload.

I was able to accomplish this using Laravel/Vue. I don't know what your environment is but I've posted my solution which should help, especially if your server is using PHP.

Configuring Uppy to Use Multipart Uploads with Laravel/Vue

Answer 2

I am sharing code snippets for AWS S3 Multipart [github]

If you add Companion to the mix, your users will be able to select files from remote sources, such as Instagram, Google Drive, and Dropbox, bypassing the client (so a 5 GB video isn’t eating into your users’ data plans), and then uploaded to the final destination. Files are removed from Companion after an upload is complete, or after a reasonable timeout. Access tokens also don’t stick around for long, for security reasons.

Setup companion server:

1: Setup s3 configuration.

Uppy automatically generates the upload URL and puts the file in the uploads directory.

s3: {
      getKey: (req, filename) =>{
        return `uploads/${filename}`;
      },
      key: 'AWS KEY',
      secret: 'AWS SECRET',
      bucket: 'AWS BUCKET NAME',
    },

2: Support upload from a remote resource Uppy handles everything for us. We just need to provide a secret key and token from different remote resources like Instagram, drive, etc.

example: Drive upload

• Generate google key and secrete from google and add it to code
• Add redirect URL for authentication

3: Run node server locally

const fs = require('fs')
const path = require('path')
const rimraf = require('rimraf')
const companion = require('@uppy/companion')
const app = require('express')()

const DATA_DIR = path.join(__dirname, 'tmp')

app.use(require('cors')({
  origin: true,
  credentials: true,
}))
app.use(require('cookie-parser')())
app.use(require('body-parser').json())
app.use(require('express-session')({
  secret: 'hello planet',
}))

const options = {
  providerOptions: {
    drive: {
    key: 'YOUR GOOGLE DRIVE KEY',
    secret: 'YOUR GOOGLE DRIVE SECRET'
    },
    s3: {
      getKey: (req, filename) =>{
    return `uploads/${filename}`;
      } ,
      key: 'AWS KEY',
      secret: 'AWS SECRET',
      bucket: 'AWS BUCKET NAME',
    },
  },
  server: { host: 'localhost:3020' },
  filePath: DATA_DIR,
  secret: 'blah blah',
  debug: true,
}

try {
  fs.accessSync(DATA_DIR)
} catch (err) {
  fs.mkdirSync(DATA_DIR)
}
process.on('exit', () => {
  rimraf.sync(DATA_DIR)
})
app.use(companion.app(options))
// handle server errors
const server = app.listen(3020, () => {
  console.log('listening on port 3020')
})

companion.socket(server, options)

Setup client:

1: client HTML code: This code will allow upload from the drive, webcam, local, etc. You can customize it to support more remote places.

• Add companion URL as your above node server running URL(http://localhost:3020)

<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Uppy</title>
    <link href="https://releases.transloadit.com/uppy/v1.29.1/uppy.min.css" rel="stylesheet">
  </head>
  <body>
    <div id="drag-drop-area"></div>

    <script src="https://releases.transloadit.com/uppy/v1.29.1/uppy.min.js"></script>
    <script>

    Uppy.Core({
        debug: false,
        autoProceed: false,
        restrictions: {
        maxNumberOfFiles: 5,

        }
    }).
    use(Uppy.AwsS3Multipart, {
        limit: 4,
        companionUrl: 'http://localhost:3020'
    }).
    use(Uppy.Dashboard, {
        inline: true,
        showProgressDetails: true,
        showLinkToFileUploadResult: false,
        proudlyDisplayPoweredByUppy: false,
        target: '#drag-drop-area',
    }).use(Uppy.GoogleDrive, { target: Uppy.Dashboard, companionUrl: 'http://localhost:3020' })
    .use(Uppy.Url, { target: Uppy.Dashboard, companionUrl: 'http://localhost:3020' })
    .use(Uppy.Webcam, { target: Uppy.Dashboard, companionUrl: 'http://localhost:3020' });
    </script>
  </body>
</html>