1

I'm getting a 504 Gateway Timeout when I try to upload a large file in "chunks" using the PUT object resumable API endpoint in Autodesk forge.

https://forge.autodesk.com/en/docs/data/v2/reference/http/buckets-:bucketKey-objects-:objectName-resumable-PUT/

My console log output is below, showing the 504 response, along with the PUT request I send.

I've tried various methods, both using the functions in the forge-apis SDK in Node, (eg uploadChunk method from ObjectsAPI) and also written various functions using request, request-promise-native, to check if using await/async, promises vs callbacks can help solve the problem.

In all cases, I get the same response - a long pause in the log and then a 504 Gateway Timeout - which appears to be coming from the Forge end.

For context, my Node JS application sits on Heroku, and is triggered from a python script running locally on the client - so Heroku is the 'middleman' - authentication is passed from client-side via python to the node app and then passed on to Forge to authenticate. Other operations in the flow are successful so I'm confident authentication is working correctly.

Can anyone see anything from my logs below that could be causing the problem? Anybody else had similar 504 issues with chunked uploads of large files? (~130MB in this case)

The first log shows my console log output - I'm sending 5mb chunks at 1-second intervals, and sending the chunk as a Buffer object (bytes) sliced to match the Content-Range

2020-06-22T10:42:33.389835+00:00 app[web.1]: Ready to upload chunk...
2020-06-22T10:42:33.389896+00:00 app[web.1]: simulating waiting for 1000 milliseconds
2020-06-22T10:42:34.391034+00:00 app[web.1]: done waiting
2020-06-22T10:42:34.392893+00:00 app[web.1]: contentRange bytes 129999974-130056191/130056192
2020-06-22T10:42:34.393393+00:00 app[web.1]: requestParams {
2020-06-22T10:42:34.393394+00:00 app[web.1]: headers: {
2020-06-22T10:42:34.393395+00:00 app[web.1]: Authorization: 'Bearer eyJhbGciOiJIxxxxxxxxx',
2020-06-22T10:42:34.393396+00:00 app[web.1]: 'Content-Type': 'application/octet-stream',
2020-06-22T10:42:34.393396+00:00 app[web.1]: 'Content-Range': 'bytes 129999974-130056191/130056192',
2020-06-22T10:42:34.393397+00:00 app[web.1]: 'Content-Length': '130056192',
2020-06-22T10:42:34.393397+00:00 app[web.1]: 'Session-Id': '-75601742'
2020-06-22T10:42:34.393397+00:00 app[web.1]: },
2020-06-22T10:42:34.393398+00:00 app[web.1]: uri: 'https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable',
2020-06-22T10:42:34.393399+00:00 app[web.1]: url: 'https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable',
2020-06-22T10:42:34.393399+00:00 app[web.1]: method: 'PUT',
2020-06-22T10:42:34.393400+00:00 app[web.1]: body: <Buffer 6b b1 b4 e4 d7 f5 1e 59 a0 07 0c db 68 7d 9e 98 75 8e 5f fc e8 0c e3 78 a0 9c b9 59 3f e9 0f 28 e5 5a f0 eb 75 7b 68 16 3d 4c c1 ca 20 7a ba 67 f0 69 ... 56167 more bytes>,
2020-06-22T10:42:34.393400+00:00 app[web.1]: resolveWithFullResponse: true
2020-06-22T10:42:34.393400+00:00 app[web.1]: }
2020-06-22T10:42:34.393466+00:00 app[web.1]: Ready to upload chunk...

The response comes back like this for each chunk:

2020-06-22T10:43:34.443453+00:00 app[web.1]: Uploading 504 > GATEWAY_TIMEOUT
2020-06-22T10:43:34.443834+00:00 app[web.1]: err:  null
2020-06-22T10:43:34.443969+00:00 app[web.1]: res:  {
2020-06-22T10:43:34.443969+00:00 app[web.1]: ----"statusCode": 504,
2020-06-22T10:43:34.443970+00:00 app[web.1]: ----"body": "",
2020-06-22T10:43:34.443970+00:00 app[web.1]: ----"headers": {
2020-06-22T10:43:34.443971+00:00 app[web.1]: --------"content-length": "0",
2020-06-22T10:43:34.443971+00:00 app[web.1]: --------"connection": "Close"
2020-06-22T10:43:34.443971+00:00 app[web.1]: ----},
2020-06-22T10:43:34.443972+00:00 app[web.1]: ----"request": {
2020-06-22T10:43:34.443972+00:00 app[web.1]: --------"uri": {
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"protocol": "https:",
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"slashes": true,
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"auth": null,
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"host": "developer.api.autodesk.com",
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"port": 443,
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"hostname": "developer.api.autodesk.com",
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"hash": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"search": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"query": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"pathname": "/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable",
2020-06-22T10:43:34.443976+00:00 app[web.1]: ------------"path": "/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable",
2020-06-22T10:43:34.443976+00:00 app[web.1]: ------------"href": "https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable"
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------},
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------"method": "PUT",
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------"headers": {
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Authorization": "Bearer eyJhbGciOiJxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Type": "application/octet-stream",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Range": "bytes 129999974-130056191/130056192",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Length": "130056192",
2020-06-22T10:43:34.443979+00:00 app[web.1]: ------------"Session-Id": "-75601742"
2020-06-22T10:43:34.443979+00:00 app[web.1]: --------}
2020-06-22T10:43:34.443979+00:00 app[web.1]: ----}
2020-06-22T10:43:34.443979+00:00 app[web.1]: }
Ed Green
  • 33
  • 5

2 Answers2

1

Thanks to my colleague Luis Felipe Paris - we managed to resolve the problem, after looking at forge-server-utils as per Petr's advice.

By changing our uploadChunk function to use the module axios instead of request or request-promise - we found the 504 timeout error disappeared.

We were using the npm module request and request-promise (even though legacy / deprecated these days) as it supports piping streams from one place to another, and this was working with all other requests we were doing, except uploading a 'resumable' file in separate chunks.

We had to convert the file payload we were sending from a Buffer object to an ArrayBuffer, and slice it correctly to match the Content-Range of each chunk.

Some of the older forge examples use request - so I guess this might be useful for other users - if request gives you problems, try axios

Ed Green
  • 33
  • 5
0

EDIT: when using the resumable upload feature, make sure that the Content-Length request header specifies the length of just the chunk you're sending, not the entire size of the uploaded file.

The Forge APIs do have some rate throttling but in that case you should probably get a 4xx status code, not 504.

Can you try a couple of things for us?

And if you're still unable to upload the file, could you please share it with us (confidentially) via forge (dot) help (at) autodesk (dot) com? We would be happy to investigate it on our end.

Petr Broz
  • 8,891
  • 2
  • 15
  • 24
  • Hi Petr, I'm glad and honoured to have your help! You may remember this from the Forge Accelerator London - we are working on transferring files from BIM360 Teams to BIM 360 Docs... our current challenge is to handle the CompositeDesign files - which are essentially Revit host files which contained linked rvt files, and appear as .zip when we migrate them using our transfer app. – Ed Green Jun 23 '20 at 10:37
  • Our current challenge is to handle the CompositeDesign files - which are essentially Revit host files which contained linked rvt files, and appear as .zip in 360 Docs when we migrate them using our transfer app. Currently our node app is unzipping the file, and then uploading the unzipped .rvt file to BIM360, and creating a new 'version' of the corresponding file - so it's slightly more tricky than just uploading a file by itself... – Ed Green Jun 23 '20 at 10:44
  • We do get the same 504 error when we test locally, without heroku (we also thought it might be something to do with Heroku talking to Forge) I've been looking at the forge-server-utils methods - they seem to be sending the same request information etc... Same 504 occurs with several different files, so we are pretty sure its not a problem with the specific file Does any of that info shed any more light? – Ed Green Jun 23 '20 at 10:44
  • Hi Ed, yeah I remember :) The issue you're seeing is strange. I remember uploading files over 100MB without any problems. Have you tried uploading the file with the vscode extension? It uses the resumable upload by default, so just to rule out any potential issues in the uploading logic. And have you tried smaller chunk sizes? – Petr Broz Jun 23 '20 at 10:58
  • Hi Petr, sure, we will try with the vscode extension to try to debug the steps.. currently our chunk size is 5MB so I think should be small enough? (2MB is the minimum if I recall correctly) Meanwhile if you can think of any other reason why we might get a 504, let me know - it's as if the Forge server does not answer, perhaps we are missing something in the request? (you can see what we are sending in my first screenshot) – Ed Green Jun 23 '20 at 11:31
  • I'll ping the engineering team. I can't think of any other reason why the server would keep returning a 504. Yep, that's why I wanted to check some of the existing solutions with the resumable upload, to make sure that it's not some missing configuration on the client side (causing the Forge servers to stall instead of returning a proper error code). – Petr Broz Jun 23 '20 at 11:59
  • Ed, so the stalling seems to be related to the way you're setting the `Content-Length` header. Instead of specifying the size of the individual chunk, your code seems to be setting the header to the entire 130MB, which makes the Forge servers wait for the rest of the data after the 5MB you actually send. – Petr Broz Jun 23 '20 at 12:21
  • Hi Petr @petr-broz, thanks for the update - I saw it after posting the answer below... we did also try setting the `Content-Length` in the header to match the chunk length - with the same 504 error.. moving from `request` to `axios` seems to resolve it... not sure exactly why though! – Ed Green Jun 25 '20 at 12:21