-1

An application running on the cloud (heroku) doesn't have a persistent local file system.

Getting Started on Heroku with Node.js says ...

Use a database or object storage instead of writing to your local filesystem

... and Using AWS S3 to Store Static Assets and File Uploads says how to configure S3 (i.e. create a bucket) -- but it doesn't say how to use it!

So if a Node.js application wants to use a bucket (to write and read data from fles), what API does it use?

  • Can it use a file system API (e.g. the fs package) -- for example is the S3 bucket mounted (visible to software to the application in the Heroku "Dyno") as a file system drive, named e.g. \data?
  • Or must the application use an Amazon-specific API, e.g. one of the Amazon S3 client library for Node.js which are mentioned in answers to this topic?
  • Or use Amazon's S3 REST API?

I'd mostly like to know whether I can access it via the fs package, i.e. whether it's mounted as a file system drive (with a name like /data so that e.g. I might read and write /data/hello.txt).

If it can be accessed as a drive, perhaps that's the simplest or most familiar API -- and the easiest to develop, e.g. because then the application can use the local file system (instead of S3) when it's running on a development machine, and just the same except use the /data drive when deployed to the cloud.

ChrisW
  • 54,973
  • 13
  • 116
  • 224
  • 1
    Did the [section of that same guide discussing file uploads](https://devcenter.heroku.com/articles/s3#file-uploads) not help? It talks about two general approaches and then links to articles about direct file uploads using Ruby, Node.js, Python, Java, and PHP. (To answer the question in your title, Heroku doesn't impose any particular library on you. Pick one.) – ChrisGPT was on strike Apr 27 '19 at 11:04
  • I think that section tells me how to let the server's clients (i.e. my application's users) upload to S3 instead of their uploads passing through the server on Heroku. It doesn't say how the server itself might do I/O using S3 (in lieu of its having a persistent local file system). – ChrisW Apr 27 '19 at 11:15
  • @Chris `Pick one.` If I picked "fs" is that supported? Is S3 mounted as a file system drive? – ChrisW Apr 27 '19 at 11:16

1 Answers1

1

AWS S3 is an object storage service using an http based API.

Can it use a file system API (e.g. the fs package) -- for example is the S3 bucket mounted (visible to software to the application in the Heroku "Dyno") as a file system drive, named e.g. \data?

It is not possible to mount a S3 bucket in your filesystem. You can still save the file in your file system, then asynchronously copy it to your bucket, but beware that Heroku uses an ephemeral storage, meaning it's only accessible from a single Dyno, and if this Dyno is stopped, restarted or moved, all files in the local system will be lost.

The recommended way, as you guessed, is to either use the S3 http API or aws-sdk. It is explained in more details here.

Anthony Garcia-Labiad
  • 3,531
  • 1
  • 26
  • 30
  • Thanks. I think that network storage on Azure is accessible as a network drive. I couldn't find documentation about that for Heroku+S3, and your answer explains why (i.e. because it isn't supported, doesn't exist). – ChrisW Apr 27 '19 at 11:49