java.io.FileNotFoundException (Read-only file system) // Uploading files to S3

Question

I am trying to create csv files from a list of maps and uploading them to S3 bucket through a lambda function. Following is the code:

public void createCSV(List<Map<String, AttributeValue>> changedRecords, Context context, String tableName)
            throws IOException {
        Calendar calendar = Calendar.getInstance();
        SimpleDateFormat formatter = new SimpleDateFormat("yyyyMMddHHmmss");
        String outputName = tableName + "_" + formatter.format(calendar.getTime()) + ".csv";

        List<String> headers = changedRecords.stream().flatMap(map -> map.keySet().stream()).distinct()
                .collect(Collectors.toList());

        try (FileWriter writer = new FileWriter(outputName, true);) {
            for (String string : headers) {
                writer.write(string);
                writer.write(",");
            }
            writer.write("\r\n");
            for (Map<String, AttributeValue> lmap : changedRecords) {
                for (Entry<String, AttributeValue> string2 : lmap.entrySet()) {
                    writer.write(string2.getValue().getS());
                    writer.write(",");
                }
                writer.write("\r\n");
            }
        }
        catch (Exception e) {
            e.printStackTrace();
        }
        s3.putObject(new PutObjectRequest("bucket_name", "data/" + outputName, outputName));
    }

Getting the following fileNotFound exception:

java.io.FileNotFoundException: data_20200227192207.csv (Read-only file system) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at java.io.FileOutputStream.(FileOutputStream.java:133) at java.io.FileWriter.(FileWriter.java:78) at com.amazonaws.lambda.demo.PLMLambda.createCSV(PLMLambda.java:84) at com.amazonaws.lambda.demo.PLMLambda.handleRequest(PLMLambda.java:54) at com.amazonaws.lambda.demo.PLMLambda.handleRequest(PLMLambda.java:1) at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:178) at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:906) at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:341) at lambdainternal.AWSLambda.(AWSLambda.java:63) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:114)

score 2 · Answer 1 · answered Feb 27 '20 at 21:11

2

Change the line:

try (FileWriter writer = new FileWriter(outputName, true);) {

to

try (FileWriter writer = new FileWriter("/tmp" + outputName, true);) {

In Lambda you can only write to the /tmp directory.

answered Feb 27 '20 at 21:11

stdunbar

16,263
11
31
53

Minor, but probably better to use `Paths.get("/tmp", outputName)` than string concatenation. – jarmod Feb 28 '20 at 13:38
Thanks. Does the file automatically gets deleted from this tmp directroy? – miserable Feb 28 '20 at 15:16
The a Lambda instance is re-used, the file is available in tmp directory. But AWS doesn't guarantee it,So do not assume the file is always there for re-used. If the lambda instance got terminated, the file will get deleted too. – Nghia Do Feb 28 '20 at 16:37

score 1 · Answer 2 · answered Feb 28 '20 at 13:28

1

If your CSV content is not so huge (like 4GB), you can simple use StringWriter instead of FileWriter with AWS Lambda, and then just put directly the string to S3.

answered Feb 28 '20 at 13:28

Nghia Do

2,588
2
17
31

java.io.FileNotFoundException (Read-only file system) // Uploading files to S3

2 Answers2