how can we have dynamic output file name in u-sql in azure data lake based on timestamp job is excuted

Question

how can we have dynamic output file name in u-sql in azure data lake based on timestamp when job is executed.Thanks for help.My code as below:

 OUTPUT @telDataResult
    TO 
    @"wasb://blobcontainer@blobstorage.blob.core.windows.net/**yyyymmdd**_TelDataOutput.Csv"

    USING Outputters.Csv();

If we run u-sql job again and again with same output file name which is static name then is job going to overwrite file data or is it going to give error "file already exists" or is it going to append data in same file.Please let me know asap. — user3714403, Jun 29 '16 at 12:36

score 6 · Accepted Answer · answered Jun 23 '16 at 19:11

This feature is currently in development but not available yet. Feel free to add your vote to the feature request: https://feedback.azure.com/forums/327234-data-lake/suggestions/10550388-support-dynamic-output-file-names-in-adla

Once it becomes available, you would do the inverse of the fileset syntax on EXTRACT. Let's assume you have a column that is called eventdate in your rowset @res. Then the following OUTPUT statement would generate the files:

OUTPUT @res
TO "/{eventdate:yyyy}/{eventdate:MM}/{eventdate:dd}/result.csv"
USING Outputters.Csv();

how can we have dynamic output file name in u-sql in azure data lake based on timestamp job is excuted

1 Answers1

Linked