0

So when using DataflowRunner, we are staging files to GCS using the filesToStage method, however this does not happen in DirectRunner. Is there a way to have DirectRunner stage files to GCS and use those files similar to the DataflowRunner perhaps by perhaps using ClassLoader or another method?

Chase
  • 267
  • 1
  • 6
  • 20

1 Answers1

1

No, direct runner simply runs locally, so it doesn't stage files to GCS, it just uses the local files to run the software. My best suggestion is to write a tool that looks for the files in two possible places, detecting if its running on dataflowrunner or directrunner by locating the files

Alex Amato
  • 1,591
  • 4
  • 19
  • 32