0

I have created a TFRecord dataset file consisting elements and their corresponding labels. I want to use it for training model on Colab using free TPU. I can load the TFRecord file and even run an iterator just to see the contents however, before the beginning of the epoch it throws following error-

UnimplementedError: From /job:worker/replica:0/task:0:
File system scheme '[local]' not implemented (file: '/content/gdrive/My Drive/data/encodeddata_inGZIP.tfrecord')
     [[{{node MultiDeviceIteratorGetNextFromShard}}]]
     [[RemoteCall]]
     [[IteratorGetNextAsOptional_1]]

In my understanding, it wants the TFRecord file on the TPU bucket, I don't know how to do that on Colab. How can one use a TFRecord file directly on Colab TPU?

Bob Smith
  • 36,107
  • 11
  • 98
  • 91
Rishabh Sahrawat
  • 2,437
  • 1
  • 15
  • 32

1 Answers1

1

You need to host it on Google Cloud Storage:

All input files and the model directory must use a cloud storage bucket path (gs://bucket-name/...), and this bucket must be accessible from the TPU server. Note that all data processing and model checkpointing is performed on the TPU server, not the local machine.

As mentioned on Google's troubleshooting page: https://cloud.google.com/tpu/docs/troubleshooting#cannot_use_local_filesystem

Hope this helps!

Michielver
  • 116
  • 4
  • Yes, thank you for your answer, I also posted it in Tensorflow on Github and the developers said so. Here is the [link](https://github.com/tensorflow/tensorflow/issues/32651). – Rishabh Sahrawat Nov 14 '19 at 10:16
  • Actually, the "usual" way of TPU + colab + tfrecord on GCS stop working recently, I am in the middle of investigating this. I got an error about cloud TPU service account is lacking permission for the bucket. Not sure if this is "real" since it all used to work fine. – kawingkelvin Jun 18 '22 at 20:36