0

This is my code about loading data from tfrecord:

def read_tfrecord(tfrecord, epochs, batch_size):

    dataset = tf.data.TFRecordDataset(tfrecord)

    def parse(record):
        features = {
            "image": tf.io.FixedLenFeature([], tf.string),
            "target": tf.io.FixedLenFeature([], tf.int64)
        }
        example = tf.io.parse_single_example(record, features)
        image = decode_image(example["image"])
        label = tf.cast(example["target"], tf.int32)
        return image, label

    dataset = dataset.map(parse)
    dataset = dataset.shuffle(buffer_size=10000)        
    dataset = dataset.prefetch(buffer_size=batch_size)  #
    dataset = dataset.batch(batch_size, drop_remainder=True)
    dataset = dataset.repeat(epochs)

    return dataset


x_train, y_train = read_tfrecord(tfrecord=train_files, epochs=EPOCHS, batch_size=BATCH_SIZE)

I got the following error:

ValueError: too many values to unpack (expected 2)

My question is:

How to unpack the data from dataset?

qinlong
  • 731
  • 8
  • 19

3 Answers3

1
dataset = read_tfrecord(tfrecord=train_files, epochs=EPOCHS, batch_size=BATCH_SIZE)
# the above should return an iterator
for x, y in dataset:
    print(x)
    print(y)
    # now the unpacking parsing happens
convolutionBoy
  • 811
  • 5
  • 18
1

You can try this solution:

dataset = read_tfrecord(tfrecord=train_files, epochs=EPOCHS, batch_size=BATCH_SIZE)

iterator = iter(dataset)

x, y = next(iterator)
stlaza
  • 79
  • 5
0

TensorFlow's get_single_element() is finally around which can be used to unpack datasets.

This avoids the need of generating and using an iterator using .map() or iter() (which could be costly for big datasets).

get_single_element() returns a tensor (or a tuple or dict of tensors) encapsulating all the members of the dataset. We need to pass all the members of the dataset batched into a single element.

This can be used to get features as a tensor-array, or features and labels as a tuple or dictionary (of tensor-arrays) depending upon how the original dataset was created.

Check this answer on SO for an example that unpacks features and labels into a tuple of tensor-arrays.

manisar
  • 103
  • 1
  • 7