Stack Overflow
Stack Overflow

Questions tagged [mini-batch]

Use mini-batch when working with neural networks and in particular with mini-batch stochastic gradient descent.

A mini-batch is a subset of the input data (batch) broken up into many smaller pieces (mini-batch) that are used to train the neural network.

References

84 questions
12
votes
1 answer

Efficient PyTorch DataLoader collate_fn function for inputs of various dimensions

I'm having trouble writing a custom collate_fn function for the PyTorch DataLoader class. I need the custom function because my inputs have different dimensions. I'm currently trying to write the baseline implementation of the Stanford MURA paper.…
user7213391
10
votes
4 answers

Tensorflow: Convolutions with different filter for each sample in the mini-batch

I would like to have a 2d convolution with a filter which depends on the sample in the mini-batch in tensorflow. Any ideas how one could do that, especially if the number of sample per mini-batch is not known? Concretely, I have input data inp of…
patapouf_ai
  • 17,605
  • 13
  • 92
  • 132
7
votes
1 answer

Zero predictions despite masking support for zero-padded mini batch LSTM training in keras

Problem Statement I’m training a many-to-many LSTM in keras with tensorflow backend (tf version 1.13.1) on tagged text sequences to predict the tag of each element in the sequence using pretrained GloVe embeddings. My training regime involves mini…
Des Grieux
  • 520
  • 1
  • 5
  • 31
5
votes
2 answers

Confused usage of dropout in mini-batch gradient descent

My question is in the end. An example CNN trained with mini-batch GD and used the dropout in the last fully-connected layer (line 60) as fc1 = tf.layers.dropout(fc1, rate=dropout, training=is_training) At first I thought the tf.layers.dropout or…
kbxu
  • 127
  • 1
  • 9
4
votes
1 answer

Is there a Python library where I can import a gradient descent function/method?

One way to do gradient descent in Python is to code it myself. However, given how popular a concept it is in machine learning, I was wondering if there is a Python library that I can import that gives me a gradient descent method (preferably…
Kevin Trinh
  • 55
  • 1
  • 5
4
votes
1 answer

Tensorflow: create minibatch from numpy array > 2 GB

I am trying to feed minibatches of numpy arrays to my model, but I'm stuck with batching. Using 'tf.train.shuffle_batch' raises an error because the 'images' array is larger than 2 GB. I tried to go around it and create placeholders, but when I try…
Dávid Papp
  • 97
  • 7
3
votes
1 answer

Does xarray.Dataset.to_array() load the array into memory and how efficiently sample mini batches from an xarray?

I am currently trying to load a big multi-dimensional array (>5 GB) into a python script. Since I use the array as training data for a machine learning model, it is important to efficiently load the data in mini batches but avoid loading the whole…
Peter
  • 31
  • 1
3
votes
1 answer

Masking and computing loss for a padded batch sent through an RNN with a linear output layer in pytorch

Although a typical use case, I can't find one simple and clear guide on what is the canonical way to compute loss on a padded minibatch in pytorch, when sent through an RNN. I think a canonical pipeline could be: 1) The pytorch RNN expects a padded…
user3435407
  • 1,019
  • 4
  • 15
  • 31
3
votes
1 answer

tensorflow.python.framework.errors_impl.InternalError: Dst tensor is not initialized

I am following this Link to implement a cDCGAN on my own dataset. My dataset contains almost 391510 images. The image size of my dataset is 64 whereas the MNIST used in this link is 28. My dataset has 2350 labels where as the MNIST dataset has…
Ammar Ul Hassan
  • 826
  • 1
  • 18
  • 48
3
votes
2 answers

How to update weights when using mini batches?

I am trying to implement mini batch training to my neural network instead of the "online" stochastic method of updating weights every training sample. I have developed a somewhat novice neural network in C whereby i can adjust the number of…
C Geeeee
  • 71
  • 5
3
votes
1 answer

C# CNTK Sparse (and others) using MinibatchSource

I just started using CNTK C# API and, so far, I have had no problem loading files if I write them in a way that I can read using full matrix readers, e.g., the MNIST tutorial example: MinibatchSource minibatchSourceExistModel =…
douglas125
  • 71
  • 1
  • 4
2
votes
2 answers

pytorch loss accumulated when using mini-batch

I am new to pytorch. May I ask what is the difference between adding 'loss.item()' or not? The following 2 parts of code: for epoch in range(epochs): trainingloss =0 for i in range(0,X.size()[1], batch_size): indices =…
G-09
  • 345
  • 2
  • 13
2
votes
0 answers

How to do minibatch gradient descent in sklearn?

Is it possible to perform minibatch gradient descent in sklearn for logistic regression? I know there is LogisticRegression model and SGDClassifier (which can use log loss function). However, LogisticRegression is fitted on whole dataset and…
Jan Musil
  • 508
  • 5
  • 15
2
votes
1 answer

Does TensorFlow optimizer minimize API implemented mini-batch?

Does Tensorflow minimize API for, say GradientDescentOptimizer implement mini-batch already when we feed the input tensor with a minibatch size of data? I was reading this bolg which indicated that the minibatch is not implemented in the minimize…
Wenlong.Wang
  • 33
  • 4
2
votes
0 answers

Why torch.nn package not supporting inputs that are a single sample

I am trying to understand deep-learning with pytorch. I read the pytorch tutorial: https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html, and its written: ''torch.nn only supports mini-batches. The entire torch.nn package only…
miki
  • 639
  • 2
  • 6
  • 16
1
2 3 4 5 6