1

I have a numpy array A of shape (550,10). I have batch size of 100 i.e how much data rows I want from A. In each iteration I want to extract 100 rows from A. But when I reach last 50 rows, I want last 50 and first 50 rows from A.

I have a function like this:

def train(index, batch_size):

    if(batch_size + index < A.shape(0)):
          data_end_index = index + batch_size
          batch_data = A[index:batch_end_index,:]
    else:
          data_end_index = index + batch_size - A.shape(0) #550+100-600 = 50
          batch_data = A[500 to 549 and 0 to 49] # How to slice here ?

How to perform last step ?

Pawandeep Singh
  • 830
  • 1
  • 13
  • 25
  • 1
    Possible duplicate of [wrapping around slices in Python / numpy](https://stackoverflow.com/questions/17739543/wrapping-around-slices-in-python-numpy) – FlyingTeller Feb 09 '18 at 09:40

2 Answers2

5

you can try:

import numpy as np
data=np.random.rand(550,10)
batch_size=100

for index in range(0,data.shape[0],batch_size):
    batch=data[index:min(index+batch_size,data.shape[0]),:]
    print(batch.shape)

output:

(100, 10)
(100, 10)
(100, 10)
(100, 10)
(100, 10)
(50, 10)
riccardo nizzolo
  • 601
  • 1
  • 7
  • 21
  • To avoid overlaps between batches, I prefer `for index in range(0, data.shape[0], batch_size+1)` – Davis Apr 20 '21 at 20:41
1

stealing riccardo's example data, using numpy.split: 

data=np.random.rand(550,10)
batch_size=100

q, block_end = data.shape[0] // batch_size, q * batch_size

batch = np.split(data[:block_end], q) + [data[block_end:]]

[*map(np.shape, batch)]
Out[89]: [(100, 10), (100, 10), (100, 10), (100, 10), (100, 10), (50, 10)]
f5r5e5d
  • 3,656
  • 3
  • 14
  • 18