Split array into equal sized windows

Question

I am trying to split an numpy.array of length 40 into smaller, equal-sized numpy.arrays, in which the number of the smaller arrays is given by the user. It is allowed to have some overlap between the smaller arrays, as situations can occur where the full length is only divisible by the splits given some form of overlap of the smaller arrays.

If I had an array np.array([range(40)]) And I had to split it into 37 sub arrays, the list of subarrays should be like this:

[1, 2, 3], [3, 4, 5], [5, 6, 7], ... [38, 39, 40]

I tried using numpy.split but this only works when the length is divisible by the size, and numpy.array_split generates uneven sizes.

Example using numpy.split

>> import numpy as np
>>> a = np.random.randint(6,size=(40))
>>> b = np.split(a,37)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/shape_base.py", line 508, in split
    'array split does not result in an equal division')
ValueError: array split does not result in an equal division

And with numpy.array_split

>>> a = np.random.randint(5,size=(40))
>>> b = np.array_split(a,37)
>>> print len(b)
37
>>> print b[0].shape
(2,)
>>> print b[3].shape
(1,)
>>> print b[5].shape
(1,)
>>> print b[6].shape
(1,)
>>> print b[30].shape
(1,)
>>>

numpy.array_split don't equally divide them.

Any solution?

On the contrary: those functions do exactly what you say they don't. Show us the output you get and the output you want. — Arya McCarthy, Apr 10 '17 at 01:22
`array_split`'s documentation says that it handles uneven splits; `split` is designed to throw an error. — Arya McCarthy, Apr 10 '17 at 01:27

score 3 · Accepted Answer · edited May 23 '17 at 10:30

3

What you're describing is called a (sliding) window, not a split.

See this answer: https://stackoverflow.com/a/15722507/7802200

What you want is to use the window_stack function developed there with a width of len(a) - n_splits + 1.

edited May 23 '17 at 10:30

Community

1
1

answered Apr 10 '17 at 02:04

Arya McCarthy

8,554
4
34
56

ahh. that make sense.. sliding windows.. then i just need a way i can determine the stride.. such that it match with the number of sub arrays i want – Somthin Apr 10 '17 at 02:10
I think I address that in the last part of my answer. In your question, you said an array of 40 with 37 subgroups should get you a width of 3, but it actually needs to be 4. It's `len(a) - n_groups + 1`. – Arya McCarthy Apr 10 '17 at 02:19

Matthew Ciaramitaro · Answer 2 · 2017-04-10T01:52:38.940

0

numpy.split() should generate equal sized sub arrays of exactly the size specified in the parameter indices_or_sections which is the second input of the function. How have you tried using the split functions? If you want to split the array arrinto 4 subarrays of size 5, use

numpy.split(arr,4)

The reason your example failed is that the array size must be divisible by the number of arrays you want. Which makes sense, because you can only split into equal sizes when the number of elements divided by the number of subarrays is an integer.

Now that you've clarified your question, I think there is no simple solution. It is easy to divide the array into equally sized groups, but if the user requests 37 equally sized groups from a set of 40 with overlap allowed, there is more than one way to do this, and you have not given us criteria to determine such methods as to help you. There won't be any easy numpy function for the task you are looking for, you'll need a search algorithm (DFS?) to figure out how to divide the set into 37 (or whatever is requested) equally sized groups.

edited Apr 10 '17 at 01:52

answered Apr 10 '17 at 01:32

Matthew Ciaramitaro

1,184
1
13
27

I don't want to specify the size of the array.. I only want to provide it the number of subarrays I want. – Somthin Apr 10 '17 at 01:36
And about not being dividable. Overlapping is allowed. So it should be dividable. – Somthin Apr 10 '17 at 01:38
I never said you'd need to specify the size of the array, And I see the part about overlap now and think you need a searching algorithm for this. – Matthew Ciaramitaro Apr 10 '17 at 01:55
Try to build N subarrays by using an algorithm that first chooses all arrays of size 1, then when it fails to produce N, it increases the length by making all overlapping size 2, then make them not overlap one at a time, testing for a size of N, if that fails go to size 3 and continue until size N at which you raise an exception. Of course there would be ways to make that much more efficient but I won't go into that. DFS links [here](https://www.tutorialspoint.com/data_structures_algorithms/depth_first_traversal.htm) and also [here](https://www.cs.usfca.edu/~galles/visualization/DFS.html) – Matthew Ciaramitaro Apr 10 '17 at 02:06

Walid Bousseta · Answer 3 · 2021-11-04T10:25:58.527

this is another way if you want to split the array with a windowing size

from numpy.lib.stride_tricks import sliding_window_view

def unfold(array, size, step):
    """
    params:
       array: 1d array 
       size : size of each chunk
       step : windowing size
    """
    assert array.ndim == 1, f"Not implemented yet, expecting 1d array but got {array.ndim}d"
    return sliding_window_view(array, window_shape=size)[::step]

find more here

Split array into equal sized windows

3 Answers3

Linked