divide array into smaller parts

Question

I would like to divide a large byte array into smaller chunks (say 64 bytes). Please help me with this.

Answer 1

Damian Vash's first method (the one using Arrays.copyOfRange()) adds zeros to the end of the last chunk if the input is not exactly a multiple of chunksize.

You might want to use this instead:

public static List<byte[]> divideArray(byte[] source, int chunksize) {

    List<byte[]> result = new ArrayList<byte[]>();
    int start = 0;
    while (start < source.length) {
        int end = Math.min(source.length, start + chunksize);
        result.add(Arrays.copyOfRange(source, start, end));
        start += chunksize;
    }

    return result;
}

and in case it's useful, the same thing using ArrayList's:

  public static List<List<String>> divideList(List<String> source, int chunksize) {
    List<List<String>> result = new ArrayList<List<String>>();
    int start = 0;
    while (start < source.size()) {
      int end = Math.min(source.size(), start + chunksize);
      result.add(source.subList(start, end));
      start += chunksize;
    }
    return result;
  }

Answer 2

You can use the method Arrays.copyOfRange(original, from, to)

 public static byte[][] divideArray(byte[] source, int chunksize) {


        byte[][] ret = new byte[(int)Math.ceil(source.length / (double)chunksize)][chunksize];

        int start = 0;

        for(int i = 0; i < ret.length; i++) {
            ret[i] = Arrays.copyOfRange(source,start, start + chunksize);
            start += chunksize ;
        }

        return ret;
    }

Or You can use as Max suggested the System.arraycopy

public static byte[][] divideArray(byte[] source, int chunksize) {


        byte[][] ret = new byte[(int)Math.ceil(source.length / (double)chunksize)][chunksize];

        int start = 0;

        for(int i = 0; i < ret.length; i++) {
            if(start + chunksize > source.length) {
                System.arraycopy(source, start, ret[i], 0, source.length - start);
            } else {
                System.arraycopy(source, start, ret[i], 0, chunksize);
            }
            start += chunksize ;
        }


        return ret;
    }

Answer 3

If you are looking save some memory, a slight modification to Damian Vash's answer would help (in this case any remaining chunk is not allocated a complete 64 byte block size, as well...)

private byte[][] splitChunks(byte[] source)
{
    byte[][] ret = new byte[(int)Math.ceil(source.length / (double)CHUNK_SIZE)][];
    int start = 0;
    for(int i = 0; i < ret.length; i++) {
        if(start + CHUNK_SIZE > source.length) {
            ret[i] = new byte[source.length-start];
            System.arraycopy(source, start, ret[i], 0, source.length - start);
        } 
        else {
            ret[i] = new byte[CHUNK_SIZE];
            System.arraycopy(source, start, ret[i], 0, CHUNK_SIZE);
        }
        start += CHUNK_SIZE ;
    }
    return ret;
}

Answer 4

Well, System.arraycopy(src, fromPos, dest, toPos, length) is generally considered faster than Arrays.copyOfRange.

byte[] source = ...read it from somewhere...;
byte[] newArray = new byte[64];
System.arraycopy(source, 0, newArray, 0, 64);

Answer 5

You have two choices:

• System.arraycopy(...)
• Array.copyOfRange(...)

both of them work the same way but while first one only manages copy, second one is meant to be used to allocate the new chunk at the same time.

I benchmarked them with a result that System.arraycopy is faster if you manage to allocate chunks all together before splitting your array but slightly slower if you allocate them whle you copy: in this case you should use Array.copyOfRange.

Answer 6

See Arrays.copyOfRange for help. You could use this in a loop to split your array into several smaller chunks.

Answer 7

This is another possible way of using Arrays.copyOfRange to divide a byte array in chunks (splitting the "data" in blockSize chunks):

byte[] data = { 2, 3, 5, 7, 8, 9, 11, 12, 13 };
// Block size in bytes (default: 64k)
int blockSize = 64 * 1024;
int blockCount = (data.length + blockSize - 1) / blockSize;

byte[] range;

try {
  
    for (int i = 1; i < blockCount; i++) {
        int idx = (i - 1) * blockSize;
        range = Arrays.copyOfRange(data, idx, idx + blockSize);
        System.out.println("Chunk " + i + ": " Arrays.toString(range));
    }

} finally {
    
    // Last chunk
    int end = -1;
    if (data.length % blockSize == 0) {
            end = data.length;
    } else {
            end = data.length % blockSize + blockSize * (blockCount - 1);
    }
        
    range = Arrays.copyOfRange(data, (blockCount - 1) * blockSize, end);

    System.out.println("Chunk " + blockCount + ": " Arrays.toString(range));
}

Answer 8

This will do...

    byte[] source = new byte[2048];
    byte[] target = new byte[1024];  

// fill source with some data...

    Array.Copy(source, buffer, 1024);