Is there a way to generate new array variables in a loop in PHP?

Question

I am looking for a way to create new arrays in a loop. Not the values, but the array variables. So far, it looks like it's impossible or complicated, or maybe I just haven't found the right way to do it.

For example, I have a dynamic amount of values I need to append to arrays.
Let's say it will be 200 000 values. I cannot assign all of these values to one array, for memory reasons on server, just skip this part.
I can assign a maximum amount of 50 000 values per one array.
This means, I will need to create 4 arrays to fit all the values in different arrays. But next time, I will not know how many values I need to process.

Is there a way to generate a required amount of arrays based on fixed capacity of each array and an amount of values?
Or an array must be declared manually and there is no workaround?

What I am trying to achieve is this:

$required_number_of_arrays = ceil(count($data)/50000);

for ($i = 1;$i <= $required_number_of_arrays;$i++) {

 $new_array$i = array();

 foreach ($data as $val) {
  $new_array$i[] = $val;
 }

}

// Created arrays: $new_array1, $new_array2, $new_array3

Answer 1

A possible way to do is to extend ArrayObject. You can build in limitation of how many values may be assigned, this means you need to build a class instead of $new_array$i = array();

However it might be better to look into generators, but Scuzzy beat me to that punchline.

The concept of generators is that with each yield, the previous reference is inaccessible unless you loop over it again. It will be in a way, overwritten unlike in arrays, where you can always traverse over previous indexes using $data[4].

This means you need to process the data directly. Storing the yielded data into a new array will negate its effects.

Fetching huge amounts of data is no issue with generators but one should know the concept of them before using them.

Answer 2

Based on your comments, it sounds like you don't need separate array variables. You can reuse the same one. When it gets to the max size, do your processing and reinitialize it:

$max_array_size = 50000;

$n = 1;
$new_array = [];

foreach ($data as $val) {
    $new_array[] = $val;

    if ($max_array_size == $n++) {
        // process $new_array however you need to, then empty it
        $new_array = [];
        $n = 1;
    }
}
if ($new_array) {
    // process the remainder if the  last bit is less than max size
}

Answer 3

You could create an array and use extract() to get variables from this array:

$required_number_of_arrays = ceil($data/50000);
$new_arrays = array();
for ($i = 1;$i <= $required_number_of_arrays;$i++) {
   $new_arrays["new_array$i"] = $data;
}
extract($new_arrays);

print_r($new_array1);
print_r($new_array2);
//...

Answer 4

I think in your case you have to create an array that holds all your generated arrays insight.

so first declare a variable before the loop.

$global_array = [];

insight the loop you can generate the name and fill that array.

$global_array["new_array$i"] = $val;

After the loop you can work with that array. But i think in the end that won't fix your memory limit problem. If fill 5 array with 200k entries it should be the same as filling one array of 200k the amount of data is the same. So it's possible that you run in both ways over the memory limit. If you can't define the limit it could be a problem.

ini_set('memory_limit', '-1');

So you can only prevent that problem in processing your values directly without saving something in an array. For example if you run a db query and process the values directly and save only the result.

You can try something like this:

foreach ($data as $key => $val) {
   $new_array$i[] = $val;
   unset($data[$key]);
}

Then your value is stored in a new array and you delete the value of the original data array. After 50k you have to create a new one.

Easier way use array_chunk to split your array into parts.

https://secure.php.net/manual/en/function.array-chunk.php

Answer 5

There's non need for multiple variables. If you want to process your data in chunks, so that you don't fill up memory, reuse the same variable. The previous contents of the variable will be garbage collected when you reassign it.

$chunk_size = 50000;
$number_of_chunks = ceil($data_size/$chunk_size);
for ($i = 0; $i < $data_size; $i += $chunk_size) {
    $new_array = array();
    foreach ($j = $i * $chunk_size; $j < min($j + chunk_size, $data_size); $j++) {
        $new_array[] = get_data_item($j);
    }
}

$new_array[$i] serves the same purpose as your proposed $new_array$i.

Answer 6

You could do something like this:

$required_number_of_arrays = ceil(count($data)/50000);
for ($i = 1;$i <= $required_number_of_arrays;$i++) {
 $array_name = "new_array_$i";
 $$array_name = [];
 foreach ($data as $val) {
  ${$array_name}[] = $val;
 }
}