Optimize foreach loop in laravel

Question

I need bulk insert in subscriber table and subscriber field table in which data comes from CSV file.

The subscriber table need the only email and subscriber fields table need subscriber id from subscriber table, field id from field table and other CSV fields.

So, I write this code for this job and here a lot of foreach loop, that's why it takes a lot of time for execute when I insert a large CSV file.

Now, Any possible to optimize this code?

Here is my code :

$emails = array();
foreach ($data as $item) {
    array_push($emails, $item['email']);
}

$subscribers = Subscriber::select('email')->whereIn('email', $emails)->where('mail_list_id', $this->id)->get();

//check exists mail in database
$exists_mail = array();
foreach ($subscribers as $item) {
    $exists_mail[$item->email] = true;
}

//get subscriber unique key
$subscriber_uid = array();

//get all field value
$total_field = array();

//bulk insert in subscribers table

$subscriberBatch = array();
foreach ($data as $item) {
    if (!isset($exists_mail[$item['email']])) {
        $column['uid'] = uniqid();
        $column['mail_list_id'] = $this->id;
        $column['email'] = $item['email'];
        $column['status'] = Subscriber::STATUS_SUBSCRIBED;
        $column['subscription_type'] = Subscriber::SUBSCRIPTION_TYPE_IMPORTED;

        array_push($subscriberBatch, $column);
        array_push($subscriber_uid, $column['uid']);

        $fieldKey['uid'] = $column['uid'];
        foreach ($fields as $field) {
            $fieldKey[Str::lower($field['tag'])] = $item[Str::lower($field['tag'])];
        }

        array_push($total_field, $fieldKey);
    }
}

Subscriber::insert($subscriberBatch);

//bulk insert in subscriber fields table
$totalSubscriber = Subscriber::select('id')->whereIn('uid', $subscriber_uid)->get()->toArray();

foreach ($totalSubscriber as $index => $subscriber) {
    $total_field[$index]['subscriber_id'] =  $subscriber['id'];
}

$fieldBatch = array();
foreach ($total_field as $item) {
    foreach ($fields as $field) {
        $fieldColumn['subscriber_id'] = $item['subscriber_id'];
        $fieldColumn['field_id']      = $field['id'];
        $fieldColumn['value']         = $item[Str::lower($field['tag'])];
        array_push($fieldBatch, $fieldColumn);
    }
}

SubscriberField::insert($fieldBatch);

https://stackoverflow.com/questions/12702812/bulk-insertion-in-laravel-using-eloquent-orm — The Alpha, Feb 18 '20 at 12:25

score 0 · Answer 1 · answered Feb 18 '20 at 13:24

//set all email in a array
$emails = array();
foreach ($data as $item) {
    array_push($emails, $item['email']);
}

$exists_mail = array();

$subscribers = Subscriber::whereIn('email', $emails)
->where('mail_list_id', $this->id)
->pluck('email')
->toArray();

//get subscriber unique key
$subscriber_uid = array();

//get all field value
$total_field = array();

//bulk insert in subscribers table
$subscriberBatch = array();
foreach ($data as $item) {
    if(!in_array($item['email'], $subscribers)){
        $column['uid'] = uniqid();
        $column['mail_list_id'] = $this->id;
        $column['email'] = $item['email'];
        $column['status'] = Subscriber::STATUS_SUBSCRIBED;
        $column['subscription_type'] = Subscriber::SUBSCRIPTION_TYPE_IMPORTED;

        array_push($subscriberBatch, $column);
        array_push($subscriber_uid, $column['uid']);

        $fieldKey['uid'] = $column['uid'];
        foreach ($fields as $field) {
            $fieldKey[Str::lower($field['tag'])] = $item[Str::lower($field['tag'])];
        }

        array_push($total_field, $fieldKey);
    }
}

Subscriber::insert($subscriberBatch);

//bulk insert in subscriber fields table
$totalSubscriber = Subscriber::select('id')
->whereIn('uid', $subscriber_uid)
->get()
->toArray();

$fieldBatch = array();

foreach ($totalSubscriber as $index => $subscriber) {
    $total_field[$index]['subscriber_id'] =  $subscriber['id'];
}


foreach ($total_field as $item) {
    foreach ($fields as $field) {
        $fieldColumn['subscriber_id'] = $item['subscriber_id'];
        $fieldColumn['field_id']      = $field['id'];
        $fieldColumn['value']         = $item[Str::lower($field['tag'])];
        array_push($fieldBatch, $fieldColumn);
    }
}

SubscriberField::insert($fieldBatch);

I remove below code and use pluck

$subscribers = Subscriber::select('email')->whereIn('email', $emails)->where('mail_list_id', $this->id)->get();

//check exists mail in database
$exists_mail = array();
foreach ($subscribers as $item) {
    $exists_mail[$item->email] = true;
}

We more optimize and we can remove below code

foreach ($total_field as $item) {
    foreach ($fields as $field) {
        $fieldColumn['subscriber_id'] = $item['subscriber_id'];
        $fieldColumn['field_id']      = $field['id'];
        $fieldColumn['value']         = $item[Str::lower($field['tag'])];
        array_push($fieldBatch, $fieldColumn);

} }

Like:

//bulk insert in subscriber fields table
$totalSubscriber = Subscriber::select('id')
->whereIn('uid', $subscriber_uid)
->get()
->toArray();

$fieldBatch = array();

foreach ($totalSubscriber as $index => $subscriber) {
    //$total_field[$index]['subscriber_id'] =  $subscriber['id'];
    foreach ($fields as $field) {
        $fieldColumn['subscriber_id'] = $subscriber['id'];
        $fieldColumn['field_id']      = $field['id'];
        $fieldColumn['value']         = $item[Str::lower($field['tag'])];
        array_push($fieldBatch, $fieldColumn);
    }
}

But I don't know what type of data in $fields array if you provide field array the I will try more optimize.

Optimize foreach loop in laravel

1 Answers1