PHP best performance to delete duplicate array values

Question

In my web application I have a function that is executed more than 100 times in a minute, and I'm trying to find a better performing approach.

public static function removeDuplicateFeeds($id, $feeds, $todo)
{
    $actionsHistory = ActionsHistory::whereAccountId($id)->whereActionName($todo)->get();
    if (!empty($actionsHistory)) {
        foreach ($actionsHistory as $history) {
            foreach ($feeds as $key => $feed) {
                if ($history->action_name == $feed['pk']) {
                    unset($feeds[$key]);
                }
            }
        }
    }
}

I want to remove all the elements from $feeds that are in $actionsHistory as well.

UPDATE:

in this test code first index of $feeds array as "pk":"7853740779" is stored on my database and after remove duplicate this item should be removed, but i have all of $feeds items into $filteredFeeds too

$userAccountSource = InstagramAccount::with('user', 'schedule')->get();
$feeds = [
    json_decode('{"pk":"7853740779","username":"teachkidss","full_name":"..."}'),
    json_decode('{"pk":"7853740709","username":"teachkidss","full_name":"..."}'),
    json_decode('{"pk":"7853740009","username":"teachkidss","full_name":"..."}')
];

$filteredFeeds = AnalyzeInstagramPageController::removeDuplicateFeeds($userAccountSource[0]->id, $feeds, 'like');

public function removeDuplicateFeeds($id, $feeds, $todo)
{
    $feeds = collect($feeds); // If $feeds is not already a collection
    $actionsHistory = ActionsHistory::whereAccountId($id)
        ->whereActionName($todo)
        ->whereIn('action_name', $feeds->pluck('pk')) // retrieves only duplicate records
        ->select('action_name')  // reducing the select improves performance
        ->get(); // Should return a eloquent collection instance
    if (!empty($actionsHistory)) {
        return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
    }
    return $feeds;
}

check out this answer: https://stackoverflow.com/a/28879677/8144595 — EvilsEmpire, Jul 21 '18 at 12:31
Maybe the best solution would be to improve the query so it returns only what you want — RiggsFolly, Jul 21 '18 at 14:05
It would also help people formulate an answer if they knew what sort of data is being returned, even if you just show us an example of the arrays being processed it would help — RiggsFolly, Jul 21 '18 at 14:06

Azeame · Answer 1 · 2018-07-21T14:12:56.573

Without knowing the number of feed or database records you'll have to test these for performance against your dataset and see if these are more performant.

public static function removeDuplicateFeeds($id, $feeds, $todo)
{
    $feeds = collect($feeds); // If $feeds is not already a collection
    $actionsHistory = ActionsHistory::whereAccountId($id)
        ->whereActionName($todo)
        ->select('action_name')  // reducing the select improves performance
        ->get(); // Should return an eloquent collection instance
    if (!empty($actionsHistory)) {
        return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
    }
    return $feeds;
}

or if your database query returns significantly more records than you have feeds you could try leveraging mysql's faster query speed instead of using php's slower array_filter/foreach speed.

public static function removeDuplicateFeeds($id, $feeds, $todo)
{
    $feeds = collect($feeds); // If $feeds is not already a collection
    $actionsHistory = ActionsHistory::whereAccountId($id)
        ->whereActionName($todo)
        ->whereIn('action_name', $feeds->pluck('pk')) // retrieves only duplicate records
        ->select('action_name')  // reducing the select improves performance
        ->get(); // Should return a eloquent collection instance
    if (!empty($actionsHistory)) {
        return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
    }
    return $feeds;
}

If either of these works, it would be good to know how much faster this was for you. Let us know. Good luck.

not sure that I can help you any further with the comparison other than to suggest that you look into using Redis instead of the database to query. But after the comparison are you using `ActionsHistory::create()` or another one at a time create/insert? If so, try a bulk insert like `DB::table('action_histories')->insert($filteredFeeds)`. — Azeame, Jul 21 '18 at 17:20

score 0 · Answer 2 · answered Jul 21 '18 at 17:01

0

You may try array_unique function from PHP. Source.

answered Jul 21 '18 at 17:01

HenonoaH

399
2
10

score 0 · Answer 3 · answered Jul 22 '18 at 08:16

0

You can use build in laravel unique() function from collection, see more here : https://laravel.com/docs/5.6/collections#method-unique

doing that you cove will maintain clean and elegant.

answered Jul 22 '18 at 08:16

fico7489

7,931
7
55
89

i need to merge two array or collection and get unlike items from them to single output which last result should be not the same in my `$feeds` variable – DolDurma Jul 22 '18 at 08:20

PHP best performance to delete duplicate array values

3 Answers3