0

i'm looking for away to see if the query 'id' has a folder with the same id as name in the file system, i did it but it will slow down the drive in the future with lots of files

$query = Model::all();

if(Input::get('field') == 'true'){

   $filenames = scandir('img/folders');
   $query->whereIn('id', $filenames);
}

as you can see this will scan and get names of all folders inside the 'folders' directory and create an array with it, now my app is going to have hundreds of thousands of folders in the future and i would like to resolve it before it happens, thanks for further help

ps: other propositions to do it differently are welcome

adam
  • 174
  • 3
  • 12

1 Answers1

1

Do you have good reason to believe that scandir on a directory with a large number of folders will actually slow you down?

You can do your query like this:

if(Input::has('field')){

   $filenames = scandir('img/folders');

   $query = Model::whereIn('id', $filenames)->get();
}

Edit 1

You may find these links useful:

PHP: scandir() is too slow

Get the Files inside a directory

Edit 2

There are some really good suggestions in the links which you should be able to use for guidance to make your own implementation. As I see it, based on the links included from the first edit I made, your options are use DirectoryIterator, readdir or chunking with scandir.

This is a very basic way of doing it but I guess you could do something with readdir like this:

$ids = Model::lists('id');

$matches = [];

if($handle = opendir('path/to/folders'))
{
    while (($entry = readdir($handle)) !== false) 
    {
        if(count($ids) === 0)
        {
            break;
        }

        if ($entry != "." && $entry != "..") 
        {
            foreach ($ids as $key => $value) 
            {
                if($value === $entry)
                {
                    $matches[] = $entry;

                    unset($ids[$key]);
                }
            }
        }
    }

    closedir($handle);
}

return $matches;
Community
  • 1
  • 1
haakym
  • 12,050
  • 12
  • 70
  • 98
  • this code will generate the same query as mine, it would be great if i can only check the existence of that id inside the whereIn like so : whereIn('id',$thisquery->id) , do you know if this is possible ? – adam Jun 14 '15 at 20:21
  • I'm confused with what you're trying to do. Are you trying to return all entries in the db with the id that match your folder names? – haakym Jun 14 '15 at 20:54
  • Your query will need to hit the db twice. Mine will only once – haakym Jun 14 '15 at 20:55
  • just once because i grabbed the first query ($query->whereIn('id', $filenames)) and the query contains many conditional where clauses what i want is to check if the file with that id exists and then stop checking, without the need of putting all the thousands names in the RAM and then repeat this process every single request – adam Jun 14 '15 at 21:42
  • and to answer your last question : YES, but note that a query can have only one folder with that name or not, only one this is why im trying to avoid returning all the folders but instead check it while in the disk – adam Jun 14 '15 at 21:45
  • So you're saying: Using all ids in your DB check for a matching folder name BUT if you found all matches STOP because there will only ever be one match - right? – haakym Jun 15 '15 at 08:23
  • yes correct, do you know how to implement functions like readdir instead ? or another solution ? – adam Jun 15 '15 at 20:22
  • 1
    this code will not only check the existence but first hit the database and grab all the id's in an array charge it to RAM and then again check the id of every query if it exists in that array so it's basically slower, after deep looking, this is impossible in sql language, mysql can't check the filesystem while working with the tables, so you need to scan and charge the RAM, i will use cache, thanks (y) – adam Jun 16 '15 at 23:08