How can I search all files in a directory for a list of strings(numbers)?

Question

I have a list of numbers in a csv formatted like this:

I also have a directory with 100+ documents. I need to check each document for each string in the csv. If one of the numbers is in one of the documents, it needs to move it to a different file. This is what I have so far:

$files = Get-ChildItem "\\wh1-app\SPS EDI\ARCHIVE IN\*.txt"


foreach ($file in $files) {

    $file = Get-Content $file

    $pos = Import-Csv -Path C:\Users\ttilton\Desktop\pos.csv -OutVariable string -ErrorAction SilentlyContinue

    foreach ($po in $pos) {

       $containsWord = $file | %{$_ -match $po.po}

        if ($containsWord -contains $true){ 

            write-host $file + " is " + $po.po

            #Copy-Item $file C:\Users\ttilton\Desktop\pos\

        }
    }

But it doesn't seem to be working. It's running without error, but its not really doing anything.

Any ideas? Thanks!!

Change ErrorAction to Stop so you can see possible exceptions occurring. — C. Helling, Oct 12 '17 at 18:30
The `Copy-Item` will not be executed, as the `#` turns the whole line into a comment. — Manuel Batsching, Oct 12 '17 at 18:31
@C.Helling As far as I understand the code `$containsWord` is an array of boolean values. With your suggestion the ìf` clause would always be true. — Manuel Batsching, Oct 12 '17 at 18:32
@ManuelBatsching Valid point, I've amended my comment. Still unsure what `$po.po` is supposed to be though. — C. Helling, Oct 12 '17 at 18:34
@ManuelBatsching I know it won't be executed, I just wanted to check that it was working with the write-host first, but nothing is printing out. and I KNOW that there are matches in that directory. — ty Til, Oct 12 '17 at 18:34
Reusing `$file` for both the name and the content is confusing, but overall I can't see anything wrong. Run it in the PowerShell ISE debugger - does it find any *.txt files? Does Import-Csv actually import any data? Does `$po.po` become the right numbers you expect? You could do the same thing with `$nums = (import-csv ___).po -join '|'; select-string -pattern $nums -path "\\wh1-app\SPS EDI\ARCHIVE IN\*.txt"` — TessellatingHeckler, Oct 12 '17 at 18:36
@C.Helling import-csv returns an array of objects with properties. The badly formatted example at the top of the post shows that `po` is the column header of the csv, so each `$po in $pos` will be a pscustomobject with one property named for the column header - that is, `$po.po` — TessellatingHeckler, Oct 12 '17 at 18:37
@C.Helling the $po part of $po.po is just from the foreach loop. the .po is the label of the column in the csv. without it the $po would be {po=123} instead of just 123. — ty Til, Oct 12 '17 at 18:38
@TessellatingHeckler everything seems to be in working order. it does get info from both the csv and text files. It just sits there an runs though without doing anything - including errors. I am running it in the ise debugger. — ty Til, Oct 12 '17 at 18:40
Unless it's just slow - the testing of every line of every file, generating an array of number-of-lines long for every file might be enormously slow... how big are the files? Select-String should be a big improvement on that front. — TessellatingHeckler, Oct 12 '17 at 18:40
@TessellatingHeckler it may just be slow, there are a lot of files in the directory, the files themselves are pretty small. I'll try the select-string in a few minutes. Thanks! — ty Til, Oct 12 '17 at 18:45
@tyTil Select-String should be significantly faster, but change your code to use `$containsWord = $file | ?{$_ -match $po.po} | select -first 1` and then check `if ($containsWord)` should reduce the amount of work considerably - only testing until the first found thing and not keeping something for every line. And try some "write-host" in each loop, to show you the current $po and current $file so you can see how fast it's progressing. — TessellatingHeckler, Oct 12 '17 at 18:51
@TessellatingHeckler that definitely sped things up! Thanks!!! — ty Til, Oct 12 '17 at 19:37

iRon · Answer 1 · 2017-10-12T19:48:56.527

For what I have test it should work and I guess that there is something wrong with the (accessibility) of you remote location (\\wh1-app\SPS EDI\ARCHIVE IN\*.txt) that contains your .txt files.

To troubleshoot this I would either put a few of the source files local (e.g. C:\temp\*.txt)

Or add you additional Write-Host commands in your script. e.g. here:

...
foreach ($file in $files) {
    Write-Host $file.name    # Is there really a file picked up here?
    $file = Get-Content $file
    ...

Some general recommendations to speedup you function:

put $pos = Import-Csv -Path C:\Users\ttilton\Desktop\pos.csv -OutVariable string -ErrorAction SilentlyContinue out of the foreach ($file in $files) {... loop, you only need to read the .csv file once.
Do not (continue to) read the .txt if you found a match (one single $True in $containsWord is enough to validate if ($containsWord -contains $true){.... To stop a ForEach loop see: https://stackoverflow.com/a/10420522/1701026

Yeah I've taken a chunk of files out and am now running on it locally. It's still slow, but works better that before thanks to the advice offered by @TessellatingHeckler in the comments above. Thanks! — ty Til, Oct 12 '17 at 19:52

How can I search all files in a directory for a list of strings(numbers)?

1 Answers1