2

The problem is that I cannot record the output of my Python script using Tee-Object to both terminal and a file.

I have multiple files containing lines of data. I need to verify this data via an HTTP request to a server. There are 8 large files and given I expect this will take a day to run since I don't want to flood the server.

  1. Importing file strings and running Python script is producing output to terminal. Completed as follows:

    $db = Import-Csv C:\Users\xxxx\documents\bunnies\foo.txt
foreach ($i in $db.StringName) {
    & py -2.7 myscript.py -option $i
}

    $db is the file. $i is the string (line) in the file. script prints to terminal.

    Since the output is going to be over several days, I need to know that it will be recorded. Tee-Object has not created a file after an hour of output.

    foreach ($i in $db.StringName) {
    & py -2.7 myscript.py -option $i
} > Tee-Object -FilePath .../bunnyrabbit.txt

  2. I assume that > Tee-Object -FilePath .../bunnyrabbit.txt appended should create the file immediately and write in an ongoing manner?

    I need to be able to check the output is okay as the program runs.

Additional: filtering output

The output per line of the script is simply "x is correct" or "x is incorrect". If I want to filter all the corrects into one file and the incorrects into another how to go about this?

My original plan was simply to re-read the output file in python and do it in a language I know.

lgjmac
  • 133
  • 1
  • 10
  • 1
    `> Tee-Object` -> `| Tee-Object`, and you need to replace your `foreach` loop with a `ForEach-Object` b/c the former doesn't write to the pipeline. – Ansgar Wiechers May 01 '18 at 22:36
  • HI thank you for being so quick. I have an error 'unexpected token 'in'' now: `foreach-object($i in $db.Email){& py -2.7 gxlu.py -s $i} | Tee-Object -....` – lgjmac May 01 '18 at 22:48
  • `ForEach-Object` is not a drop-in replacement for `foreach`. [Related](https://stackoverflow.com/q/29148462/1630171). – Ansgar Wiechers May 01 '18 at 22:50

1 Answers1

2

You'll need the | pipeline operator, not the > redirection operator.

Additionally, either move Tee-Object inside the foreach body and use the -Append switch, or change the script to use the ForEach-Object cmdlet instead of a loop statement as suggested by Ansgar Wiechers:

$db = Import-Csv C:\Users\xxxx\documents\bunnies\foo.txt
foreach ($i in $db.StringName) {
    & py -2.7 myscript.py -option $i |Tee-Object -FilePath ..\bunnyrabbit.txt -Append
}

or

$db = Import-Csv C:\Users\xxxx\documents\bunnies\foo.txt
$db.StringName |ForEach-Object {
    & py -2.7 myscript.py -option $_ 
} |Tee-Object -FilePath ..\bunnyrabbit.txt
Mathias R. Jessen
  • 157,619
  • 12
  • 148
  • 206
  • Thank you. I think I am about to understand something fundamental about powershell that I never understood before: – lgjmac May 01 '18 at 23:09
  • What exactly is the object? Some variable representing the line or the actual content of the line and how do you work it out for every cmdlet? The get-member property isn't quite the same thing. The help files are really bad on for each! I didn't even understand the scripting guy posts about it. – lgjmac May 01 '18 at 23:11
  • The "object" in question is whatever `ForEach-Object` receives as pipeline input - if the expression `$db.StringName` results in an array of say, 5 strings, then the `ForEach-Object` scriptblock will execute 5 times, with `$_` referring to the current object in the pipeline each time. Does that make more sense? – Mathias R. Jessen May 01 '18 at 23:19
  • 1
    [This might be helpful in understanding how cmdlets/functions in a pipeline behaves in PowerShell](https://stackoverflow.com/questions/48522415/are-the-cmdlets-in-a-pipeline-executing-in-parallel/48523475#48523475) – Mathias R. Jessen May 01 '18 at 23:21
  • Yes it does. And thank you for taking the time. In order to access each string's content, would I need to use an object property like $_.Name or something, or is the actual object $_ the string? thank you ill check out the link. i really appreciate it! – lgjmac May 01 '18 at 23:21
  • That link is REALLY helpful thank you Anyone needing help on objects go to that link. Pipelines process concurrently. – lgjmac May 01 '18 at 23:25
  • `$_` is whatever you piped in - since you're already passing bare strings, `$_` will be a string, yes :) – Mathias R. Jessen May 01 '18 at 23:28
  • Thank you Mathias. Please allow me a favour.Could you up vote me somewhere? I have another question ban!! It's like they are shooting the ones that can't keep up ! – lgjmac May 06 '18 at 19:24