0

I'm trying to write a PowerShell script to quickly generate a lot of tests for a C++ program and also verify its output. main.cpp is the program I'm verifying, brute.cpp is a brute solution I know for sure works and gen.cpp generates tests. Here's the code:

g++ -o gen.exe gen.cpp
g++ -o brute.exe brute.cpp
g++ -o main.exe main.cpp
for ($i = 0; ; $i++) {
    Write-Output "$i"
    Start-Process .\gen.exe -RedirectStandardOutput ".\cmake-build-debug\in.in" -NoNewWindow -Wait
    Start-Process .\brute.exe -RedirectStandardInput ".\cmake-build-debug\in.in" -RedirectStandardOutput ".\cmake-build-debug\out1" -NoNewWindow -Wait
    Start-Process .\main.exe -RedirectStandardInput ".\cmake-build-debug\in.in" -RedirectStandardOutput ".\cmake-build-debug\out2" -NoNewWindow -Wait
    if ((Get-FileHash  ".\cmake-build-debug\out1" ).Hash -ne (Get-FileHash ".\cmake-build-debug\out2").Hash) {
        Compare-Object (Get-Content ".\cmake-build-debug\out1") (Get-Content ".\cmake-build-debug\out2")
        break
    }
}

This works but it every test takes around 6 seconds, I would want something around 50 times faster. (the programs themselves take up a matter of miliseconds.)

Jeff
  • 97
  • 4
  • Have you tried to profile the individual statements? How long does `g++` take to compile the binaries? How many rounds does the loop usually run before a discrepancy is encountered? What's the size or the output/input files? – Mathias R. Jessen Jan 07 '22 at 15:19
  • This might be a good usage case for For-EachObject -Parallel, which will thread and run the loops with multiple threads at once. Keep in mind though that you'll then have multiple instances of gen, brute and main running at once, which will be competing for resources. That's not as good of a test, IMHO – FoxDeploy Jan 07 '22 at 15:43
  • Size of the input/output files is a few characters. Nr of rounds the loop makes doesn't really matter since I'm trying to improve time per round, which is a few seconds right now. Compile time before the loop starts is a few seconds per file. I don't know how to profile something like this line per line, otherwise I would give you more exact information. – Jeff Jan 07 '22 at 15:46
  • if you can eliminate the hashing that'll help a little bit with the loop time `if((gc out1 -raw) -eq (gc out2 -raw))` – Gregor y Jan 07 '22 at 15:54
  • also if your relative path's are pointing to a network share, then working off local temp files could be faster `$out1=[System.IO.Path]::GetTempFileName();rm $out1` – Gregor y Jan 07 '22 at 16:21
  • when main.cpp eventually produces the same results as brute.cpp, would this loop infinitely or does gen.cpp create a stopping 'test-passed' condition? – Gregor y Jan 07 '22 at 16:38
  • You could compare the lengths of files first, although since you’ve said they’re typically only a few bytes each that might not actually save much time in your case. – mclayton Jan 07 '22 at 17:27
  • [Windows CreateProcess overhead](https://stackoverflow.com/questions/10710912/what-is-the-process-creation-overhead-in-windows) is around 20-30ms. Your desire 50x faster is right around the limit of what launching six processes could take. And if you say the processes are very fast, what is it in your script which is taking the extra time? Do you have any AntiVirus software running? "*I don't know how to profile something like this line per line, otherwise I would give you more exact information*" - comment each bit of the script out and see if any of them make a significant difference? – TessellatingHeckler Jan 07 '22 at 17:38
  • @TessellatingHeckler there are some more accurate ways, but the easy way is `$ElapsedTime = (Measure-Command {a line/block of code}).TotalMilliseconds` – Gregor y Jan 07 '22 at 18:15

0 Answers0