continuous read from subprocess.Popen

Question

In Python, I have a question about the subprocess.Popen function, my problem is that I can't get my head around a continuous read of stdout stream. When I use communicate() at the end of my function, I get my preferred output. But I have two problems here. First, communicate() buffers the whole output before it prints anything and it would be nice to get an continuous output. Second, I had read in the communicate() documentation that communicate() isn't meant for big data streams which is in my scenario the case.

#!/usr/bin/python

import os
import sys
from subprocess import *
import itertools


def combinate(hash_mode,hash_file,directory):
erg = Popen(['hashcat', '-a', '0', '-m', hash_mode, hash_file, '-O', '--potfile-disable'],
                     stdin=PIPE,
                     stdout=PIPE,
                     stderr=PIPE,
                     universal_newlines=True)
file = []
with os.scandir(directory) as listOfEntries:
    for entry in listOfEntries:
        if entry.is_file() and entry.name is not ".DS_Store":
            file.append(open(directory+entry.name).readlines())
    file = list(itertools.permutations(file))
    for b in range(0, len(file)):
        for i in itertools.product(*file[b]):
            test = '\n'.join(i).replace("\n", "")
            erg.stdin.writelines(test+'\n')

print(erg.communicate()[0])

This is my output with communicate:

Session..........: hashcat
Status...........: Cracked
Hash.Type........: SHA-512
Hash.Target......:          7ba4e9da57a7d3bd8b1b43c0b028a96d77721f6b33e3b85f0b2...298b56
Time.Started.....: Sat Feb 24 03:52:05 2018 (0 secs)
Time.Estimated...: Sat Feb 24 03:52:05 2018 (0 secs)
Guess.Base.......: Pipe
Speed.Dev.#2.....:   969.7 kH/s (0.13ms)
Recovered........: 1/1 (100.00%) Digests, 1/1 (100.00%) Salts
Progress.........: 384
Rejected.........: 0
Restore.Point....: 0
Candidates.#2....: telefon1telefon3telefon2 -> tasse2tasse3tasse1

And this is my output with an for loop with an stdout.readline:

Session..........: hashcat
Status...........: Running
Hash.Type........: SHA-512
Hash.Target......:  7ba4e9da57a7d3bd8b1b43c0b028a96d77721f6b33e3b85f0b2...298b56
Time.Started.....: Sat Feb 24 04:14:30 2018 (10 secs)
Time.Estimated...: Sat Feb 24 04:14:40 2018 (0 secs)
Guess.Base.......: Pipe
Speed.Dev.#2.....:        0 H/s (0.00ms)
Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 0
Rejected.........: 0
Restore.Point....: 0
Candidates.#2....: [Copying]

As you can see, I get an output, but the hashcat process doesn't get my stdin stream or doesn't process it and I don't know why.

How can I realize an continuous output with my code?

I think you need to post the `stdout.readline` code as `communicate` never can meet your requirements, it's useless to post this code here. BTW, if you could post a runnable code, that will be fine. — atline, Feb 24 '18 at 05:39
Yes, that's exactly what I wanted to say with my long description. I already came to the conclusion that communicate() isn't right. And that's (more or less) runnable code, you just have to call combinate() and give the function parameter values for the 3 variables. The first two parameters are typical for hashcat, so if you know hashcat, you know what values the parameters need. And the third parameter just is a string path to a directory, where the function iterates over the found password dictionaries and combines them which each other. Sorry if there was something unclear ^^ — friluk, Feb 24 '18 at 09:47
You should consider using threads and have `stdout` and `stdin` run in separate threads, as e.g. in my answer here: https://stackoverflow.com/a/48777349/7738328 — JohanL, Feb 24 '18 at 10:29
WOW, you don't know how thankful I'm right now. The whole night I have tried many threading examples and yours is the first one which did exactly what I wanted. Thank you — friluk, Feb 24 '18 at 10:41

score 0 · Answer 1 · edited Feb 25 '19 at 09:18

0

The answer from JohanL in the comments was my solution. Great thanks for that.

You should consider using threads and have stdout and stdin run in separate threads, as e.g. in my answer here: [link of post][stackoverflow.com/a/48777349/7738328 – JohanL]

edited Feb 25 '19 at 09:18

azro

53,056
7
34
70

answered Feb 24 '18 at 10:53

friluk

1
2

continuous read from subprocess.Popen

1 Answers1