How to Communicate with a Chess engine in Python?

Question

On Windows 7 I can communicate with a chess engine via command line. Small example session with Stockfish on Win 7:

C:\run\Stockfish>stockfish-x64.exe
Stockfish 2.2.2 JA SSE42 by Tord Romstad, Marco Costalba and Joona Kiiski
quit

C:\run\Stockfish>

The first line was output by the engine and the 'quit' was what I typed to quit the engine (There are other things I can do, but that's clear to me).

Now I want to communicate with that engine from python:

import subprocess
engine = subprocess.Popen(
    'stockfish-x64.exe',
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
)
for line in engine.stdout:
    print(line.strip())
engine.stdin.write('quit\n')

and I get

C:\run\Stockfish>communicate.py
b'Stockfish 2.2.2 JA SSE42 by Tord Romstad, Marco Costalba and Joona Kiiski'

But it doesn't quit the engine (no C:\run\Stockfish> prompt), it keeps waiting for input. I have to close the window by hand. It seems not to get my quit message (last line of the python script) written to stdin.

In other words, I can read from stdout but when I write to stdin nothing happens.

What am I doing wrong and how to do it right?

Answer 1

You've got a deadlock: the subprocess is waiting for input, while your program is waiting for it to output more lines in

for line in engine.stdout:
    print(line.strip())

This loop only stops when the subprocess closes its stdout.

Answer 2

You might want to use asyncio like python-chess does. See

engine.py

and the example from the documentation

import asyncio
import chess
import chess.engine

    async def main():
        transport, engine = await chess.engine.popen_uci("/usr/bin/stockfish")

        board = chess.Board()
        while not board.is_game_over():
            result = await engine.play(board, chess.engine.Limit(time=0.1))
            board.push(result.move)

        await engine.quit()

    asyncio.set_event_loop_policy(chess.engine.EventLoopPolicy())
    asyncio.run(main())

Answer 3

I move my edit in the question into this answer.

Thanks to larsmans´ help, I solved it.

Example Python script:

import subprocess, time

engine = subprocess.Popen(
    'stockfish14.exe',
    universal_newlines=True,
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    bufsize=1,
)

def put(command):
    print('\nyou:\n\t'+command)
    engine.stdin.write(command+'\n')
    
def get():
    # using the 'isready' command (engine has to answer 'readyok')
    # to indicate current last line of stdout
    engine.stdin.write('isready\n')
    print('\nengine:')
    while True:
        text = engine.stdout.readline().strip()
        if text == 'readyok':
            break
        if text !='':
            print('\t'+text)

get()
put('uci')
get()
put('setoption name Hash value 128')
get()
put('ucinewgame')
get()
put('position startpos moves e2e4 e7e5 f2f4')
get()
put('go infinite')
time.sleep(3)
get()
put('stop')
get()
put('quit')

Output:

C:\run\chessengines\Stockfish>communicate.py

engine:
        Stockfish 14 by the Stockfish developers (see AUTHORS file)

you:
        uci

engine:
        id name Stockfish 14
        id author the Stockfish developers (see AUTHORS file)
        option name Debug Log File type string default
        option name Threads type spin default 1 min 1 max 512
        option name Hash type spin default 16 min 1 max 33554432
        option name Clear Hash type button
        option name Ponder type check default false
        option name MultiPV type spin default 1 min 1 max 500
        option name Skill Level type spin default 20 min 0 max 20
        option name Move Overhead type spin default 10 min 0 max 5000
        option name Slow Mover type spin default 100 min 10 max 1000
        option name nodestime type spin default 0 min 0 max 10000
        option name UCI_Chess960 type check default false
        option name UCI_AnalyseMode type check default false
        option name UCI_LimitStrength type check default false
        option name UCI_Elo type spin default 1350 min 1350 max 2850
        option name UCI_ShowWDL type check default false
        option name SyzygyPath type string default <empty>
        option name SyzygyProbeDepth type spin default 1 min 1 max 100
        option name Syzygy50MoveRule type check default true
        option name SyzygyProbeLimit type spin default 7 min 0 max 7
        option name Use NNUE type check default true
        option name EvalFile type string default nn-3475407dc199.nnue
        uciok

you:
        setoption name Hash value 128

engine:

you:
        ucinewgame

engine:

you:
        position startpos moves e2e4 e7e5 f2f4

engine:

you:
        go infinite

engine:
        info string NNUE evaluation using nn-3475407dc199.nnue enabled
        info depth 1 seldepth 1 multipv 1 score cp 90 nodes 33 nps 33000 tbhits 0 time 1 pv e5f4
        info depth 2 seldepth 2 multipv 1 score cp 336 nodes 72 nps 36000 tbhits 0 time 2 pv e5f4 a2a3 d8h4 e1e2
        info depth 3 seldepth 3 multipv 1 score cp 322 nodes 115 nps 57500 tbhits 0 time 2 pv e5f4 g2g3 f4g3
        info depth 4 seldepth 4 multipv 1 score cp 160 nodes 438 nps 219000 tbhits 0 time 2 pv e5f4 d2d3 d8h4 e1e2
        info depth 5 seldepth 5 multipv 1 score cp 38 nodes 1463 nps 365750 tbhits 0 time 4 pv e5f4 g1f3 g8f6 b1c3 d7d5
        info depth 6 seldepth 6 multipv 1 score cp 41 nodes 2424 nps 404000 tbhits 0 time 6 pv e5f4 g1f3 g8f6 b1c3 d7d5 e4d5 f6d5
        info depth 7 seldepth 7 multipv 1 score cp 41 nodes 3409 nps 426125 tbhits 0 time 8 pv e5f4 g1f3 g8f6 b1c3 d7d5 e4d5 f6d5
        info depth 8 seldepth 11 multipv 1 score cp 43 nodes 4937 nps 448818 tbhits 0 time 11 pv d7d5 g1f3 d5e4 f3e5 f7f6 b1c3 f6e5 d1h5 e8d7
        info depth 9 seldepth 11 multipv 1 score cp 4 nodes 14604 nps 486800 tbhits 0 time 30 pv d7d5 g1f3 e5f4 e4d5 g8f6 f1c4 f6d5 e1g1 d5b6
        info depth 10 seldepth 13 multipv 1 score cp 24 nodes 16663 nps 490088 tbhits 0 time 34 pv d7d5 g1f3 e5f4 e4d5 g8f6 f1c4 f6d5 e1g1
        info depth 11 seldepth 16 multipv 1 score cp 49 nodes 24347 nps 486940 tbhits 0 time 50 pv e5f4 f1c4 d7d5 e4d5 g8f6 b1c3 c7c6
        info depth 12 seldepth 19 multipv 1 score cp 46 nodes 58792 nps 477983 tbhits 0 time 123 pv e5f4 g1f3 g7g5 h2h3 d7d5 e4d5 d8d5 b1c3 d5e6 d1e2
        info depth 13 seldepth 19 multipv 1 score cp 45 nodes 88777 nps 482483 tbhits 0 time 184 pv e5f4 f1c4 d7d6 b1c3 d8h4 e1f1 c8e6 c3d5 h4d8 g1f3
        info depth 14 seldepth 17 multipv 1 score cp 50 nodes 120394 nps 483510 tbhits 0 time 249 pv e5f4 f1c4 d7d5 c4d5 d8h4 e1f1 b8c6 d2d4 g7g5 b1c3 f8g7 g1f3 h4h5
        info depth 15 seldepth 19 multipv 1 score cp 52 nodes 192828 nps 480867 tbhits 0 time 401 pv e5f4 f1c4 g8f6 b1c3 c7c6 d2d4 d7d5 e4d5 c6d5 d1e2 f8e7 c4b3 e8g8 c1f4 e7b4 g1f3
        info depth 16 seldepth 22 multipv 1 score cp 59 nodes 309617 nps 475602 tbhits 0 time 651 pv e5f4 f1c4 g8f6 b1c3 c7c6 d2d4 d7d5 e4d5 c6d5 c4b5 b8c6 c1f4 f8d6 f4d6 d8d6 d1e2 c8e6 b5c6
 b7c6 g1f3
        info depth 17 seldepth 26 multipv 1 score cp 34 nodes 955333 nps 476237 hashfull 65 tbhits 0 time 2006 pv e5f4 f1c4 b8c6 d2d4 d8h4 e1f1 d7d6 g1f3 c8g4 c4b5 h4h5 b1c3 e8c8 b5c6 b7c6 c
1f4 f7f5 d1d3 g8f6 d4d5 f5e4 d3a6 c8d7
        info depth 18 seldepth 28 multipv 1 score cp 44 nodes 1132938 nps 477227 hashfull 78 tbhits 0 time 2374 pv e5f4 f1c4 c7c6 d2d4 d7d5 e4d5 c6d5 c4b5 b8c6 c1f4 g8f6 b1c3 f8b4 g1f3 b4c3
b2c3 e8g8 e1g1 c8g4

you:
        stop

engine:

you:
        quit

C:\run\chessengines\stockfish>