x11 forwarding with paramiko

Question

I'm trying to run a command with paramiko that should be able to open an X window. The script I'm using would something as follows:

import paramiko                                    

ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect('192.168.122.55', username='user', password='password')
transport = ssh_client.get_transport()
session = transport.open_session()

session.request_x11()
stdin = session.makefile('wb')
stdout = session.makefile('rb')
stderr = session.makefile_stderr('rb')
session.exec_command('env; xterm')
transport.accept()

print 'Exit status:', session.recv_exit_status()
print 'stdout:\n{}'.format(stdout.read())
print 'stderr:\n{}'.format(stderr.read())
session.close()

Unfortunately, when I run the script above, I get this output:

Exit status: 1
stdout:
SHELL=/bin/bash
XDG_SESSION_COOKIE=8025e1ba5e6c47be0d2f3ad6504a25ee-1347286654.617967-1932974971
SSH_CLIENT=192.168.122.1 58654 22
USER=user
MAIL=/var/mail/user
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=/home/user
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/user
LOGNAME=user
SSH_CONNECTION=192.168.122.1 58654 192.168.122.55 22
DISPLAY=localhost:10.0
_=/usr/bin/env

stderr:  
xterm: Xt error: Can't open display: localhost:10.0

If I run the following command in a terminal:

ssh -X user@192.168.122.55 'env; xterm'

then I get the same environment variables (some ports changed, though), so I'd say that my environment is correct. However, I'm still missing something to make paramiko work with x11 forwarding.

A couple of things I tried are:

• Use the handler parameter in request_x11: aside from printing values, I didn't get any further than with the default handler.
• Use the auth_cookie parameter in request_x11: tried to hardcode a cookie value that was being used according to the xauth list output. The idea of doing this was to avoid problems that might happen according to the documentation string in paramiko itself:

If you omit the auth_cookie, a new secure random 128-bit value will be generated, used, and returned. You will need to use this value to verify incoming x11 requests and replace them with the actual local x11 cookie (which requires some knoweldge of the x11 protocol).

Is there some other thing I could do to make it work or troubleshoot the problem?

Note: This has been previously asked in:

• superuser: the only response points to the request_x11 documentation I've already tried to use to no avail.
• stackoverflow: the accepted response suggests to use the handler parameter, but it's wrong.
• github: no answer provided for more than a year.

Answer 1

Reading the paramiko code, I realized that paramiko only implements a way to establish an x11 channel. It does not connect the channel to the local x11 display. That is left to you.

Here is a small implementation that I have just written:

#!/usr/bin/env python

import os
import select
import sys

import paramiko
import Xlib.support.connect as xlib_connect


local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])
local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])


ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect('server', username='username', password='password')
transport = ssh_client.get_transport()
session = transport.open_session()
session.request_x11(single_connection=True)
session.exec_command('xterm')
x11_chan = transport.accept()

session_fileno = session.fileno()
x11_chan_fileno = x11_chan.fileno()
local_x11_socket_fileno = local_x11_socket.fileno()

poller = select.poll()
poller.register(session_fileno, select.POLLIN)
poller.register(x11_chan_fileno, select.POLLIN)
poller.register(local_x11_socket, select.POLLIN)
while not session.exit_status_ready():
    poll = poller.poll()
    if not poll: # this should not happen, as we don't have a timeout.
        break
    for fd, event in poll:
        if fd == session_fileno:
            while session.recv_ready():
                sys.stdout.write(session.recv(4096))
            while session.recv_stderr_ready():
                sys.stderr.write(session.recv_stderr(4096))
        if fd == x11_chan_fileno:
            local_x11_socket.sendall(x11_chan.recv(4096))
        if fd == local_x11_socket_fileno:
            x11_chan.send(local_x11_socket.recv(4096))

print 'Exit status:', session.recv_exit_status()
while session.recv_ready():
    sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
    sys.stdout.write(session.recv_stderr(4096))
session.close()

Some notes:

• I'm using some helper functions from python-Xlib. This is a pure python implementation of Xlib. See this question for details on installing it: How do you install Python Xlib with pip?

• Some of the details of how I have implemented this make me believe it will only work for 1 x11 connection (hence session.request_x11(single_connection=True).) I would like to keep working at this to get it to handle multiple connections, but that will have to wait for another day.

• This code essentially connects the following channels/sockets together in a async fashion using select.poll:

  • session.stdout -> sys.stdout
  • session.stderr -> sys.stderr
  • x11channel -> local_x11_socket
  • local_x11_socket - > x11channel

• The paramiko module outputs alot of usefull debuging info to the logging module. You can view this by configuring the logging module:

  import logging
  logging.basicConfig(level=logging.DEBUG)
  

Answer 2

• the x11 request may use a MIT-MAGIC-COOKIE-1 that you may not handled properly
• using ssh directly I saw it needed to confirm the x11 request (cookie challenge?)
• the .Xauthority file may also be an issue
• you can try to strace ssh process and see a normal flow
• in your script, you can replace xterm with strace xterm and compare with the above.

some links:

• http://en.wikipedia.org/wiki/X_Window_authorization
• http://tech.groups.yahoo.com/group/ssh/message/6747
• http://answers.tectia.com/questions/523/how-do-i-enable-x11-forwarding-for-users-without-a-home-directory

good luck.

EDIT: building on top of Gary's answer, with multiple x11 connections.

#!/usr/bin/env python

import os
import select
import sys
import getpass
import paramiko
import socket
import logging
import Xlib.support.connect as xlib_connect
LOGGER = logging.getLogger(__name__)

# connection settings
host = '192.168.122.55'
user = 'user'
password = getpass.getpass()

ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(host, username=user, password=password)
del password

# maintain map
# { fd: (channel, remote channel), ... }
channels = {}

poller = select.poll()
def x11_handler(channel, (src_addr, src_port)):
    '''handler for incoming x11 connections
    for each x11 incoming connection,
    - get a connection to the local display
    - maintain bidirectional map of remote x11 channel to local x11 channel
    - add the descriptors to the poller
    - queue the channel (use transport.accept())'''
    x11_chanfd = channel.fileno()
    local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])
    local_x11_socket_fileno = local_x11_socket.fileno()
    channels[x11_chanfd] = channel, local_x11_socket
    channels[local_x11_socket_fileno] = local_x11_socket, channel
    poller.register(x11_chanfd, select.POLLIN)
    poller.register(local_x11_socket, select.POLLIN)
    LOGGER.debug('x11 channel on: %s %s', src_addr, src_port)
    transport._queue_incoming_channel(channel)

def flush_out(session):
    while session.recv_ready():
        sys.stdout.write(session.recv(4096))
    while session.recv_stderr_ready():
        sys.stderr.write(session.recv_stderr(4096))

# get local disply
local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])
# start x11 session
transport = ssh_client.get_transport()
session = transport.open_session()
session.request_x11(handler=x11_handler)
session.exec_command('xterm')
session_fileno = session.fileno()
poller.register(session_fileno, select.POLLIN)
# accept first remote x11 connection
transport.accept()

# event loop
while not session.exit_status_ready():
    poll = poller.poll()
    # accept subsequent x11 connections if any
    if len(transport.server_accepts) > 0:
        transport.accept()
    if not poll: # this should not happen, as we don't have a timeout.
        break
    for fd, event in poll:
        if fd == session_fileno:
            flush_out(session)
        # data either on local/remote x11 socket
        if fd in channels.keys():
            channel, counterpart = channels[fd]
            try:
                # forward data between local/remote x11 socket.
                data = channel.recv(4096)
                counterpart.sendall(data)
            except socket.error:
                channel.close()
                counterpart.close()
                del channels[fd]

print 'Exit status:', session.recv_exit_status()
flush_out(session)
session.close()

Answer 3

Given that you asked for a minimal version, which I understand as make it as easy to to use as possible. Here is a version based in both codes, but this separate the x11 session commands from the general code, making the main program simple and the session code reusable:

import paramiko
import os
import select
import sys
import Xlib.support.connect as xlib_connect

def run(transport, session, command):
    def x11_handler(channel, (src_addr, src_port)):
        x11_fileno = channel.fileno()
        local_x11_channel = xlib_connect.get_socket(*local_x11_display[:3])
        local_x11_fileno = local_x11_channel.fileno()

        # Register both x11 and local_x11 channels
        channels[x11_fileno] = channel, local_x11_channel
        channels[local_x11_fileno] = local_x11_channel, channel

        poller.register(x11_fileno, select.POLLIN)
        poller.register(local_x11_fileno, select.POLLIN)

        transport._queue_incoming_channel(channel)

    def flush_out(channel):
        while channel.recv_ready():
            sys.stdout.write(channel.recv(4096))
        while channel.recv_stderr_ready():
            sys.stderr.write(channel.recv_stderr(4096))

    local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])

    channels = {}
    poller = select.poll()
    session_fileno = session.fileno()
    poller.register(session_fileno)

    session.request_x11(handler=x11_handler)
    session.exec_command(command)
    transport.accept()

    # event loop
    while not session.exit_status_ready():
        poll = poller.poll()
        if not poll: # this should not happen, as we don't have a timeout.
            break
        for fd, event in poll:
            if fd == session_fileno:
                flush_out(session)
            # data either on local/remote x11 channels/sockets
            if fd in channels.keys():
                sender, receiver = channels[fd]
                try:
                    receiver.sendall(sender.recv(4096))
                except:
                    sender.close()
                    receiver.close()
                    channels.remove(fd)

    flush_out(session)
    return session.recv_exit_status()

if __name__ == '__main__':
    ssh_client = paramiko.SSHClient()
    ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh_client.connect('192.168.122.55', username='user', password='password')
    transport = ssh_client.get_transport()
    session = transport.open_session()
    run(transport, session, 'xterm')

I know you could do it by yourself. However, but just by copying the function run anybody could use it without hassle.

The right answer is https://stackoverflow.com/a/12903844/278878. This example is to make it easier for newcomers.

Answer 4

Thanks Gary van der Merwe and dnozay for their code. The code below heavily relies on it and serves for running X programs on Windows. The notable difference is using select.select instead of poll, as poll is not available in Windows. Any improvements or corrections are welcome.

import select
import sys
import paramiko
import Xlib.support.connect as xlib_connect
import os
import socket
import subprocess



# run xming
XmingProc = subprocess.Popen("C:/Program Files (x86)/Xming/Xming.exe :0 -clipboard -multiwindow")
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(SSHServerIP, SSHServerPort, username=user, password=pwd)
transport = ssh_client.get_transport()
channelOppositeEdges = {}

local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])
inputSockets = []

def x11_handler(channel, (src_addr, src_port)):
    local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])
    inputSockets.append(local_x11_socket)
    inputSockets.append(channel)
    channelOppositeEdges[local_x11_socket.fileno()] = channel
    channelOppositeEdges[channel.fileno()] = local_x11_socket
    transport._queue_incoming_channel(channel)

session = transport.open_session()
inputSockets.append(session)
session.request_x11(handler = x11_handler)
session.exec_command('xterm')
transport.accept()

while not session.exit_status_ready():
    readable, writable, exceptional = select.select(inputSockets,[],[])
    if len(transport.server_accepts) > 0:
        transport.accept()
    for sock in readable:
        if sock is session:
            while session.recv_ready():
                sys.stdout.write(session.recv(4096))
            while session.recv_stderr_ready():
                sys.stderr.write(session.recv_stderr(4096))   
        else: 
            try:
                data = sock.recv(4096)
                counterPartSocket  = channelOppositeEdges[sock.fileno()]
                counterPartSocket.sendall(data)
            except socket.error:
                inputSockets.remove(sock)
                inputSockets.remove(counterPartSocket)
                del channelOppositeEdges[sock.fileno()]
                del channelOppositeEdges[counterPartSocket.fileno()]
                sock.close()
                counterPartSocket.close()

print 'Exit status:', session.recv_exit_status()
while session.recv_ready():
    sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
    sys.stdout.write(session.recv_stderr(4096))
session.close()
XmingProc.terminate()
XmingProc.wait()

Answer 5

For those working in Mac OS X Leopard, there is no select.poll(). Here is a modified version of dnozay's answer using kqueue instead of poll. Any improvements/corrections on this would be appreciated.

#!/usr/bin/env python

import os
import select
import sys
import paramiko
import socket
import Xlib.support.connect as xlib_connect

# get local display
local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])

ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect('hostname', port=22, username='username', password='password')

channels = {}
kq = select.kqueue()

def x11Handler(x11_chan, (src_addr, src_port)):
    x11_chan_fileno = x11_chan.fileno()
    local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])
    local_x11_socket_fileno = local_x11_socket.fileno()
    channels[x11_chan_fileno] = x11_chan, local_x11_socket
    channels[local_x11_socket_fileno] = local_x11_socket, x11_chan

    ev = [select.kevent(x11_chan_fileno, filter=select.KQ_FILTER_READ, flags=select.KQ_EV_ADD), select.kevent(local_x11_socket_fileno, filter=select.KQ_FILTER_READ, flags=select.KQ_EV_ADD)]
    kevents = kq.control(ev, 0, None)
    transport._queue_incoming_channel(x11_chan)

def flushOut(session):
    while session.recv_ready():
        sys.stdout.write(session.recv(4096))
    while session.recv_stderr_ready():
        sys.stderr.write(session.recv_stderr(4096))

# start x11 session
transport = ssh_client.get_transport()
session = transport.open_session()
session.request_x11(handler=x11Handler)
session.exec_command('xterm')

# accept first remote x11 connection
x11_chan = transport.accept()
session_fileno = session.fileno()

session_ev = [select.kevent(session_fileno, 
    filter=select.KQ_FILTER_READ,
    flags=select.KQ_EV_ADD)] 

kevents_session = kq.control(session_ev, 0, None)

# event loop
while not session.exit_status_ready():
    r_events = kq.control(None, 4)
    # accept subsequent x11 connections if any

    if len(transport.server_accepts) > 0:
        transport.accept()
    if not r_events: # this should not happen, as we don't have a timeout.
        break
    for event in r_events:
        print event
        if event.ident & session_fileno:
            flushOut(session)
        # data either on local/remote x11 socket
        if event.ident in channels.keys():
            x11_chan, counterpart = channels[event.ident]
            try:
                # forward data between local/remote x11 socket.
                data = x11_chan.recv(4096)
                counterpart.sendall(data)
            except socket.error:
                x11_chan.close()
                counterpart.close()
                del channels[event.ident]

flushOut(session)
kq.close()
session.close()