0

I've spent dozens of hours trying various online examples and ChatGPT code yet can't make this work. SSH somehow is a black box to me that I can't figure out.

The goal is to forward back a dask scheduler Bokeh page to my computer through a jump server. I am using a jupyter notebook for the code to setup my dask tasking and submit the job to the remote. In order to submit job I need to setup the dask client at its remote listening port 8786 on the AWS server to submit the job. I did play a bit with some dask examples that supposedly handle the ssh connections with no luck either.

Tunnel ports 8786/8787:

[ PC ] <----> [ Jump Server w password auth] <----> [ AWS Server with Cert Auth ]

So far I can successfully connect and send ssh commands and retrieve the stdout. However, the paramiko code is not forwarding any port other than 22 it appears. Of note is that I tried using transport_instance.get_transport().request_port_forward('', 8787, handler=paramiko.proxy.ProxyCommand(f"ssh -W localhost:8787 {ec2_user}@{ec2_ip}")) and in the debug logs see that the port forward was rejected due to "port in use".

When I use the terminal with my jump server setup in the ssh/config file, the ports forward no problem. I am executing inside a Jupyter Notebook.

I am willing to use other more simpler methods if available. Paramiko seems like really under the hood method but not easy. The subprocess.call with the same ssh command I use for the terminal using my ssh config also works fine which I may just decide to use instead. subprocess.call('ssh -F .ssh/config aws-remote')

My ssh config is as follows:

    Host aws-dask
      HostName <aws IP>
      User ec2-user
      IdentityFile aws-provided.pem
      ProxyJump bastion-server
      LocalForward 8787 localhost:8787
      LocalForward 8786 localhost:8786
    
    Host bastion-server
      HostName <ip>
      User <username>

Code:


    import paramiko
    
    # SSH configuration
    jump_host = jump_ip
    target_host = ec2_host
    jump_user = jump_user
    target_user = aws_user
    private_key_path = ec2_private_key
        
    try:
        # Connect to the jump server
        jump_client = paramiko.SSHClient()
        jump_client.load_system_host_keys()
        jump_client.connect(jump_host, username=jump_user, password=jump_password)
    
        # Connect to the target AWS server through the jump server
        transport = jump_client.get_transport()
        target_session = transport.open_channel("direct-tcpip", (target_host, 22), ("127.0.0.1", 0))
    
        # Create a client for the target server
        target_client = paramiko.SSHClient()
        target_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        target_client.connect(target_host, username=target_user, key_filename=private_key_path, sock=target_session)
    
        # Send test command
        stdin, stdout, stderr = target_client.exec_command(test_cmd)
        print(stdout.read().decode())
        print(stderr.read().decode())
    
    
    except Exception as e:
        print("An error occurred:", str(e))
Leland
  • 147
  • 1
  • 7
  • See https://stackoverflow.com/q/11294919/850848 – Martin Prikryl Aug 29 '23 at 05:53
  • Further to the comment: you might well find more luck and better help using CLI ssh, which is much better supported than via paramiko. – mdurant Aug 29 '23 at 13:12
  • @mdurant Thank you I have seen this and tried the forward.py example, but it appears to not work for some reason. It appears to be coded for a single-hop server. I have the transport established over the jump server with password using paramiko, but possibly the forward is requiring the password again but not providing stdin/out? A method I did use that works is subprocess.call('ssh -F .ssh/config aws-remote') which uses my ssh/config. The tunnel is held open for the duration. Just need to figure out how to run in background. – Leland Aug 30 '23 at 17:46

0 Answers0