SSH tunnel forwarding with jump host and remote database

Question

I have a remote MySQL database hosted on Amazon RDS ("D"). For security purposes, it is only accessible through a remote server ("C"). C is accessible via ssh through a jump host "B". I need a double ssh tunnel to then access a remote SQL host.

[A: local host] -> [B: jump host] -> [C: target host] => [D: RDS MySQL host]

I would like to access D through Python, using paramiko and/or sshtunnel. All of the information I can find involves:

• a single ssh tunnel and a remote SQL host (ex. A -> C => D, no jump host)
  • ssh first with mysqldb in python
  • python mysql connectivity via ssh
• a double ssh tunnel to an SQL host (ex. A -> B -> C, D is hosted on C).
  • Connecting to remote Postgresql database over ssh tunnel using python
  • Paramiko: Port Forwarding Around A NAT Router
  • Nested SSH session with Paramiko

So far, I'm using paramiko with a proxy command to get from A to C. I can access D by executing a command on C, but not by connecting with mysqldb or sqlalchemy (my ultimate goal).

My current code:

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
proxy = paramiko.ProxyCommand("ssh -A B_username@B_host -W C_host:12345")
ssh.connect("C_host", username="C_username", sock=proxy)

stdin, stdout, stderr = ssh.exec_command("mysql -u D_username -p D_password -h D_host_rds")
print("STDOUT:\n{}\n\nSTDERR:\n{}\n".format(stdout.read(), stderr.read()))
# successfully prints out MySQL welcome screen

I'm looking for something like this (modified from example 2 in the sshtunnel docs):

import paramiko
from sshtunnel import SSHTunnelForwarder

with SSHTunnelForwarder(
    intermediate = {
        ("B_host", 22),
        ssh_username = "B_username",
        ssh_password = "B_password")},
    remote = {
        ("C_host", 12345),
        ssh_username = "C_username",
        ssh_password = "C_password")},
    remote_bind_address=("D_host_rds", 3306),
    local_bind_address=("0.0.0.0", 3307)) as server:

    conn = MySQLdb.connect(
        user = "D_username",
        passwd = "D_password",
        db = "my_database",
        host = "127.0.0.1",
        port = 3307)

tl;dr: How do I forward a port through two ssh jumps in Python?

Answer 1

I figured it out. It works with a combination of ssh config settings and the SSHTunnelForwarder context manager from the sshtunnel library.

Using the following model and naming conventions:

[A: local host] -> [B: jump host] -> [C: target host] => [D: RDS MySQL host]

I set up my ~/.ssh/config to get from A to C through B:

Host C_ssh_shortcut
    HostName C_host
    User C_user
    Port 22
    ForwardAgent yes
    ProxyCommand ssh B_user@B_host -W %h:%p

I added the key/keys I used to log in to B and C to my ssh-agent:

ssh-add

And finally I set up SSHTunnelForwarder:

import sqlalchemy
from sshtunnel import SSHTunnelForwarder

with SSHTunnelForwarder(
    "C_ssh_shortcut",                     # The SSHTunnelForwarder "ssh_address_or_host" argument, which takes care of bypassing B through the ProxyCommand set up in ~/.ssh/config
    remote_bind_address=(D_host, 3306),   # Points to your desired destination, ie. database host on 3306, which is the MySQL port
    local_bind_address=('', 1111)         # Gives a local way to access this host and port on your machine. '' is localhost / 127.0.0.1, 1111 is an unused port
) as server:
    connection_string = "mysql+pymysql://D_user:D_password@localhost:1111/D_dbname"  # note that D_host and D_port were replaced by the host and port defined in "local_bind_address"
    engine = sqlalchemy.create_engine(connection_string)
    # do your thing

From here, I am able to use my engine as usual to interact with my database.

Answer 2

This code work for me

import pymysql
import paramiko
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine

#ssh config
mypkey = paramiko.RSAKey.from_private_key_file('your/user/location/.ssh/id_rsa')             
ssh_host = 'your_ssh_host'
ssh_user = 'ssh_host_username'
ssh_port = 22  

#mysql config         
sql_hostname = 'your_mysql_host name'
sql_username = 'mysql_user'
sql_password = 'mysql_password'
sql_main_database = 'your_database_name'
sql_port = 3306
host = '127.0.0.1'



with SSHTunnelForwarder(
        (ssh_host, ssh_port),
        ssh_username=ssh_user,
        ssh_pkey=mypkey,
        remote_bind_address=(sql_hostname, sql_port)) as tunnel:              

    engine = create_engine('mysql+pymysql://'+sql_username+':'+sql_password+'@'+host+':'+str(tunnel.local_bind_port)+'/'+sql_main_database)
    connection = engine.connect()
    print('engine creating...')
    sql = text(""" select * from nurse_profiles np limit 50""")
    nurseData = connection.execute(sql)
    connection.close()


    nurseList = []
    for row in nurseData:
        nurseList.append(dict(row))
    print('nurseList len: ', len(nurseList))
    print('nurseList: ', nurseList)

Answer 3

I use this code for PostgreSQL database and it works. I am sure it will work too if use MySQL database. I change the PostgreSQL database part here to MySQL, here is the code:

import pymysql
import paramiko
import sqlalchemy
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
import pandas as pd

#SSH config
mypkey = paramiko.RSAKey.from_private_key_file('id_rsa_file', password = 'id_rsa_password')
ssh_host = 'your_ssh_host'
ssh_user = 'your_ssh_host_username'
ssh_port = 22

#SQL config         
sql_hostname = 'your_sql_host_name'
sql_username = 'sql_user'
sql_password = 'sql_password'
sql_main_database = 'your_database_name'
sql_port = 3306
host = '127.0.0.1'

with SSHTunnelForwarder((ssh_host, ssh_port),
                        ssh_username=ssh_user,
                        ssh_pkey=mypkey,
                        remote_bind_address=(sql_hostname, sql_port)) as tunnel:
    #Connect to SQL
    local_port = str(tunnel.local_bind_port)
    engine = create_engine(f'mysql+pymysql://{sql_username}:{sql_password}@127.0.0.1:' + local_port +f'/{sql_main_database}')
    Session = sessionmaker(bind = engine)
    session = Session()
    print('Database session created!')

    #To inspect the schemas and tables in your database
    inspector = inspect(engine)
    schemas = inspector.get_schema_names()
    for schema in schemas:
        print(f'schema:{schema}')
        for table_name in inspector.get_table_names(schema = schema):
            print(f'table: {table_name}')

    query_code = "your query code from SQL here"

    #Execute query code
    exec_database = session.execute(query_code)
    df = pd.DataFrame(exec_database.fetchall())
    df.columns = exec_database.keys()
    
    print('Dataframe created from database!')
    session.close()
    engine.dispose()

You can also change the part below:

#Execute query code    
exec_database = session.execute(query_code)
df = pd.DataFrame(exec_database.fetchall())
df.columns = exec_database.keys()

to read SQL query directly using pandas using code below:

df = pd.read_sql_query(query_code, engine)

Additionally, part of code below:

#To inspect the schemas and tables in your database
inspector = inspect(engine)
schemas = inspector.get_schema_names()
for schema in schemas:
    print(f'schema:{schema}')
    for table_name in inspector.get_table_names(schema = schema):
        print(f'table: {table_name}')

is only necessary when you don't have any idea what schemas and tables are in your database. You can use those codes above to inspect and show them.