Python: SFTP: List the directories and files recursively based on the regex pattern

Question

I am automating my task by writing scripts using Python 2.7. As part of my work, I should fetch the appropriate files from the directories in a SFTP server. The directories with the file name are mentioned in the table in a regex pattern format. The directory depth, regex pattern for the files and directories differ from one file to the other.

The regex pattern in the table is as below:

file_pattern
/dir1/dir2/dir3/(\d{10}/[^/]*/(test_\d{12}_full).txt.gz$
/dir1/dir2/dir3/test_[a-z]{4}_\d{12}_full).txt.gz$

I am planning to iterate over the file_pattern column values to fetch the file pattern using which I check if there are appropriate files matching the regex pattern.

Below is my Python script:

import MySQLdb
import paramiko

def connect_to_ftp():
    cursor = db.cursor()
    db = MySQLdb.connect(<ip_address>, <user_name>, <password>, <db_name>)
    file_pattern = "select file_pattern from {0}".format(<table name>)
    cursor.execute(file_pattern)
    file_pattern_all = cursor.fetchall()
    link_ip_addr = paramiko.Transport(<FTP_ip_address>, <port_number>) 
    link_ip_addr.connect(<user_name>, <password>)
    sftp_conn =  paramiko.SFTPClient.from_transport(transport)
    ftp_list = sftp_conn.listdir()
    print ftp_list
connect_to_ftp()

I am able to connect to the SFTP server, list the directories present in the depth 1. I need a solution that would help me go to the depth matching the regex pattern and list the appropriate files in the SFTP server.

The answers for the post: Recursive directory download with Paramiko? is for the file path that do not have regex pattern. I need a similar solution for the file path and file name with the regex pattern.

Please help me with the relevant solution.