0

i'm trying to read sasdataset using pyreadstat directly from sftp server, as i couldn't download the data, i'm trying this approach:

import pysftp


class My_Connection(pysftp.Connection):
    def __init__(self, *args, **kwargs):
        try:
            if kwargs.get('cnopts') is not None:
                return
            kwargs['cnopts'] = pysftp.CnOpts()
            kwargs['cnopts'].hostkeys = None
        except pysftp.HostKeysException as e:
            self._init_error = True
            print('Warning Failed to load Host-keys')
        else:
            self._init_error = False

        self._sftp_live = False
        self._transport = None
        super().__init__(*args, **kwargs)

    def __del__(self):
        if not self._init_error:
            self.close()
with My_Connection(SFTP_HOST,username=SFTP_USER,password=SFTP_PASSWORD,) as conn:
    conn.cwd('/sas/sasdata/sasdev/sasdatasets')
    with open("/sas/sasdata/sasdev/sasdatasets/statfile.sas7bdat", 'r') as fp:
        df = pyreadstat.read_sas7bdat(fp)

can anyone help me how we can read sasdataset directly in server, when file size is big?

Rakesh Dash
  • 1
  • Can you use SASpy instead? https://sassoftware.github.io/saspy/ – Reeza Nov 19 '21 at 17:05
  • Is it an SFTP server or SAS server? Those are very different use cases. – Reeza Nov 19 '21 at 17:06
  • Is this a SAS server that you have access to (or could be given access to)? If so then Reeza's suggestion of SASPy or SWAT (SASPy for 9.4 and SWAT for Viya) is the best choice by far. – Joe Nov 19 '21 at 17:12
  • @Reeza in my organization we don't run locally, mostly in remote server , how we can connect to saspy locally, i was trying to connect through swat its not taking the port 22 which i generally while logging to sas using htpps url, i tried another port not working though, do you have any idea on this please suggest, really appreciate your thoughts on this. – Rakesh Dash Nov 20 '21 at 07:06

1 Answers1

0

The open function still loads a local file, no matter what magic you attempt before.

You need to use pysftp Connection.open:

with conn.open("statfile.sas7bdat", 'r') as fp:

Though pysftp seems dead project. Consider using Paramiko instead.

Paramiko also has its open, the SFTPClient.open. See for example:
Reading .csv file to memory from SFTP server using Python Paramiko

Also your hostkeys magic seems suspicious. Make sure you know what you are doing, not to make your connection insecure. See Verify host key with pysftp.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992