1

I have a python code that needs to get file or directory information on a remote host that's Unix/Linux (OS names: HP-UX/RedHat/SunOS/AIX/Linux/etc).

The program SSH into the remote host (using paramiko library) and executes ls -l or ls -ld depending on if it's a file or a directory.

Information I need are:

  1. Permission (user/group/other)
  2. User owner
  3. Group owner
  4. Last Modified
  5. File name

However, the problems with ls are:

  1. Output is different from platform so special handling is needed which makes code verbose using checks.
  2. File size unit may be different in the output depend on environment variable (for GNU coreutils, BLOCK_SIZE), or some may even not support this. This also requires platform-specific checks.

I am looking for a python library or simple portable executable if there is one.

Solutions I considered (but seems infeasible)

  1. Use regex to check format of the output and process if it matches konwn format. However, this seems to be error prone due to try-checks.
  2. Also try-check environment variables for the file size and found out the file size unit in the output. (e.g. echo few characters to a file and check the unit. If 4 char is written and size says 1 then I know for sure the unit is greater than 1 bytes. Repeat the steps). This seems to be also error prone
  3. Install cross-platform compiler on each host, compile and then execute. Can't do it since if the host OS is reinstalled or restored to the point that doesn't have compiler, this installation process needs to be repeated.

Any suggestions?

Harry Cho
  • 2,309
  • 4
  • 20
  • 28

3 Answers3

3

Would stat do the trick? It seems like it has all the features you're looking for and should already be installed.

http://ss64.com/bash/stat.html

If you want to work in python, python has a built in library called stat which provides similar functionality:

https://docs.python.org/2/library/stat.html

Christopher Shroba
  • 7,006
  • 8
  • 40
  • 68
  • HP-UX and AIX doesn't have `stat` installed. And due to limitation, we cannot install new packages on the system. – Harry Cho Dec 05 '16 at 23:52
  • You may want to try the python library. It will likely still work, since it uses the stat Linux library call, not the executable. See my edited answer – Christopher Shroba Dec 05 '16 at 23:54
  • Does `stat` python package works with remote hosts? I think it's only for local files – Harry Cho Dec 05 '16 at 23:55
  • Well the only thing that is working with a remote host is SSH. When you connect to a remote host over SSH, everything you run in that SSH session is happening *locally* on the remote host. In the example you gave, you ran ls locally on the remote host. So in a similar way, I'm proposing you connect to your remote host over SSH, and from that session, run a python program which uses stat. – Christopher Shroba Dec 06 '16 at 00:06
  • Here's an example of using stat from python to get the modification time of an object on a remote host: http://i.imgur.com/PSPbmTc.png – Christopher Shroba Dec 06 '16 at 00:06
1

SFTP is a standard file transport protocol built into SSH-2. So if you can SSH into the remote hosts, then most likely you can use SFTP in a standard way to list and stat files. SFTP is widely supported by SSH servers including OpenSSH.

paramiko is probably the most popular SSH/SFTP wrapper for Python. Here is an example script using paramiko to do an SFTP stat:

import paramiko

ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.set_missing_host_key_policy(paramiko.WarningPolicy()) # Only warn on no known_hosts
ssh.connect("localhost", username="sam")
sftp = ssh.open_sftp()
try:
  listing = sftp.listdir_attr(".")
  print(listing[0:10])
  my_stat = sftp.stat(".")
  print(my_stat)
  print(my_stat.st_size, my_stat.st_mtime, my_stat.st_atime, my_stat.st_uid, my_stat.st_gid)
except IOError:
  pass
ssh.close()

Of course this is just a simple example and there is a lot more you can do with paramiko SFTP API - docs.

Update: Related post: SFTP in Python? (platform independent)

spinkus
  • 7,694
  • 4
  • 38
  • 62
0

You probably don't have python installed on all those machines either, but I bet you have some ancient crusty version of perl on even the HPUX box.

~/tmp/t3 $perl -e 'print (join (",", (stat("t.awk"))), "\n");'
655368,160089,33204,1,16257,200,0,178,1480727842,1480710575,1480710575,4096,8

~/tmp/t3 $stat t.awk
  File: `t.awk'
  Size: 178             Blocks: 8          IO Block: 4096   regular file
Device: a0008h/655368d  Inode: 160089      Links: 1
Access: (0664/-rw-rw-r--)  Uid: (16257/mcgowan)   Gid: (  200/    users)
Access: 2016-12-02 17:17:22.000000000 -0800
Modify: 2016-12-02 12:29:35.000000000 -0800
Change: 2016-12-02 12:29:35.000000000 -0800

Here's the cross reference for the perl stat fields:

  0 dev      device number of filesystem
  1 ino      inode number
  2 mode     file mode  (type and permissions)
  3 nlink    number of (hard) links to the file
  4 uid      numeric user ID of file's owner
  5 gid      numeric group ID of file's owner
  6 rdev     the device identifier (special files only)
  7 size     total size of file, in bytes
  8 atime    last access time in seconds since the epoch
  9 mtime    last modify time in seconds since the epoch
 10 ctime    inode change time in seconds since the epoch (*)
 11 blksize  preferred I/O size in bytes for interacting with the
             file (may vary from file to file)
 12 blocks   actual number of system-specific blocks allocated
             on disk (often, but not always, 512 bytes each)
Ian McGowan
  • 3,461
  • 3
  • 18
  • 23
  • And of course you'll have to do some extra work to turn uid/gid into names and walk the directory tree if you need all the entries in a directory. Should all be doable with a very minimal perl install, no CPAN modules required... – Ian McGowan Dec 06 '16 at 01:43