5

Currently i am working on a SFTP protocol.I have created SFTP client Using Jsch Library and SFTP Server using Apache Mina Sshd library.I have made connection between them and can successfully send files to SFTP server.Now i am working on creating a SFTP server side file handler that handles the incoming files.As a example let say SFTP server can receive files from SFTP client but currently in my implementation there is no way to notify when file is arrived into server.I just go server root folder and see if there is a files available.That is how i know if files are arrived.

I would like to implement that when files arrive into server it will notify user to files are arrived and files content.(file Name and other details).But the problem is that i am new to Apache Mina sshd API.i have gone through documentation but i couldn't figured it out.

Please I would like know that if there are any already implemented listeners for handle incoming files in Apache Mina Sshd server or if not how can i implement my own listener for incoming files.

SFTP Server Code

public class SftpServerStarter {

    private SshServer sshd;
    private final static Logger logger = LoggerFactory.getLogger(SftpServerStarter.class);

    public void start(){


        sshd = SshServer.setUpDefaultServer();
        sshd.setPort(22);
        sshd.setHost("localhost");

        sshd.setPasswordAuthenticator(new MyPasswordAuthenticator());
        sshd.setPublickeyAuthenticator(new MyPublickeyAuthenticator());
        sshd.setKeyPairProvider(new SimpleGeneratorHostKeyProvider());
        sshd.setSubsystemFactories(Arrays.<NamedFactory<Command>>asList(new SftpSubsystem.Factory()));
        sshd.setCommandFactory(new ScpCommandFactory());
        sshd.setFileSystemFactory(new VirtualFileSystemFactory("C:/root"));


        try {
            logger.info("Starting ...");
            sshd.start();
            logger.info("Started");
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            logger.info("Can not Start Server");
        }
    }

}
gihan-maduranga
  • 4,381
  • 5
  • 41
  • 74

3 Answers3

14

I started out using @gihan's suggestion, but I ran into some problems with the File Monitors processing the file before the user was finished uploading it on some clients.

Here is the solution I found from poking around in Mina's source code. In spite of the sparse, useless documentation on Apache Mina's site, I think this is the way they intended for developers to use their library.

NOTE: Since your needs are probably different than mine, keep in mind that this may not be a copy-and-paste solution. You will probably need to adapt this code to suit your needs, but I'm fairly confident that this code does provide the key to the solution you're looking for.

Step 1: Implement SftpEventListener

Create your own class that implements org.apache.sshd.server.subsystem.sftp.SftpEventListener. Here's mine as an example. My implementation is set up to run a series of registered FileUploadCompleteListener methods whenever a file is newly uploaded or overwritten, and block user attempts to navigate or create directories.

public class SFTPServiceSFTPEventListener implements SftpEventListener {

    Logger logger = Logger.getLogger(SFTPServiceSFTPEventListener.class);

    SFTPService service;

    public SFTPServiceSFTPEventListener(SFTPService service) {
        this.service = service;
    }

    public interface FileUploadCompleteListener {
        void onFileReady(File file);
    }

    private List<FileUploadCompleteListener> fileReadyListeners = new ArrayList<FileUploadCompleteListener>();

    public void addFileUploadCompleteListener(FileUploadCompleteListener listener) {
        fileReadyListeners.add(listener);
    }

    public void removeFileUploadCompleteListener(FileUploadCompleteListener listener) {
        fileReadyListeners.remove(listener);
    }

    @Override
    public void initialized(ServerSession serverSession, int version) {

    }

    @Override
    public void destroying(ServerSession serverSession) {

    }

    @Override
    public void open(ServerSession serverSession, String remoteHandle, Handle localHandle) {
        File openedFile = localHandle.getFile().toFile();
        if (openedFile.exists() && openedFile.isFile()) {
        }
    }

    @Override
    public void read(ServerSession serverSession, String remoteHandle, DirectoryHandle localHandle, Map<String,Path> entries) {

    }

    @Override
    public void read(ServerSession serverSession, String remoteHandle, FileHandle localHandle, long offset, byte[] data, int dataOffset, int dataLen, int readLen) {

    }

    @Override
    public void write(ServerSession serverSession, String remoteHandle, FileHandle localHandle, long offset, byte[] data, int dataOffset, int dataLen) {
    }

    @Override
    public void blocking(ServerSession serverSession,  String remoteHandle, FileHandle localHandle, long offset, long length, int mask) {
    }

    @Override
    public void blocked(ServerSession serverSession, String remoteHandle, FileHandle localHandle, long offset, long length, int mask, Throwable thrown) {
    }

    @Override
    public void unblocking(ServerSession serverSession, String remoteHandle, FileHandle localHandle, long offset, long length) {
    }

    @Override
    public void unblocked(ServerSession serverSession, String remoteHandle, FileHandle localHandle, long offset, long length, Boolean result, Throwable thrown) {
    }

    @Override
    public void close(ServerSession serverSession, String remoteHandle, Handle localHandle) {
        File closedFile = localHandle.getFile().toFile();
        if (closedFile.exists() && closedFile.isFile()) {
            logger.info(String.format("User %s closed file: \"%s\"", serverSession.getUsername(), localHandle.getFile().toAbsolutePath()));
            this.service.UserWroteFile(serverSession.getUsername(), localHandle.getFile());

            for (FileUploadCompleteListener fileReadyListener : fileReadyListeners) {
                fileReadyListener.onFileReady(closedFile);
            }
        }
    }

    @Override
    public void creating(ServerSession serverSession, Path path, Map<String,?> attrs) throws UnsupportedOperationException {
        logger.warn(String.format("Blocked user %s attempt to create a directory \"%s\"", serverSession.getUsername(), path.toString()));
        throw new UnsupportedOperationException("Creating sub-directories is not permitted.");
    }

    @Override
    public void created(ServerSession serverSession, Path path, Map<String,?> attrs, Throwable thrown) {
        String username = serverSession.getUsername();
        logger.info(String.format("User %s created: \"%s\"", username, path.toString()));
        service.UserWroteFile(username, path);
    }

    @Override
    public void moving(ServerSession serverSession, Path path, Path path1, Collection<CopyOption> collection) {

    }

    @Override
    public void moved(ServerSession serverSession, Path source, Path destination, Collection<CopyOption> collection, Throwable throwable) {
        String username = serverSession.getUsername();
        logger.info(String.format("User %s moved: \"%s\" to \"%s\"", username, source.toString(), destination.toString()));
        service.UserWroteFile(username, destination);
    }

    @Override
    public void removing(ServerSession serverSession, Path path) {

    }

    @Override
    public void removed(ServerSession serverSession, Path path, Throwable thrown) {

    }

    @Override
    public void linking(ServerSession serverSession, Path source, Path target, boolean symLink) throws UnsupportedOperationException {
        logger.warn(String.format("Blocked user %s attempt to create a link to \"%s\" at \"%s\"", serverSession.getUsername(), target.toString(), source.toString()));
        throw new UnsupportedOperationException("Creating links is not permitted");
    }

    @Override
    public void linked(ServerSession serverSession, Path source, Path target, boolean symLink, Throwable thrown) {

    }

    @Override
    public void modifyingAttributes(ServerSession serverSession, Path path, Map<String,?> attrs) {

    }

    @Override
    public void modifiedAttributes(ServerSession serverSession, Path path, Map<String,?> attrs, Throwable thrown) {
        String username = serverSession.getUsername();
        service.UserWroteFile(username, path);
    }
}

Step 2: Add an instance of your listener to your server

Once you've implemented your class, all you need to do is instantiate it and add it to your server using an SftpSubsystemFactory before calling start() on your server:

// Your SSHD Server
SshServer sshd = SshServer.setUpDefaultServer();

SftpSubsystemFactory sftpSubsystemFactory= new SftpSubsystemFactory();

// This is where to put your implementation of SftpEventListener
SFTPServiceSFTPEventListener sftpEventListener = new SFTPServiceSFTPEventListener(this);
sftpEventListener.addFileUploadCompleteListener(new SFTPServiceSFTPEventListener.FileUploadCompleteListener() {
    @Override
    public void onFileReady(File file) {
        try {
            doThingsWithFile(file);
        } catch (Exception e) {
            logger.warn(String.format("An error occurred while attempting to do things with the file: \"%s\"", file.getName()), e);
        }
    }
});
sftpSubsystemFactory.addSftpEventListener(sftpEventListener);

List<NamedFactory<Command>> namedFactoryList = new ArrayList<NamedFactory<Command>>();
namedFactoryList.add(sftpSubsystemFactory);
sshd.setSubsystemFactories(namedFactoryList);

// Do your other init stuff...

sshd.start();

Once you've done that, your implementation of SftpEventListener will start automatically responding to the events you've implemented. Mine basically just responds to when the user closes the file (which occurs when the file upload is complete), but as I said, you can feel free to implement the other methods to respond to other events.

brokethebuildagain
  • 2,162
  • 1
  • 22
  • 44
  • This code won't work. You call a function `onFileReady` and then invoke `onFileReadyForUpload`. This also adds confusion. Is this a service for handling an upload occurring, or handling when an upload is about to happen? Is it ready for upload or uploaded? – christopher Nov 15 '17 at 11:38
  • 1
    Whoops; thanks! I manually refactored the code before posting it to SO and missed the function name. It's supposed to say `onFileReady`. The reason it was called `onFileReadyForUpload` before is because the way I was using this code was for a server that uploaded the files somewhere else once it received them via SFTP. Hopefully my latest edit will clear that up. :) – brokethebuildagain Nov 15 '17 at 19:15
  • Given that close(...) is called at the end of reading a file as well as writing one, what is going to stop doThingsWithFile(...) every time someone does an sftp get? – PapaLazarou Feb 22 '18 at 14:25
  • @PapaLazarou nothing; but for my case it didn't matter. If it does for you, you can easily keep track of when files are created/moved/etc with the `creating`, `moving`, or `write` handlers. Just be careful to ensure that the client is finished with the file before doing anything with it. As I said, adapt the solution to fit your needs. :) – brokethebuildagain Feb 22 '18 at 18:09
  • Off the top of my head, if I were needing to do that, I would have a set of file paths that I updated whenever `write`, `created` , or `moved` (possibly also `modifiedAttributes` if needed) was called on a file. Then in my `close` method, I'd check to see if the filename was in that set. if it was, I'd remove the file path from the set and trigger the event. – brokethebuildagain Feb 22 '18 at 18:31
  • works for me, but when i add some SftpEventListener, the default one is still being invoked – Darlyn Jan 18 '20 at 17:40
  • @Darlyn, that shouldn't be happening. I would suggest looking at how you are attaching your SftpEventListener. It sounds like you have attached the default SftpEventListener instead of your custom implementation. Or you may have attached both. – brokethebuildagain Jan 20 '20 at 18:08
  • @CullenJ i described my process here: https://stackoverflow.com/questions/59803046/overriding-sftpeventlistener-using-apache-mina-sshd , but when you look at the source code, method that handles evens always invokes custom eventHandlers, and then default functionality.. so i guess its default behavior – Darlyn Jan 21 '20 at 14:56
  • @Darlyn I don't think the default remove operations are default event listeners--they're part of mina's core functionality. The EventListener interface just gives you hooks to execute custom code in the process before or after Mina performs actions with files. I have posted an answer to your question. – brokethebuildagain Jan 21 '20 at 18:39
  • 1
    Very nice solution, thanks! Best way to actually use Mina Listeners. I did not use the delegate you introduced, but just implemented the interface, which is enough. In newer versions, all methods are default methods, so it's enough to implement what you need, eg. created, written, ... – Ice09 Apr 22 '20 at 18:19
2

I finally found a solution but it is not coming from Apache Mina SSHD API. Here is the concept: We can monitor the server's root directory for file changes. If there is an file changed in server folder, it will trigger an event. There are plenty of API's available to do this. In my code snippet, I'm using org.apache.commons.io.monitor.

SFTPFileListner Class

public static void startMonitor(String rootFolder) throws Exception {

        //every 5 seconds it will start monitoring
        final long pollingInterval = 5 * 1000;

        File folder = new File(rootFolder);

        if (!folder.exists()) {

            throw new RuntimeException("ERROR : Server root directory not found: " + rootFolder);
        }

        FileAlterationObserver observer = new FileAlterationObserver(folder);
        FileAlterationMonitor monitor = new FileAlterationMonitor(pollingInterval);
        FileAlterationListener listener = new FileAlterationListenerAdaptor() {

            @Override
            public void onFileCreate(File file) {
                try {

                    System.out.println("[SFTPFileListner] Received :"+ file.getName());
                    System.out.println("[SFTPFileListner] Received File Path :"+ file.getCanonicalPath());



                } catch (IOException e) {
                    throw new RuntimeException("ERROR: Unrecoverable error when creating files " + e.getMessage(),e);
                }
            }

        };

        observer.addListener(listener);
        monitor.addObserver(observer);
        monitor.start();
    }

After creating the monitor class, you can call implemented method in your SFTP server class.

SFTP server Class

//pass server root directory 
SFTPFileListner.startMonitor("C:/root");
brokethebuildagain
  • 2,162
  • 1
  • 22
  • 44
gihan-maduranga
  • 4,381
  • 5
  • 41
  • 74
-1

As I mentioned in my other post, Mina didn't directly give us the capability to handle triggers on receipt or partial receipt of an incoming file but our needs were very specific. So our only option was to step outside of Mina, which your solution above is doing. It might be worth pushing that as a pull request / feature for Mina or developing it further as an open source complimenting solution. I think its a common problem that people would face to have an active notification / trigger system when something lands in their server. Good luck with the rest of your development!

Leigh Griffin
  • 84
  • 1
  • 6
  • 1
    Thanks.In Apache Mina incoming files handle SftpSubsystem class.if you clearly study it you can find how they handle files and related information.but the problem is that can not use directly. – gihan-maduranga Jul 17 '15 at 07:07