One way is that you can add command line task to build defintion to call curl command to get file over SSH. If you are using on-premise build agent, you could refer to this thread to install curl in windows.
Another way (if you are familiar with typescript), based on the source code of copyfilesoverssh, it gets necessary information of SSH (e.g. username, password), then based on that information to connect to SSH server and upload files.
var sshEndpoint = tl.getInput('sshEndpoint', true);
var username:string = tl.getEndpointAuthorizationParameter(sshEndpoint, 'username', false);
var password:string = tl.getEndpointAuthorizationParameter(sshEndpoint, 'password', true); //passphrase is optional
var privateKey:string = tl.getEndpointDataParameter(sshEndpoint, 'privateKey', true); //private key is optional, password can be used for connecting
var hostname:string = tl.getEndpointDataParameter(sshEndpoint, 'host', false);
var port:string = tl.getEndpointDataParameter(sshEndpoint, 'port', true);
Also, based on this article’s sample, it can download file from server.
client.scp('admin:password@example.com:/home/admin/file.txt', './', function(err) {
})
So, you could refer to those code to build a custom build task to achieve download files from remote host over SSH/SCP.
Articles about add build task to VSTS: https://www.visualstudio.com/en-us/docs/integrate/extensions/develop/add-build-task and http://blog.devmatter.com/custom-build-tasks-in-vso/