I would like to send files to FTPS server using Apache Camel. The problem is that this FTPS server requires that the TLS/SSL session is to be reused for the data connection. And I can't set 'TLSOptions NoSessionReuseRequired' option for security reason to solve the issue.
As far as I know, Apache Camel uses Apache Common Net class FTPSClient internally to communicate to FTPS servers and Apache Common Net doesn't support this feature as described here
So I has implemented this workaround. Here is code of my custom FTPSClient:
public class SSLSessionReuseFTPSClient extends FTPSClient {
// adapted from: https://trac.cyberduck.io/changeset/10760
@Override
protected void _prepareDataSocket_(final Socket socket) throws IOException {
if (socket instanceof SSLSocket) {
final SSLSession session = ((SSLSocket) _socket_).getSession();
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
// final Method getHostMethod = socket.getClass().getDeclaredMethod("getHost");
Method getHostMethod;
try {
getHostMethod = socket.getClass().getDeclaredMethod("getPeerHost");
} catch (NoSuchMethodException e) {
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object host = getHostMethod.invoke(socket);
final String key = String.format("%s:%s", host, String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT);
putMethod.invoke(cache, key, session);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
}
It works brilliantly as standalone FTPS client in JDK 8 and JDK 11 as shown:
public class FTPSDemoClient {
public static void main(String[] args) throws IOException {
System.out.println("Java version is: " + System.getProperty("java.version"));
System.out.println("Java vendor is: " + System.getProperty("java.vendor"));
final SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
System.setProperty("jdk.tls.client.protocols", "TLSv1,TLSv1.1,TLSv1.2");
System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2");
//System.setProperty("javax.net.debug", "all");
ftps.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
ftps.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
ftps.connect("my_ftps_server");
System.out.println("Connected to server");
ftps.login("user", "password");
System.out.println("Loggeded to server");
ftps.setFileType(FTP.BINARY_FILE_TYPE);
// Use passive mode as default because most of us are
// behind firewalls these days.
ftps.enterLocalPassiveMode();
ftps.setUseEPSVwithIPv4(true);
// Set data channel protection to private
ftps.execPROT("P");
for (final String s : ftps.listNames("directory1/directory2")) {
System.out.println(s);
}
// send file
try (final InputStream input = new FileInputStream("C:\\testdata\\olympus2.jpg")) {
ftps.storeFile("directory1/directory2/olympus2.jpg", input);
}
// receive file
try (final OutputStream output = new FileOutputStream("C:\\testdata\\ddd.txt")) {
ftps.retrieveFile(""directory1/directory2/ddd.txt", output);
}
ftps.logout();
if (ftps.isConnected()) {
try {
ftps.disconnect();
} catch (final IOException f) {
// do nothing
}
}
}
}
Now I am ready to use this custom FTPSClient in my Apache Camel route, first I create custom FTPSClient instance and make it available for Apache Camel:
public final class MyFtpClient {
public static void main(String[] args) {
RouteBuilder routeBuilder = new MyFtpClientRouteBuilder();
System.out.println("Java version is: " + System.getProperty("java.version"));
System.out.println("Java vendor is: " + System.getProperty("java.vendor"));
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", String.valueOf(false));
System.setProperty("jdk.tls.client.protocols", "TLSv1,TLSv1.1,TLSv1.2");
System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2");
SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
ftps.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
// ftps.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
ftps.setRemoteVerificationEnabled(false);
ftps.setUseEPSVwithIPv4(true);
SimpleRegistry registry = new SimpleRegistry();
registry.bind("FTPClient", ftps);
// tell Camel to use our SimpleRegistry
CamelContext ctx = new DefaultCamelContext(registry);
try {
ctx.addRoutes(routeBuilder);
ctx.start();
Thread.sleep(5 * 60 * 1000);
ctx.stop();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
And use it in Apache Camel Route:
public class MyFtpClientRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
// lets shutdown faster in case of in-flight messages stack up
getContext().getShutdownStrategy().setTimeout(10);
from("ftps://my_ftps_server:21/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
// use system out so it stand out
System.out.println("*********************************************************************************");
System.out.println("Use ctrl + c to stop this application.");
System.out.println("*********************************************************************************");
}
}
And it works! But, when I add another route in the same java code by adding second from clause like this:
from("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
from("file://c:/testdata?noop=true&readLock=changed&delay=30s")
.to("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&stepwise=false&ftpClient=#FTPClient") // changed from FTPClient to FTPClient1
.log("Upload file ${file:name} complete.");
it ruins my code, it throws exception:
org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: null Socket is closed. Code: 226
...
Caused by: java.net.SocketException: Socket is closed
at java.net.Socket.setSoTimeout(Socket.java:1155) ~[?:?]
at sun.security.ssl.BaseSSLSocketImpl.setSoTimeout(BaseSSLSocketImpl.java:637) ~[?:?]
at sun.security.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:74) ~[?:?]
at org.apache.commons.net.ftp.FTP._connectAction_(FTP.java:426) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:668) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:658) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.ftp.FTPSClient._connectAction_(FTPSClient.java:221) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.SocketClient._connect(SocketClient.java:254) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.commons.net.SocketClient.connect(SocketClient.java:212) ~[commons-net-3.8.0.jar:3.8.0]
at org.apache.camel.component.file.remote.FtpOperations.doConnect(FtpOperations.java:125) ~[camel-ftp-3.4.1.jar:3.4.1]
Files, anyway are transferred to and from FTPS server by Apache Camel.
Interesting thing, when I don't share my custom FTPSClient and use one instance exactly for one route like this:
SSLSessionReuseFTPSClient ftps = new SSLSessionReuseFTPSClient();
...
SSLSessionReuseFTPSClient ftps1 = new SSLSessionReuseFTPSClient();
...
SimpleRegistry registry = new SimpleRegistry();
registry.bind("FTPClient", ftps);
registry.bind("FTPClient1", ftps1);
from("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&ftpClient=#FTPClient") // #FTPClient
.to("file://c:/testdata?noop=true&readLock=changed")
.log("Downloaded file ${file:name} complete.");
from("file://c:/testdata?noop=true&readLock=changed&delay=30s")
.to("ftps://my_ftps_server/directory1/directory2?username=user&password=RAW(password)"
+ "&localWorkDirectory=/tmp&autoCreate=false&passiveMode=true&binary=true&noop=true&resumeDownload=true"
+ "&bridgeErrorHandler=true&throwExceptionOnConnectFailed=true&maximumReconnectAttempts=0&transferLoggingLevel=OFF"
+ "&readLock=changed&disconnect=true&stepwise=false&ftpClient=#FTPClient1")
.log("Upload file ${file:name} complete.");
it works perfectly!
So, I have couple of questions:
- Why does Apache Camel (I mean Apache Common Net) developers refuse (or can't) to add usage of same TLS session functionality to
FTPSClientclass since 2011? - Am I the only person who uses Apache Camel to work with FTPS server with data connection using same TLS session? I haven't managed to find solution anywhere.
- Is it possible to force Apache Camel not to share custom
FTPSClientinstance what, I suppose is the root of the problem, but to create new instance ofFTPSClientevery time then route are processed? My solution doesn't seem elegant. - What is wrong in my custom
FTPSClientimplementation that leads to this error then I use instance of this class in Apache Camel? StandardFTPClienthasn't this issue, of course.
