How to write byte or stream to Apache Camel FTP to transfer file

Question

In my code currently, I get data from the database and then I write a file out of the data. I have this kind of camel route and working solution:-

private static final String INPUT_FILE_DIRECTORY_URI = "file:" + System.getProperty("user.home")
        + "/data/cdr/?noop=false";

private static final String SFTP_SERVER = "sftp://" +System.getProperty("user.name")
        + "@sftp_server_url/data/cdr/?privateKeyFile=~/.ssh/id_rsa&passiveMode=true";

from(INPUT_FILE_DIRECTORY_URI)
            .streamCaching()
            .log("Sending file to local sftp")
            .to(SFTP_SERVER);

I don't want to write a file in the local disk. Instead, I want to write file data directly to the SFTP server. I don't know how to do it? But I imagine it should be possible to do it. Can you tell me is it possible? If yes, how to do it?

score 1 · Accepted Answer · answered Dec 11 '18 at 07:43

I managed to solve this problem in another way. It is more suitable for my particular problem.

             byte[] csvData = csvStringBuilder.toString().getBytes();
             Routes.withProducer(producer)
                    .withHeader(Exchange.FILE_NAME, myCsvFile.csv)
                    .withBody(csvData)
                    .to(SFTP_SERVER).request(byte[].class);

Thomas · Answer 2 · 2018-12-04T12:57:47.347

0

You shouldn't use streamCaching unless you really using it. It store your file in memory, use it if you need to consume multiples times your input.

You can use Jpa component or a custom bean getting your data. Load it from database and then send it to your ftp server.

With Jpa :

@Entity 
@NamedQuery(name = "data", query = "select x from Data x where x.id = 1") 
public class Data { ... }

After that you can define a consumer uri like this one:

from("jpa://org.examples.Data?consumer.namedQuery=data")
.to("SFTP_SERVER");

EDIT : to convert a list to csv and send it to ftp :

from("jpa://org.examples.Data?consumer.namedQuery=data")
.marshal()
.csv()
.to("sftp://" +System.getProperty("user.name") + 
"@sftp_server_url/data/cdr/myFile.csv?" +"privateKeyFile=~/.ssh/id_rsa&passiveMode=true");

See CSV component who convert a list to a csv file.

edited Dec 04 '18 at 12:57

answered Dec 04 '18 at 08:40

Thomas

1,008
3
16
34

Thanks. But actually, I don't get my data from the database. I get it as a list from another source. There is no camel component for it. So the question should be how do I write a list to the sftp endpoint? The list data should be saved as CSV file at sftp. – masiboo Dec 04 '18 at 10:45
You need to marshall your list with the camel csv component and then send it to your ftp server. – Thomas Dec 04 '18 at 12:58

score 0 · Answer 3 · answered Sep 22 '20 at 10:55

Yes it is possible :) To do this send the file inputStream in a camel DIRECT component and in the associated route make the copy to FTP. I use this case, to upload a file and directly copy it to ftp with from(directInputStreamName).to(yourFtpUri). This is an sample code :

Your service

@Service
public class FileService {
  @Produce(uri = PfnumDownloadConstants.CAMEL_DIRECT_UPLOAD)
  private ProducerTemplate producer;

  public void sendFileToFtp(File fileToSend, String ftpDestinationUri) throws IOException {
    Map<String, Object> headers = new HashMap<>();
    //In this variable you can init the ftp destination uri or you can hard code it in route
    headers.put("destinationUri", ftpDestinationUri);
    //set filename to name your file in ftp
    headers.put(Exchange.FILE_NAME_ONLY, file.getName());
    InputStream targetStream = new FileInputStream(file);
    //send stream as body and list of headers to direct 
    producer.sendBodyAndHeaders(targetStream, headers);
  }
}

Your Camel route

@Component
public class FileUploadRoute extends RouteBuilder {
  @Override
  public void configure() throws Exception {
    //Manage camel exception in a dedicated processor
    onException(Exception.class).process(exceptionProcessor).log("error :: ${exception}");
  
    from(CAMEL_DIRECT_UPLOAD)
    .log("file copy to ftp '${header.CamelFileNameOnly}' in  process")
    .toD("file:/mnt?fileName=${header.CamelFileNameOnly}&delete=false")
    .log("copy done");
  }
}

How to write byte or stream to Apache Camel FTP to transfer file

3 Answers3