Marklogic DMSDK Transform Module?

Question

Usecase : I want to get all the contents of my documents in db and store it in one zip file.

I used the ml-java-util to convert the content into zip file. My server side transformation module logic is like below:

1. I went into each document and extracted only required fields using node Xpath.
2. But when I executed , it created a zip file when I opened, it has set of files based on Uri name and with extracted content. Rather than merging all the file contents into one it is creating new file for every URI.

How can I override that behavior? I want all the content (i.e.extracted values) in all the documents to be in a single file that too converted to a zip.

Tried to apply For loop but thought it is of no use since the function itself getting uri's (i.e. context.uri) one by one.

Any help is appreciated.

Thanks

Answer 1

You can use ExportToWriterListener! ExportToWriterListener exports all the contents retrieved by QueryBatcher and writes to a File.

DatabaseClient client = DatabaseClientFactory.newClient("localhost", 8012,
    new DatabaseClientFactory.DigestAuthContext("admin", "admin"));
DataMovementManager moveMgr = client.newDataMovementManager();
ServerTransform transform = new ServerTransform("transformName");
File outputFile = new File("output.txt"); // pass in your file here
String collection = "customers";
StructuredQueryDefinition query = new  StructuredQueryBuilder().collection(collection); // Substitute your query here
try (FileWriter writer = new FileWriter(outputFile)) {
  ExportToWriterListener exportListener = new ExportToWriterListener(writer)
    .withRecordSuffix("\n")
    .withTransform(transform) // pass in your Server Transform here
    .onGenerateOutput(
      record -> {
        String contents = record.getContentAs(String.class); 
        return contents; // return the content as it is which is the server transformed documents' content
      }
    );

  QueryBatcher queryJob =
    moveMgr.newQueryBatcher(query)
      .withThreadCount(5)
      .withBatchSize(10)
      .onUrisReady(exportListener)
      .onQueryFailure( throwable -> throwable.printStackTrace() );
  moveMgr.startJob( queryJob );
  queryJob.awaitCompletion();
  moveMgr.stopJob(queryJob);
}

Then you can create a zip out of the file.