1

I would like to right unit and integration test cases to test all the functional flow. Can someone share your thoughts on how I can create a mock endpoint to watch directory for file creation and further process them via multiple routes and move the file to different directories.

  1. Monitoring the /hello directory for new file creation.
  2. If there is a file based on the file prefix, updated the headers with container name.
  3. Upload to appropriate azure blob container based on file name.
  4. Make an api call and move the file to /success directory once its processed.

FileWatcherRoute.java

@Service
public class FileWatcherRoute extends RouteBuilder {

  @Value("${watcher-base-url}")
  private String baseUrl;

  @Value("${watcher-subscription-key}")
  private String subscriptionKey;

  @Override
  public void configure() {

    Processor logResponse = exchange -> log
        .info("The response code is: {}", exchange.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE));

    from("file-watch:hello?events=CREATE&antInclude=**/*.csv&recursive=true")
        .routeId("fileWatch")
        .to("direct:updateHeaders")
        .end();

    from("direct:updateHeaders")
        .routeId("updateHeaders")
        .choice()

            .when((exchange -> exchange.getIn().getHeader(Exchange.FILE_NAME).toString().trim().matches("\\d{8}_\\d{4}(_Inventory.csv)")))
            .setHeader("CamelAzureStorageBlobContainerName", constant(AppConstants.STORE))

                        .when(exchange -> exchange.getIn().getHeader(Exchange.FILE_NAME).toString().trim().matches("\\d{8}-\\d{6}_Idle_Inventory_\\d{4}.csv"))
            .setHeader("CamelAzureStorageBlobContainerName",constant(AppConstants.IDLE_FILE))

        .toD("direct:uploadFileToBlob")
        .end();

    from("direct:uploadFileToBlob")
        .routeId("uploadFile")
        .log("Container Name: ${header.CamelAzureStorageBlobContainerName}")
        .toD("azure-storage-blob://{accName}/${header.CamelAzureStorageBlobContainerName}?blobName=${header.CamelFileName}&operation=uploadBlockBlob&serviceClient=#serviceClient")
        .to("direct:startRestApi")
        .log("${header.CamelFileName} Uploaded to ${header.CamelAzureStorageBlobContainerName} Container Successfully")
        .end();


    from("direct:startRestApi")
        .routeId("restCall")
        .setHeader(Exchange.HTTP_METHOD, constant("GET"))
        .setHeader("Content-Type",constant("application/json"))
        .setHeader("Ocp-Apim-Subscription-Key",constant(subscriptionKey))
        .to(baseUrl)
        .to("direct:processedFiles")
        .process(logResponse)
        .end();

    from("direct:processedFiles")
        .routeId("fileProcessing")
        .choice()
        .when(exchange -> exchange.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE).equals(200))
        .to("file://success")
        .otherwise()
        .to("file://error")
        .end();
  }
}

FileWatcherRouteTest.java

@CamelSpringBootTest
@SpringBootTest
@MockEndpoints
@UseAdviceWith
public class FileWatcherRouteTest {

  @Autowired
  CamelContext camelContext;

  @Autowired
  ProducerTemplate producerTemplate;

  @EndpointInject("mock:processedFiles")
  MockEndpoint mockEndpoint;

  @Test
  void when_new_file_created_should_update_header_with_container_name_and_upload_to_container() throws Exception {

    AdviceWith.adviceWith(camelContext, "fileWatch", routeBuilder -> {
      routeBuilder.replaceFromWith("direct:file-watch");
    });

    camelContext.start();
    mockEndpoint.assertIsSatisfied();

  }

}
TechGeek
  • 480
  • 1
  • 8
  • 22

1 Answers1

4

MockEndpoint only supports producer endpoints. This is due to the fact that it is designed to perform assertions to Exchanges and Messages.

For example:

  • Does the given MockEndpoint receive the correct amount of messages
  • Does the message body match with what is expected.
  • Where the required headers provided to the MockEndpoint

When testing a route you should use ProducerTemplate send methods to start the route under test with whatever body, headers and properties you want to test with it.

In your case you can use java file operations to read a file from test resources folder as string, byte array or stream and use send it as body along with headersCamelFileName and CamelFileEventType to target route.

I would also recommend treating your direct routes like functions to make them easier to test. Meaning that instead of jumping from route to route you could have parent route that just calls the child routes in correct order.

from("file-watch:hello?events=CREATE&antInclude=**/*.csv&recursive=true")
    .routeId("fileWatch")
    .to("direct:processNewFile")

// "Parent route"
from("direct:processNewFile")
    .routeId("processCreatedFile")
    .to("direct:updateHeaders")
    .to("direct:uploadFileToBlob")
    .to("direct:processedFiles")

This allows you easily write individual tests for direct:updateHeaders, direct:uploadFileToBlob and direct:processedFiles

For example:

  • Test that direct:uploadFileToBlob throws proper exception if one of the required headers is missing or somehow invalid.
  • Test that direct:uploadFileToBlob properly handles connection exceptions with azure.
  • Test that direct:processedFiles places files to correct folders.

If you want to test file output you can use JUnits TemporaryFolder to create temporary folder which you can then target with your file producer endpoint. After route has completed you can use basic assertEquals and file operations to check whether the output meets your test requirements.

Apache Commons IO is also pretty handy library for tests for reading files from resources and copying files to TemporaryFolder.

Couple examples with files, ProducerTemplate, Commons IO and CamelTestSupport:

package com.example;

import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

import org.apache.camel.Exchange;
import org.apache.camel.RoutesBuilder;
import org.apache.camel.builder.AdviceWithRouteBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.apache.commons.io.FileUtils;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;

public class ExampleTest  extends CamelTestSupport {
    
    @Rule
    public TemporaryFolder temporaryFolder = new TemporaryFolder();
    
    File inputFolder;
    File outputFolder;

    @Test
    public void temporaryFolderExampleTest() throws Exception {

        context.adviceWith(context.getRouteDefinition("processInputFile"), 
            new AdviceWithRouteBuilder(){

                @Override
                public void configure() throws Exception {
                    
                    replaceFromWith("direct:start");

                    weaveAddLast()
                        .to("mock:result");
                }
            }
        );

        MockEndpoint resultMockEndpoint = getMockEndpoint("mock:result");
        resultMockEndpoint.expectedMessageCount(1);
        resultMockEndpoint.message(0).body().isEqualTo("Hello world!");

        InputStream body = fetchFileFromResourcesFolderAsStream("test-files/Hello.txt");
        Map<String, Object> headers = new HashMap<>();
        headers.put(Exchange.FILE_NAME, "Hello.txt");

        startCamelContext();
        template.sendBodyAndHeaders("direct:start", body, headers);

        resultMockEndpoint.assertIsSatisfied();

        File resultFile = new File(outputFolder, "Hello.txt");
        assertEquals(true, resultFile.exists());

        // FileUtils from commons-io/commons-io/2.11.0
        String result = FileUtils.readFileToString(resultFile, StandardCharsets.UTF_8);
        assertEquals("Hello world!", result);
    }

    @Test
    public void pollEnrichExampleTest() throws Exception {

        context.adviceWith(context.getRouteDefinition("pollEnrichExample"), 
            new AdviceWithRouteBuilder(){

                @Override
                public void configure() throws Exception {

                    weaveAddLast()
                        .to("mock:result");
                }
            }
        );

        MockEndpoint resultMockEndpoint = getMockEndpoint("mock:result");
        resultMockEndpoint.expectedMessageCount(1);
        resultMockEndpoint.message(0).body().isEqualTo("Hello");

        File resourceFile = fetchFileFromResourcesFolder("test-files/Hello.txt");
        File testFile = new File(outputFolder, "Hello.txt");
        FileUtils.copyFile(resourceFile, testFile);
        
        startCamelContext();
        template.sendBody("direct:pollEnrichExample", null);

        resultMockEndpoint.assertIsSatisfied();
    }

    @Override
    protected RoutesBuilder createRouteBuilder() throws Exception {
    
        return new RouteBuilder() {

            @Override
            public void configure() throws Exception {
             
                from("file:{{file.input}}")
                    .routeId("processInputFile")
                    .convertBodyTo(String.class)
                    .setBody(simple("${body} world!"))
                    .log("${body}")
                    .to("file:{{file.output}}");

                from("direct:pollEnrichExample")
                    .routeId("pollEnrichExample")
                    .pollEnrich("file:{{file.output}}", 3000)
                    .convertBodyTo(String.class)
                    .log("${body}");
            }
        };
    }

    // use of placeholder properties and configuration files is highly encouraged
    @Override
    protected Properties useOverridePropertiesWithPropertiesComponent() {
        
        try {
            inputFolder = temporaryFolder.newFolder("input");
            outputFolder = temporaryFolder.newFolder("output");

        } catch (Exception e) {
            e.printStackTrace();
        }

        Properties prop = new Properties();
        prop.setProperty("file.input", inputFolder.getPath());
        prop.setProperty("file.output", outputFolder.getPath());
        return prop;
    }

    @Override
    public boolean isUseAdviceWith() {
        return true;
    }

    public File fetchFileFromResourcesFolder(String pathInResources){
        
        ClassLoader classLoader = ExampleTest.class.getClassLoader();
        return new File(classLoader.getResource(pathInResources).getFile());
    }

    public InputStream fetchFileFromResourcesFolderAsStream(String pathInResources){

        try {
            return new FileInputStream(fetchFileFromResourcesFolder(pathInResources));   
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }
}

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.11.0</version>
</dependency>

Above examples using Camel 3.4.4

Pasi Österman
  • 2,002
  • 1
  • 6
  • 13