How to use split on a stream in Apache Camel?

Question

When selecting a very large amount of data from a db, I want to use camel's split-component. It should split the stream into 100 objects and get it in iterator format.

I successfully received multiple data from db, however failed during split. I'm getting data from <to>, but I don't know where to write this <to> syntax.

<routes xmlns="http://camel.apache.org/schema/spring">
    <route id="DbToKafka" streamCache="true">
        <from uri="timer:foo?repeatCount=1" />
        <split streaming="true">
            <tokenize token="{" />
            <to uri="sql:classpath:sql/get.sql" />
        </split>
        <log message="DATA : ${body}" />
        <marshal>
            <json library="Jackson" />
        </marshal>
        <log message="JSON : ${body}" />
        <to uri="kafka:test-topic?brokers={{kafka.brokers}}" />
        <log message="data sended : ${body}" />
    </route>
</routes>

2019-09-06 13:29:29.380 ERROR 10260 --- [3 - timer://foo] o.a.camel.processor.DefaultErrorHandler : Failed delivery for (MessageId: ID-DESKTOP-225HFIE-1567744166599-0-2 on ExchangeId: ID-DESKTOP-225HFIE-1567744166599-0-1). Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException: source

Message History --------------------------------------------------------------------------------------------------------------------------------------- RouteId ProcessorId Processor
Elapsed (ms) [DbToKafka ] [DbToKafka ] [timer://foo?repeatCount=1
] [ 9] [DbToKafka ] [split1 ] [split[tokenize{body() using token: ,}]
] [ 7]

run without the split statement, get this result: [{"ROW_ID":"520","COM_CD_NM":"사용중"},{"ROW_ID":"521","COM_CD_NM":"메모지 수수 "},{"ROW_ID":"522","COM_CD_NM":"상호대화
"},{"ROW_ID":"523","COM_CD_NM":"물품수수
"},{"ROW_ID":"524","COM_CD_NM":"기타
"},{"ROW_ID":"525","COM_CD_NM":"자격증변조 "},...]

score 1 · Answer 1 · answered Sep 06 '19 at 07:28

Welcome to StackOverflow!

The errors is because you are trying to split nothing. You need to retrieve the data before you go into the split, and then every operation you want to operate on the split data needs to be within the split element.

I think it won't work anyway, since your marshalling won't work as the tokenize will remove the token { and therefore not be real JSON.

Looking at Apache Camel with Json Array split will give you an example of how to do it. I suspect you need some thing like this - the marshalling may not be necessary as you are splitting by JSONpath anyway.

<route id="DbToKafka" streamCache="true">
    <from uri="timer:foo?repeatCount=1" />
    <to uri="sql:classpath:sql/get.sql" />
    <split streaming="true">
        <jsonpath>$</jsonpath>
        <log message="DATA : ${body}" />
        <marshal>
            <json library="Jackson" />
        </marshal>
        <log message="JSON : ${body}" />
        <to uri="kafka:test-topic?brokers={{kafka.brokers}}" />
        <log message="data sended : ${body}" />
    </split>
</route>

How to use split on a stream in Apache Camel?

1 Answers1