Can I inject a single, static item into a Spring Batch item reader?

Question

We have a Spring Batch job that pulls a dynamic list of recipients from a file. We want to add a single extra recipient to serve as a quality control. I thought about adding a new tasklet that just spits out this record and passes it along to the real reader. I've read a few questions here, articles elsewhere and the documentation about transferring data between Spring Batch steps, but I'm not sure that's the easiest, or best way to accomplish this.

_{Like the official documentation using listeners, this article using autowired components and different listeners, and this question and answers.}

If I did get a generator tasklet set up and pass its data into the reader, how would I insert it into the reader's actual records?

Some snippets of the code we're working with – it's purely annotation driven, no XML config setup anywhere.

Step builder

public Step loadRecipients() {
    return stepBuilderFactory.get("loadRecipients").<Recipient, Recipient>chunk(chunkSize)
            .reader(recipientsItemReader)
            .processor(recipientsItemProcessor)
            .writer(recipientsWriter)
            .taskExecutor(taskExecutor)
            .throttleLimit(1)
            .build();
}

Reader config

@StepScope
public FlatFileItemReader<Recipient> recipientItemReader() {

    FlatFileItemReader<Recipient> itemReader = new FilePrefixItemReader<>(
            "theFilePath",
            staticResourceLoader(),
            FunctionUtils.propagateExceptions((org.springframework.core.io.Resource resource) -> new GZIPInputStream(resource.getInputStream()))
    );

    userCategoryItemReader.setLineMapper(userCategoriesDefaultLineMapper);

    return userCategoryItemReader;

}

Should I just finagle my extra record into the resource input stream with some funky wrapper? Is there some other Spring magic I can use to add my static record?

score 1 · Answer 1 · answered Apr 04 '16 at 07:39

wrap/extend the Writer and add the static item there, rough sourcecode:

public class AddStaticItemWriter implements ItemWriter<String> {

    @Override
    public void write(final List<? extends String> items) throws Exception {
        // check some funky condition
        if (addStaticItem) {
            items.add(STATIC_ITEM);
        }
        // business code
        // or delegate to underlying writer
    }
}

some hints (pros, cons):

the added item is not known to spring batch, might lead to some weird things with roll-back scenarios (skip, re-try)
like above, you could wrap the reader and add the item there

This is a perfect answer for a job that operates in a single batch, but for a job with multiple batches your `funky condition` you mention also has to be gated to inject it only once. That said, you could abstract and generify the class fairly easily for lots of re-use. — Patrick M, May 16 '16 at 20:45

score 0 · Accepted Answer · answered May 16 '16 at 20:43

Rather than perverting an item writer, I ended up making a specific tasklet for this. The major downside for the item writer approach was that the current implementation is very lean and has a lot of reused code. Extending the item writer added some code that didn't really belong there.

The major upside of the tasklet was upholding the a single-responsibility principle. It was very easy to get the tasklet to write to the database resource. If the writer was writing to a more complicated resource (such as a REST template or file destination), the hybridized writer would have been much cleaner. (Note, there was more code needed to get all the recipient parameters in order, this is just a basic tasklet example.

/**
 * Inject the internal email recipient, for monitoring and informational purposes.
 */
public class InjectInternalEmailRecipientTasklet implements Tasklet{

    public static final Float DEFAULT_MAX_AFFINITY_SCORE = 1.0f;

    private UserCategoryRepository userCategoryRepository;

    public InjectInternalEmailRecipientTasklet(RecipientRepository recipientRepository) {
        this.recipientRepository = recipientRepository;
    }

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {

        // We can safely inject this record even on non-prod environments because the email processor obfuscates all emails on
        // non-prod environments. N.B. we do not want the internal user to receive TEST emails/placements.

        recipeintRepository.bulkInsert(new Recipient("testemail@example.com");
        return RepeatStatus.FINISHED;
    }
}

And adding the tasklet step to the job config is straightforward as well.

public Job loadRecipients() {
    return jobs.get("loadRecipients")
            .start(truncateRecipientsStep())
            .next(injectStaticAnalyticsUserCategoryStep())
            .next(loadRecipients())
            .preventRestart()
            .build();
}

public Step injectInternalEmailRecipientStep() {
    return stepBuilderFactory.get("injectAnalyticsEmailUserCategoryStep")
            .tasklet(injectInternalEmailRecipientTasklet())
            .build();
}

public Tasklet injectInternalEmailRecipientTasklet() {
    return new InjectInternalEmailRecipientTasklet(recipientRepository);
}

Job configuration is so verbose for the sake of following patterns that serve the more complicated jobs well.

Can I inject a single, static item into a Spring Batch item reader?

2 Answers2