3

I have a spring batch application that reads and writes into the same table. I have used pagination for reading the items from the table as my data volume is quite high. When I set the chunk size as more than 1 then my pagination number is getting updated wrongly and hence failing to read some items from the table. Any idea?

@Bean
    @StepScope
    public RepositoryItemReader<DedWorkFlowResponse> dedWorkFlowResponseItemReader() {

        return new RepositoryItemReaderBuilder<DedWorkFlowResponse>()
                .name("dedWorkFlowResponseItemReader")
                .methodName("findByPickedIs")
                .arguments(Collections.singletonList(false))
                .pageSize(100)
                .repository(dedWorkFlowResponseRepository)
                .sorts(Collections.singletonMap("id", Sort.Direction.ASC))
                .build();
    }


public interface DedWorkFlowResponseRepository extends JpaRepository<DedWorkFlowResponse, Long> {
    public Page<DedWorkFlowResponse> findByPickedIs(boolean picked,Pageable pageable);
}


@Component
public class Writer implements ItemWriter<DedWorkFlowResponse> {

    @Autowired
    ResponseRepository responseRepository;

    @Autowired
    DedWorkFlowResponseRepository dedWorkFlowResponseRepository;


    @Override
    public void write(List<? extends DedWorkFlowResponse> list) throws Exception {
        List<Response> responses = new ArrayList<>();

        list.stream().forEach(dedWorkFlowResponse ->{

            Response response = new Response();
            response.setName(dedWorkFlowResponse.getName());
            response.setProcessedDate(new Date(System.currentTimeMillis()));

            System.out.println(response.toString());
            responses.add(response);

            dedWorkFlowResponse.setPicked(true);
            dedWorkFlowResponseRepository.saveAndFlush(dedWorkFlowResponse);
        } );

        responseRepository.saveAll(responses);
    }
}

In writer I am updating some value in DedWorkFlowResponse Table

tausif
  • 141
  • 1
  • 8
  • What do you mean by "my pagination number is getting updated wrongly"? How does you job fail to read items? Please post the stacktrace of the error. – Mahmoud Ben Hassine Jan 06 '20 at 10:13
  • 1
    I am using `RepositoryItemReader`. There is no error and Job is completing successfully. The problem is that some rows are not read by the `RepositoryItemReader`. Suppose I have set page size 10. Then it will read page 1 to 10 and skip 11 to 20. I have created the git repository for the same. Please have a look at the code https://github.com/mdtausifahmad/springboot-batch.git. [Click Here to go to Repository ](https://github.com/mdtausifahmad/springboot-batch.git) – tausif Jan 07 '20 at 05:05
  • Thanks. I was able to run your code and understand your issue. You are hitting the same problem as: https://stackoverflow.com/questions/26509971. So I'm marking this question as duplicate. – Mahmoud Ben Hassine Jan 08 '20 at 11:30
  • Does this answer your question? [Spring batch jpaPagingItemReader why some rows are not read?](https://stackoverflow.com/questions/26509971/spring-batch-jpapagingitemreader-why-some-rows-are-not-read) – Mahmoud Ben Hassine Jan 08 '20 at 11:30
  • It looks similar but have to explore more to find the solution. Thanks for the question link – tausif Jan 08 '20 at 17:03
  • @tausif did you find a solution to it? I am facing a similar problem – Sunit Chatterjee Apr 14 '21 at 14:32
  • @MahmoudBenHassine I went through the link - https://stackoverflow.com/questions/26509971. - but it talks about using a `JpaPagingItemReader` whereas I want to use the `RepositoryItemReader` – Sunit Chatterjee Apr 14 '21 at 14:33
  • @SunitChatterjee The same could happen with `RepositoryItemReader` as well as it is a paging reader (see `pageSize()` method in its builder). – Mahmoud Ben Hassine Apr 15 '21 at 11:06

0 Answers0