1

How to read files from s3 directly from FlatFileItemReader.

below is my FileItemreader. I am using java annotation . I followed the below link http://cloud.spring.io/spring-cloud-aws/spring-cloud-aws.html#_resource_handling

code

public  static FlatFileItemReader<Person> reader( String filenameStandard) {
            FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
            reader.setLinesToSkip(1);
            System.out.println("Job parameter ++++++++++++++++++++++"+filenameStandard);
            //working configuration with ClassPathResource
           //reader.setResource(new ClassPathResource(filenameStandard));

          /* String location= "s3://"+ "iebr-input"+"/" +"test_data.csv";
           reader.setResource(new AmazonS3Resource(amazonS3ClientInitializer, "iebr-input", "test_data.csv"));*/



            reader.setRecordSeparatorPolicy(new DefaultRecordSeparatorPolicy());
            reader.setLineMapper(new DefaultLineMapper<Person>() {
                {

                    setLineTokenizer(new DelimitedLineTokenizer() {
                        {
                        //  setNames(new String[] { "id", "firstName", "lastName" ,"harmScore", "removabilityScore", "kowScore"});
                            setNames(new String[] {"case", "name", "ccd_case", "date_of_birth", "family", "fullname"
                                          });
                        }
                    });
                    setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
                        {
                            setTargetType(Person.class);
                        }
                    });
                }
            });
            return reader;
        }
Angel
  • 57
  • 1
  • 10
  • https://stackoverflow.com/questions/30832041/spring-batch-read-files-from-aws-s3 – StanislavL Aug 04 '17 at 07:22
  • Hi , i tried both the approaches, one creating custom resources and also SPring cloud AWS .if you can point me to a sample implementation ot git repo that would be great – Angel Aug 04 '17 at 08:44
  • See https://github.com/mminella/S3JDBC – Michael Minella Aug 05 '17 at 16:36
  • Thanks @MichaelMinella ,I am currently working on another story so , I will try to do this example later this week , if I find any issues will update here. – Angel Aug 05 '17 at 21:59
  • @MichaelMinella ,Meanwhile I know your are the SME on spring batch ..I created another question [[link] https://stackoverflow.com/questions/45141204/spring-boot-with-spring-batch-and-jpa-configuration] could you please look/advice at this ..or point me to an example of **Spring batch with spring data JPA** .. I checked PRO SPRING BATCH and SPRING BATCH IN ACTION ..I check your learning spring batch in THAT Interface -ItemProcessor example 29yh video uses Spring JPA only ..Please kindly point me to an example or git repo for SPRING BATCH with Spring Data JPA ,,using repository classes – Angel Aug 05 '17 at 23:44
  • @MichaelMinella - any update on the spring batch with spring data spa example – Angel Aug 09 '17 at 13:02

0 Answers0