Limit CSV file size using openCsv StatefulBeanToCSV

Question

I have a list of objects let's say List of Product which I get from my Db. Im able to write those List to a CSV file using openCSV StatefulBeanToCSV writer for 0.5M records it takes less than 10s to write the data to CSV file.. but the problem is StatefulBeanToCSV writes all those 0.5M records to a single CSV file whose size becomes larger.. so I need a way to restrict the CSV file to 20Mb. If the size of the CSV file reaches 20mb I need StatefulBeanToCSV to write to a new CSV file. any help is highly appreciated.

public static void buildProductCsv(final List<Product> product,
                final String filePath) {

            try {

                Writer writer = new FileWriter(filePath);

                // mapping of columns with their positions
                ColumnPositionMappingStrategy<Product> mappingStrategy = new ColumnPositionMappingStrategy<Product>();
                // Set mappingStrategy type to Product Type
                mappingStrategy.setType(Product.class);
                // Fields in Product Bean
                String[] columns = new String[] { "productCode", "MFD", "EXD" };
                // Setting the colums for mappingStrategy
                mappingStrategy.setColumnMapping(columns);

                StatefulBeanToCsvBuilder<Product> builder = new StatefulBeanToCsvBuilder<Product>(writer);

                StatefulBeanToCsv<Product> beanWriter = builder.withMappingStrategy(mappingStrategy).build();
                // Writing data to csv file
                beanWriter.write(product);
                writer.close();

                log.info("Your csv file has been generated!");

            } catch (Exception ex) {
                log.warning("Exception: " + ex.getMessage());
            }

        }

maybe this can help [link](https://stackoverflow.com/a/39093221/7546121) — Amir Schnell, Apr 18 '20 at 05:25
Does this answer your question? [Limit file size while writing in java](https://stackoverflow.com/questions/39092861/limit-file-size-while-writing-in-java) — Amir Schnell, Apr 18 '20 at 05:25

score 2 · Answer 1 · answered Apr 18 '20 at 05:46

This might not be the best approach, but you can simply check the size of one Product object and then you will have an abstract idea on what should be the maximum sublist size that can be written to one CSV. Then you can iterate the list creating multiple CSV files.

To get the size of one product you can use the below method. Note that Product object should be serializable.

  private static long getObjectSize( Object value ) // Pass product here
  {
    byte[] serializedObject;
    ByteArrayOutputStream byteArrayOutputStream = null;
    ObjectOutputStream objectOutputStream = null;

    try
    {
        byteArrayOutputStream = new ByteArrayOutputStream();
        objectOutputStream = new ObjectOutputStream( byteArrayOutputStream );
        objectOutputStream.writeObject( value );
        objectOutputStream.flush();
        serializedObject = byteArrayOutputStream.toByteArray();
        byteArrayOutputStream.flush();

        return Math.round( ( double ) serializedObject.length / ( 1024 * 1024 ) ); // Size in MB
    }
    catch( IOException e )
    {
        e.printStackTrace();
    }
    finally
    {
        try
        {
            if( byteArrayOutputStream != null )
            {
                byteArrayOutputStream.close();
            }
            if( objectOutputStream != null )
            {
                objectOutputStream.close();
            }
        }
        catch( IOException e )
        {
        }
    }
    return 0;
}

Limit CSV file size using openCsv StatefulBeanToCSV

1 Answers1