1

I am trying to parse a csv file with fast-csv and at the end store all the lines in a database with sequelize

Here is my script to parse the data:

module.exports = async () => {
  try {
let csvData = [];

    csv
      .parseFile(__basedir + "/test.csv", { headers: true })
      .on('data', (data) => {
        csvData.push(data);
      })
      .on('end', () => {
          Card_Report.bulkCreate(csvData)
      });
} catch (e) {
    console.error('Something went wrong', e)
  }
}

Here is my table model:

  const Test= sequelize.define(
      'test',
      {
        id: {
          type: DataTypes.UUID,
          primaryKey: true,
          defaultValue: uuidv4(),
          allowNull: false
        },
        name: {
          type: DataTypes.STRING,
        },
        age: {
          type: DataTypes.INTEGER,
        }
        }, {
            createdAt: false,
            updatedAt: false,        
            tableName: 'test'
         }
    )
    return Test
  }

and here is my csv file :

name, age  
nina, 24
sarah, 23

when I run the script I get this error :

Unhandled rejection SequelizeUniqueConstraintError: Validation error

But when there is only one row in my csv file it adds the row to the table correctly

How can I fix this?

Cissus
  • 15
  • 6
  • Looks like an issue with a primary key, since Sequelize says about `UniqueConstraint`. Please check if your PK in the database is not `null` for both parsed persons. Also, do you have `uuid` package installed, because you call its `uuidv4` function? – Ilya Kushlianski Jul 01 '21 at 14:58
  • As a side note, instead of try/catch I'd listen to 'error' event during the stream – Ilya Kushlianski Jul 01 '21 at 15:01
  • @Ilya Kushlianski it returns the same primary key like this : INSERT INTO `card_report` (`id`,`name`,,`age`) VALUES ('2ab2c36b-8907-4c15-9b0d-966d04c82a67','nina','24'),('2ab2c36b-8907-4c15-9b0d-966d04c82a67','sarah', '23'); what can cause this as I have set a default value in the model defaultValue: uuidv4(), – Cissus Jul 01 '21 at 15:05
  • @Ilya Kushlianski yes for the uuid I declared it this way const { v4: uuidv4 } = require('uuid'); – Cissus Jul 01 '21 at 15:10

1 Answers1

1

It seems you are getting the same id, because all models in Sequelize are "prepared" once, and your function uuidv4() runs only once. Thus for every row you insert into the DB it will have the same uuid. You could try mitigating it like this:

id: {
  type: DataTypes.UUID,
  defaultValue: DataTypes.UUIDV4, <- try something like this
  primaryKey: true,
},

Thus you are shifting responsibility for generating a unique UUID for each row on the database

P.S.: You might need a uuid extension for your DB, something like this:

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
Ilya Kushlianski
  • 780
  • 9
  • 15