Convert nested json data into CSV file using csv-writer

Question

This is the logic that I have followed. I am getting proper results when I am just passing { id: 'title', title: 'Title' } into csvFields. But I am not getting any result when I try to get task.model i.e nested fields into my output csv file. Do I need to make any changes in document before passing it into writeRecords?

npm package used: csv-writer

let totalCount = await req.list.model.find(where).countDocuments();
const batchSize = 50;
const batchCount = Math.ceil(totalCount / batchSize);
const csvFields = [
    { id: 'title', title: 'Title' },
    { id: 'task.model', title: 'task' }, // Does not work
];

// To work on how to display nested fields in csv.
for (let i = 0; i < 1; i++) {
    let batch = await req.list.model.find(where).skip(batchSize * i).populate('task').limit(2);

    const csvWriter = createCsvWriter({
        header: csvFields,
        path: `batch_${i}.csv`,
    });

    csvWriter.writeRecords(batch)
        .then(() => console.log('csv written successfully'))
        .catch((error) => console.log(error));
};

This is the document JSON that I am getting by consoling batch

[ { title: 'Dummy Task',
    starred: false,
    isTicket: true,
    isDraft: false,
    isDeleted: true,
    closed: false,
    logs: [ 60c891a95372b913afdc8ed2, 60c893c75372b913afdc93c2 ],
    created_at: 2021-06-15T11:40:25.122Z,
    updated_at: 2021-06-15T11:49:27.261Z,
    _id: 60c891a95372b913afdc8ed1,
    ticketTat: null,
    closed_at: null,
    city: 60c70bf72fa78254cbf4d410,
    region: 60c70bf72fa78254cbf4d40f,
    camera: 60c891085372b913afdc8d82,
    schedule: 60c891505372b913afdc8e2c,
    company: 60c70b882fa78254cbf4d3e7,
    task:
     { complianceType: [Object],
       monitoringType: 'Self',
       monitoringMode: 'Watch',
       status: true,
       created_at: 2022-03-30T12:56:46.619Z,
       updated_at: 2022-03-30T12:56:46.619Z,
       _id: 60c890c85372b913afdc8d71,
       checklist: 604217fb6f4c96665e780f0a,
       referenceId: 604710f054fca40eadffe735,
       model: ' Task',
       endpoint: '',
       taskType: 'Artificial Intelligence',
       actionType: 'Activity Detection',
       recordingType: 'Video',
       taggedMediaType: [],
       cooldownPeriod: '180',
       tat: null,
       tag: null,
       slug: 'wopipe',
       threshold: [],
       pushType: [Array],
       __v: 0,
       faceRecognition: false,
       version: '1',
       incidentInfo: [],
       thirdPartyIntegration: null,
       modelType: [Object],
       optimalRangeTitle: 'Ideal Value' },
    location: 60c70bf72fa78254cbf4d411,
    taskType: 'Artificial Intelligence',
    checklist: 604217fb6f4c96665e780f0a,
    assigned_at: 2021-06-15T11:40:25.122Z,
    __v: 0,
    confidence: 'high',
    uid: '7f57f0e0-05e4-4338-902a-a8b3cc1934bf',
    metadata:
     { task: 'Task',
       city: 'Buffalo',
       location: 'Lower West Side',
       region: 'New York',
       checklist: 'd1',
       schedule: 'Simulation',
       raisedBy: '' },
    timezone: 'Pacific/Honolulu',
    modelType: { type: 'process-based', label: 'Process based' },
    incidentInfo: [],
    assignTask: 60c891545372b913afdc8e36,
    raisedBy: null,
    archived: true,
    personDetected: [] },
  { title: 'Dummy2Task',
    starred: false,
    isTicket: true,
    isDraft: false,
    isDeleted: true,
    closed: false,
    logs: [ 60c893b45372b913afdc93ad, 60c893c75372b913afdc93c3 ],
    created_at: 2021-06-15T11:49:08.898Z,
    updated_at: 2021-06-15T11:49:27.261Z,
    _id: 60c893b45372b913afdc93ac,
    raisedFrom: 'Artificial Intelligence',
    eventStatus: '',
    ticketNo: '002',
    ticketStatus: 60c70b882fa78254cbf4d3f3,
    ticketTag: 60c70b882fa78254cbf4d3f2,
    ticketTat: null,
    closed_at: null,
    city: 60c70bf72fa78254cbf4d410,
    region: 60c70bf72fa78254cbf4d40f,
    camera: 60c891085372b913afdc8d82,
    schedule: 60c891505372b913afdc8e2c,
    company: 60c70b882fa78254cbf4d3e7,
    task:
     { complianceType: [Object],
       monitoringType: 'Self',
       monitoringMode: 'Watch',
       status: true,
       created_at: 2022-03-30T12:56:46.619Z,
       updated_at: 2022-03-30T12:56:46.619Z,
       _id: 60c890c85372b913afdc8d71,
       checklist: 604217fb6f4c96665e780f0a,
       referenceId: 604710f054fca40eadffe735,
       model: 'Task2',
       endpoint: '',
       taskType: 'Artificial Intelligence',
       actionType: 'Activity Detection',
       recordingType: 'Video',
       taggedMediaType: [],
       cooldownPeriod: '180',
       tat: null,
       tag: null,
       slug: 'wopipe',
       threshold: [],
       pushType: [Array],
       company: 60c70b882fa78254cbf4d3e7,
       __v: 0,
       faceRecognition: false,
       version: '1',
       incidentInfo: [],
       thirdPartyIntegration: null,
       modelType: [Object],
       optimalRangeTitle: 'Ideal Value' },
    location: 60c70bf72fa78254cbf4d411,
    taskType: 'Artificial Intelligence',
    checklist: 604217fb6f4c96665e780f0a,
    assigned_at: 2021-06-15T11:49:08.898Z,
    __v: 0,
    confidence: 'high',
    uid: '14b2b91f-308d-43be-8f5a-d9269b9139a6',
    metadata:
     { task: 'Task2',
       city: 'Buffalo',
       location: 'Lower West Side',
       region: 'New York',
       checklist: 'Pipe',
       schedule: 'Simulation2',
       raisedBy: '' },
    timezone: 'Pacific/Honolulu',
    modelType: { type: 'process-based', label: 'Process based' },
    incidentInfo: [],
    assignTask: 60c891545372b913afdc8e36,
    raisedBy: null,
    archived: true,
    personDetected: [] } ]

score 0 · Answer 1 · answered May 14 '23 at 19:06

You don't need to change the input, just add headerIdDelimiter in the configuration object. Now you can use the . to get to nested properties. Source.

const csvWriter = createCsvWriter({
                      header: csvFields,
                      headerIdDelimiter: '.', // Add this
                      path: `batch_${i}.csv`,
                   });

Convert nested json data into CSV file using csv-writer

1 Answers1