How to filter an array by several params only if they are present

Question

I have a function to filter results by several params, such as name, status, type and date range (startDate and endDate). I would like to be able to filter results by these params together, but only if they are present, i. e. I can pass name and status, but don't pass type. I don't know how to do this with a date range. Now the filter is working only if I pass startDate and endDate, in all other cases, even if other params are present and there is corresponding data in the array, it returns null. How can I make startDate and endDate optional?

Here is my filter:

if (params.name || params.status || params.type || params.startDate && params.endDate) {
  const startDate = new Date(params.startDate).setHours(0,0,0);
  const endDate = new Date(params.endDate).setHours(23,59,59);
  dataSource = tableListDataSource.filter(
    (data) =>
      data.name.match(new RegExp(params.name, 'ig')) &&
      data.status.includes(params.status || '') &&
      data.type.includes(params.type || '') &&
      (
        new Date(data.createdAt).getTime() > startDate && new Date(data.createdAt).getTime() < endDate
      )
  );
}

Thank you for your help!

EDIT:

I'm using this filter inside a function on backend:

function getRule(req, res, u) {
  let realUrl = u;

  if (!realUrl || Object.prototype.toString.call(realUrl) !== '[object String]') {
    realUrl = req.url;
  }

  const params = parse(realUrl, true).query;

  if (params.name || params.status || params.type || params.startDate && params.endDate) {
    const startDate = new Date(params.startDate).setHours(0,0,0);
    const endDate = new Date(params.endDate).setHours(23,59,59);
    dataSource = tableListDataSource.filter(
      (data) =>
        data.name.match(new RegExp(params.name, 'ig')) &&
        data.status.includes(params.status || '') &&
        data.type.includes(params.type || '') &&
        (
          new Date(data.createdAt).getTime() > startDate && new Date(data.createdAt).getTime() < endDate
        )
    );
  }

  const result = {
    data: dataSource,
    success: true,
  };

  return res.json(result);
}

Answer 1

By using your current approach, you can make the startDate and endDate optional by doing as following;

&& (
  (params.startDate && params.endDate) ? 
    (new Date(data.createdAt).getTime() > startDate && new Date(data.createdAt).getTime() < endDate) :
    true
)

So, what above does is basically check if params.startDate and params.endDate have no falsy values;

• If they don't, then do your existing filter with the dates;
• Otherwise if one of them do have falsey value, ignore the date-related filter by returning true.

This is how your final code will look like;

if (params.name || params.status || params.type || params.startDate && params.endDate) {
  const startDate = new Date(params.startDate).setHours(0,0,0);
  const endDate = new Date(params.endDate).setHours(23,59,59);
  dataSource = tableListDataSource.filter(
    (data) =>
      data.name.match(new RegExp(params.name, 'ig')) &&
      data.status.includes(params.status || '') &&
      data.type.includes(params.type || '') &&
      (
        (params.startDate && params.endDate) ? 
          (new Date(data.createdAt).getTime() > startDate && new Date(data.createdAt).getTime() < endDate) :
          true
      )
  );
}

Edit:

Normally, I'd suggest that filtering should not be done in FE, but rather in BE stack. Therefore you could only fetch the needed data along with pagination supports.

However, if you insist doing it in FE - I'd suggest that to encapsulate both filter function and handling params to filter data source. Blacklist everything and whitelist only the accepted params, and scale along the way as needed.

The following is an example of how I'd do it.

Note that; the filterDataSource complexity increases with the amount of fields you would support to be filtered. The fields iteration inside it equals to stacking multiple if conditions with extra steps.

/** 
 * @description Filters dataSource with provided fields
 * @param dataSource An array containing the data source
 * @param fields An key-value pair containing { [dataSourceField]: callbackFn(value) | "string" | number }
 */
const filterDataSource = (dataSource, fields) => {
  if (dataSource && dataSource.length) {
    return dataSource.filter((row) => {
      const rowFiltered = [];

      /**
       * @todo Scale the type of filter you want to support and how you want to handle them
       */
      for (const fieldName in fields) {
        if (Object.hasOwnProperty.call(fields, fieldName) && Object.hasOwnProperty.call(row, fieldName)) {
          const filter = fields[fieldName];

          if (typeof filter === 'function') {
            /** Call the callback function which returns boolean */
            rowFiltered.push(!!filter(row));
          }
          else if (typeof filter === 'object' && filter instanceof RegExp) {
            /** Predicate by regex */
            rowFiltered.push(!!row[fieldName].match(filter));
          }
          else if (typeof filter === 'string') {
            /** Exact match of string */
            rowFiltered.push(!!row[fieldName].match(new RegExp(filter, 'ig')));
          }
          else if (typeof filter === "number") {
            /** Exact match of number */
            rowFiltered.push(row[fieldName] === filter);
          }
        }
      }

      /** If this row is part of the filter, ONLY return it if all filters passes */
      if (rowFiltered.length > 0) {
        /** This will check if all filtered return true */
        return rowFiltered.every(Boolean);
      }
      else {
        /** If this row is NOT part of the filter, always return it back */
        return true;
      }
    });
  }

  return dataSource;
}


/**
 * @description Filter your datasource with pre-defined filter function for supported params
 * @param dataSource An array of object containing the data
 * @param params A set of object containing { [param]: value }
 * @todo Safely guard the wathched params here, encode them if needed.
 */
const filterDataByParams = (dataSource, params) => {
  const fieldsToFilter = {};

  if (params.name) {
    fieldsToFilter['name'] = new RegExp(params.name, 'ig');
  }
  
  if (params.status) {
    fieldsToFilter['status'] = params.status;
  }
  
  if (params.type) {
    fieldsToFilter['type'] = params.type;
  }
  
  if (params.startDate && params.endDate) {
    /**
     * Assuming createdAt is EPOCH
     * @todo What is the type of row.createdAt and params.startDate? 
     * @todo Adjust the logic here and apply validation if needed.
     */
    const startMillis = new Date(params.startDate).getTime() / 1e3, // Millis / 1e3 = EPOCH
      endMillis = new Date(params.endDate).getTime() / 1e3; // Millis / 1e3 = EPOCH

    /** Should we give a nice warning if invalid date value is passed? */
    if (isNaN(startMillis) && isNaN(endMillis)) {
      console.error('Invalid date params passed. Check it!');
    }

    /** Random defensive - remove or add more */
    if (startMillis && endMillis && startMillis > 0 && endMillis > 0 && startMillis < endMillis) {
      fieldsToFilter['createdAt'] = (row) => {
        return row.createdAt >= startMillis && row.createdAt <= endMillis;
      };
    }
  }

  if (Object.keys(fieldsToFilter).length) {
    return filterDataSource(dataSource, fieldsToFilter);
  }
  else {
    return [...dataSource];
  }
}


/** 1k Set of mocked data source with createdAt between 1 Jan 2019 to 13 February 2021 */
fetch('https://api.jsonbin.io/b/6027ee0987173a3d2f5c9c3d/3').then((resp) => {
  return resp.json();
}).then((mockDataSource) => {
  mazdaFilteredData = filterDataByParams(mockDataSource, {
    'name': 'Mazda',
    'startDate': '2019-05-04T19:06:20Z',
    'endDate': '2020-08-09T19:06:20Z'
  });
  
  hondaFilteredData = filterDataByParams(mockDataSource, {
    'name': 'honda',
    'startDate': '2019-10-05T00:00:00Z',
    'endDate': '2020-12-09T23:23:59Z'
  });
  
  mercedezFilteredData = filterDataByParams(mockDataSource, {
    'name': 'merce',
    'startDate': '2020-01-01T00:00:00Z',
    'endDate': '2021-12-31T23:23:59Z'
  })
  
  console.log({mazdaFilteredData, hondaFilteredData, mercedezFilteredData});
});

Answer 2

function getPresentParams(param, checkList) {
  // we start with an empty Array
  let output = [];
  // for each item in checkList
  checkList.forEach(item => {
  
    // if the item is an array, multiple params are needed 
    if (Array.isArray(item)) {
        // the item is an itemList
      let allItemsPresent = true;
      for (const itemListItem of item) {
        // if one of the list is not there
        if (!param.hasOwnProperty(itemListItem)) {
            allItemsPresent = false;
          // then stop the loop
            break;
          }
      }
      // else all are matching
      if (allItemsPresent) {
        // add all to our output
                output.push(...item);
      }
    }
    // else if the item is not an Array
    else {
        // simple check if the param is present
        if (param.hasOwnProperty(item)) {
        output.push(item);
      }
    }
  })
  return output;
}

const params = {type: "car", color:"red", tires:4};

// any of "type", "tires" or "windows" should be searched
console.log(getPresentParams(params, ["type", "tires", "windows"]));
// output is ["type", "tires"]

// only show matches with "color" AND "type"
console.log(getPresentParams(params, [["color", "type"]]));
// output is ["color", "type"]

// show matches with "color" or ["windows" AND "tires"]
console.log(getPresentParams(params, ["color", ["windows", "tires"]]));
// output is ["color"]

with this function you get an Array with all present params you are searching for. By passing 2 or more paramNames as Array it will only add them to the list if ALL of them are present.

Then you can simply check in your filter function if the param is present, make your check and if it fails return false.

const foundParams = getPresentParams(params, ["type", "status", ["startDate", "endDate"], "name"]);

And then in your filter function:

if (foundParams.includes("startDate") && foundParams.includes("endDate")) {
    // if your dates are not matching return false
}
if (foundParams.includes("name")) {
    // if your name isn't matching return false
}
// and so on

// at least if all checks have passed
return true;

Thats only one solution. There are some more like an object with key: arrow functions and some iteration. But I think with this solution you get an better idea what you are doing