How to create a dynamic sitemap with mongodb, node.js, express and EJS?

Question

I'm trying to create a dynamic sitemap for my website, it has lots of pages that change often. The sitemap needs to be accessed from www.mywebsite.com/sitemap.xml

My current attempt queries the database for all the pages, get's each pages url and passes it to an EJS template that creates what looks like XML.

I have two problems here

The route to the page cannot have a file suffix. e.g. '.xml'
The page is automatically treated as html

I realise that there are other options for creating a sitemap using modules like "express-sitemap," but I haven't been able to find any easily understood (i am new to this) documentation for them, and this seems like a good way of doing things to me

which version of nodejs are you using? – Muhammad Faizan Apr 10 '18 at 10:02 — Muhammad Faizan, Apr 10 '18 at 10:02
hi, I'm using version, v6.11.2 – CopyPasteThankYou Apr 10 '18 at 13:41 — CopyPasteThankYou, Apr 10 '18 at 13:41

Muhammad Faizan · Answer 1 · 2020-11-05T13:03:33.757

yes you can use express-sitemap

To generate Sitemap automatically

var sitemap = require('express-sitemap')();
 
var app = require('express')();
 
sitemap.generate(app);

To generate dynamically..

for suppose you have products pages and you have specified url for them.. You can create a dynamic file everytime and place it in your public folder.

const Product = require('./model/product')
const sitemap = require('sitemap');  
let sitemapData;
const generateSitemap = async () => {  
    const products = await Product.find({},{path: 1});
    const urls = products.map({path} => `/products/${path}`)
    sitemapData = sitemap.createSitemap ({
        hostname: 'http://example.com',
        cacheTime: 600000,        // 600 sec - cache purge period 
        urls
    });
}

You can use this function in a routine or with a cron and generate sitemap regularly..

setInterval(generateSitemap, 360000); //running every hour

other thing that you can do is:

use sitemapData variable and do things like this.

sitemapData.add({url: '/product-a/'}); // when some product is added
sitemapData.add({url: '/product-b/', changefreq: 'monthly', priority: 0.7});
sitemapData.del({url: '/product-c/'}); // when something is removed
sitemapData.del('/product-d/');

you can serve it in a route like this:

app.get('/sitemap.xml', function(req, res) {
  sitemapData.toXML( function (err, xml) {
      if (err) {
        return res.status(500).end();
      }
      res.header('Content-Type', 'application/xml');
      res.send( xml );
  });
});

score 0 · Answer 2 · edited Sep 23 '20 at 12:25

Here is how I made a txt sitemap. I find Google Search Console has an easier time fetching txt sitemaps vs xml sitemaps. But if you want to make a xml sitemap, you can look at this blog for the correct formatting. This code uses Mongoose and is saved as /pages/sitemap.txt.js.

// pages/sitemap.txt.js

import dbConnect from "../utils/dbConnect";
import Pduct from "../models/Pduct";

const createSitemap = (posts) => `${posts
  .map(({ slug }) => {
    return `https://[YOUR DOMAIN]/${slug}`;
  })
  .join("\n")}
    `;

export async function getServerSideProps({ res }) {
  await dbConnect();
  const request = await Pduct.find({}, "slug").lean();
  res.setHeader("Content-Type", "text");
  res.write(createSitemap(request));
  res.end();
}

export default () => null;

How to create a dynamic sitemap with mongodb, node.js, express and EJS?

2 Answers2

To generate Sitemap automatically

To generate dynamically..