0

I have one requirement to send MongoDB collection data in chunks from one microservice to another microservice by calling POST API

so, I tried this

microservice - A (Accept data and store in db)

app.post("/savedata", async(req, res) => {
    console.log("Body => %o", req.body);
    // insert data into db 
    res.status(200).send({status: "ok"});
});

microservice - B (Get data from mongo and post to microservice-A)

app.post("/postdata", async(req, res) => {
    let db = await client.db("Student");
    let collection = db.collection("details");
    let query = [
        {
            "$project": {
                _id:0
            }
        }
    ];
    let mongoStream = collection.aggregate(query).stream();
    const url = "http://microservice_A_HOST/savedata"; 
    axios({
        url: url,
        method: "post",
        data: mongoStream,
        headers: {
            'Content-Type': 'application/json'
        }
    }).then((response) => {
        console.log("Resp %j =>", response.status);
    }).catch(err => {
        console.log("err => ", err);
    });
    res.status(200).send({status: "ok"});
});

I am getting this error

TypeError: data should be a string, Buffer or Uint8Array
    at RedirectableRequest.write (/root/sample_code/node_modules/follow-redirects/index.js:84:11)
    at AggregationCursor.ondata (_stream_readable.js:726:22)
    at AggregationCursor.emit (events.js:210:5)
    at addChunk (_stream_readable.js:308:12)
    at readableAddChunk (_stream_readable.js:289:11)
    at AggregationCursor.Readable.push (_stream_readable.js:223:10)

please help me out Thanks

Abhishek Nilange
  • 11
  • 2
  • Maybe you should try to use "transform" in stream() as in answer https://stackoverflow.com/questions/49806740/how-to-pipe-with-mongodb-node-js-driver-cursor-and-aggregatecursor – Валентин Никин Jun 08 '21 at 21:24
  • I *guess* this is because axios check if `data` is a instance of the nodejs "stream". Not sure how mongodb has implemented the stream interface. Have you tried to wrap the mongodb stream in a nodejs [`PassThrough`](https://nodejs.org/dist/latest-v14.x/docs/api/stream.html#stream_class_stream_passthrough) stream? – Marc Jun 09 '21 at 10:22

0 Answers0