Google Cloud function to fetch data from third party server

Question

I am new to Google Cloud Functions features and implementation. So I want to know that is it possible to make HTTP or HTTPS request to third party server API using Cloud function and if yes then how? And when I receive data in response can I store it into my firebase database using same cloud function instance?

And how can I make this request to be called periodically or schedule it? Thanks in advance

@arudzinska I know about what cloud functions, it has events and triggers. But I do not know whether I can make HTTP request through cloud function or no. And how to schedule this HTTP request? — Ankit Gupta, Jan 09 '19 at 13:31
Cloud Functions are pretty much node scripts that run in a managed container. So you can do most things in Cloud Functions that you can do in a node script. If ever you wonder: can something be done in Cloud Functions, I recommend searching for how to do the same thing in Node.js and then try to do that in Cloud Functions. Most often that will work, if not: post a question with the use-case. — Frank van Puffelen, Jan 09 '19 at 15:42
@FrankvanPuffelen Thanks a lot. For showing me this path to how to search queries related to cloud functions in future. This is a big help. — Ankit Gupta, Jan 10 '19 at 11:52

score 14 · Answer 1 · answered Dec 27 '20 at 23:38

14

Here's how to do it using node-fetch.

Your Cloud Function:

const fetch = require('node-fetch');

exports.functionName= (req, res) => {
  const fetchFromURL = async () => await (await fetch('https://yourURL.com')).json();

  fetchFromURL().then((data) => {
    // do something with data (received from URL).
  });
};

You will need to add the "node-fetch" dependency to your function's package.json as well.

Your package.json:

{
  "name": "sample-http",
  "version": "0.0.1",
  "dependencies": {
    "node-fetch": "^2.6.1"
  }
}

answered Dec 27 '20 at 23:38

Mimina

2,603
2
29
21

3

or `import fetch from 'node-fetch'` – Jonathan Apr 03 '21 at 03:18
For anyone still having an issue, it could be that you installed the v3 version of node-fetch, switching to the latest v2 worked for me, using `npm install node-fetch@2`: https://stackoverflow.com/a/70541151/2139770 – AEQ Mar 03 '23 at 08:34

score 13 · Accepted Answer · edited Jun 20 '20 at 09:12

UPDATE on 8 May 2020

request-promise being now deprecated, I recommend to use axios.

You can use the node.js request-promise library to do so.

You could do something along these lines, for example:

.....
var rp = require('request-promise');
.....

exports.yourCloudFunction = functions.database.ref('/parent/{childId}')
    .onCreate((snapshot, context) => {
      // Grab the current value of what was written to the Realtime Database.
      const createdData = snapshot.val();

      var options = {
          url: 'https://.......',
          method: 'POST',
          body: ....  
          json: true // Automatically stringifies the body to JSON
      };

      return rp(options);

    });

If you want to pass parameters to the HTTP(S) service/endpoint you are calling, you can do it through the body of the request, like:

  .....
  const createdData = snapshot.val();

  var options = {
      url: 'https://.......',
      method: 'POST',
      body: {
          some: createdData.someFieldName
      },
      json: true // Automatically stringifies the body to JSON
  };
  .....

or through some query string key-value pairs, like:

  .....
  const createdData = snapshot.val();
  const queryStringObject = { 
     some: createdData.someFieldName,
     another: createdData.anotherFieldName
  };

  var options = {
      url: 'https://.......',
      method: 'POST',
      qs: queryStringObject
  };
  .....

IMPORTANT:

Note that if you plan to call a non Google-owned service (like the "third party server" you mentioned), you need to be on the "Flame" or "Blaze" pricing plan.

As a matter of fact, the free "Spark" plan "allows outbound network requests only to Google-owned services". See https://firebase.google.com/pricing/ (hover your mouse on the question mark situated after the "Cloud Functions" title)

UPDATE FOLLOWING YOUR COMMENT:

If you want to trigger a call to the third party server and then populate the Firebase Realtime Database with data received from this server you could do as follows. I took an example of call to an API from the request-promise documentation: https://github.com/request/request-promise#get-something-from-a-json-rest-api.

You would then call this Cloud Function regularly with an online CRON job like https://www.easycron.com/.

exports.saveCallToAPI = functions.https.onRequest((req, res) => {
  var options = {
    uri: 'https://api.github.com/user/repos',
    headers: {
      'User-Agent': 'Request-Promise'
    },
    json: true // Automatically parses the JSON string in the response
  };

  rp(options)
    .then(repos => {
      console.log('User has %d repos', repos.length);

      const dbRef = admin.database().ref('userName'); //For example we write to a userName node
      var newItemRef = dbRef.push();
      return newItemRef.set({
        nbrOfRepos: repos.length
      });
    })
    .then(ref => {
      response.send('Success');
    })
    .catch(error => {
      response.status(500).send(error);
    });
});

Thanks for the answer, but I am looking for the opposite way. What I mean is to first make a request and then make changes to the firebase database. — Ankit Gupta, Jan 10 '19 at 12:02
any reason you prefer `request-promise` over `axios`? thanks! — Crashalot, May 08 '20 at 01:06
Hi @Crashalot. No, and actually it is now preferred to use Axios since `request-promise has been deprecated. I’ll update the answer. — Renaud Tarnec, May 08 '20 at 05:29

Google Cloud function to fetch data from third party server

2 Answers2

UPDATE on 8 May 2020

Linked