4

Has anyone scraped or used the FB API to access data from a public profile's wall post? If you just go to the URL you can inspect the element to see most of the data as well as the ajax calls for the infinite scroll on the wall.

How would you I go about accomplishing this?

The following posts did not get answered or were just not same scenario but are close:

As an example of what I would like to accomplish, this public profile https://www.facebook.com/aurelia.filion has a wall post from November 21, 2013 that I would hypothetically like to get the data from. How can that be done?

Community
  • 1
  • 1
MicFin
  • 2,431
  • 4
  • 32
  • 59
  • there is no way to get user wall postings without a user token. this is for privacy reasons, there is no situation where you would need data from another user account in a serious way without authorizing that user. also, scraping is not allowed on facebook. – andyrandy Aug 23 '14 at 08:14
  • about scraping: https://www.facebook.com/apps/site_scraping_tos_terms.php – andyrandy Aug 23 '14 at 08:14
  • 2
    @luschn Thank you, that is the common response I am hearing. "there is no situation where you would need data from another user account in a serious way without authorizing that user" is not true in this case. Thank you for the information though. It is interesting to think that I can see something on my screen but not be allowed to capture it. Hypothetically, I could screen shot it manually I suppose. I'll stick to Twitter for now, friendlier API. – MicFin Aug 23 '14 at 18:22
  • friendlier for the developer maybe (although the twitter api got limits that are a real pain), but facebook tries to protect the privacy of the user. it is totally understandable that you are not allowed to scrape things you see with your browser imho. apps should have zero business with the wall of a user profile. you would not be allowed to use it anywhere anyway. – andyrandy Aug 24 '14 at 01:12

3 Answers3

1

Crawling Facebook is prohibited unless you have express written permission.

Facebook's Automated Data Collection Terms ( copied date 19 march 2017 )

  1. These terms govern your collection of data from Facebook through automated means, such as through harvesting bots, robots, spiders, or scrapers ("Automated Data Collection"), as well as your use of that data.

  2. You will not engage in Automated Data Collection without Facebook's express written permission.

  3. By obtaining permission to engage in Automated Data Collection you agree to abide by these Automated Data Collection Terms, which incorporate by reference the Statement of Rights and Responsibilities.

  4. You agree that your use of data you collect through Automated Data Collection will be confined solely to search indexing for display on the Internet unless granted separate approval by Facebook for alternative usage and display on the Internet.

  5. You agree that you will not sell any data collected through, or derived from data collected through, Automated Data Collection.

  6. You agree that you will not transfer data collected through Automated Data Collection in aggregated or bulk form.

  7. You agree that you will destroy all data you have collected through Automated Data Collection upon Facebook's written request and that you will certify such destruction under penalty of perjury.

  8. You agree that Facebook may revoke any permission granted at anytime for any reason and you agree to immediately cease collection and use of data collected through Automated Data Collection on notice of such revocation.

  9. You agree to provide an accounting of all uses of data collected through Automated Data Collection within ten (10) days of your receipt of Facebook's request for such an accounting.

  10. You agree that you will not circumvent any measures implemented by Facebook to prevent violations of these terms.

  11. You agree that you will not violate the restrictions in any robot exclusion header.

  12. You agree that you will only use your own true IP address/useragent identity and will not mask your services under the IP address/useragent string of another service.

  13. You agree that you will not transfer any approved IP address or useragent to any party without Facebook's express written consent.

  14. You agree that any violation of these terms may result in your immediate ban from all Facebook websites, products and services. You acknowledge and agree that a breach or threatened breach of these terms would cause irreparable injury, that money damages would be an inadequate remedy, and that Facebook shall be entitled to temporary and permanent injunctive relief, without the posting of any bond or other security, to restrain you or anyone acting on your behalf, from such breach or threatened breach. Nothing herein shall be construed as preventing Facebook from pursuing any and all remedies available to it, including the recovery of money damages.

  15. Nothing herein shall be construed to confer any grant to, or license of, any intellectual property rights, whether by estoppel, by implication, or otherwise.

Read complete list here https://www.facebook.com/apps/site_scraping_tos_terms.php

Ansen E Anand
  • 107
  • 1
  • 5
  • the link in his answer is pretty easy to understand though. and of course this agreement is for everyone, not just facebook users. scraping stuff from any website without agreement from the website owner is a general no go. – andyrandy Aug 24 '14 at 01:14
0

FaceBook have closed off most of the data you can scrape from the API. Unless you are friends it is limited to their names, username, locale, and if I remember correctly gender. Those are the default settings. If they have specifically chosen to share more (with applications in general, which is very few people) there will be more info.

I think that to get posts, you need that person to allow your app (developer ID) to access their posts.

As for scraping. You can do that, but again FB are wise to this. You can scrape about 1,000-2,000 pages before they show you a captcha. Then you also need multiple accounts and proxies.

To start, you will need CURL. I recommend the following for PHP:

http://semlabs.co.uk/journal/object-oriented-curl-class-with-multi-threading

http://semlabs.co.uk/journal/multi-threaded-stack-class-for-php

There are basic examples there. I believe I have some posts in my history about logging in. Scrapping is quite simple. I prefer RegEx to get details from the page like:

preg_match_all( '~<div.*? class="this-data".*?>(.*?)<\/div>~is', $page, $mats );
Kohjah Breese
  • 4,008
  • 6
  • 32
  • 48
0

To get data from facebook walls, you have to use Facebook Open Graph API and Facebook OAuth to login from outside of facebook. You will get complete detail here facebook developer panel

Salman Mohammad
  • 182
  • 1
  • 14