5

I've build chrome extension where it takes out newsfeeds from social media pages. However, I'd like to keep posts from specific social media accounts that users follow w/o injecting but filtering. The problem is that sometimes(if not most of the times) the posts might not be in the preloaded feed & all social media companies use lazy loading. Therefore, I am really looking for creative solutions to solve this intriguing problem.

Unsuccessful methods I've tried:

  • deleting posts so new one comes
  • scrolling inside context
Daniyal dehleh
  • 2,314
  • 2
  • 7
  • 22
  • 3
    Can you show some code? Hard to tell what the problem is without being able to see it. – I wrestled a bear once. Sep 20 '21 at 20:56
  • You need to list your code here for us to help – Transformer Sep 21 '21 at 02:30
  • How about simply using the site's APIs to load new posts, i.e look into the network requests being made when you scroll to the bottom and try to replicate them. – Utkarsh Dixit Sep 21 '21 at 03:40
  • Hi All! Thanks for your responses & really sorry for my lack of code. I really wanted to provide but it would've been +1000 Lines code minimum to do complexity of FB newsfeed. Thus, I thought it can be tried on our own FB dev console. I would be posting the answer soon. – Daniyal dehleh Sep 22 '21 at 14:44

1 Answers1

2

This question does not really fit that well within Stack Overflow, IMHO, because it is not about code, but about programming principles in general.

Thus, no-one can give you an answer showing how your code should look like. But at least we can come up with advises on how to deal with your scenario.

My advise would be for you to:

  1. Debug the network traffic for each and every social media site your are supporting. Look for XHR requests at the network inspector, and inspect the responses to find which endpoint is responsible for fetching more content.
  2. In the Sources tab of the web inspector, Add a new XHR/fetch breakpoint for this URL
  3. Inspect the stack trace backwards, until you find a function or method call which is responsible for calling the content fetch
  4. Verify that when you call this function, content will load via XHR and will be injected in your network feed. If not, you may have to drill further down the rabbit hole, or continue single-stepping the execution to find which method/function is responsible for injection the newly loaded content into the document.
  5. When you have successfully managed to reliably reproduce this, your extension have to inject code into the page itself in order to trigger the content loading
Xyz
  • 5,955
  • 5
  • 40
  • 58
  • Aron, thanks a lot for your well-detailed response! That's certainly a mindful way to go about it especially if you're planning to filter content natively in the app itself via a VPN. Please correct me if I am wrong as the next step is going natively with Apple's local VPN as I am not sure how visible the traffic would be with encryption. In terms of chrome extension, I believe I found a much easier way where you delete posts which by itself triggers the lazy loader & you stop when the target post is reached. The ultimate key is to learn how indexing works. I would be posting the code soon. – Daniyal dehleh Sep 22 '21 at 14:39