0

I'll start with an example: Say you're searching an online store for a product that is [XYZ] compatible. But the result page or product listing page don't display that information, and item compatibility is not searchable, so you have to click each product and check. This can be very arduous when you have 100+ products to look through.

I have come across numerous browser plugins for search result pages (such as eBay or Craiglist) of which will crawl through and visit each result link, and then display a piece of information on the result page for each item. Now I know what is [XYZ] compatible before clicking!

I want to write a script for a particular website's search results.

  • Can I achieve this using vanilla JavaScript? How? (I'd prefer something simple like a bookmarklet)
  • If not, what would I need to accomplish this?

I just want to know which result is XYZ compatible, and then highlight it on the result page. I imagine it would be something like using cURL to load each link, and then using string slicing to find the compatibility text.

Bort
  • 493
  • 1
  • 8
  • 26
  • I think there is a miscommunication. What I want already exists. There are plenty of craigslist scripts out there that will scan results for specific things, and then highlight items on the result page for you. Can you explain what you are getting at? – Bort May 29 '16 at 21:07
  • OK, so, you have a list of links and you want to highlight portions of text that match a string word (query criteria)? What means `XYZ compatible`? Can you explain better? I'm a bit confused. – Roko C. Buljan May 29 '16 at 21:12
  • Sort of...I don't want to highlight the original text though, I want to simply mark the results of the search, because the original text will be on a different URL. – Bort May 29 '16 at 21:13
  • Now I'm sad I removed my first comment. Anyways it's still unclear what you're after. – Roko C. Buljan May 29 '16 at 21:15
  • Yeah lol. So you are saying that I shouldn't be doing this? Because again, I'm fairly certain there is no other way those craigslist plugins could be doing what they do without crawling each result. And for that matter, I'm going to end up clicking all the results anyway! – Bort May 29 '16 at 21:17
  • Okay it should be very clear: Say you want to buy a phone case. But when you search some site for "Hard plastic phone case", the phone model isn't displayed on the results page. So now you have to click each result to make sure it fits your phone. But a script could just visit the results for you, and highlight all the cases compatible with your phone. – Bort May 29 '16 at 21:20
  • If I got your question well, 1. I enter a search word. 2. you show some links (this is unclear...) 3. than you want to perform a link scraping to all those links to find some text content inside those static pages? If yes, imagine you have 200 links results and 300 active users. 200*300 fetch queries.... madness! If you have a **Relational database** logic than it's easy. Every item has both Case and Phone-Model relation. The database knows instantly which links are a case-model-match! – Roko C. Buljan May 29 '16 at 21:22
  • Step 2 is unclear?? If you search craiglist for a phone case, the results are links. I think the confusion is that you think I OWN this website! I'm trying to do this for website(s) that I don't own. I don't have access to any database. – Bort May 29 '16 at 21:24
  • Another way around is that: 1. Generate i.e: 200 links 2. Add some `data-model="phone model here"` to those links. Than using JS you can 1. get the user's selected phone model 2. Highlight the links that match the `data` attribute. Just an idea... – Roko C. Buljan May 29 '16 at 21:25
  • Sh*t. Than if you don't own the website, but you're building a service, than yes. Collect all links into an Array, Visit (AJAX) every link and retrieve a desired `id` or `class` element where you can find/read the phone model name... Want an example? – Roko C. Buljan May 29 '16 at 21:27
  • Yes I would love any help that points me in the right direction! – Bort May 29 '16 at 21:28
  • You want to do a sub-filtering of the links shown. Why not including your term on the original search? – LordNeo May 29 '16 at 21:29
  • I guess you could do some curl magic to get the contents of each site to then only filter those who contain your second seach term – LordNeo May 29 '16 at 21:30
  • @LordNeo Sometimes, the thing I am searching for is not searchable. For example, the shipping method of a particular item. (Good question!) – Bort May 29 '16 at 21:31

1 Answers1

1

I'll use jQuery for simplicity but you can translate to plain JS if you will.

Let's say your <a> links are inside some #list parent, and the data that could result a positive match is stored in pages under some #phoneModel element...

jsBin demo

var query = "samsung s6";
var regex = new RegExp(query, "i");

$("#list a").each(function() { // Every link in #list parent...

  var $link = $(this);

  $.ajax({
    url: $link.attr("href"),
    dataType: 'html',
    success: function (data) {
      if( data.match(regex) ){      
        $link.addClass("match");
      }
    }
  });

});

...than in CSS you simply need some class .match{ color:red; }.

But again... if you have lot of links it's a really bad idea to do such expensive AJAX calls...

Roko C. Buljan
  • 196,159
  • 39
  • 305
  • 313
  • When you say "bad idea", bad in what way? What have I to lose? – Bort May 29 '16 at 23:25
  • Performance. AJAX requests will take some time and if you are going to call 100 of them, this will make your browser slow. Alternatively you can do a [XMLHTTPREQUEST](https://developer.mozilla.org/de/docs/Web/API/XMLHttpRequest) inside a web worker. – dude May 30 '16 at 05:42