Need help understanding results returned by Bing Web Search API v5

Question

I've updated the following snippet that I created earlier with the beta version of the Bing Web Search API to use the newer domain name: api.cognitive.microsoft.com/bing/v5.0/search that Bing Web Search API uses now -

Please replace your own Bing API key to run the sample

<!DOCTYPE html>
<html>
<head>
    <title>Bing Search v5 - show all results</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>
<script>

var total;
var ofst = 0;
var pgcnt=50;
var results = '';

var burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=Bill Gates&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us&freshness=Month"; //737 results
//var burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=Bill Gates&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us"; //304 results
//var burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=site:mvark.blogspot.com&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us&freshness=Month"; 

$(function() { 

function xhr_get(url) {
  return $.ajax({
  url: url,
  beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","abc123"); //replace value with your own key
            },
  type: "GET",

  })
 .done(function(data) {
        total = data.webPages.totalEstimatedMatches; 
  len = data.webPages.value.length
  for (i=0; i<len; i++ )
  {
     results += "<p><a href='" + data.webPages.value[i].url + "'>" + data.webPages.value[i].name + "</a>: " + data.webPages.value[i].snippet + "</p>";
  }
  $('#content').html(results);
 ofst += pgcnt;
 if (total > ofst) { 

 burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=Bill Gates&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us&freshness=Month"; 
 //burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=Bill Gates&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us"; 
 //burl = "https://api.cognitive.microsoft.com/bing/v5.0/search/?q=site:mvark.blogspot.com&count=" + pgcnt + "&offset=" + ofst + "&mkt=en-us&freshness=Month"; 
 xhr_get(burl);
 }
 else { console.log('No more results to show'); 
 }
  })
}

xhr_get(burl);

});
</script> 
Results:  <div id="content">Fetching...</div>
</body>
</html>

I see a difference between the results that were returned earlier & results fetched now. I would like to know if there is something wrong with my code that is causing the changed behavior and the answers to the following:

For some search keywords, I've seen that the maximum results that are returned are now exactly 1000 (totalEstimatedMatches=1000) though there are more if I search through Bing's website. Is 1000 the maximum limit and is there some throttling?
When freshness=Month request parameter is added it returns more results that I would get if I didn't use it? Isn't the default behavior to fetch all results if freshness is not specified?

I observe less web contents to be found. Does anyone else observe the same? — Jakob Alexander Eichler, Aug 10 '16 at 17:54
I'm not a js dev but I'm writing this--> (https://github.com/rtruxal/py-cog-serv) in python, and when I replicate your first query I'm getting totalEstimatedMatches of 179000 rather than the 737 your code produces. Unfortunately that's all I got. Can't read JS super-well so I can't ID the issue. — Rob Truxal, Nov 17 '16 at 20:02

score 1 · Answer 1 · answered Nov 23 '16 at 19:48

Bing does not have a max value for totalEstimatedMatches. Although it's important to keep in mind it's an estimate and we don't know what estimate means specifically. It could potentially mean 500 < totalEstimatedMatches < 2000 for all we know. How they 'estimate' this number is not in the documentation.
I stared at your second question for a long time before figuring out where the dissonance is. Try appending &responseFilter=Webpages to the end of all the URLs you're making. When tabulating your responses, it looks like you're counting only webresults but not specifying that only web results are to be returned by your query.

Need help understanding results returned by Bing Web Search API v5

1 Answers1