I've created API with Kimono Labs, to generate RSS feed from a website. It is working ok, crawling data every hour, however every several days it just stop working. No errors, nothing. It the crawl history i can see, that previous crawls was successful, and then API just stop crawling the data. Until i launch manual crawl. Then API start working again, but only for a several days. And then all again, it stops, i initiate manual crawl, it's working for some time. What can cause such a behavior?
Asked
Active
Viewed 234 times
2 Answers
0
It's intended behaviour described in every API's under (?) popover:
<p>Auto-run frequency <span class="icon-question-circle" data-html="true" popover="Specify how often this API will automatically fetch new data from the target page(s). APIs are limited to 1 URL for a hourly auto-run, <1000 URLs for a daily auto-run, and <10,000 URLs for a weekly auto-run."></span></p>
AndriuZ
- 648
- 6
- 26
-
I don't see, how this is relevant? – Andrew Shtern Oct 30 '15 at 07:32
-
my answer was to Your unedited question :-) if it's not intended behaviour - report the bug directly – AndriuZ Oct 30 '15 at 11:31
0
Anyway, it was a kimono issue, that is fixed now. I've got an e-mail from support
This is a crawling bug that we've now implemented handling for.
We are running a script that will check for queued scheduled crawls every hour
and start them if they are not running.
Andrew Shtern
- 11
- 3