0

I don't think this fits into any category so putting on flame suit and see what happens.

I have a game store. I wrote a php script to pull the xml file from the parent company, process the file and post to my blog. Simple right? This has worked flawless with a cron job for FIVE(5) YEARS!! no problem.

As of this year I noticed some were things happening. Video links disappearing, as they have migrated those to youtube. That is fine if you are visiting their site, but not good if your are creating a post that has a video. But not a big deal, that was easily fix able.

Why I am here today is because of the following. At first I was getting 503 ERRORS and thought it was godaddy, but it turns out, it was something to do with the XML file. I know this because the posting stopped due to errors. Most were missing tags, which of course produced a domino effect of other erros that could not complete.

I changed the time out, as well as the accepted file size (ini_set(mem 128m)), but that did not work. I have a suspicion, I do not know, thus why I am here, that the file has become to big to process.

This is the code that has worked for years with no issues. not working now

function getXMLFeed($xml_feed)
    {
        ############################################################################
        ##### GRAB NEW RELEASES  XML BY CURL  
        ############################################################################
        //This is the file where we save the    information
        $fp = fopen('data/gamedb.xml','w');
        $ch = curl_init($xml_feed);
        curl_setopt($ch, CURLOPT_TIMEOUT, 50);
        // write curl response to file
        curl_setopt($ch, CURLOPT_FILE, $fp); 
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
        // get curl response
        $data = curl_exec($ch); 
        curl_close($ch);
        fclose($fp);
        return $data;
    }

the "xmlfeed" is the link to my rss account assigned to me from the parent company. couple tips I could use - links if you have them, is there a way to just pull out the recent game(s) instead of the whole library once a day? This is a remote file (for those that ask) Also, Am I correct that the file has gotten to big - it takes about 30-45 seconds to process - which is a long time.

I'll await your replies! Again, this is worked for 5+ years with no hands-on. I wrote it to be fully automated and it works. this issue started a few weeks ago. Keep this in mind when asking questions or offering suggestions.

EDIT: There is a shorter rss link that we can use, but is it no where comprehensive as the xml, as it shows only 8 games and just a few did bits. Using that for now with a few modification. and the file is only 7k tops.

EDIT x2 here are some of the errors is produces remotely or locally when running it:

Warning: SimpleXMLElement::__construct(): Entity: line 1: parser error : Start tag expected, '<' not found in D:\xampp\htdocs\bfgbutler\bfg.loader.php on line 92

Warning: SimpleXMLElement::__construct(): data/game.db in D:\xampp\htdocs\bfgbutler\bfg.loader.php on line 92

Warning: SimpleXMLElement::__construct(): ^ in D:\xampp\htdocs\bfgbutler\bfg.loader.php on line 92

Fatal error: Uncaught exception 'Exception' with message 'String could not be parsed as XML' in D:\xampp\htdocs\bfgbutler\bfg.loader.php:92 Stack trace: #0 D:\xampp\htdocs\bfgbutler\bfg.loader.php(92): SimpleXMLElement->__construct('data/game.db') #1 D:\xampp\htdocs\bfgbutler\runmaint.php(17): importGames('klyxgaming', 'en') #2 {main} thrown in D:\xampp\htdocs\bfgbutler\bfg.loader.php on line 92

It should not be throwing these errors. I can only assume that they are coming because of changes in the xml file.

Cain
  • 27
  • 1
  • 2
  • 9
  • What happens when you use a standard browser to request that data? – VolkerK Jan 31 '16 at 18:21
  • Hi, thanks for asking, if it just put the address in the browser, takes about 30 seconds and the whole library shows up as it should. that is the puzzling part. – Cain Jan 31 '16 at 18:25
  • Check webserver and php logfiles – Charlotte Dunois Jan 31 '16 at 18:27
  • @CharlotteDunois I did an edit on my OP. see if that helps for some clues – Cain Jan 31 '16 at 18:32
  • Looks like your XML is corrupted. I would suggest saving what you get returned from XML in a file (what you currently do) and post the contents here. – Charlotte Dunois Jan 31 '16 at 18:34
  • It's almost like it is getting garbled now when getting the file... I agree , but why now? someting in the source must have changed. – Cain Jan 31 '16 at 18:34
  • I dont think it would fit, I think it is like 25MB – Cain Jan 31 '16 at 18:36
  • this might help someone, here is line #92 $data = new SimpleXMLElement('data/game.db'); – Cain Jan 31 '16 at 18:39
  • 30 seconds might hit the time limit, see http://docs.php.net/manual/en/info.configuration.php#ini.max-execution-time and http://docs.php.net/set_time_limit – VolkerK Jan 31 '16 at 18:40
  • OK found the prob - but still dont have a solution: @CharlotteDunois is correct, my database xml is cut off at the bottom, so it is not getting the whole file. I have the ini's set at set_time_limit(0); ini_set('memory_limit', '2048M'); but doesnt seem to help. since the file is over 25mb i'll adjut the mem limit - if that is correct and see what happens – Cain Jan 31 '16 at 18:42
  • http://stackoverflow.com/questions/1167062/best-way-to-process-large-xml-in-php – Charlotte Dunois Jan 31 '16 at 18:43
  • thank you @CharlotteDunois, I think that might be what I need. Not familiar with SAX but good time as any - also I think the suggestion from VolkerK works too, the max_execution_time got the rest of the file, but other issues are at hand I can deal with later. Thank you for your guys input - i'll hammer it out from here (ergo why i have low post - i can usually find my solution here :-) - cheers! – Cain Jan 31 '16 at 18:53

0 Answers0