Parsehub API PHP

Question

How can I dump the results into a MySql db? Specially decode gzip and parse it to an HP array then dump into a db.

<?php

$connection = mysqli_connect("localhost","root","Aruns@cal");
if (!$connection) {
    die("Database connection failed: " . mysqli_error());
}

$db_select = mysqli_select_db($connection, "arunslocal");
if (!$db_select) {
    die("Database selection failed: " . mysqli_error());
}

$params = http_build_query(array(
  "api_key" => "xxxxxxx",
  "format" => "JSON"
));

$result = file_get_contents(
    'https://www.parsehub.com/api/v2/projects/tmnx0bN0Ty0U/last_ready_run/data?'.$params,
    false,
    stream_context_create(array(
        'http' => array(
            'method' => 'GET'
        )
    ))
);

echo "1"; 
$arr = json_decode($result, true);
echo "2"; 
foreach((array)$arr as $item){
    echo "3"; 
    $name = $item['name'];
    $value = $item['value'];
    echo "4"; 

    $sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES(1,3)";
    //$sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES('$name','$value')";

        if(!mysqli_query($connection,$sql))
        {
            die('Error : ' . mysqli_error($connection));
        }



    }
echo "5"; 
?>

How can I dump the results into a MySql db? Specially decode gzip and parse it to an HP array then dump into a db.

This is the responce i get ��8ZtqHycvgxB98L.csv]�1 �0н��v��N�:A<��J��`*�O�/ �s^�q�7�+?3��O6^>Np{�݅�w��'��!�l�;�� ewaJIp_8�즃-T;Cu�<6W=��S��[��mQ$�G��BX — Arun Prasan, Dec 19 '17 at 05:22
depends on what insert you are referring to.. I know SQL one though — Arun Prasan, Dec 19 '17 at 05:24
when i remove the format param i get a json string so the api works — madalinivascu, Dec 19 '17 at 05:25
I want to get the results into a table. But the results i get in encoded or something. Plus the json result is not loading into an array — Arun Prasan, Dec 19 '17 at 05:26
https://www.parsehub.com/api/v2/projects/tZELovmM4nqU/last_ready_run/data?api_key=tmgT6na5v3Me Can use this to view the data in json — Arun Prasan, Dec 19 '17 at 05:27
$arr = json_decode($results, true); foreach((array)$arr as $item){ echo "3"; $name = $item['name']; $value = $item['value']; } — Arun Prasan, Dec 19 '17 at 05:34
That could be compressed. Check the MIME type in the result. — tadman, Dec 19 '17 at 05:46
You also put your API key on the public internet. **Regenerate it immediately**. — tadman, Dec 19 '17 at 05:47

score 2 · Accepted Answer · edited Jun 20 '20 at 09:12

The data is gzip encoded

Your problem is the response data is gzip encoded.

As per parsehub's API documentation,

The Content-Encoding of this response is always gzip.

Solution

Try decoding the gzip response.

You can use gzdecode to decode the gzip response.

Also fetch the API with format json instead.

You can then json_decode to convert it into an array and use it.

Example

<?php
    $params = http_build_query(array(
      "api_key" => "xxxxxx",
      "format" => "json"
    ));

    $result = file_get_contents(
        'https://www.parsehub.com/api/v2/projects/tmnx0bN0Ty0U/last_ready_run/data?'.$params,
        false,
        stream_context_create(array(
            'http' => array(
                'method' => 'GET'
            )
        ))
    );
    $result = gzdecode($result);
    echo $result;
    $parsed_result = json_decode($result, true);
    $yield_data = $parsed_result['yield_data'];
    foreach($yield_data as $item){
        $name = $item['name'];
        $value = $item['value'];

        $sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES('{$name}',{$value})";

        if(!mysqli_query($connection,$sql)) {
            die('Error : ' . mysqli_error($connection));
        }
    }
?>

Output

{
 "yield_data": [
  {
   "name": "Sri Lanka 3M",
   "value": "7.760"
  },
  {
   "name": "Sri Lanka 6M",
   "value": "8.500"
  },
  {
   "name": "Sri Lanka 1Y",
   "value": "9.054"
  },
  {
   "name": "Sri Lanka 2Y",
   "value": "9.283"
  },
  {
   "name": "Sri Lanka 3Y",
   "value": "9.588"
  },
  {
   "name": "Sri Lanka 4Y",
   "value": "9.850"
  },
  {
   "name": "Sri Lanka 5Y",
   "value": "10.000"
  },
  {
   "name": "Sri Lanka 6Y",
   "value": "10.100"
  },
  {
   "name": "Sri Lanka 7Y",
   "value": "10.150"
  },
  {
   "name": "Sri Lanka 8Y",
   "value": "10.200"
  },
  {
   "name": "Sri Lanka 9Y",
   "value": "10.250"
  },
  {
   "name": "Sri Lanka 10Y",
   "value": "10.300"
  },
  {
   "name": "Sri Lanka 15Y",
   "value": "10.575"
  }
 ]
}

Updated the question with the modifications. how to handle the gzip part? — Arun Prasan, Dec 19 '17 at 06:13
glad that I could be of help, If the solution worked, you can accept the answer. Thanks machi. — RamC, Dec 19 '17 at 06:29

madalinivascu · Answer 2 · 2017-12-19T05:41:42.520

0

You need to loop over the yield_data array Try the following:

$arr = json_decode($results, true);
 foreach($arr['yield_data'] as $item) { 
   $name = $item['name'];
   $value = $item['value'];
 }

demo

edited Dec 19 '17 at 05:41

answered Dec 19 '17 at 05:36

madalinivascu

32,064
4
39
55

what echo 3 and 4? – madalinivascu Dec 19 '17 at 06:18

Parsehub API PHP

2 Answers2

The data is gzip encoded