0

I have a non-user-specific large multidimensional array ~100Mb-150Mb that I currently save it on one JSON file. This array has to be updated every minute using data coming from an API.

I'm not sure if I should use $_COOKIE or $_SESSION for storing it, and avoid file-writings (i.e., fopen(); fwrite(); fclose();) for performance.

There may not be a memory issue since the data is already being collected with this setup on the top of following script:

ini_set('max_execution_time', 0);
ini_set('memory_limit', '-1');
set_time_limit(0);

Data Collection

// Config class for path and other constants
require_once __DIR__ . "/ConstEQ.php";

class EquityRecords extends EQ implements ConstEQ
{

    public static function getSymbols()
    {

        //***************** START: ALL SYMBOLS ARRAY ********************** //
        // var: is a filename path directory, where there is an md file with list of equities
        $list_of_equities_file = __DIR__ . self::SYMBOLS_PATH;

        // var: is content of md file with list of equities
        $content_of_equities = file_get_contents($list_of_equities_file);

        // var is an array(3) of equities such as: string(4) "ZYNE", string(10) "2019-01-04", string(27) "ZYNERBA PHARMACEUTICALS INC"

        $symbols_array = preg_split('/\R/', $content_of_equities);
        //***************** END: ALL SYMBOLS ARRAY ********************** //

        // child and mother arrays are created to help calling equities in batches of 100, which seems to be the API limit.
        $child = array();
        $mother = array();
        // var: is 100 counter
        $limit_counter = self::NUMBER_OF_STOCKS_PER_REQUEST;
        foreach ($symbols_array as $ticker_arr) {
            $limit_counter = $limit_counter - 1;
            $symbols_array = preg_split('/\t/', $ticker_arr);
            array_push($child, $symbols_array);

            if ($limit_counter <= 0) {
                $limit_counter = self::NUMBER_OF_STOCKS_PER_REQUEST;
                array_push($mother, $child);
                $child = array();
            }

        }
        return $mother;
    }

    public static function allEquitiesSignleJSON()
    {
        $equity_arrays = EquityRecords::getSymbols();
        $base_url = self::BASE_URL . self::TARGET_QUERY;

        $current_time = date("Y-m-d-H-i-s");
        $all_equities = array();
        // ticker: AAPL, GE, AMD
        foreach ($equity_arrays as $ticker_arr) {
            $ticker = array_column($ticker_arr, 0);
            $equity_url = $base_url . implode("%2C", $ticker) . self::END_POINT;
            $raw_eauity_json = file_get_contents($equity_url);
            $raw_equity_array = json_decode($raw_eauity_json, true);
            $all_equities = array_merge($all_equities, $raw_equity_array);
        }

        $all_equities_json = json_encode($all_equities);

        $symbols_dir = __DIR__ . self::SYMBOLS_DIR;

        if (!is_dir($symbols_dir)) {mkdir($symbols_dir, 0755, true);}

        $raw_equity_file = $symbols_dir . "/" . $current_time . ".json";
        $fp = fopen($raw_equity_file, "x+");
        fwrite($fp, $all_equities_json);
        fclose($fp);
        echo "YAAAY! Equity JSON file success at " . __METHOD__ . " !  " . self::NEW_LINE;
    }

}
  • It seems by default max capacity of $_SESSION is 128Mb. Is max capacity of $_COOKIE the same as $_SESSION?
  • Which one might be faster to use or should I use, or is there other ways to avoid file-writings for improving performance?
Emma
  • 27,428
  • 11
  • 44
  • 69
  • 1
    since sessions are server-side and cookies are client-side, wouldn't the answer be completely dependent on whether you need the data on the server or the client, since 100-150mb are quite a lot to transfer via WAN? and wouldn't the flag "php" imply you're asking about server-side computation, therefore answering your question? – Franz Gleichmann Apr 13 '19 at 18:51
  • 1
    Cookies are constrained to a maximum of 50 á 4KB. So, that kinda cuts down on the options. Notably PHPs session store is also filesystem-bound, so 100MB per user is certainly inadvisable. – mario Apr 13 '19 at 18:54
  • 1
    You cannot write in `$_COOKIE[]`, only read from it. You can set a cookie using `setcookie()`, it is put into the response sent to the browser, the browser sends it back to you on the next request and you'll find it in `$_COOKIE[]` on the next page. Assuming you didn't know how cookies work (you just learned) do you think sending 100-150 MB back and forth in a cookie (assuming it is possible, which is not) is the way to go? Think again. – axiac Apr 13 '19 at 19:01

1 Answers1

2

I'm not sure if I should use $_COOKIE or $_SESSION for storing it…

Quite likely neither of these things.

$_COOKIE is mapped to cookies in the user's browser. It is limited to 4 kB per cookie, and a total of roughly 80 kB across all cookies. Setting large cookies will make requests to your web site slow, and is not recommended.

$_SESSION is stored in a file on the web server and has looser size limits, but is stored separately for each user. Unless the data you're gathering from this API is user-specific, this is not a good choice either.

If your data is truly just being collected from an API and should be shared across all users, storing it in a file (like you're already doing!) is a perfectly reasonable approach. 100 MB won't fit into cookies, and storing it in the session won't improve anything.