Create MongoDB ObjectID from date in the past using PHP driver

Question

I have to import a lot of data into MongoDB from MySQL and I'd like to use the timestamp from the ObjectID instead of storing it in a separate key/value (as it is in the existing data). In order to do this I'd need to create an ObjectID for the existing data with a date from the past. I also need to do this using the PHP driver. I've read that there might be a way to do this in Python, Java and Node.JS so I thought maybe there was an equivalent method in PHP.

If this is possible - is it safe to do? Meaning and I going to have issues with duplicate or invalid ObjectIDs? Thanks.

In Node.JS:

var timestamp = Math.floor(new Date().getTime()/1000);
var objectId = new ObjectID(timestamp);

Below is from: MongoDB using timestamps to sort

In Python:

gen_time = datetime.datetime(2010, 1, 1)
dummy_id = ObjectId.from_datetime(gen_time)

In Java:

Date d = new Date(some timestamp in ms);
ObjectId id = new ObjectId(d)

Answer 1

Right now, the PHP driver has no built in functionality for this, the __set_state() that the other answer mentioned is only for being able to session-deserialize the ID and doesn't allow you to create it through the specific components.

You will have to do the following to automatically create an ID:

<?php
function createId( $yourTimestamp )
{
    static $inc = 0;

    $ts = pack( 'N', $yourTimestamp );
    $m = substr( md5( gethostname()), 0, 3 );
    $pid = pack( 'n', posix_getpid() );
    $trail = substr( pack( 'N', $inc++ ), 1, 3);

    $bin = sprintf("%s%s%s%s", $ts, $m, $pid, $trail);

    $id = '';
    for ($i = 0; $i < 12; $i++ )
    {
        $id .= sprintf("%02X", ord($bin[$i]));
    }
    return new MongoID($id);
}

var_dump( createId( time() ) );
?>

Answer 2

If you are using the MongoId just for comparison, for example, selecting all records within a date range, you don't need a fully valid id. So you could simply do:

$id = new \MongoId(dechex($timestamp) . str_repeat("0", 16));

Just be sure to never insert this id and just use it for $gte/$gt/$lt/$lte queries.

edit

My bad, the above snippet would work with dates before about 1979 as dechex($timestamp) doesn't always return 8 characters, so a better snippet would be:

$id = new \MongoId(sprintf("%08x%016x", $timestamp, 0));

Answer 3

This could create problems since one of the factors that makes the objectid unique is the time part however it should increment the last byte (or so) on multiple inserts with the same time.

It looks like there was some movement towards allowing a ObjectId to be created with some params and infact it kinda almost talks about it here: http://php.net/manual/en/mongoid.set-state.php

Theoretically, an array of properties used to create the new id. However, as MongoId instances have no properties, this is not used.

However there is no way, atm, I believe to achieve what you can do in other languages without writing your own ObjectId generator.

Answer 4

I found the solution marked as correct here does not work for me. So, I'm providing my solution that will work. Anyway, before running the code, ensure that mongodb and mongodb php drivers are installed following these two official tutorials:

(1) https://www.mongodb.com/docs/manual/administration/install-community/

(2) https://www.mongodb.com/docs/drivers/php/

Then open your project directory and run the command in terminal/cmd:

  composer require mongodb/mongodb

(as mentioned in link no (2) above). The above command will automatically configure all your dependencies.

After all the above operations are done, run the following code snippet to create an ObjectId based on timestamp:

<?php
require_once __DIR__ . '/vendor/autoload.php';

// connect to mongodb
$con = new MongoDB\Client("mongodb://localhost:27017");  

function createObjectIdFromTimestamp($timeStamp) {
    $next_first_4bytes = (string) dechex(rand(1, 65535));
    $next_second_4bytes = (string) dechex(rand(1, 65535));
    $next_third_4bytes = (string) dechex(rand(1, 65535));
    $next_fourth_4bytes = (string) dechex(rand(1, 65535));
    $timeStamp = $timeStamp . $next_first_4bytes . $next_second_4bytes . $next_third_4bytes . $next_fourth_4bytes;
    $newId = new MongoDB\BSON\ObjectId($timeStamp);
    return $newId;
}

// creating a timestamp for 15th March 2013, 03:45:23 am:
$date = date_create_from_format("d-M-Y h:i:s a","15-Mar-2013 03:45:23 am"); 
// just change the above statement to a date and time you want to create your timestamp for.
    /*
        To create a date in the format PHP can understand,
        the above statement was written. '$date' is a variable
        that has stored the date in the format that PHP understands.
    
        In the function: date_create_from_format(),
        d = date of the month (in 2 digits), e.g: 15 (in the above statement),
        M = name of month (in 3 letters), e.g: Mar (in the above statement),
        Y = year (in 4 digits), e.g: 2013 (in the above statement),
        h = hour (in 12 hrs format) (in 2 digits), e.g: 03 (in the above statement),
        i = minutes (in 2 digits), e.g: 45 (in the above statement),
        s = seconds (in 2 digits), e.g. 23 (in the above statement),
        a = am or pm, e.g. am (in the above statement)
    */
$timestmp = substr((string) dechex($date -> getTimestamp()), 0, 8);
$id = createObjectIdFromTimestamp($timestmp);
// echo "\$id  = " . $id . "<br>";
    
?>

If you want to create an object id in the current date, then just replace the whole code snippet above with a php statements shown below:

<?php
require_once __DIR__ . '/vendor/autoload.php';

// connect to mongodb
$con = new MongoDB\Client("mongodb://localhost:27017"); 
$id = new MongoDB\BSON\ObjectId();
?>

I've tested my solution in Ubuntu.