How to create different record for every update in FIWARE-STH Comet?

Question

I just started using FIWARE and I'm having trouble with the data I want to persist on MongoDB. I want to use STH Comet in minimal mode. I am using a Python script that creates my context data with a specific id and then updates certain attributes of this data. While I can't see any problem when I observe the JSON data from this script, and HTTP requests but MongoDB keeps only the last updated version of this data in the related collection. In addition, the "sth" labeled collections does not show up. However I can see the "credate" and "moddate" metadatas in attributes. But I need different records for every update.

Here is the first POST request that I send to Orion to create context data:

{"id": "urn:ngsi-ld:entity:001", 
"type": "Log",
"a": {"type": "Datetime", "value": "17/09/2021"}, 
"b": {"type": "Datetime", "value": "12:00:18.0"}, 
"c": {"type": "Integer", "value": 49.51}, 
"d": {"type": "Integer", "value": 175.35}, 
"e": {"type": "Integer", "value": 24.25}, 
"f": {"type": "Integer", "value": 999.1}, 
"g": {"type": "Integer", "value": 85.87}, 
"h": {"type": "Integer", "value": -0.01}, 
"i": {"type": "Integer", "value": 37.41}, 
"j": {"type": "Integer", "value": 60.65}}

Here is what I get in MongoDB collection, attrNames has been made dummy for the question:

{
  "_id": {
    "id": "urn:ngsi-ld:entity:001",
    "type": "Log",
    "servicePath": "/"
  },
  "attrNames": [
    "a",
    "b",
    "c",
    "d",
    "e",
    "f",
    "g",
    "h",
    "i",
    "j"
  ],
  "attrs": {
    "a": {
      "type": "Datetime",
      "creDate": 1645514416,
      "modDate": 1645514416,
      "value": "17/09/2021",
      "mdNames": []
    },
    "b": {
      "value": "12:00:21.0",
      "type": "Datetime",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514597
    },
    "c": {
      "value": 666.47,
      "type": "Integer",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514640
    },
    "d": {
      "value": 175.55,
      "type": "Integer",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514597
    },
    "e": {
      "value": 24.27,
      "type": "Integer",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514597
    },
    "f": {
      "value": 999.28,
      "type": "Integer",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514597
    },
    "g": {
      "type": "Integer",
      "creDate": 1645514416,
      "modDate": 1645514416,
      "value": 85.87,
      "mdNames": []
    },
    "h": {
      "type": "Integer",
      "creDate": 1645514416,
      "modDate": 1645514416,
      "value": -0.01,
      "mdNames": []
    },
    "i": {
      "type": "Integer",
      "creDate": 1645514416,
      "modDate": 1645514416,
      "value": 37.41,
      "mdNames": []
    },
    "j": {
      "value": 1111.47,
      "type": "Integer",
      "mdNames": [],
      "creDate": 1645514416,
      "modDate": 1645514640
    }
  },
  "creDate": 1645514416,
  "modDate": 1645514640,
  "lastCorrelator": "679e1cba-93b0-11ec-be0f-0242ac120102"

This is the data output I can obtain with STH Comet Query:

{
    "contextResponses": [
        {
            "contextElement": {
                "attributes": [
                    {
                        "name": "c",
                        "values": [
                            {
                                "_id": "621ca977165813000740898e",
                                "recvTime": "2022-02-28T10:52:39.012Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.47
                            },
                            {
                                "_id": "621ca97d165813000740899e",
                                "recvTime": "2022-02-28T10:52:45.581Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.61
                            },
                            {
                                "_id": "621ca98316581300074089a7",
                                "recvTime": "2022-02-28T10:52:51.239Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.46
                            },
                            {
                                "_id": "621ca98916581300074089ac",
                                "recvTime": "2022-02-28T10:52:57.662Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.57
                            },
                            {
                                "_id": "621ca98d16581300074089b9",
                                "recvTime": "2022-02-28T10:53:01.852Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.47
                            },
                            {
                                "_id": "621ca99316581300074089c7",
                                "recvTime": "2022-02-28T10:53:07.242Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.41
                            },
                            {
                                "_id": "621ca9a116581300074089d2",
                                "recvTime": "2022-02-28T10:53:21.305Z",
                                "attrName": "c",
                                "attrType": "Integer",
                                "attrValue": 49.51
                            }
                        ]
                    }
                ],
                "id": "urn:ngsi-ld:entity:001",
                "isPattern": false,
                "type": "Log"
            },
            "statusCode": {
                "code": "200",
                "reasonPhrase": "OK"
            }
        }
    ]
}

As you can see, I can obtain the MongoDB _id in the returned query, but the problem is, I cannot find the location of these seperate datas in related MongoDB database.

I've gone through all the docs but either couldn't find the solution or am I missing something in the architectural logic. How should I go about the issue?

Thanks!

I'm not sure of understanding the case... According to the response from STH you include, the STH is behaving correctly and for each timestamp of attribute `c` is showing the value of such attribute at that time. What's the problem? :) — fgalan, Mar 03 '22 at 20:30
PD. I strongly encourage to NOT use the NGSI-v1 API in STH. This is a deprecated API. You should use NGSI-v2, see https://fiware-sth-comet.readthedocs.io/en/latest/raw-data-retrieval/index.html — fgalan, Mar 03 '22 at 20:32
Hi @fgalan ! Thanks a lot for the support, I really appreciate it. The problem was that the persisted data did not appear as historical records on MongoDB. I must have fixed a point I missed by redoing my configurations from scratch, now the problem is solved. I also mentioned a different situation that I encountered after solving this problem and which I thought might be a bug, as an issue in the GitHub repository where we first contacted. :) Thank you again for your interest. :) — YamcoTheGreat, Mar 04 '22 at 08:12
Great to know you finally solved the problem. I'd recommend you to selt-answer your question to explain it. I'd have a look to STH github to see the issue you have created. Thanks! — fgalan, Mar 04 '22 at 10:30

score 1 · Accepted Answer · answered Mar 04 '22 at 11:01

To put it simply, I realized that the problem I was experiencing was due to the parameters that I did not change in the config.js file of the STH Comet I set up on the remote server.

So, I changed the host parameter (from "localhost" to the "server's own IP") in config.server part and the URI parameter (from "localhost:27017" to "serversIP:27017") in the config.database part in the file.

The system is now working as expected.

How to create different record for every update in FIWARE-STH Comet?

1 Answers1