Local CosmosDB connectivity with pymongo for gramex

Question

Not able to connect cosmos db using gramex mongo adapter. From python, We can connect to local cosmosDB instance using:

# --------------------------------------
import pymongo
uri = r"mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D@localhost:10255/admin?ssl=true"
client = pymongo.MongoClient(uri,
                             tls=True,
                             tlsCAFile='./documentdbemulatorcert.cer')
db = client.admin
print(db.command("serverStatus"))
# --------------------------------------

Please mind “tlsCAFile” parameter

However in gramex when I connect using:

# --------------------------------------
url:
  envvartest-app-data:
    pattern: /$YAMLURL/appdata
    handler: FormHandler
    kwargs:
      url: "mongodb://localhost:Tm%2BOzpb8BrV7DrZHSrGm3GMKyx9r%2Frl5ue9letmD1XRUUiafHFUyIQNenAQDla85nqVDrb8tr%2FtB0LR4azi1FQ%3D%3D@localhost:10255/admin?ssl=true"
      database: galaxy-dev
      collection: threats
      id: record_number
      connect_args:
        tls: True
        tlsCAFile: $YAMLPATH/documentdbemulatorcert.cer
        # ssl:
        #   ssl_ca: $YAMLPATH/documentdbemulatorcert.cer
# --------------------------------------

the connection fails reading:

ERROR   17-Feb 14:05:06 formhandler 9988 envvartest-app-data: filter failed
Traceback (most recent call last):
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
    result[key] = yield val
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
    value = future.result()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
    return self.__get_result()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 235, in filter
    data = method(url=url, controls=controls, args=args, query=query, **kwargs)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1474, in _filter_mongodb
    meta_cols = pd.DataFrame(list(table.find().limit(100)))
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1238, in next
    if len(self.__data) or self._refresh():
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 1130, in _refresh
    self.__session = self.__collection.database.client._ensure_session()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1935, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1883, in __start_session
    server_session = self._get_server_session()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1921, in _get_server_session
    return self._topology.get_server_session()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 520, in get_server_session
    session_timeout = self._check_session_support()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 502, in _check_session_support
    None)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\topology.py", line 220, in _select_servers_loop
    (self._error_message(selector), timeout, self.description))
pymongo.errors.ServerSelectionTimeoutError: localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091), Timeout: 30s, Topology Description: <TopologyDescription id: 620e089c395cf9273486aa57, topology_type: Single, servers: [<ServerDescription ('localhost', 10255) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091)')>]>
ERROR   17-Feb 14:05:06 __init__ 9988 500 GET /appdata (127.0.0.1) 30795.58ms envvartest-app-data

Not able to connect cosmos db using gramex mongo adapter.

Steps to reproduce

Install CosmosDB emulator
Generate access key:
.\Microsoft.Azure.Cosmos.Emulator.exe /GenKeyFile=D:\99exps\cosmosdb\key
Start cosmosDB emulator with mongoDB support:
.\Microsoft.Azure.Cosmos.Emulator.exe /FailOnSslCertificateNameMismatch /EnableMongoDbEndpoint=3.6 /EnableMongoDbEndpoint=3.2 /KeyFile=D:\99exps\cosmosdb\key
Create Gramex Application with above mentioned configuration
Observe the error

Edit 1:

The error is changed now:

ERROR   19-Feb 00:10:33 formhandler 9988 galaxy-app-data: filter failed
Traceback (most recent call last):
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\handlers\formhandler.py", line 158, in get
    result[key] = yield val
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\tornado\gen.py", line 1133, in run
    value = future.result()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
    return self.__get_result()
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 239, in filter
    columns=columns, **kwargs)
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\gramex\data.py", line 1521, in _filter_mongodb
    cols = [k for k in table.find().limit(1)[0].keys()]
  File "C:\Users\shraddheya.shrivasta\Anaconda3\lib\site-packages\pymongo\cursor.py", line 694, in __getitem__
    raise IndexError("no such item for Cursor instance")
IndexError: no such item for Cursor instance
WARNING 19-Feb 00:10:33 web 500 GET /appdata (::1): IndexError('no such item for Cursor instance')
ERROR   19-Feb 00:10:33 __init__ 9988 500 GET /appdata (::1) 846.11ms galaxy-app-data

Could you please remove the `connect_args` parameter and move the `tls*` arguments directly under `kwargs`? Gramex doesn't recognize a `connect_args` parameter. — S Anand, Feb 17 '22 at 09:00

score 1 · Answer 1 · answered Feb 20 '22 at 07:56

As of Gramex 1.76.0 (Feb 2022), FormHandler does not connect to empty MongoDB collections, since there is no schema defined.

There is a plan to support empty MongoDB collections by specifying the schema explicitly.

But for now, you should be able to access the collection once you've added at least one row to it.

Local CosmosDB connectivity with pymongo for gramex

Edit 1:

1 Answers1