Can `gramex` `storelocations` create parent folder structure?

Question

Suppose I want to have custom storelocations location. I can surely pass that path in the yaml. However, If the path does not exist, gramex throws me an error

Code I want to use is in gramex.yaml as:

storelocations:
  projects: &PROJECTS
    url: $GRAMEXDATA/apps/custom_app/store
    table: projects
    columns:
      id:
        type: INTEGER
        primary_key: true
      projectname: TEXT
      summary: TEXT
      datecreated: INTEGER
url:
  test-end-point:
    pattern: /$YAMLURL/
    handler: FormHandler
    kwargs:
      id: id
      <<: *PROJECTS

When I run gramex The error I got is:

Traceback (most recent call last):
  File "<win_user>\.conda\envs\base39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "<win_user>\.conda\envs\base39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "<win_user>\.conda\envs\base39\Scripts\gramex.exe\__main__.py", line 7, in <module>

  File "<win_user>\.conda\envs\base39\lib\site-packages\gramex\__init__.py", line 184, in commandline
    return init(cmd=config)
  File "<win_user>\.conda\envs\base39\lib\site-packages\gramex\__init__.py", line 309, in init
    callback = getattr(services, key)(conf[key])
  File "<win_user>\.conda\envs\base39\lib\site-packages\gramex\services\__init__.py", line 565, in storelocations
    gramex.data.alter(**subconf)
  File "<win_user>\.conda\envs\base39\lib\site-packages\gramex\data.py", line 1193, in alter
    engine = create_engine(url, **kwargs)
  File "<win_user>\.conda\envs\base39\lib\site-packages\gramex\data.py", line 660, in create_engine
    _ENGINE_CACHE[url] = create(url, **kwargs)
  File "<string>", line 2, in create_engine
  File "<win_user>\.conda\envs\base39\lib\site-packages\sqlalchemy\util\deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
  File "<win_user>\.conda\envs\base39\lib\site-packages\sqlalchemy\engine\create.py", line 514, in create_engine
    u = _url.make_url(url)
  File "<win_user>\.conda\envs\base39\lib\site-packages\sqlalchemy\engine\url.py", line 738, in make_url
    return _parse_url(name_or_url)
  File "<win_user>\.conda\envs\base39\lib\site-packages\sqlalchemy\engine\url.py", line 799, in _parse_url
    raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: Could not parse SQLAlchemy URL from string '<win_user>\AppData\Local\Gramex Data/apps/custom_app/store'

Implying that storelocations cannot create the folder tree if needed. What is the remedy for this?
Is there any inbuilt method I am missing?
Is there better / more robust way to do it?

score 0 · Answer 1 · answered Aug 06 '23 at 05:41

Gramex storelocations currently relies on SQLAlchemy's create_engine() to connect to the data store.

create_engine() creates files if the directory exists, but not if the directory does not exist.

So you need to create the directory first, but the database itself will be auto-created. You could do that before running gramex. Doing this inside a schedule: will not work since storelocations are processed before schedules.

Can `gramex` `storelocations` create parent folder structure?

1 Answers1