0

Quite new to this as I am trying to deploy MemCachier and I am quite new to code in general.

Followed this guide https://devcenter.heroku.com/articles/flask-memcache

I have a function in a Flask app that will store a df in cache. I am working localy and didn't deploy the changes yet. The app is working fine and I can access the df in my app, but MemCachier GUI on Heroku does not display any data stored, so I am assuming its using cache.init_app(app, config={'CACHE_TYPE': 'simple'})

My code + function is:

#set memcache in Heroku
cache_servers = os.environ.get('MEMCACHIER_SERVERS')
if cache_servers == None:
    cache.init_app(app, config={'CACHE_TYPE': 'simple'})
else:
    cache_user = os.environ.get('MEMCACHIER_USERNAME') or ''
    cache_pass = os.environ.get('MEMCACHIER_PASSWORD') or ''
    cache.init_app(app,
        config={'CACHE_TYPE': 'saslmemcached',
                'CACHE_MEMCACHED_SERVERS': cache_servers.split(','),
                'CACHE_MEMCACHED_USERNAME': cache_user,
                'CACHE_MEMCACHED_PASSWORD': cache_pass,
                'CACHE_OPTIONS': { 'behaviors': {
                    # Faster IO
                    'tcp_nodelay': True,
                    # Keep connection alive
                    'tcp_keepalive': True,
                    # Timeout for set/get requests
                    'connect_timeout': 2000, # ms
                    'send_timeout': 750 * 1000, # us
                    'receive_timeout': 750 * 1000, # us
                    '_poll_timeout': 2000, # ms
                    # Better failover
                    'ketama': True,
                    'remove_failed': 1,
                    'retry_timeout': 2,
                    'dead_timeout': 30}}})

def symbol_search():

    flo = BytesIO()

    directory = 'symboldirectory'
    filenames = ('otherlisted.txt', 'nasdaqlisted.txt')

    ftp = FTP('ftp.nasdaqtrader.com')
    ftp.login()
    ftp.cwd(directory)

    #Create pandas dataframes from the nasdaqlisted and otherlisted files.
    for item in filenames:
        nasdaq_exchange_info=[]
        ftp.retrbinary('RETR ' + item, flo.write)
        flo.seek(0)
        nasdaq_exchange_info.append(pd.read_fwf(flo))
    ftp.quit()

    # Create pandas dataframes from the nasdaqlisted and otherlisted files.
    nasdaq_exchange_info=pd.concat(nasdaq_exchange_info, axis=1)
    nasdaq_exchange_info[['symbol', 'name', 'Exchange', 'Symbol', 'etf', 'Lot_size', 'Test', 'NASDAQ_Symbol']]=nasdaq_exchange_info['ACT Symbol|Security Name|Exchange|CQS Symbol|ETF|Round Lot Size|Test Issue|NASDAQ Symbol'].str.split('|', expand=True)
    nasdaq_exchange_info=nasdaq_exchange_info.drop(nasdaq_exchange_info.columns[[0]], axis=1).dropna()
    nasdaq_exchange_info=nasdaq_exchange_info[(nasdaq_exchange_info['Test'] != 'Y') & (nasdaq_exchange_info['symbol'] != 'Y') & (~nasdaq_exchange_info.symbol.str.contains('symbol', 'file')) & (~nasdaq_exchange_info.name.str.contains('%', 'arrant'))]
    nasdaq_exchange_info=nasdaq_exchange_info.drop(['Symbol', 'Exchange', 'Lot_size', 'Test', 'NASDAQ_Symbol', 'etf'], axis = 1)
    nasdaq_exchange_info=nasdaq_exchange_info[['name', 'symbol']].values.tolist()
    return cache.set("nasdaq_exchange_info", nasdaq_exchange_info)

symbol_search()

What I am missing here? and how can I upload the cache to MemCachier that it will be visible in the GUI?

Oris
  • 73
  • 8
  • Did you provision the add-on? Do you see a `MEMCACHIER_SERVERS` environment variable / config var having been set? – ChrisGPT was on strike Mar 18 '22 at 10:37
  • Thanks for helping. I forgot to set the env variable so I did set it for MEMCACHIER_SERVERS, but now the FLask app won't run. I get an error no memcache module found althogh I installed unsing pip install Flask-Caching. Any idea what need to be installed? – Oris Mar 18 '22 at 16:47
  • If you provisioned the add-on you shouldn't need to _set_ `MEMCACHIER_SERVERS`. The add-on should set that for you automatically. – ChrisGPT was on strike Mar 19 '22 at 13:11
  • In any case, you can't `pip install` directly on Heroku. All dependencies must be declared in your `requirements.txt` or `Pipfile` and `Pipfile.lock`. Update those files, commit, and redeploy. – ChrisGPT was on strike Mar 19 '22 at 13:13
  • Thanks a lot Chris, I have ended up using the steps described here https://devcenter.heroku.com/articles/memcachier#python and changed the code, after installing python-binary-memcached now it's working as intended – Oris Mar 19 '22 at 21:51

0 Answers0