0

Context:

I am trying to code my own money aggregator because most of available tools on the market does not cover all financial websites yet. I am using python 2.7.9 on a raspberrypi.

I managed to connect to some of my accounts so far thanks to requests or Dryscrap library. The website I am trying to aggregate now is giving me hard time (more that the one I posted one 3 month ago ) and its name is https://www.linxo.com (which is is actually an aggregator itself).

This time I decided to use requests (might be a bad choice, I am not sure)

Issue:

When running this code by miming the curl request from a browser

import requests

with requests.Session() as s:
    headers = {
        'Accept-Encoding': 'gzip, deflate, br',
        'Accept-Language': 'en-US,en;q=0.8,fr;q=0.6',
        'Upgrade-Insecure-Requests': '1',
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/59.0.3071.109 Chrome/59.0.3071.109 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'Referer': 'https://wwws.linxo.com/auth.page',
        'Connection': 'keep-alive',
    }

    r_init = s.get('https://wwws.linxo.com/auth.page#Login', headers=headers)

    linxoSession = r_init.cookies.items()[1][1]  

    r_connect = s.get('https://wwws.linxo.com/secured/overview.page', headers=headers) 

    headers = {
        'Origin': 'https://wwws.linxo.com',
        'Accept-Encoding': 'gzip, deflate, br',
        'Accept-Language': 'en-US,en;q=0.8,fr;q=0.6',
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/59.0.3071.109 Chrome/59.0.3071.109 Safari/537.36',
        'Content-Type': 'text/x-gwt-rpc; charset=UTF-8',
        'Accept': '*/*',
        'X-GWT-Module-Base': 'https://wwws.linxo.com/secured/js/',
        'X-GWT-Permutation': 'DB8126A36E6BF1903AACA5D5D293D391',
        'Referer': 'https://wwws.linxo.com/secured/overview.page',
        'Connection': 'keep-alive',
    }

    data = '7|0|7|https://wwws.linxo.com/secured/js/|XXXXXXXX|net.customware.gwt.dispatch.client.standard.StandardDispatchService|execute|net.customware.gwt.dispatch.shared.Action|com.linxo.gwt.rpc.client.auth.CheckSessionAction/4080764126|' + linxoSession + '|1|2|3|4|1|5|6|7|'
    r_connect = s.post('https://wwws.linxo.com/secured/js/dispatch', headers=headers, cookies=r_connect.cookies, data=data)

the result for the last request is

 //EX[2,0,0,1,["com.linxo.gwt.rpc.client.exception.InvalidSessionException/3836580376","No context while handler is supposed to be logged-in"],0,7]

Questions:

  • Why do have I have such error message and how can I login to the
    website properly to retrieve the data I am looking for?
  • Should I use another library such as dryscrap?
JB Rolland
  • 87
  • 10

1 Answers1

0

I ended up using selenium on raspberry pi with Firefox to debug and PhantomJS to run the program headless. It drastically accelerates my coding time but the performance might likely to be lower which I do not really mind

JB Rolland
  • 87
  • 10