Python Requests - how use post method?

Question

How do I make an inquiry from the Academic Articles list? using Python and Requests

search for the name Maria Helena

site: http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search

Can you show us what you have done so far? Also, did you read the documentation for requests? — hikerjobs, Mar 11 '18 at 05:16
Possible duplicate of [How to send POST request?](https://stackoverflow.com/questions/11322430/how-to-send-post-request) — iBug, Mar 11 '18 at 05:22

score 2 · Answer 1 · answered Mar 11 '18 at 05:24

Here are some steps you can try:

Visit the webpage with your browser and examine what request is made using the google developer tools.

Note that the response is HTML so you may need to process the data with a scraping library like beautiful soup. Could there be a json api (left as an exercise)?

Making a request is not too bad it will be something like (full implementation is left as an exercise):

requests.post("http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search", ...)

Note that I left the payload as an exercise but feel free to ask for hints if you need (try searching for how to input form data with requests in google)

score 1 · Accepted Answer · answered Mar 11 '18 at 05:43

import json

import requests
from bs4 import BeautifulSoup

URL = 'http://bibliotecadigital.fgv.br/ojs/index.php/abpt/search/search'


def get_search_result(query):
    r = requests.post(URL, data={'query': query})
    r.raise_for_status()
    return r.text


def parse_search_result(html):
    soup = BeautifulSoup(html, 'lxml')
    div = soup.find('div', {'id': 'results'})

    result = []
    for row in div.find_all('tr', {'valign': 'top'}):
        edition, title, links = row.find_all('td')
        all_links = {}
        for link in links.find_all('a'):
            all_links[link.text.strip().lower()] = link['href']

        result.append({
            'edition': edition.a['href'],
            'title': title.text,
            **all_links
        })
    return result


if __name__ == '__main__':
    search_result = get_search_result('Maria Helena')
    parsed_result = parse_search_result(search_result)

    print(len(parsed_result))
    print(json.dumps(parsed_result, indent=4))

Console:

25
[
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1055",
        "title": "Considera\u00e7\u00f5es s\u00f4bre o \"Sistema de reabilita\u00e7\u00e3o dos incapacitados\" da Gr\u00e3-Bretanha",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13779",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13779/12679"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1084",
        "title": "Como transmitir aos pais os resultados dos testes",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14685",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14685/13582"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1060",
        "title": "Resumo da participa\u00e7\u00e3o do Brasil ao V Congresso Interamericano de Psicologia Aplicada",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13915/12814"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1097",
        "title": "Estudo evolutivo dos n\u00edveis de segrega\u00e7\u00e3o perceptiva na crian\u00e7a",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14989/13886"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1058",
        "title": "An\u00e1lise das concord\u00e2ncias e discord\u00e2ncias entre a s\u00edntese da entrevista e a s\u00edntese final, no processo de orienta\u00e7\u00e3o",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13879",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13879/12779"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1102",
        "title": "The Psychological Development of the Child",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15168",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15168/14054"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1094",
        "title": "Impress\u00f5es do XVII Congresso Internacional de Psicologia e da 71\u00aa Conven\u00e7\u00e3o Anual da Associa\u00e7\u00e3o Americana de Psicologia",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14952/13850"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1111",
        "title": "Perspectivas atuais da Psicologia da Educa\u00e7\u00e3o",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15740/14599"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1101",
        "title": "Realidade do mundo atual e posi\u00e7\u00e3o do jovem",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15142/14027"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1058",
        "title": "Aspectos da readapta\u00e7\u00e3o profissional",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/13877/12777"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1094",
        "title": "A dislexia e o problema da lateralidade",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14949/13847"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1110",
        "title": "Psicologia Educacional",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15704",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15704/14568"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1118",
        "title": "Psicological testing",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16094",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16094/14916"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1092",
        "title": "Estudo comparativo das aspira\u00e7\u00f5es profissionais dos jovens e das profiss\u00f5es aconselhadas pelo Instituto de Sele\u00e7\u00e3o e Orienta\u00e7\u00e3o Profissional",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14835/13735"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1110",
        "title": "Fontes de tendenciosidade cognitiva nas rela\u00e7\u00f5es interpessoais",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15693/14557"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1082",
        "title": "Problemas de linguagem e comunica\u00e7\u00e3o",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14659/13561"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1118",
        "title": "Lideran\u00e7a e din\u00e2mica de grupo",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16085/14907"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1107",
        "title": "An\u00e1lise e contr\u00f4le do teste de Brosson em grupos de adolescente",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15460/14344"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1100",
        "title": "Readings in Child Development and Personality",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15118",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15118/14004"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1099",
        "title": "Cognition: Theory, Research, Promise",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15074",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15074/13965"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1088",
        "title": "Princ\u00edpios psicol\u00f3gicos b\u00e1sicos da linguagem",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/14758/13656"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1099",
        "title": "A car\u00eancia afetiva e sua repercuss\u00e3o na adapta\u00e7\u00e3o escolar",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15066/13958"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1098",
        "title": "A livre atividade art\u00edstica e o desenvolvimento do adolescente",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15057/13949"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1097",
        "title": "ADAPTATION - LEVEL THEORY",
        "resumo": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15000",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/15000/13897"
    },
    {
        "edition": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/issue/view/1115",
        "title": "Esteri\u00f3tipos em rela\u00e7\u00e3o a alunos de Psicologia num campus universit\u00e1rio",
        "pdf": "http://bibliotecadigital.fgv.br/ojs/index.php/abpt/article/view/16011/14836"
    }
]

Python Requests - how use post method?

2 Answers2