Spynner: get html of second page after submitting form

Question

I have just started using Spynner to scrape webpages and am not finding any good tutorials out there. I have here a simple example where I type a word into Google and then I want to see the resulting page.

But how do I go from clicking the button to actually getting the new page?

import spynner

def content_ready(browser):
    if 'gbqfba' in browser.html:
        return True #id of search button

b = spynner.Browser()
b.show()
b.load("http://www.google.com", wait_callback=content_ready)
b.wk_fill('input[name=q]', 'soup')
# b.browse() # Shows the word soup in the input box
with open("test.html", "w") as hf: # writes the initial page to a file
    hf.write(b.html.encode("utf-8"))
b.wk_click("#gbqfba") # Clicks the google search button (or so I think)

But now what? I'm not even sure that I have clicked the google search button, although it does have id=gbqfba. I have also tried just b.click("#gbqfba"). How do I get the search results?

I have tried just doing:

 with open("test.html", "w") as hf: # writes the initial page to a file
    hf.write(b.html.encode("utf-8"))

but that still prints the initial page.

Answer 1

i solved this by sending Enter to the input and waiting two seconds. not ideal, but it works

import spynner
import codecs
from PyQt4.QtCore import Qt

b = spynner.Browser()
b.show()
b.load("http://www.google.com")
b.wk_fill('input[name=q]', 'soup')
# b.browse() # Shows the word soup in the input box

b.sendKeys("input[name=q]",[Qt.Key_Enter])
b.wait(2)
codecs.open("out.html","w","utf-8").write(b.html)

Answer 2

The recommended method is to wait for the new page to load:

b.wait_load()