Creating a spell check, need to access words.txt in App Engine

Question

I'm working on creating an app that a user can input a paragraph of text and I run spellcheck on the paragraph. I've implemented this easily enough in a local document just using python by opening the file 'words.txt' which contains my dictionary. I'm trying to figure out how to open a word list for implementing my spell check.

My first attempt has been to place it as a static file in my directory and do a fetch on the file, but I keep getting an error.

Here are my files:

app.yaml:

application: *******
version: 1
runtime: python
api_version: 1

handlers:

- url: /(.*\.txt)
  mime_type: text/plain
  static_files: static/\1
  upload: static/(.*\.txt)

- url: /.*
  script: helloworld.py

Excerpts from helloworld.py

import os
from google.appengine.ext.webapp import template
import cgi
import datetime
import urllib
import wsgiref.handlers
from google.appengine.api import urlfetch 

from google.appengine.ext import db
from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

....

result=urlfetch.fetch(os.path.join(os.path.dirname(__file__),'words.txt'))
words=set(result.splitlines())

What is wrong? I keep getting an error.

possible duplicate of [Read a file on App Engine with Python?](http://stackoverflow.com/questions/2630205/read-a-file-on-app-engine-with-python) — Wooble, Feb 10 '12 at 12:43

score 2 · Answer 1 · answered Dec 05 '12 at 15:23

You have a bunch of different file access approaches mixed up here. When you are doing:

result=urlfetch.fetch(os.path.join(os.path.dirname(__file__),'words.txt'))

the path you are creating for 'words.txt' would be the correct path for a file you have uploaded in the same directory as the handler script, if you had not declared it to be part of a static_files, although it appears from your config that you would want the path to be more like:

os.path.join(os.path.dirname(__file__), 'static', 'words.txt')

However, you have two problems here. First, static_files are served from a different process than your handler, and (may) be loaded onto an entirely different server. You will not have filesystem access to them (I presume this is why you are using urlfetch.) The path you are building, however, is for filesystem access, not urlfetch access. To use urlfetch in production, you will need to use a network url, as suggested by @brianmichelich -- however, be aware that this will not work using the development server, which can process only one request at a time.

If you remove the static directives from your app.yaml, however, you should be able to use that path as part of a regular with/open block.

score 2 · Answer 2 · answered Feb 10 '12 at 07:43

2

You can not access files you've defined as static in app.yaml from within your code. Instead either remove the static mapping, or include another copy within your code. Then you can access the file using something such as:

f = open('words.txt')
words = f.read()
f.close()

answered Feb 10 '12 at 07:43

Robert Kluin

8,282
25
21

This doesn't seem to work for me. I currently have 4 files all in the same directory (Project). But when I use these lines I get a page with (I also tried using './words.txt' as in the next comment, but it didn't work either): Traceback (most recent call last): IOError: [Errno 2] No such file or directory: 'words.txt' – Danny Feb 13 '12 at 04:40
@Danny Have you checked what the current path is? Does it match what you expect? If the files are in a subdirectory, are you specifying the path correctly? Are you also sure that the files are not getting caught by a static handler in app.yaml? – Robert Kluin Feb 13 '12 at 07:38

score 0 · Answer 3 · answered Feb 10 '12 at 05:59

0

You can put local files on Google App Engine and open them for read.
If your file is in the top level you need todo something like this:

f = open('./words.txt')

You don't need to fetch the file using urlfetch.

answered Feb 10 '12 at 05:59

Shay Erlichmen

31,691
7
68
87

Got caught up fixing the fetch call. This also answers the question without using urlfetch: http://stackoverflow.com/questions/2630205/read-a-file-on-app-engine-with-python – Brian Michelich Feb 10 '12 at 06:05

score -1 · Answer 4 · answered Feb 10 '12 at 05:53

You should be fetching a fully qualified url, something like http://myapp.appspot.com/words.txt

from google.appengine.api import urlfetch, app_identity

url = 'http://%s/words.txt' % app_identity.get_default_version_hostname()
result=urlfetch.fetch(url)
words=result.content

Creating a spell check, need to access words.txt in App Engine

4 Answers4