The problem might not be "real". You have a submission which is missing city_src even if all forms are fine, but there are some other ways in which this can happen:
- the form is filled via Javascript and some customer had it disabled. The form was submitted without some fields.
- the connection hiccuped and the packet did not get through completely. I know, I know, TCP ought to prevent this. It normally does. I remember seeing a couple of SO questions where somehow (thanks to proxies, browser extensions and undoubtedly gremlins too) it did not.
- the form is submitted via GET (maybe through AJAX), which has a size limitation. Unlikely, yet possible.
- The most likely: there are jillions of bots out there that will try and force any form they happen upon, to see whether anything interesting drops out - a XSS opening, some spam, the possibility of sending spam, possibly some symptom of SQL injection vulnerability. Whatever. The result is that form endopoints are wont to gather all forms of rubbish, and the logs too.
Detail of the attack scenario:
Joe Q. Bot arrives on a given page containing a form. The form is complete and has all fields, of course. And in the normal course of events, all fields would be submitted to the server. A browser would do this.
But the bot is not a browser; using whatever logic its creator employed, it may try and submit the form as is, or can try and fiddle with it in order to obtain information or cause errors.
For example, it's not unheard of for an incomplete form used to build an INSERT into a database to throw an error and leak information about said database (e.g. the address of the database server, which could be, say, on Amazon Elastic and be insufficiently guarded - who's going to guess our IP?).
Then again, since the bot is usually on the crappy side of software engineering, it might fail to cope with more than "enough" fields, so that only the first half of a long form is submitted. Or it may fail to parse properly the form itself due to a number of reasons.
The result is that a field that should be there is actually not there, and your code chokes when analyzing the request.
A simple workaround is, as soon as you're sure that there's no normal and honest way of getting a broken form, wrap the whole procedure in a try/catch block and send any exception privately to your inbox.
===
In this specific case I'd try and check the web server log in order to see what calls were made (e.g. GET /site/customers/app/whatever) just before the error popped up. Then you can try and replay those calls and see whether the error pops up again. You'll then have a clearer idea about the route the call has taken.
You might also want to supply incomplete forms yourself to see whether those elicit the same error, and what can be done to route the response back to something meaningful ("There has been a submission error, please retry" or blocking an incoming bot by sending its IP address to, say, an iptables script or a firewall utility webservice on the DMZ side)
