1

This is somewhat similar to 403 Forbidden Error, probably ModSecurity

I have a form on a webpage, this is for submitting maths equations and content. When used legitimatly the fields will contain the unsafe characters ", <, >, &, ' and ` I was escaping these with

 &#x22;, &#x3C;, &#x3E;, &#x26;, &#x27; and &#x60;

I was encoding/escaping the unsafe chars with the above HTML entities before submitting the form from the web page to the PHP script. So even though the fields legitimally contained the unsafe characters the submitted form did not. In my PHP script I first checked the $_POST['formFieldData']; and if it contained any of the unsafe characters then I immediatly and silently aborted processing for that entire post. My thinking was that a legitimate use of the form on my webpage would not contain any of unsafe characters. So if anyone set up their own page with a form aimed at my PHP script they would not be able to send it unsafe characters. I only ever write the data to the browser screen with the unsafe characters HTML entity encoded.

Originally (a few years ago) when I set the form up the raw data (legitimatly containing unsafe characters) was posted to the PHP script and what was echoed back to the browser after encoding the unsafe characters, then the data was emailed to me for further processing and storage. I was recieving a substantial number of scamming/phishing/junk emails that originated from a robot or some other form. To cut down on the volume of these and still allow legitimate ones through I implemented the encoding on my webpage and rejected incomming posts containing unsafe characters. Over night the volume of these went to zero. I must add that I also considered the . : ; / \ @ unsafe and encoded/escaped them and dropped the post if it contained them in unencoded form.

Recently even with the above cleaning/escaping I started recieving false positive mod-security trips for XSS filtering that blocked the submissions from reaching the php file for processing. The message was "Potential Cross Site Scripting Attack'"

Unfortunatly the hosting company do not have these trips logged in the users log file, all the told me was that there are trips and they sent me the rule

I have pasted the rule at the end of this post as there is a huge amount of text in the rule.

The hosting company had two suggestions: White list the rule so that it does not apply to my site. Turning mod-security off. I would rather not do one of these if possible.

Towards the end of the rule is the code t:htmlEntityDecode, This gave me the idea that the rule is unescaping the content therefore defeating my html escaping.

I set about jumbling up the content that I had escaped using the Javascript replace .replace(/&#x/g, "x#&"); for example the &#x22; becomes x#&22; on the server in the PHP form processing file then I undid this jumbling up with $form_Data = str_replace("x#&", "&#x", $form_Data);

This worked perfectly, that is the mod-security filter did not trip for the content that previously caused false positives.

This is a hack but I don't think I have any choice but to do some kind of hack.

The rule as detailed below looks like one giant regular expression plus some directives to unesape HTMLentities, compressWhitespace etc.

I would be most obliged if someone that knows about these filters could offer some suggestions on any methods that I can use to preprocess the form content before posting it from the webpage to make it more resilient to false positives.

Thanks in advance.

This rule is:

Rule 340147: Generic XSS filter
SecRule ARGS|ARGS_NAMES|!ARGS:js_includes|!ARGS:/m1_source/|!ARGS:/geodir/|!ARGS:/banner_block/
|!ARGS:/introcopy/|!ARGS:eingabe|!ARGS:ausgabe|!ARGS:/previewdata/|!ARGS:/tracking_extra/
|!ARGS:/^groups/|!ARGS:video|!ARGS:/google_map/|!ARGS:/field_map/|!ARGS:/gacode/|!ARGS:code1
|!ARGS:ga_code|!ARGS:customized|!ARGS:code_analytics|!ARGS:uvod|!ARGS:/^field_video/|!ARGS:q
|!ARGS:/^textarea-video/|!ARGS:leirro|!ARGS:lomake|!ARGS:vastaus|!ARGS:/^texte$/|!ARGS:vraag
|!ARGS:qti_data|!ARGS:tracklist|!ARGS:i_google|!ARGS:code_area_text|!ARGS:/log_code/
|!ARGS:/^ADVERT_/|!ARGS:UserData|!ARGS:areas|!ARGS:templatecode|!ARGS:/prevObject/
|!ARGS:/replaceAll/|!ARGS:/insertBefore/|!ARGS:/insertAfter/|!ARGS:/prependTo/|!ARGS:/appendTo/
|!ARGS:/mapcode/|!ARGS:googleCode|!ARGS:/sidebar/|!ARGS:/ad_code/|!ARGS:/^recipient/
|!ARGS:optional_head|!ARGS:/^form/|!ARGS:/^var_value/|!ARGS:variable_data|!ARGS:/^instance/
|!ARGS:/customfield/|!ARGS:notice|!ARGS:/formcode/|!ARGS:/ajax/|!ARGS:all|!ARGS:allowedTags
|!ARGS:/google_analytics/|!ARGS:/widget/|!ARGS:ad_code|!ARGS:/keycaptcha_code/|!ARGS:/jscode/
|!ARGS:postcontents|!ARGS:/adsense/|!ARGS:video1|!ARGS:/updateAds/|!ARGS:map|!ARGS:gmapcode
|!ARGS:/^Sidebar/|!ARGS:/^wpTextbox/|!ARGS:paragrafo|!ARGS:/question/|!ARGS:/style/
|!ARGS:tracking_code|!ARGS:whats-new|!ARGS:analyticscode|!ARGS:top_news|!ARGS:data[config]
|!ARGS:fulltext|!ARGS:introtext|!ARGS:offertext|!ARGS:block|!ARGS:livezillacode|!ARGS:/embed/
|!ARGS:/desc/|!ARGS:/script/|!ARGS:/^p_process_chats/|!ARGS:obj_itop|!ARGS:/cms/
|!ARGS:eventDescription|!ARGS:/^product/|!ARGS:match_report|!ARGS:eip_value|!ARGS:/^usergroup/
|!ARGS:sendDescription|!ARGS:email_id|!ARGS:obj_itop|!ARGS:sml_prt_1|!ARGS:pay_inst_1
|!ARGS:/^jform/|!ARGS:phpcode|!ARGS:intro|!ARGS:Snippet|!ARGS:oid|!ARGS:Submit2|!ARGS:/^obj_/
|!ARGS:layout|!ARGS:pageset|!ARGS:contact_form_information|!ARGS:/^site_/|!ARGS:/^translations/
|!ARGS:create_tables|!ARGS:insertfile|!ARGS:video_credits|!ARGS:input[Desarrollo]|!ARGS:move2
|!ARGS:hoperation|!ARGS:login_form|!ARGS:/product_benefits/|!ARGS:/custom_code/|!ARGS:arg2
|!ARGS:resumoDetalhe|!ARGS:bbcode_tpl|!ARGS:Right_photo_1|!ARGS:embedVideo|!ARGS:/^K2ExtraField/
|!ARGS:mentorhelp|!ARGS:/submitcode/|!ARGS:beschrijving|!ARGS:custombannercode|!ARGS:bannercode
|!ARGS:privatecapacity|!ARGS:diz|!ARGS:FormLayout|!ARGS:/^fck/|!ARGS:parent_name
|!ARGS:/^code_tscript/|!ARGS:_qf_Group_next|!ARGS:project_company|!ARGS:categories_title
|!ARGS:antwoord|!ARGS:project_company|!ARGS:signature|!ARGS:paepdc|!ARGS:tpl_source
|!ARGS:teaser_js|!ARGS:/^autoDS/|!ARGS:FrmSide|!ARGS:mainKeywords|!ARGS:/VB_announce/
|!ARGS:guardar|!ARGS:/serendipity/|!ARGS:omschrijving|!ARGS:resolution|!ARGS:newyddionc
|!ARGS:bericht|!ARGS:property_copy|!ARGS:/^outpay/|!ARGS:bedrijfsprofiel|!ARGS:s_query
|!ARGS:finish_survey|!ARGS:photolater|!ARGS:ticket_response|!ARGS:/element/
|!ARGS:option[vbpclosedreason]|!ARGS:/introduction/|!ARGS:/contenido/|!ARGS:/sql/
|!ARGS:prefix|!ARGS:query|!ARGS:c_features|!ARGS:/tekst/|!ARGS:embeddump|!ARGS:other_clubs
|!ARGS:/^elm/|!ARGS:/^saes/|!ARGS:dlv_instructions|!ARGS:/^cymr/|!ARGS:_qf_Register_upload
|!ARGS:/^elm/|!ARGS:verbiage|!ARGS:news|!ARGS:/^wz/|!ARGS:tiny_vals|!ARGS:sSave|!ARGS:/article/
|!ARGS:/about/|!ARGS:/Summarize/|!ARGS:/^product_options/|!ARGS:/SiteStructure/|!ARGS:/anmerkung/
|!ARGS:/summary/|!ARGS:/edit/|!ARGS:reply|!ARGS:/story/|!ARGS:resource_box|!ARGS:navig
|!ARGS:preview__hidden|!ARGS:/page/|!ARGS:order|!ARGS:/post/|!ARGS:youtube|!ARGS:reply
|!ARGS:business|!ARGS:/homePage/|!ARGS:pagimenu_inhoud|!ARGS:/note/|!ARGS:Post|!ARGS:/^field_id/
|!ARGS:area|!ARGS:/detail/|!ARGS:/comment/|!ARGS:LongDesc|!ARGS:ta|!ARGS:Returnid|!ARGS:busymess
|!ARGS_NAMES:/^V\*/|!ARGS_NAMES:/^S\*/|!ARGS:/^quickrise_advertise/|!ARGS:rt_xformat
|!ARGS:/wysiwyg/|!ARGS:contingut|!ARGS:/^werg/|!ARGS:/body/|!ARGS:/css/|!ARGS:/^section/
|!ARGS:/msg/|!ARGS:t_cont|!ARGS:/^doc/|!ARGS:/xml/|!ARGS:tekst|!ARGS:formsubmit
|!ARGS:invoice_snapshot|!ARGS:submit|!ARGS:/html/|!ARGS:/content/|!ARGS:/footer/|!ARGS:/header/
|!ARGS:/footer/|!ARGS:/link/|!ARGS:text|!ARGS:txt|!ARGS:/refer/|!ARGS:/referrer/|!ARGS:/template/
|!ARGS:/ajax/ "(?:< ?script|(?:<|< ?/)(?:(?:java|vb)script|about|applet|activex|chrome)|
< ?/?i?frame|\%env)" \Potential Cross Site Scripting Attack'
"phase:2,deny,status:406,t:none,t:removeNulls,t:utf8toUnicode,t:urlDecodeUni,t:htmlEntityDecode,
t:jsdecode,t:cssdecode,t:replaceComments,t:compressWhitespace,t:lowercase,capture,id:340147,
rev:137,severity:2,msg:'Atomicorp.com WAF Rules: ,logdata:'%{TX.0}'"
Community
  • 1
  • 1
user1558796
  • 293
  • 4
  • 19
  • A web application firewall does not make proper contextual data handling obsolete. However, proper contextual data handling can make a web application firewall obsolete. – Gumbo Dec 31 '14 at 16:19
  • Thank you Gumbo. I do not know what you mean by "proper contextual data handling". I would be obliged if you would elaborate. – user1558796 Jan 02 '15 at 10:36
  • You cannot treat all data equally because whether some data poses a security threat depends on how it’s processed. Take StackOverflow as an example: you can write a question/answer/comment containing `` and this is not a problem because the submitted text is properly processed according to its use (i. e., printed into an HTML document while special characters are getting properly encoded). However, a generic web application firewall would reject the HTTP request as it contains a ``. – Gumbo Jan 02 '15 at 10:43
  • Gumbo, I now know what you mean, I, in my opinion was handling the data knowing that legitimate use contains unsafe characters but in encoded form and illigemate use may contain unsafe characters in unencoded form. I have edited my original post to provide info that should have been provided in the first place. I appreciate your valuable comments. – user1558796 Jan 02 '15 at 11:38
  • My point it: web application firewalls don’t know how the data is processed and thus treat all data the same – with the result of many false positives. – Gumbo Jan 02 '15 at 11:57
  • I understand you now Gumbo – user1558796 Jan 02 '15 at 12:03

1 Answers1

2

I would be most obliged if someone that knows about these filters could offer some suggestions on any methods that I can use to preprocess the form content before posting it from the webpage to make it more resilient to false positives.

base64?

When used legitimatly the fields will contain the unsafe characters ", <, >, &, ' and `

Then mod_security isn't a good choice of tool for you. (Personally my view is that input filtering rules like these are rarely a good idea for anyone.)

This is a hack but I don't think I have any choice but to do some kind of hack.

Is someone imposing mod_security on you? By tunnelling through it you are making it ineffective in any case, so it is (even more than normal) a hopeless waste of time.

bobince
  • 528,062
  • 107
  • 651
  • 834
  • Bobince, I have today tried base64 and it seems that will be perfect for the job if I continue to try to hack my way through the mod_security. I have edited my post to provide a bit more info as to why I was encoding at source. – user1558796 Jan 02 '15 at 11:41
  • Bobince, mod_security is not "imposed" on me as such. They silently implemented it and do not have a mechanism in place to inform users about mod_security rules being triggered "as to do so would likely be more hindrance than help" they do not even log triggers. I only found out by accident that legitimate submissions were intercepted and abandoned by the hosting company without informing their customer. – user1558796 Jan 02 '15 at 11:53
  • Bobince, My choice is to either turn off mod_security or leave it on with preprocessing (encoding at the form side) so a legimate submission that contained unsafe chars in the fields is not blocked but any submission with unsafe characters in the posted data is blocked by mod_security. Based on your input and the fact that the hosting company do not provide feedback to me I think it best that I turn off mod_security. They do log these triggers in their own logs but not in the logs that a customer sees. – user1558796 Jan 02 '15 at 11:58
  • There is no such thing as an inherently “unsafe” character. You need to make sure you are correctly handling all characters in your application code, and you need to do that whether or not you use mod_security, because (a) WAFs like mod_security are not watertight and (b) you are at present deliberately providing an unprotected tunnel through the WAF, ensuring the result is no more or less secure than when you started. Handling characters correctly means, for one, escaping them appropriately for any context you inject them into, for example replacing `<` with `<` when writing to HTML. – bobince Jan 02 '15 at 16:50
  • Stopping form spam is another question really unrelated to input filtering and output escaping. A common approach is to implement a rate-limiting CAPTCHA, but this has obvious UX disadvantages. A range of other techniques aim to stop the stupider bots by making forms behave in unusual ways they are not expecting from standard scripts (eg honeypot fields, JavaScript validation, time-limited CSRF tokens). See eg http://stackoverflow.com/questions/2387496/how-to-prevent-robots-from-automatically-filling-up-a-form – bobince Jan 02 '15 at 16:56
  • Thank you bobince. Apoligies for taking so long to accept your answer – user1558796 Jan 16 '15 at 16:22