4

I've created a huge Rails app 2 years ago which is host on Heroku Cedar stack:

ruby '1.9.3'
gem 'rails', '3.1.10'

I'm using Airbrake to catch exceptions. Here's the list of unresolved errors created in the past 11 days:

5 occurences, last about 12 hours ago
ArgumentError: invalid %-encoding (~”¼%“„èš�šE�BW€ôNÎÙ®9öÈS��e:G{`ÂÖú‘ÉÆql½‘‡<‘†I«ó� l)
training#show

9 occurences, last about 14 hours ago
ArgumentError: invalid byte sequence in UTF-8
intra#receive_proposal

1 occurences, last about 21 hours ago
ArgumentError: invalid byte sequence in UTF-8
why_choose_us#page

6 occurences, last 2 days ago
ArgumentError: invalid %-encoding (%E4%B8%AA%E4%BA%BA%E4%B8%BB%E9%A1%)
intra#receive_proposal

1 occurences, last 4 days ago
ArgumentError: invalid %-encoding (�    *†H†÷ �����0L1 0���U����GlobalSign Root CA - R21�0���U� � GlobalSign1�0���U��� GlobalSig... - R21�0���U� � GlobalSign1�0���U��� GlobalSign0‚�"0 �   *†H†÷ ������‚���0‚� �‚���¦Ï$�¾.o(™EBÄ«>!T›�Ó„pú�³Ë¿‡_ƆӲ0\Öý­ñ{Üåø`
info#contact

3 occurences, last 5 days ago
ArgumentError: invalid byte sequence in UTF-8
training#show

1 occurences, last 5 days ago
ArgumentError: invalid %-encoding (�/�‚Q#ËPýv‚Ûs�7B�Œª²�βšî¿ƒô<W)‹ÈØ}1Û�’‚çøªXx•o΄®J�÷�*Rå<2ˆë<®¿V1žÈ}м¯‹¾üh?j¯J_–ŸWÁj >»e...ƒ«}ßö:öK�24½åóìmÛaM¼6oQZìÛaà$U³u�³Xó@ç�ék�{€aZÿú6VìÀ‰cN á��·÷Jûå¬Þ˜�9 2˜Ž¥¨õÁ�$Ï.6�[œÄêB²,B�átT r�%�¸�������������������
training#domain

1 occurences, last 5 days ago
ArgumentError: invalid byte sequence in UTF-8
training#problem

1 occurences, last 6 days ago
ArgumentError: invalid byte sequence in UTF-8
info#user_sitemap

1 occurences, last 6 days ago
ArgumentError: invalid byte sequence in UTF-8
content#show

1 occurences, last 7 days ago
ArgumentError: invalid byte sequence in UTF-8
info#contact

1 occurences, last 8 days ago
ArgumentError: invalid byte sequence in UTF-8
intra#intra_curricula

3 occurences, last 8 days ago
ArgumentError: invalid %-encoding (09��U���2www.entrust.net/cps incorp. by ref. (limits liab.)1%0#��U����(c) 1999 entrust.net limited1:08��U���1entrust.net secure server certification authoritycom- g3.com)
video#show

1 occurences, last 9 days ago
ArgumentError: invalid %-encoding (�    *†H†÷ �����0W1�0 ��U����BE1�0���U� ��GlobalSign nv-sa1�0���U����Root CA1�0���U����Global...�U����BE1�0���U� ��GlobalSign nv-sa1�0���U����Root CA1�0���U����GlobalSign Root CA0‚�"0 �    *†H†÷ ������‚���0‚� �‚���Ú�æ™Î£ãOŠ
training#problem

1 occurences, last 9 days ago
ArgumentError: invalid byte sequence in UTF-8
intra#corporate_training

1 occurences, last 10 days ago
ArgumentError: invalid byte sequence in UTF-8
home#routing_error

1 occurences, last Jul 01 2014, 07:14:23 PM
ArgumentError: invalid byte sequence in UTF-8
training#town

As you can see, something is very wrong. Remote addresses are wall over the world (US, china, Italy, ...)

Here's the backtrace of the last error:

/vendor/ruby-1.9.3/lib/ruby/1.9.1/uri/common.rb:898 in "decode_www_form_component"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:40 in "unescape"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:98 in "block (2 levels) in parse_nested_query"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:98 in "map"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:98 in "block in parse_nested_query"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:97 in "each"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/utils.rb:97 in "parse_nested_query"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/request.rb:302 in "parse_query"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/http/request.rb:289 in "parse_query"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/request.rb:190 in "POST"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/http/request.rb:251 in "POST"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/http/parameters.rb:10 in "parameters"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/http/filter_parameters.rb:33 in "filtered_parameters"
/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.5.0/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:33 in "process_action"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/abstract_controller/base.rb:121 in "process"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/abstract_controller/rendering.rb:45 in "process"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_controller/metal.rb:193 in "dispatch"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_controller/metal/rack_delegation.rb:14 in "dispatch"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_controller/metal.rb:236 in "block in action"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/routing/route_set.rb:71 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/routing/route_set.rb:71 in "dispatch"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/routing/route_set.rb:35 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb:152 in "block in call"
/vendor/bundle/ruby/1.9.1/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:96 in "block in recognize"
/vendor/bundle/ruby/1.9.1/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:75 in "optimized_each"
/vendor/bundle/ruby/1.9.1/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:95 in "recognize"
/vendor/bundle/ruby/1.9.1/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb:141 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/routing/route_set.rb:538 in "call"
/vendor/bundle/ruby/1.9.1/gems/sass-3.2.5/lib/sass/plugin/rack.rb:54 in "call"
/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.5.0/lib/new_relic/rack/browser_monitoring.rb:12 in "call"
/vendor/bundle/ruby/1.9.1/gems/pdfkit-0.5.2/lib/pdfkit/middleware.rb:16 in "call"
/vendor/bundle/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:35 in "block in call"
/vendor/bundle/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:34 in "catch"
/vendor/bundle/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:34 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/best_standards_support.rb:17 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/etag.rb:23 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/conditionalget.rb:25 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/head.rb:14 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/params_parser.rb:21 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/flash.rb:243 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/session/abstract/id.rb:195 in "context"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/session/abstract/id.rb:190 in "call"
/lib/middleware/flash_session_cookie_middleware.rb:16 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/cookies.rb:331 in "call"
/vendor/bundle/ruby/1.9.1/gems/activerecord-3.1.10/lib/active_record/query_cache.rb:64 in "call"
/vendor/bundle/ruby/1.9.1/gems/activerecord-3.1.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:477 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/callbacks.rb:29 in "block in call"
/vendor/bundle/ruby/1.9.1/gems/activesupport-3.1.10/lib/active_support/callbacks.rb:392 in "_run_call_callbacks"
/vendor/bundle/ruby/1.9.1/gems/activesupport-3.1.10/lib/active_support/callbacks.rb:81 in "run_callbacks"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/callbacks.rb:28 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/sendfile.rb:101 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/remote_ip.rb:48 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/show_exceptions.rb:47 in "call"
/vendor/bundle/ruby/1.9.1/gems/railties-3.1.10/lib/rails/rack/logger.rb:13 in "call"
/vendor/bundle/ruby/1.9.1/gems/quiet_assets-1.0.1/lib/quiet_assets.rb:20 in "call_with_quiet_assets"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/methodoverride.rb:24 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/runtime.rb:17 in "call"
/vendor/bundle/ruby/1.9.1/gems/activesupport-3.1.10/lib/active_support/cache/strategy/local_cache.rb:72 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/lock.rb:15 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-rewrite-1.2.1/lib/rack/rewrite.rb:20 in "call"
/vendor/bundle/ruby/1.9.1/gems/actionpack-3.1.10/lib/action_dispatch/middleware/static.rb:61 in "call"
/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:136 in "forward"
/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:245 in "fetch"
/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:185 in "lookup"
/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:66 in "call!"
/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:51 in "call"
/vendor/bundle/ruby/1.9.1/gems/railties-3.1.10/lib/rails/engine.rb:456 in "call"
/vendor/bundle/ruby/1.9.1/gems/railties-3.1.10/lib/rails/application.rb:143 in "call"
/vendor/bundle/ruby/1.9.1/gems/railties-3.1.10/lib/rails/railtie/configurable.rb:30 in "method_missing"
/vendor/bundle/ruby/1.9.1/gems/rack-1.3.10/lib/rack/deflater.rb:13 in "call"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/connection.rb:80 in "block in pre_process"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/connection.rb:78 in "catch"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/connection.rb:78 in "pre_process"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/connection.rb:53 in "process"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/connection.rb:38 in "receive_data"
/vendor/bundle/ruby/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:256 in "run_machine"
/vendor/bundle/ruby/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:256 in "run"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/backends/base.rb:61 in "start"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/server.rb:159 in "start"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/controllers/controller.rb:86 in "start"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/runner.rb:185 in "run_command"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/lib/thin/runner.rb:151 in "run!"
/vendor/bundle/ruby/1.9.1/gems/thin-1.3.1/bin/thin:6 in "<top (required)>"
/vendor/bundle/ruby/1.9.1/bin/thin:23 in "load"
/vendor/bundle/ruby/1.9.1/bin/thin:23 in "<main>"

Can you help me to get rid of this?

jww
  • 97,681
  • 90
  • 411
  • 885
fro_oo
  • 1,610
  • 4
  • 24
  • 46
  • "invalid %-encoding", "invalid byte sequence", "Remote addresses are wall over the world..." - are you certain you are not under attack? – jww Jul 12 '14 at 11:14
  • I think it is the case. I don't know how to secure that old Rails app. – fro_oo Jul 12 '14 at 22:37
  • Try http://stackoverflow.com/questions/3916931/rails-3-invalid-multibyte-char-us-ascii/9528543#9528543 – RAJ Jul 21 '14 at 07:08

1 Answers1

3

I've also been getting these recently on a site of mine. From the logs, it comes up when easou.com search spider hits the website recently.

For example, I get this exception:

An ArgumentError occurred in blog#index:

  invalid byte sequence in UTF-8

My exception_notification logs don't show anything particularly obvious, which I assumed probably was from a query string added to the end of the URL from the spider.

According to this issue in Rack, it's actually a bug inside Rails. My site is running Rails 4, but I'll guess it's the same sort of bug in older Rails versions as well.

If you prepend his invalid UTF query param to the end of a url, you should be able to replicate the error. For example:

https://gorails.com/blog?%28t%B3odei%29

A solution for this is to use the utf8 sanitizer gem which you can find here: https://github.com/whitequark/rack-utf8_sanitizer/

  1. Add this to your Gemfile:

    gem 'rack-utf8_sanitizer', '~> 1.2.2'

  2. Run bundle

  3. And then adding this to your config/application.rb:

    config.middleware.insert 0, Rack::UTF8Sanitizer

  4. Commit, redeploy, and all should be swell. You can hit the same url that was breaking before with the query param and Rails should handle it alright this time.

excid3
  • 1,658
  • 15
  • 31
  • 1
    After a day, I noticed I don't get the invalid byte sequence, but I sitll get `invalid %-encoding` errors. So this doesn't completely fix it yet. – excid3 Jul 20 '14 at 02:37