Rails: Filter sensitive data in JSON parameter from logs

Question

I am running Rails 3 and trying to filter sensitive information out of our logs which are JSON blobs that are passed as post parameters. For example, user creation might take a post param called user with a string value that is a JSON object. One of the keys in the JSON object is password and we want to filter this out of our logs. The best way I found to do this was to add a block to our filter_params, like so:

keys_to_filter = ['password', 'password_confirmation']
config.filter_parameters << lambda do |k,v|
  if v.is_a? String
    keys_to_filter.each do |key|
      # Match "key":"<filter_out>", or "key":"<filter_out>"}, allowing for whitespace
      v.sub!(/("\s*#{key}\s*")\s*:\s*"[^,\}]*"\s*([,\}])/, "\\1:\"[FILTERED]\"\\2")
    end
  end
end

This adds a block to the filter_params, which causes an error which is described in another question: Rails: ParameterFilter::compiled_filter tries to dup symbol

It appears that it is not safe to pass a block to filter_parameters, so I'm wondering if there is another way to solve this problem.

have you considered encrypting the password, then it would be safe from people simply looking at the logs — Paul Kaplan, Jun 03 '11 at 02:08
@paul-kaplan: How does that make the logs more secure? It would obscure human-readable passwords, true. But, now the server is expecting an encrypted password, but that's just another string. If I wanted to, I could take those encrypted strings from the logs, put them in a post to the server, and log into the account. I agree that it is _slightly_ better, and that the risk of exploitation is small, but it doesn't really solve the problem. — thefugal, Jun 06 '11 at 18:06
I'm having the same problem. Seems the JSON being spit to my log is heavily escaped, probably causing `filter_parameters` to fail in finding `password`. `Parameters: {"{\"user\":{\"first_name\":\"Barry\",\"last_name\":\"Hess\",\"email\":\"barry@example.com\",\"password\":\"notfilterediswear\"}}"}` — Barry Hess, Mar 15 '12 at 15:25
My problem was improper `Content-Type` (or complete lack of it) in API calls meaning Rails didn't format properly to the log. Proper `Content-Type`, proper parameter filtering. — Barry Hess, Mar 15 '12 at 15:46

score 2 · Answer 1 · answered Apr 07 '16 at 17:48

Why is @Nesbitt's answer downvoted (down below)? Sure, it references code in a test suite, but that test suite is just testing a feature documented in ActionDispatch::Http::FilterParameters:

If a block is given, each key and value of the params hash and all subhashes is passed to it, the value or key can be replaced using String#replace or similar method.

See comments in the API docs for Rails 3.x here.

I needed to do the same thing: transform a field in a JSON blob that was sent with an HTTP POST to a Rails 3 app. In my case I just wanted to include a hashed digest of a field, so in config/application.rb:

config.filter_parameters += [:password, lambda {|key, value|
  if key.to_s == 'my_key'
    value.replace(calculate_my_hash(value))
  end
}]

score 2 · Answer 2 · edited Mar 28 '14 at 14:03

2

There is an example of passing a block to filter_parameters in the Rails test suite:

config.filter_parameters += [ :foo, 'bar', lambda { |key, value|
  value = value.reverse if key =~ /baz/
 }]

edited Mar 28 '14 at 14:03

SpaceGhost

359
5
17

answered Jun 03 '11 at 18:22

Andrew Nesbitt

5,976
1
32
36

score 1 · Answer 3 · answered Jul 21 '11 at 09:12

I'm developing a Rails 3 app, and using Backbone.js. I'm passing JSON objects when I create or update a record. I overrode my Backbone model's toJSON function and hardcoded a password param just to test your issue. In my case, config.filter_parameters is just [:password], and it filters the password param correctly in the logs. At first I tested this in update which submits a PUT request. I then tested this in create with a POST request just to check if there's any special bug when submitting a POST. Password is still filtered correctly. Am I missing something?

It looks like config.filter_parameters works correctly without needing to pass a block.

score 0 · Answer 4 · answered Jul 14 '11 at 18:04

0

I have dealt with the same issue in our application. I ended up blocking the entire JSON string from logging where there was secure data, and then adding explicit loggers for any information I wanted logged.

answered Jul 14 '11 at 18:04

Joshua

5,336
1
28
42

Rails: Filter sensitive data in JSON parameter from logs

4 Answers4

Linked