10

I am working on general "architecture" related to logging and security of web applications. As far as I know, it is considered the best practice to log all request/response data (not only access logs, but the body of the requests/responses). It's good for security analysis, debugging purposes, audit and many more things.

There is an issue, that sensitive information is transferred in some requests, for example, passwords and/or credit card data.

(Please note: of course, I am using HTTPS, but passwords and/or credit card data will appear as plain text in logging or log files. And by the way, I do not store credit card data, because I am not PCI DSS compliant, we transfer this data to our partner, who is PCI DSS compliant).

Currently, I log and store offsite access logs (so logs without request/response bodies, but with GET parameters data) and I log request/response body data in application code (so I am able do decide what kind of data goes to log, and erase sensitive data before writing it to log).

However, I am thinking to implement the logging (of request/response bodies) outside the application, lets say on the server level via some module (for example, mod_dumpio or something similar), but in this case, logging the sensitive information might be a big issue.

How should I implement/configure it?

Mogsdad
  • 44,709
  • 21
  • 151
  • 275
user3489820
  • 1,459
  • 3
  • 22
  • 38
  • If you could tell us what kind of framework you are using for logging then it will be easier for us. if you are using log4j try reading this http://stackoverflow.com/questions/2461726/how-to-mask-credit-card-numbers-in-log-files-with-log4j. – RamPrakash Nov 18 '15 at 21:53
  • 3
    _"...passwords and/or credit card data will appear as plain text in logging or log files."_ Whatever else you do, DO NOT **EVER** LOG CREDIT CARD INFORMATION OR PASSWORDS!!!!!! – Mogsdad Nov 23 '15 at 21:58

2 Answers2

8

There are several best practices that you should consider when dealing with sensitive data.

First of all, minimize all of the sensitive data that you transfer at all. By hashing the password as early as possible in the architecture, you can ensure that any downstream components never even run the risk of exposing sensitive data.

Second, which is associated to the prior point, you should constrain/isolate the components that ever receive this sensitive information. For a password, only the authentication server would ever need this information - the rest of the components should really only deal with an authorization token (or something of the same ilk.) With regards to a credit card, this should likely only ever be associated with those services that would ever need it - for example the checkout/billing component of your system.

Third, for those systems that actually do need to get the actual value of the sensitive information for logic (e.g. credit card), you should consider creating/using a token-like system. This would basically be a place where you can store the sensitive information and provide an ID reference that replaces the original sensitive data element (e.g. exchange a credit card number for a credit card number id).

To close out the solution, exchange the sensitive data as close to the edge as possible (e.g. where you captured the credit card) in exchange for this token. Exchange this token for the sensitive information value (with the right authorization) to get the value back and then use it at the very latest possible moment. Finally, audit the logging that is done at those sensitive points to make sure that it's not logged in the clear.

This design pattern how some of the biggest e-commerce sites in the world do it. (Sorry, I am not allowed to disclose whom.) Hope it helps!

Shannph Wong
  • 131
  • 1
  • 5
  • Would you suggest encrypting the sensitive data residing in the database? Or are the access controls to the database sufficient? – Vidya Sep 19 '16 at 16:02
  • The old rule of "Encrypting data while in transit AND at rest" still applies, so, yes, you should encrypt the data residing in the database. There are many reasons, but maybe the simplest way is to illustrate it. If you just consider someone backing up the disk and restoring it - with the data unencrypted, that system may be open for exploitation. One last thing, you'll probably want to think about encrypting the data at the field level - not just the entire database. It will give you more flexibility in terms of how to manage that data in the future - for sharding or archiving etc. – Shannph Wong Sep 20 '16 at 17:45
4

You shouldn't store sensitive information such as user passwords or credit card details in log files, doing this with credit card details will mean you fall foul of PCI compliance (as you've noted). For user passwords, there should be no need to log them - you should be checking the user's input password, hashed using whatever method you use, with the password you store in the database.

Without knowing what technology you're using, best practice would suggest filtering out these parameters. For example, the Ruby on Rails framework has (had?) a filter_parameter_logging method which you can pass inputs you want to filter to it - these inputs will then be masked in your log files. It might be worth implementing something similar in your application, so that any field called "password", "card_number" or whatever you define your blacklist to be isn't logged.

James I
  • 198
  • 1
  • 14