How to keep an audit/history of changes to the table

Question

I've been asked to create a simple DataGrid-style application to edit a single table of a database, and that's easy enough. But part of the request is to create an audit trail of changes made, who made them, and the date/time.

How might you solve this kind of thing?

(I'll be using C# in VS2008, ADO.NET connected to SQL Server 2005, WPF and Xceed's DataGrid, if it makes any difference.)

Answer 1

There are two common ways of creating audit trails.

1. Code your data access layer.
2. In the database itself using triggers.

There are advantages and disadvantages to both. Some people prefer one over the other. It's often down to the type of app and the type of database use you can expect.

If you do it in your DA layer it's pretty much up to you. You just need to add code to every method that saves to the database to also save a log of the changes. This auditing code could be in your DA layer code, or even in your stored procs in your database if you are using stored procs for everything. Essentially the premise is the same, any time you make a change to the database, log that change.

If you want to go down the triggers route, you can write custom triggers for each table, or fashion a more generic trigger that works the same on lots of tables. Check out this article on audit triggers. This works by firing of triggers whenever a change is made, and the triggers log the changes. Remember that if you want to audit SELECT statements, you can't use triggers, you'll have to do that with in code/stored proc auditing. It's also worth remember that depending on your database, triggers may not fire in all circumstances. For example, most databases don't fire triggers during TRUNCATE statements. Check that your triggers get fired in any case that you need auditing.

Alternately, you could also take a look at using the service broker to do async auditing on a dedicated machine. This is more complex and takes a bit of configuring to set up.

Which ever way you do it you need to decide on the format the audit log will take. Normally you would save this log in your database, but you could just save it in a log file or whatever suits your requirements. You could use a single audit table that logs all changes, or you could have an audit table per main table being audited. For large scale implementations you could even consider putting the audit tables in a totally separate database. If your logging into a table, it's common to have a "change type" field which indicates if the audited change was an insert, update or delete style of change, along with the changed data, user who made the change and the date/time the change was made. Don't forget to include the old and new data for update style changes.

Answer 2

Ditto use triggers.

Anyone considering soft deletion should have a read of Richard Dingwall's The trouble with soft delete.

Answer 3

Most universal method would be to create another table for storing versions of record from the first table. Then, you can remove all the data from main table. Suppose you need versioning of a table Person(PersonId, Name, Surname):

CREATE TABLE Person 
(
   PersonId INT,                   // PK
   CurrentPersonVersion INT        // FK
);

CREATE TABLE PersonVersion
(
  PersonVersionId INT,             // PK
  PersonID                         // FK 
  Name VARCHAR,                    // actual data
  Surname VARCHAR,                 // actual data

  ChangeDate                       // logging data
  ChangeAuthor                     // logging data
)

Now any change requires inserting new PersonVersion and updating the CurrentPersonVersionID.

Answer 4

The best way to do this is set up triggers in the database that write to audit tables.

Answer 5

Solution 1: SQL Server Change Data Capture

https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/enable-and-disable-change-data-capture-sql-server?view=sql-server-2017

First you need to enable change data capture on your database

USE AdventureWorks2012
GO  
EXEC sys.sp_cdc_enable_db  
GO  

Then you can query the changes using fn_cdc_get_all_changes_ or fn_cdc_get_net_changes_.

-- ========  
-- Enumerate All Changes for Valid Range Template
-- ========  
USE AdventureWorks2012;  
GO  

DECLARE @from_lsn binary(10), @to_lsn binary(10);  
SET @from_lsn = sys.fn_cdc_get_min_lsn('HR_Department');  
SET @to_lsn   = sys.fn_cdc_get_max_lsn();  

SELECT * FROM cdc.fn_cdc_get_all_changes_HR_Department  
(@from_lsn, @to_lsn, N'all');  

---

Solution 2: SQL Server Database Auditing

Source : https://www.dbaservices.com.au/how-to-configure-sql-server-auditing/

ENABLE DATABASE AUDITING

Database auditing requires that a server audit (although not necessarily server audit specification) to be in place. The DB auditing however is created within the user database that is to be audited, rather than within the master database where the server audit gets created. Database audit specifications can be found within the DB itself under Security –> Database Audit Specifications.

To create a database audit, you’ll need to first USE the database (to select it), then the following provides an example syntax for auditing SELECT, UPDATE and DELETE operations for specific tables within that database;

USE UserDatabase
GO


CREATE DATABASE AUDIT SPECIFICATION [User_Database_Audit_Specification]  
FOR SERVER AUDIT [SQL_Server_Audit]
     ADD (SELECT , UPDATE , DELETE ON UserDatabase.dbo.Customer_DeliveryAddress BY dbo )
    ,ADD (SELECT , UPDATE , DELETE ON UserDatabase.dbo.DimCustomer_Email BY dbo )
    ,ADD (SELECT , UPDATE , DELETE ON UserDatabase.dbo.DimCustomer_Phone BY dbo )
WITH (STATE = ON) ;   
GO

The SELECT, UPDATE and DELETE operations aren’t the only things you can add to the audit specification though…

+------------+-------------------------------------------------------------------+
| Action     | Description                                                       |
+------------+-------------------------------------------------------------------+
| SELECT     | This event is raised whenever a SELECT is issued.                 |
| UPDATE     | This event is raised whenever an UPDATE is issued.                | 
| INSERT     | This event is raised whenever an INSERT is issued.                | 
| DELETE     | This event is raised whenever a DELETE is issued.                 | 
| EXECUTE    | This event is raised whenever an EXECUTE is issued.               | 
| RECEIVE    | This event is raised whenever a RECEIVE is issued.                | 
| REFERENCES | This event is raised whenever a REFERENCES permission is checked. | 
+------------+-------------------------------------------------------------------+

The full list of database events you can log is available here:

https://learn.microsoft.com/en-us/sql/relational-databases/event-classes/security-audit-event-category-sql-server-profiler?view=sql-server-2017

Answer 6

I was recently faced with a requirement to audit some tables and I opted to use triggers. Like others, I only wanted to see entries in the audit table for those fields that had actually changed, however, when updating the tables, the application was updating all the fields in row whether they'd changed or not, therefore, checking whether the fields had been updated or not availed me nothing - they all had!

What I wanted, therefore, was a method of checking the actual value in each field to see if it had changed or not and only writing it to the audit table if it had. Having been unable to find any solution to this conundrum anywhere, I came up with my own, as follows:

CREATE TRIGGER [dbo].[MyTable_CREATE_AUDIT]
ON [dbo].[MyTable]
AFTER UPDATE

AS

INSERT INTO MyTable_Audit 
(ItemID,LastModifiedBy,LastModifiedDate,field1,field2,field3,
field4,field5,AuditDate)
SELECT i.ItemID,i.LastModifiedBy,i.LastModifiedDate,

field1 = 
  CASE i.field1
    WHEN d.field1 THEN NULL
    ELSE i.field1
  END,

field2 = 
  CASE i.field2
    WHEN d.field2 THEN NULL
    ELSE i.field2
  END,

field3 = 
  CASE i.field3
    WHEN d.field3 THEN NULL
    ELSE i.field3
  END,

field4 = 
  CASE i.field4
    WHEN d.field4 THEN NULL
    ELSE i.field4
  END,  

field5 = 
  CASE i.field5
    WHEN d.field5 THEN NULL
    ELSE i.field5
  END,

GETDATE()

FROM inserted i
INNER JOIN deleted d
ON i.ItemID = d.ItemID

As you can see, I'm comparing the values of each field in the deleted and inserted tables and only writing the field value from the inserted table to the audit table if they differ, otherwise I just write NULL.

It certainly works for me. Can anyone see any issues with this approach? My team own both the application and the database so possible curved balls like schema changes are covered off.

Answer 7

The other way of doing this apart from triggers is this,

1. Have four columns, UpdFlag, DelFlag, EffectiveDate and TerminatedDate for each table you want to do an audit trail on.
2. code your sproc's in such a way that when you do an update, to pass in the all of the row's column data into the sproc, update the row by setting the TerminatedDate to the date that was updated, and mark the UpdFlag and to put in the datetime into the column
3. Then create a new row with the new data (which is really updated). and put in a new date now for the EffectiveDate and the TerminatedDate set to the max date.

Likewise if you want to do a deletion of the row, simply update the row by marking the DelFlag as set, the TerminatedDate with the datetime now. You are in effect doing a soft delete and not an actual sql's Delete.

In that way, when you want to audit the data, and to show a trail of the changes, you can simply filter the rows for those that have the UpdFlag set, or between EffectiveDate and TerminatedDate. Likewise for those that were deleted, you filter for those that have the DelFlag set or between EffectiveDate and TerminatedDate. For the current rows, filter the rows that have both flags set off. The advantage is you don't have to create another table for the audit when the trigger is used!

Answer 8

I'd go triggers route, by creating table with similar structure to updated one, with additional columns for tracking changes like ModifiedAt etc. And then adding on update trigger that will insert changes to that table. I find it easier to maintain than have everything in the application code. Ofcourse many people tend to forget about triggers when it comes to questions like 'wtf this table is changing' ;) Cheers.