4

I'm trying to implement a system-wide logging, which would log all failed Stored Procedure executions in our dabatase and I'm looking at Extended Events.

I've done some research and it seems quite easy to capture failed statements using following code:

--Create an extended event session
CREATE EVENT SESSION what_queries_are_failing ON SERVER
ADD EVENT sqlserver.error_reported (
    ACTION (sqlserver.sql_text
        , sqlserver.tsql_stack
        , sqlserver.database_id
        , sqlserver.username
        )
    WHERE ([severity] > 10)
    )
ADD TARGET package0.asynchronous_file_target (
    SET filename = 'C:\XEventSessions\what_queries_are_failing.xel'
    , metadatafile = 'C:\XEventSessions\what_queries_are_failing.xem'
    , max_file_size = 5
    , max_rollover_files = 5
    )
    WITH (MAX_DISPATCH_LATENCY = 5 SECONDS)
GO

-- Start the session
ALTER EVENT SESSION what_queries_are_failing ON SERVER STATE = START
GO

;WITH events_cte
AS (
    SELECT DATEADD(mi, DATEDIFF(mi, GETUTCDATE(), CURRENT_TIMESTAMP), xevents.event_data.value('(event/@timestamp)[1]', 'datetime2')) AS [err_timestamp]
        , xevents.event_data.value('(event/data[@name="severity"]/value)[1]', 'bigint') AS [err_severity]
        , xevents.event_data.value('(event/data[@name="error_number"]/value)[1]', 'bigint') AS [err_number]
        , xevents.event_data.value('(event/data[@name="message"]/value)[1]', 'nvarchar(512)') AS [err_message]
        , xevents.event_data.value('(event/action[@name="sql_text"]/value)[1]', 'nvarchar(max)') AS [sql_text]
        , xevents.event_data
    FROM sys.fn_xe_file_target_read_file('S:\XEventSessions\what_queries_are_failing*.xel', 'S:\XEventSessions\what_queries_are_failing*.xem', NULL, NULL)
    CROSS APPLY (
        SELECT CAST(event_data AS XML) AS event_data
        ) AS xevents
    )
SELECT *
FROM events_cte
ORDER BY err_timestamp;

However I'd like to immediately store failed statement into a table, let's call it Logs.Errors but I couldn't find a way to do it and the upper method would have to work as a scheduled job.

Right now, our procedures look like that:

CREATE PROCEDURE [dbo].[MyProcedure]
AS
BEGIN
    SET NOCOUNT ON;
    BEGIN TRY
        SELECT 1;
    END TRY
    BEGIN CATCH
        EXECUTE Logs.PrintError;
        EXECUTE Logs.LogError;
    END CATCH
END

Where Logs.LogError procedure is making use of DBCC INPUTBUFFER(); but it doesn't capture parameters, just the exact procedure that was executed. That's all I can get from it:

+----------------------------+-----------+-----------+------------------------------+
|        ErrorMessage        | EventType | Parameter |          Statement           |
+----------------------------+-----------+-----------+------------------------------+
| Incorrect syntax near '.'. | RPC Event |         0 | DbName.dbo.FailedProcedure;1 |
+----------------------------+-----------+-----------+------------------------------+

I'm looking for a way to either make DBCC INPUTBUFFER() work by forcing it to capture whole statement or XE to insert records straight into some table, if that's possible.

Any questions - let me know.

Evaldas Buinauskas
  • 13,739
  • 11
  • 55
  • 107
  • Can you please elaborate on `he upper method would have to work as a scheduled job.` – TheGameiswar Nov 28 '16 at 15:37
  • @TheGameiswar Sure. I had in mind that Extended Events could run in the background and store failed queries information to given file. Then based on a schedule (let's say hourly) I could read that file and insert records into `Logs.Errors` table. Does it make more sense now? – Evaldas Buinauskas Nov 28 '16 at 15:42
  • Yiou need not run extended events start and stop events as job,once you start it ,it runs in background – TheGameiswar Nov 28 '16 at 15:44
  • @TheGameiswar I understand that. The point of this question is how store failed queries with all the parameters into user table by either using XE or DBCC INPUTBUFFER right when the query fails. – Evaldas Buinauskas Nov 28 '16 at 15:47
  • 1
    you can use batch start,stop events ,look at my answer on dba.se http://stackoverflow.com/questions/36098446/extended-event-session-on-sql-server-2014-to-capture-stored-procedure-calls-and – TheGameiswar Nov 28 '16 at 15:50
  • 1
    this also may help:http://sqlblog.com/blogs/davide_mauri/archive/2013/03/19/extended-events-did-it-again-monitoring-stored-procedures-performance.aspx – TheGameiswar Nov 28 '16 at 15:51

1 Answers1

3

I've found that XEvents are great for monitoring events as they happen. However, they don't provide a mechanism to "handle" observed events. To fill that gap, I've used Event Notifications. I often describe them as asynchronous DDL triggers. I'll let you decide if that tl;dr definition is accurate or not.

If you want to give event notifications a try, here is a script that you can start with (sorry, it's really long). Let me know if you have any questions/issues. I'll try to reply as best as I can.

--Create these objects within a database that has service broker enabled.
USE DbaData
GO

--Drop objects first before trying to create them (in specific sequence).
IF EXISTS (
    SELECT *
    FROM sys.services
    WHERE name = 'svcUserErrorReportedNotification'
)
    DROP SERVICE svcUserErrorReportedNotification;
GO

IF EXISTS (
    SELECT *
    FROM sys.service_queues
    WHERE name = 'queUserErrorReportedNotification'
)
    DROP QUEUE queUserErrorReportedNotification;
GO

IF EXISTS (
    SELECT * 
    FROM sys.server_event_notifications 
    WHERE name = 'enUserErrorReportedEvents'
)
    DROP EVENT NOTIFICATION enUserErrorReportedEvents
    ON SERVER
GO

--Create a queue just for user error events.
CREATE QUEUE queUserErrorReportedNotification
GO

--Create a service just for user error events.
CREATE SERVICE svcUserErrorReportedNotification
ON QUEUE queUserErrorReportedNotification ([http://schemas.microsoft.com/SQL/Notifications/PostEventNotification])
GO

-- Create the event notification for user error events on the service.
CREATE EVENT NOTIFICATION enUserErrorReportedEvents
ON SERVER
WITH FAN_IN
FOR USER_ERROR_MESSAGE
TO SERVICE 'svcUserErrorReportedNotification', 'current database';
GO

IF EXISTS (
    SELECT *
    FROM INFORMATION_SCHEMA.ROUTINES r
    WHERE r.ROUTINE_SCHEMA = 'dbo' AND r.ROUTINE_NAME = 'ReceiveUserErrorReportedEvent'
)
    DROP PROCEDURE dbo.ReceiveUserErrorReportedEvent 
GO

CREATE PROCEDURE dbo.ReceiveUserErrorReportedEvent
/*****************************************************************************
* Name     : dbo.ReceiveUserErrorReportedEvent
* Purpose  : Runs when there is a USER_ERROR_MESSAGE event.
* Inputs   : None
* Outputs  : None
* Returns  : Nothing
******************************************************************************
* Change History
*   11/28/2016  DMason  Created
******************************************************************************/
AS
BEGIN
    SET NOCOUNT ON
    DECLARE @MsgBody XML

    WHILE (1 = 1)
    BEGIN
        BEGIN TRANSACTION

        -- Receive the next available message FROM the queue
        WAITFOR (
            RECEIVE TOP(1) -- just handle one message at a time
                @MsgBody = CAST(message_body AS XML)
                FROM queUserErrorReportedNotification
        ), TIMEOUT 1000  -- if the queue is empty for one second, give UPDATE and go away
        -- If we didn't get anything, bail out
        IF (@@ROWCOUNT = 0)
        BEGIN
            ROLLBACK TRANSACTION
            BREAK
        END 
        ELSE
        BEGIN
            --Grab some relevant items from the message body XML (it is EVENTDATA(), btw)
            DECLARE @Login SYSNAME;
            DECLARE @ErrMsgText VARCHAR(MAX);
            DECLARE @ApplicationName VARCHAR(MAX);
            DECLARE @Severity INT;
            DECLARE @ErrorNumber INT;
            DECLARE @DBName SYSNAME;

            SET @Login = @MsgBody.value('(/EVENT_INSTANCE/LoginName)[1]', 'VARCHAR(128)' );
            SET @ErrMsgText = @MsgBody.value('(/EVENT_INSTANCE/TextData)[1]', 'VARCHAR(MAX)' );
            SET @ApplicationName = @MsgBody.value('(/EVENT_INSTANCE/ApplicationName)[1]', 'VARCHAR(MAX)' );
            SET @Severity = @MsgBody.value('(/EVENT_INSTANCE/Severity)[1]', 'INT' );
            SET @ErrorNumber = @MsgBody.value('(/EVENT_INSTANCE/Error)[1]', 'INT' );
            SET @DBName = @MsgBody.value('(/EVENT_INSTANCE/DatabaseName)[1]', 'VARCHAR(128)' );

            --Do stuff here.
            --Log to a table, etc.

            /*
                Commit the transaction.  At any point before this, we 
                could roll back -- the received message would be back 
                on the queue AND the response wouldn't be sent.
            */
            COMMIT TRANSACTION
        END
    END
END
GO

ALTER QUEUE dbo.queUserErrorReportedNotification 
WITH 
STATUS = ON, 
ACTIVATION ( 
    PROCEDURE_NAME = dbo.ReceiveUserErrorReportedEvent, 
    STATUS = ON, 
    MAX_QUEUE_READERS = 1, 
    EXECUTE AS OWNER) 
GO
Dave Mason
  • 4,746
  • 2
  • 23
  • 24
  • Oh boy. This looks exactly what I am looking for. I'll try that on my local machine and feedback right when I can. Thanks! – Evaldas Buinauskas Nov 28 '16 at 16:10
  • Something I just thought of...the USER_ERROR_MESSAGE may generate a lot of "noise". It includes errors for all severity levels (I think). For instance, you might find a high number of events for "errors" with a severity level below 10, which you probably won't care about. This might flood the queue faster than SQL can process it. – Dave Mason Nov 28 '16 at 16:15
  • Is it possible to filter only failed procedures above specific severity? – Evaldas Buinauskas Nov 28 '16 at 16:19
  • The [USER_ERROR_MESSAGE](https://msdn.microsoft.com/en-us/library/ms190953.aspx?f=255&MSPPError=-2147217396) event should only occur when there is an error. So that should cover the "failed procedures" criteria. You can filter on severity level, based on the value retrieved from the @msgbody XML. – Dave Mason Nov 28 '16 at 16:23
  • 1
    This is really great and feels like you've directed me to the right path. Thanks a lot! – Evaldas Buinauskas Nov 28 '16 at 17:01
  • 1
    You're welcome! Please add some notes here if you run into issues. I'll be happy to update the script/answer if there are any omissions or other errors. – Dave Mason Nov 28 '16 at 19:46
  • All I can think of now would be performance improvements, if any, and then passing DBA wall to get this into production. ☺️ – Evaldas Buinauskas Nov 28 '16 at 20:23
  • Oh crap. I've noticed just now that USER_ERROR_MESSAGE doesn't include statement, that was executed, just the error message. I guess I'll have to find a way to connect both event notifications and extended events together. :| – Evaldas Buinauskas Nov 29 '16 at 08:47
  • Is there anything of use in the EVENT_INSTANCE_EXCEPTION event? I see a "TextData" field in the [schema definition](http://schemas.microsoft.com/sqlserver/2006/11/eventdata/events.xsd). – Dave Mason Dec 01 '16 at 17:45