Logging Database Changes

Chris Leon — Thu, 12 Aug 2010 00:11:08 +0000

Have you ever had team members change something without telling you and not back it up, mess something up yourself, or have a boss that thinks he’s a SQL expert that sabotages things? I know I’ve had some of those problems in the past. Some of us aren’t lucky enough to have something like Visual Studio DB Pro or Tarantino implemented yet. In fact, there’s probably a large percentage of developers that just work off the production database. The team I’m on now had something in place when I got there to help with this problem. I took the general idea and made a database trigger to keep track of changes to stored procedures, functions, views, and tables. It’s no real database versioning system, but it’s a step in the right direction.

CREATE TRIGGER [LogChangeHistory]
ON DATABASE
FOR create_procedure, alter_procedure, drop_procedure,
create_function, alter_function, drop_function,
create_view, alter_view, drop_view,
create_table, alter_table, drop_table
AS

SET NOCOUNT ON

DECLARE @data XML

SET @data = EVENTDATA()

INSERT INTO Logging.History.ChangeLog
(
    DatabaseName,
    EventType,
    ObjectName,
    ObjectType,
    SqlCommand,
    LoginName,
    UpdatedOn,
    SqlCommandText
)

VALUES
(
    @data.value('(/EVENT_INSTANCE/DatabaseName)[1]', 'varchar(256)'),
    @data.value('(/EVENT_INSTANCE/EventType)[1]', 'varchar(50)'),  -- value is case-sensitive
    @data.value('(/EVENT_INSTANCE/ObjectName)[1]', 'varchar(256)'),
    @data.value('(/EVENT_INSTANCE/ObjectType)[1]', 'varchar(25)'),
    @data.query('(/EVENT_INSTANCE/TSQLCommand)[1]'),
    @data.value('(/EVENT_INSTANCE/LoginName)[1]', 'varchar(256)'),
    GETDATE(),
    SUBSTRING(REPLACE(CONVERT(VARCHAR(MAX), @data.query('EVENT_INSTANCE/TSQLCommand/CommandText')), '', CHAR(13) + CHAR(10)),
        14,
        len(REPLACE(CONVERT(VARCHAR(MAX), @data.query('EVENT_INSTANCE/TSQLCommand/CommandText')), '', CHAR(13) + CHAR(10))) - 28)
)
GO

This is database level so you’ll need to put it the trigger each database. We made a separate database where we put all the changes from other databases on the server.

My favorite DMVs

Chris Leon — Wed, 11 Aug 2010 13:09:29 +0000

While most of my programming background is in web development, I’ve been working a lot with refactoring SQL Server things so you’ll be seeing the next few posts mostly on that topic.

One of the SQL Server features I’ve found most useful in performance tuning SQL Server is Dynamic Management Views. If you’re given a database with a general task of improving it with no additional directions, it’s a great place to start to find out the largest of the performance problems. This is a list of the DMV queries I’ve found most useful. I just had them in a text file and I’ve done my best to search the intertubes for them to attribute where I can.

Top 10 Cost of Missing Indexes (http://msdn.microsoft.com/en-us/magazine/cc135978.aspx)

Sometimes developers don’t give a thought to indexes until they start suffering performance problems. This gives you the top 10 causes of SQL Server attempting to use an index but it wasn’t there. Generally creating these will have a large performance increase.

SELECT  TOP 10
        [Total Cost]  = ROUND(avg_total_user_cost * avg_user_impact * (user_seeks + user_scans),0)
        , avg_user_impact
        , TableName = statement
        , [EqualityUsage] = equality_columns
        , [InequalityUsage] = inequality_columns
        , [Include Cloumns] = included_columns
FROM        sys.dm_db_missing_index_groups g
INNER JOIN    sys.dm_db_missing_index_group_stats s
       ON s.group_handle = g.index_group_handle
INNER JOIN    sys.dm_db_missing_index_details d
       ON d.index_handle = g.index_handle
ORDER BY [Total Cost] DESC;

Longest Elapsed Time (http://blog.sqlauthority.com/2010/05/14/sql-server-find-most-expensive-queries-using-dmv/)

This is just the queries that take the longest to run. It might denote some problems if anything in here is unexpected.

SELECT TOP 100
    qs.total_elapsed_time / qs.execution_count / 1000000.0 AS average_seconds,
    qs.total_elapsed_time / 1000000.0 AS total_seconds,
    qs.execution_count,
    SUBSTRING (qt.text,qs.statement_start_offset/2,
         (CASE WHEN qs.statement_end_offset = -1
            THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2
          ELSE qs.statement_end_offset END - qs.statement_start_offset)/2) AS individual_query,
    o.name AS object_name,
    DB_NAME(qt.dbid) AS database_name
  FROM sys.dm_exec_query_stats qs
    CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as qt
    LEFT OUTER JOIN sys.objects o ON qt.objectid = o.object_id
where qt.dbid = DB_ID()
  ORDER BY average_seconds DESC;

Another Longest Elapsed Time View (no idea where I got this one)

This just gives a slightly different view of the previous query.

SELECT  creation_time
        ,last_execution_time
        ,total_physical_reads
        ,total_logical_reads
        ,total_logical_writes
        , execution_count
        , total_worker_time
        , total_elapsed_time
        , total_elapsed_time / execution_count avg_elapsed_time
        ,SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
         ((CASE statement_end_offset
          WHEN -1 THEN DATALENGTH(st.text)
          ELSE qs.statement_end_offset END
            - qs.statement_start_offset)/2) + 1) AS statement_text
FROM sys.dm_exec_query_stats AS qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
ORDER BY total_elapsed_time / execution_count DESC;

Most I/O (http://msdn.microsoft.com/en-us/magazine/cc135978.aspx)

SELECT TOP 100
    (total_logical_reads + total_logical_writes) / qs.execution_count AS average_IO,
    (total_logical_reads + total_logical_writes) AS total_IO,
    qs.execution_count AS execution_count,
    SUBSTRING (qt.text,qs.statement_start_offset/2,
         (CASE WHEN qs.statement_end_offset = -1
            THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2
          ELSE qs.statement_end_offset END - qs.statement_start_offset)/2) AS indivudual_query,
    o.name AS object_name,
    DB_NAME(qt.dbid) AS database_name
  FROM sys.dm_exec_query_stats qs
    CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as qt
    LEFT OUTER JOIN sys.objects o ON qt.objectid = o.object_id
where qt.dbid = DB_ID()
  ORDER BY average_IO DESC;

General Query Stats by Execution per Database (http://www.mssqltips.com/tip.asp?tip=1843)

SELECT  Top 100
       sdest.dbid
       ,sdest.[text] AS Batch_Object,
       SUBSTRING(sdest.[text], (sdeqs.statement_start_offset/2) + 1,
       ((CASE sdeqs.statement_end_offset
               WHEN -1 THEN DATALENGTH(sdest.[text]) ELSE sdeqs.statement_end_offset END
                       - sdeqs.statement_start_offset)/2) + 1) AS SQL_Statement
       , sdeqp.query_plan
       , sdeqs.execution_count
       , sdeqs.total_physical_reads
       ,(sdeqs.total_physical_reads / sdeqs.execution_count) AS average_physical_reads
       , sdeqs.total_logical_writes
       , (sdeqs.total_logical_writes / sdeqs.execution_count) AS average_logical_writes
       , sdeqs.total_logical_reads
       , (sdeqs.total_logical_reads / sdeqs.execution_count) AS average_logical_lReads
       , sdeqs.total_clr_time
       , (sdeqs.total_clr_time / sdeqs.execution_count) AS average_CLRTime
       , sdeqs.total_elapsed_time
       , (sdeqs.total_elapsed_time / sdeqs.execution_count) AS average_elapsed_time
       , sdeqs.last_execution_time
       , sdeqs.creation_time
FROM sys.dm_exec_query_stats AS sdeqs
       CROSS apply sys.dm_exec_sql_text(sdeqs.sql_handle) AS sdest
       CROSS apply sys.dm_exec_query_plan(sdeqs.plan_handle) AS sdeqp
WHERE  sdeqs.last_execution_time > DATEADD(HH,-2,GETDATE())
               AND sdest.dbid = (SELECT DB_ID('DB Name'))
ORDER BY execution_count DESC

Top Waits (http://www.davewentzel.com/content/waits-and-qs-queries)

WITH Waits AS
 (
 SELECT
   wait_type,
   wait_time_ms / 1000. AS wait_time_s,
   100. * wait_time_ms / SUM(wait_time_ms) OVER() AS pct,
   ROW_NUMBER() OVER(ORDER BY wait_time_ms DESC) AS rn
 FROM sys.dm_os_wait_stats
 WHERE wait_type
   NOT IN
     ('CLR_SEMAPHORE', 'LAZYWRITER_SLEEP', 'RESOURCE_QUEUE',
   'SLEEP_TASK', 'SLEEP_SYSTEMTASK', 'SQLTRACE_BUFFER_FLUSH', 'WAITFOR',
   'CLR_AUTO_EVENT', 'CLR_MANUAL_EVENT')
   ) -- filter out additional irrelevant waits 

SELECT W1.wait_type,
 CAST(W1.wait_time_s AS DECIMAL(12, 2)) AS wait_time_s,
 CAST(W1.pct AS DECIMAL(12, 2)) AS pct,
 CAST(SUM(W2.pct) AS DECIMAL(12, 2)) AS running_pct
FROM Waits AS W1
 INNER JOIN Waits AS W2 ON W2.rn <= W1.rn
GROUP BY W1.rn,
 W1.wait_type,
 W1.wait_time_s,
 W1.pct
HAVING SUM(W2.pct) - W1.pct < 95; -- percentage threshold;

Most often blocked queries (http://msdn.microsoft.com/en-us/magazine/cc135978.aspx)

SELECT TOP 10
 [Average Time Blocked] = (total_elapsed_time - total_worker_time) / qs.execution_count
,[Total Time Blocked] = total_elapsed_time - total_worker_time
,[Execution count] = qs.execution_count
,[Individual Query] = SUBSTRING (qt.text,qs.statement_start_offset/2,
         (CASE WHEN qs.statement_end_offset = -1
            THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2
          ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)
,[Parent Query] = qt.text
,DatabaseName = DB_NAME(qt.dbid)
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as qt
ORDER BY [Average Time Blocked] DESC;

There are a lot of other views available on the Internet. These are just the ones I usually use as a starting point for diagnosing problems.

New Blog

Chris Leon — Fri, 06 Aug 2010 22:52:00 +0000

This morning I woke up and realized I hadn’t checked to see if any variations of my name were available as domain names in a while. I found chrisleon.net available and registered it as quickly as possible. I tried to change my old blog over to this name but somewhere in the labyrinth of godaddy’s menus I screwed up things to the point where it was just easier to delete everything and start over. It’s not like the world will miss my former babbling about bike movies, keyboards, and Harry Potter. I’m going to try to keep this mostly technical with a few other things thrown in. If I’ve forgotten anyone in my blogroll let me know.