Database Corruption, Transparent Data Encryption, and Trace Flag 5004

This one comes straight from the email bag.  A friend recently had a problem, they were placing TDE on a database and the encryption scan had stopped at state 2 percent_complete 0. I'm bouncing around the Charlotte NC airport facing some plane delays, and I thought what better time than to play around with a little database corruption. 

“So Balls”, you say, “What does TDE stuck in an encryption scan have to do with corruption.”

Great Question Dear Reader!  The default Page_Verify setting in SQL Server is Checksum.  This means when a page is read into memory and written back to disk a Checksum is calculated based off the pages contents.  When it is written back to disk, that Checksum is there.  When the page is read again the Checksum is used as a validation.  If the Checksum fails then it tosses an error reporting the page as a Suspect Page.

Think of this like going through the TSA Checkpoint, you’ve got your ticket and your identification.  If your ticket says ‘Serenity’, but your ID says ‘Zachary’ you will probably get flagged by the system as Suspect.  In both cases that’s where the probing begins. 

WE’VE GOT A PROBLEM

For this example I’m going to use a database that I’ve corrupted called CorruptAdventure taken from a corrupted version of AdventureWorksDW2008R2.  Horrible name for a database, it was just asking for corrupting.  We’ll start out assuming everything is fine.  The powers that be want TDE, Transparent Data Encryption, enabled on the database and we will do that.  First we’ll create our Master Key and a Database Certificate to use in the encryption.

/*

Create Master Key

and Certificate

*/

USE master

GO

Create Master Key Encryption By Password='MasterKeyPass1'

GO

Create Certificate DatabaseCertificate With Subject='Dont Put Anything Importiant in the subject'

GO

Now we’ll point to CorruptAdventure and create a Database Encryption Key and set encryption to on.  Transparent Data Encryption will read each page into memory.  If it doesn’t have a checksum one will get written.  Our page has a checksum, but it’s contents have been corrupted.  When SQL calculates a checksum to validate the current on, the page will get logged to the MSDB.dbo.Suspect_Pages table.

use CorruptAdventure

go

create database encryption key

with algorithm = aes_256

encryption by server certificate DatabaseCertificate

go

Alter Database CorruptAdventure

Set Encryption on

go

It looks like it is encrypting!

Whoa! We hit our error. 

Let’s query our Suspect_Pages table.  Just like I thought we’ve got our database ID and our page ID.  The error_type column is equal to 2, this means our page was flagged suspect during a Checksum operation. 

/*

It just stalled out

Why would this happen?

A page checksum occurs on all pages

whent the TDE scan

*/

select * from msdb.dbo.suspect_pages

Now let’s run DBCC CheckDB and verify if we really have something wrong with our database.

DBCC CHECKDB(CorruptAdventure) WITH NO_INFOMSGS

Msg 8928, Level 16, State 1, Line 1

Object ID 325576198, index ID 5, partition ID 72057594043498496, alloc unit ID 72057594044940288 (type In-row data): Page (1:3874) could not be processed.  See other errors for details.

Msg 8939, Level 16, State 98, Line 1

Table error: Object ID 325576198, index ID 5, partition ID 72057594043498496, alloc unit ID 72057594044940288 (type In-row data), page (1:3874). Test (IS_OFF (BUF_IOERR, pBUF->bstat)) failed. Values are 12716041 and -4.

Msg 8976, Level 16, State 1, Line 1

Table error: Object ID 325576198, index ID 5, partition ID 72057594043498496, alloc unit ID 72057594044940288 (type In-row data). Page (1:3874) was not seen in the scan although its parent (1:3888) and previous (1:3873) refer to it. Check any previous errors.

Msg 8978, Level 16, State 1, Line 1

Table error: Object ID 325576198, index ID 5, partition ID 72057594043498496, alloc unit ID 72057594044940288 (type In-row data). Page (1:3875) is missing a reference from previous page (1:3874). Possible chain linkage problem.

CHECKDB found 0 allocation errors and 4 consistency errors in table 'FactInternetSales' (object ID 325576198).

CHECKDB found 0 allocation errors and 4 consistency errors in database 'CorruptAdventure'.

repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB (CorruptAdventure).

Just as I suspected corruption.   We got the Database ID and page number from Suspect_Pages and DBCC CHECKDB just verified that the page is indeed corrupt. Now we can find exactly what type of data is corrupted, which will determine our strategy for handling it.  We have the Object ID and Index ID for the DBCC CHECKDB Scan.

We can do a query against sys.indexes joined to sys.objects using the IndexID, 5, and ObjectId, 32576198, provided.  We will get the table name, index name, and index type.

select

     o.name as TableName

     ,i.name as IndexName

     ,i.type_desc

from

     sys.indexes i

     left join sys.objects o

     on i.object_id=o.object_id

where

     o.object_id=325576198

     and i.index_id=5

Our corruption is on a non-clustered index.  If you ever get corruption this is one of the easiest types to fix.  We drop our non-clustered index and re-create it, and it should fix everything.

USE CorruptAdventure

GO

DROP INDEX IX_FactInternetSales_OrderDateKey ON dbo.FactInternetSales

GO

CREATE NONCLUSTERED INDEX IX_FactInternetSales_OrderDateKey ON dbo.FactInternetSales

(OrderDateKey ASC)

GO

Now let’s Run DBCC CHECKDB to get a clean bill of health.

DBCC CHECKDB(CorruptAdventure) WITH NO_INFOMSGS

Excellent, looking at our TDE status it still hasn’t moved. 

The TDE encryption scan should have paused when the Checksum error occurred.  In case it didn’t you can manually pause the encryption scan and reset it with Trace Flag 5004.  Turning Trace Flag 5004 on will stop the encryption scan right where it is.  You then need to turn Trace Flag 5004 off so you can re-issue the encryption command and watch it commence.  You might not need to use Trace Flag 5004, but I like to play this one on the safe side.

DBCC TRACEON(5004)

GO

DBCC TRACEOFF(5004)

GO

ALTER DATABASE CorruptAdventure

SET ENCRYPTION ON

Let’s check our encryption status.

We are progressing again, and it looks like now we’ve completed!  Excellent, not only did we get our database encrypted but we were able to fix corruption that we were not previously aware of.  One last peek at our TDE scan and we see it is complete and our encryption_status is now 3, no longer stuck at 2.

Well my plane just arrived, so that’s all for now Dear Reader, as always Thanks for stopping by.

Thanks,

Brad

How to Data Compress Varchar(MAX)

I talk a lot about compression.  I’ve blogged a pretty decent amount on it as well.  One of the things that often confuses people is what can and cannot be compressed.  There is a list of data types that can be Row Compressed.  That list is different between each SQL Version.  Page compression on the other hand works at the binary level, it is data type agnostic.

The big determining factor is what type of Allocation Unit your data is stored on.

“Balls,” you say “What’s an Allocation Unit?”

An Allocation unit is the structure behind the structure.  Think of real estate for a second.  Buildings and property are zoned in a city or a town.  One section is for businesses, another is zoned for residential, one may be zoned for the government.  In SQL Server we have 3 different zones IN_ROW_DATA, ROW_OVERFLOW_DATA, and LOB_DATA. 

Instead of being sized just for type, your size matters just as much.  If you are a regular every day Integer or Character field you live in IN_ROW_DATA.  You are LOB_DATA if you are a VARBINARY(MAX) that contains a 500 MB picture file.  ROW_OVERFLOW_DATA are variable length fields that start off on IN_ROW_DATA pages, but if that data grows large enough that it cannot fit on an 8 KB IN_ROW_DATA page then it gets popped off the IN_ROW_DATA Page and lands  on the ROW_OVERFLOW_DATA Page.

The data types in SQL that have a (MAX) designation, XML, or certain CLR types start off on IN_ROW_DATA pages.  They get moved off if the size grows.

HOW IN THE WIDE WIDE WORLD OF SPORTS

So how in the wide wide world of sports does this apply to Data Compression?  If your data is on an IN_ROW_DATA page it could be compressed.  Row compression still only applies to the data types that are listed per version, see row compression here at MSDN.

Page Compression only requires matching binary patterns, as long as it is IN_ROW_DATA pages we are good to go.  You can use this script to run against your database to get the Allocation Unit makeup of your tables and indexes.

SELECT

     OBJECT_NAME(sp.object_id) AS [ObjectName]

     ,si.name AS IndexName

     ,sps.in_row_data_page_count as In_Row

     ,sps.row_overflow_used_page_count AS Row_Over_Flow

     ,sps.lob_reserved_page_count AS LOB_Data

FROM

     sys.dm_db_partition_stats sps

     JOIN sys.partitions sp

           ON sps.partition_id=sp.partition_id

     JOIN sys.indexes si

           ON sp.index_id=si.index_id AND sp.object_id = si.object_id

WHERE

     OBJECTPROPERTY(sp.object_id,'IsUserTable') =1

order by sps.in_row_data_page_count desc

The higher the IN_ROW_DATA page count the more likely you have a candidate for compression. 

ON TO THE MAIN EVENT

We’ve laid the ground work now on to the main event.  First we’ll create our database and  our table and insert some data.  I’ve got two Varchar(Max) fields, we’ll put 2012 characters in each. 

/*

Select our demo database

to use

*/

use master

go

if exists(select name from sys.databases where name='demoInternals')

begin

     alter database demoInternals set single_user with rollback immediate

     drop database demoInternals

end

go

Create Database demoInternals

go

USE demoInternals

GO

/*

Create our table

*/

IF EXISTS(SELECT name FROM sys.tables WHERE name='vmaxTest')

BEGIN

     DROP TABLE dbo.vmaxTest

END

GO

CREATE TABLE vmaxTest(myid int identity(1,1)

     , mydata varchar(max) default 'a'

     ,mydata2 varchar(max) default 'b'

     ,CONSTRAINT pk_vmaxtest1 PRIMARY KEY CLUSTERED (myid))

GO

/*

Insert 5000 rows

*/

DECLARE @i INT

SET @i=0

WHILE (@i<5000)

BEGIN

     INSERT INTO vmaxTest(mydata, mydata2)

     VALUES(replicate('a',2012)+cast(@i AS VARCHAR(5)), replicate('b', 2012)+cast(@i AS VARCHAR(5)))

     SET @i=@i+1

END

GO

If you use our script from earlier then you can see we have 4950 IN_ROW_DATA Pages.

Now let’s update one of our Varchar(max) fields to 8000 characters so that we push it off of IN_ROW_DATA and over to LOB_DATA Pages.   Run our script again to get our counts.

/*

Now we'll update just the b values

to force them into row_overflow data

pages

*/

UPDATE dbo.vmaxTest

set mydata2=replicate('b',8000)

We certainly have some fragmentation, but we’ve added 5009 LOB_DATA pages to the mix.  Now let’s apply Page Compression and use our script again to see the results.

/*

Rebuild our table with

Page Compression

*/

ALTER TABLE dbo.vmaxtest

REBUILD WITH(DATA_COMPRESSION=PAGE);

GO

As you can see the IN_ROW_DATA Compressed, the LOB_DATA didn’t.  Another way that knowing thy data can help you understand what you can and should compress.

Thanks,

Brad

Book Review: On Writing A Memoir of the Craft by Stephen King

I just finished this book.  Literally.  Sitting on an airplane in route from Orlando to Charlotte NC.  I got up out of my seat, fetched my computer just so I could start writing.  It took me two weeks to finish.   I was busy, but I enjoyed every moment I could sneak reading On Writing into the day.

Are you a writer?  Technical, fiction, non-fiction, blogger, columnist, or novelist?  If you write read it. 

 Not a Stephen King fan? Read it anyway.  He is a once in a lifetime author.  He has been very successful in his line of work.  He knows something about the craft that you may not.  He knew plenty that I did not. 

“So Balls”, you say, “What’s so great about this book?”

In a word Dear Reader? Everything.

MY INTRODUCTION TO STEPHEN KING

I’m not a fan of horror movies.  I’m a bit of a wennie in that regard.  Ask my wife, she loves them.  I’ll watch them, I cringe, I jump, and sometimes I’ll make a sound. 

I don’t like the gore of the movies, but give a book that has the same elements and I’ll lap it up.  My tastes tend to shift towards the Supernatural, Sci-Fi, Horror, Mysteries, and tales of Knight’s and times such as that.  I’ve occasionally read biographies, but other than Technical Manuals, most of what I read is Fiction.

I’m not sure the first a Stephen King book I read.  The first time I remember reading something of his was a collection of short stories called Different Seasons.  It had stories that ran the gambit.  Rita Hayworth and the Shawshank Redemption (read it long before it was a movie), The Apt Pupil (ditto), Different Seasons (aka the movie Stand by Me ditto again), and the Langoliers (TV movie but ditto times four).  I always heard of Stephen King the “horror” writer, he’s been writing since before I was born and famous for just about as long. 

I also found something interesting; my favorite part of the book was the Introduction.  I liked reading the thoughts of the man himself.  He seemed funny, smart, the kind of guy you would want to hang out with.  He took me on a trip and described things in such a way that I understood them.  I liked hanging out with Stephen King.

Fast forward some years and I picked up the novelized version of the Screen Play for Storm of the Century.  Once again I got to read the comments, the thoughts that made up the man, and learn a little more about his process.  I liked the TV version, admittedly I didn’t watch it until after I’d read the screen play, but the dialogue was better in my head.  The special effects budget had no limit.  What stuck with me most was King’s description of how he had envisioned the character of Andre Linoge. 

Steve,  hope you don’t mind that I call him Steve he’s told me so many stories over the years calling him “Stephen” feels too formal.  Anyway Steve had a dream about a man sitting on the bunk of a bed in a cell block.  You could draw parallels to the Green Mile, but Andre was different from John Coffey.  He was smaller and looked quite a bit different for starters.  Instead of being gentle he was a menacing force, everyone was in greater danger when he was close by even if he was in prison cell.   The cage held a hungry tiger, not a passive giant.  The dream scared the bejesus out of him, he woke up and had to write.  Had to write about the character before it left his mind.

Think of the vivid dreams that you get.  You wake up and have to tell someone.  Good, bad, scary, crazy, a dream that leaves a mark.  How cool would it be to make a story out of the dream? 

We recently took the kids to the library to get our first round of library cards.  While we were there I was looking for a book.  I looked and eventually found Just After Sunset: Stories.  In the introduction Steve mentioned On Writing and my interest was piqued.  The next trip to the library I picked it up, I’m glad that I did.

ENOUGH GUSHING ON TO THE REVIEW: ON WRITING

http://www.flickr.com/photos/65426280@N03/6854893798/

We start out with Stevie King growing up as a kid.  He goes out of his way to show us that he wasn’t born into writing.  It was a skill he developed.  It takes practice; you have to work out the muscles that you use writing. You will fail.  Failure is part of trying.  Don’t let that discourage you from trying; Stevie King had a stack of rejection letters that he kept above his desk.  Success was not overnight or easy.  We travel through his life with great detail, to see success and failure. 

His early life is humbling.  He wrote his first novels in a trailer on a small desk by the washing machine.  There is no shortcut to being a successful writer, but we see how the man was crafted.  We see his views on literature.   We get his reflections on his life.

You need to have a tool box.  In it you need to place the tools that you will use.  As a doctor you may wield a scalpel, in IT you use a computer, and as a writer you need your tools as well.  Not just a pen and paper, or a keyboard.  Tools of vocabulary, grammar, a greater understanding of nouns, verbs (passive vs. active), lessons on adverbs and pronouns, elements of style, naturally evolving stories vs. outlined stories, and instructions and examples of how to use them.  Stephen King makes it interesting, engaging, and gasp educational.

Steve explains his thought process on connecting with the reader.  The bond between the writer and the reader is so great and so close, that it is psychic on the level of telepathy.  Don’t believe me?  I’ll give you a quick example. 

                “The old man walked through the rain.  He pulled his jacket around him close.  He had made this walk many times, up the road from the store back to his apartment.  The weather raged against him.   One look at his hands and you could see this was far from his first storm.  Life had not been easy, he never asked it to be.  He dipped his head and passed an inside out umbrella that had lost its way, discarded and forgotten.  Not unlike the old man himself.”

The umbrella, did you see it?  Was it maroon, light yellow, blue, maybe green?  You know what color you saw, I didn’t need to tell you.  How about the old man, what color was his hair?  I never told you, but yet you saw it, or was he bald?  Somehow you just knew.   You Dear Reader have a great imagination.  You can paint your own canvas, and I should let you.  That makes this story not mine, but our story.

You knew Dear Reader, magically, as if by telepathy exactly what I was thinking.  Straight down rain, sideways rain.  You knew, and you got it just right.

IN SUMMARY

Reading this book will make you look at the way that you write, and examine what you are doing.  I could do a chapter by chapter review but it wouldn’t do it justice.  It is the instructions of a teacher, and a damn fine read.  I borrowed this book from the library. I will buy a copy.  Word count 1429, final -10%=1286 (You’ll get it when you read it).

Thanks,

Brad