Monday, August 6, 2012

How to Find Bad Page Splits


http://www.flickr.com/photos/brickapolis/5474835288/


I was working with a friend last week, doing SQL related teaching and tune ups.  Earlier this week I had received an email from them regarding unusually high page splits they were receiving after deploying new code, upwards of 800 per second!

This sounds really bad at first glance, but is it?  It depends on the type of Page Split behavior.

  This is an excellent opportunity for us to look at together. We need to look at how active your database system is and determine what type of activity the Page Splits are to see if this is  bad or not.  The new code release may play into this, but then again maybe it doesn’t.

“So Balls,” you say, “What’s a Page Split?”

Great Question Dear Reader! And away we go.

DOING THE SPLITS
http://www.flickr.com/photos/kathy1243/5570985183/


First let’s talk about the setting.  Your table is either a Heap or a Clustered Index.  Page Splits can only occur in Indexes, Heaps get something called Forwarding Pointers.  A Heap, however, can have Non-Clustered Indexes.  Non-Clustered Indexes can get Page Splits on their leaf level indexes.  So regardless of the base table type, only Indexes can get Page Splits.

  Page Splits are natural behaviors to data growth.  Some cause fragmentation in indexes, we want to track these and figure out a way to keep them from happening to prevent fragmentation, which can lead to poor query performance.  

A page split can occur as an Update or an Insert statement.  There are 3 scenarios that can cause Page splits, Sequential Inserts, Non Sequential Inserts, and Updates.

“So Balls”, you ask, “Why does it matter if it is an Insert or an Update, Sequential or Non Sequential?”

Again with the great question’s Dear Reader, well played.  

The way that a Sequential Page Split operation is performed is that the Leaf page become full, and you go to insert one more record.  A Split operation is performed when the next record is attempted to be inserted on the page.  The data is placed on a newly allocated page.  The PFS, Page Free Space, Allocation Bytemap, will flip bits indicating that the old page is full as well as bits for our new page.  The table will be updated to reflect the new page has been created.  This is not a Split operation that we would not want to track it, for the purpose of causing Index Fragmentation.

The way that a, Non Sequential Insert, Mid Page Split operation is performed is the almost the same as an Update.  The leaf level of the page is already full.  An Insert operation occurs on the page.  Records are moved off the page to make room for that Non Sequential Insert. This is a Split operation that we would want to track.

The way a Split operation is performed on an update is a page already is full of data, and one of the records on the page is updated and the data grows, it pushes the another record off of the page.  This is a Split operation that we would want to track.

These Inserts and Updates can not only affect the pages that they are on but could have a cascading affect onto other pages if they are all full.

The update could cause multiple pages to be altered, and changes to our Clustered and Non Clustered Indexes, this translates to I/Ops which will cost us CPU cycles.  Our Indexs will point queries into the right direction, but we could cause fragmentation and out of date statistics that could lead to a less than optimal plan from being selected.  So most inserts are not bad, and updates that cause excessive Page Splits should be examined.

DETECTING THE  SPLITS

So the question becomes how do we track these Mid Page Splits and separate them out from the End Page Splits.  The lucky ones using SQL 2012 have an Extended Event Action sqlserver.transaction_log that they can use.  Jonathan Kehayias (@SQLPoolboy|Blog) wrote a great blog about this earlier this year, to read more on it click here.

But what if you’re not on SQL 2012? We have the Transaction log that has recorded this same event.  The LOP_DELETE_SPLIT logged operation is the logged event that tells us if a Mid Page Split has occurred.  Specifically it marks that rows were deleted from a Page as a result of a split.  Using the undocumented procedure fn_dblog we can query the Transaction Log to find these events.  


Jonathan provides a great script for producing Mid Page Splits, I recommend reading his blog, even if you’re not on 2012 it will give you a good understanding of the logged event, as well as a practice script to use.  Just run the same table scripts in SQL 2005, 2008, or 2008 R2.  If you use 2005 you’ll have to change the DateTime2 value in Jonathan’s script to a DateTime value…or at least something other than a DateTime2 value.

Once you have created some Mid Page Splits run the following query.
SELECT
     COUNT(1) AS NumberOfSplits
     ,AllocUnitName
     ,Context
FROM
     fn_dblog(NULL,NULL)
WHERE
     Operation = 'LOP_DELETE_SPLIT'
GROUP BY
     AllocUnitName, Context
ORDER BY
     NumberOfSplits DESC

Now you have the total number of operations and the full Object Name of the Index that is the cause of the Mid Page Splits.  Now you can go and rebuild your Index with a lower amount of Fill Factor to see if that keeps the splits from occurring.

Keep in mind that the LOP_DELETE_SPLIT will only stay in your transaction log so long as the VLF’s, Virtual Log Files (the internal components of the transaction log), are not over written by new transactions.

 For best results you could place this script in a SQL Agent job and run it a couple times an hour and log the results to a table, so you could analyze what indexes you have that get Mid Page Splits and check them against any nightly re-index jobs to see the level of fragmentation of those same indexes.

If you don’t have nightly index jobs that log I’d recommend grabbing Ola Hallengren’s Maintenance plans click here to visit his site.

END PAGE SPLITS, OR NEEDLE I GIVE YOU THE HAYSTACK


http://www.flickr.com/photos/gymru1958/3418449455/


So we know how to find the Mid Page Splits, but what about the End Page Splits.  After all you don't want to know one without the other right?  Well Dear Reader like most things Microsoft finding the first one is easy, the second not so much.

You would think that their is a logged event that signifies an End of Page Split.  If there is I didn't find it.  But there are other fields in the fn_dblog function that we can use to find our End Page Splits.  When you do a Select * from fn_dblog you get a lot of info back.  Parsing it can seem daunting, and if we were left with only our field Context we would truly be up the river.

However if you use the field [Transaction Name] you will find a value called 'SplitPage'.  Adding the Description field will get you more detail.  But let's take this a bit at a time.  Frist we'll create a database a table and insert just enough data that we get 1 page split.  Then we'll use the undocumented DBCC IND in order to get a full list of page number for our table.


IF EXISTS(SELECT name FROM sys.databases WHERE name='endPageSplits')
BEGIN
     ALTER DATABASE endPageSplits SET SINGLE_USER WITH ROLLBACK IMMEDIATE
     DROP DATABASE endPageSplits
END
GO
CREATE DATABASE endPageSplits
GO
USE endPageSplits
GO
IF EXISTS(SELECT NAME FROM SYS.TABLES WHERE NAME='endSplits')
BEGIN
     DROP TABLE dbo.endSplits
END
GO
CREATE TABLE endSplits(myid int identity(1,1), mychar char(2000) NOT NULL DEFAULT 'A', constraint pk_myid_endsplits primary key clustered(myid))
GO

DECLARE @i INT

SET @i=0

WHILE(@i< 1)
BEGIN
     SET  @i=@i+1

     INSERT INTO dbo.endSplits
     DEFAULT VALUES
END

DBCC IND(endPageSplits, 'endSplits', 1)
GO




We see that our Allocation Unit (PageType=10) is page 154, our First Data Page (PageType=1) is 153, our Index Page (PageType=2) is 155, and our Second Data Page (PageType=1) is 155.  If we do a query on fn_dblog looking for [Transaction Name]='SplitPage'.

For this example we'll add in our Description we need to get everything that is like 'Changed type HOBT_ROOT (0)%'.  This is only good for this current example.  Since we have a small clustered index we'll cause a root Index page to be created, and that will give us in Hexadecimal the values of our pages being inserted,




SELECT
            [Transaction Name]
            ,Description
            ,*
FROM
            FN_DBLOG(NULL,NULL)
WHERE
            [Transaction Name]='SplitPage'
            or Description like 'Changed type HOBT_ROOT (0)%'




The Changed type HOBT_ROOT will get us more than one row.  We want to look at rows 2 & 3.  Row 3 is our SplitPage logged event. Row 2 contains data in the description that is directly related to our page split.  I've pasted it's contents below.


Changed type HOBT_ROOT (0) for AU 72057594039697408 in rowset 72057594038779904 from page 0001:00000099 to page 0001:0000009b

These are Hexadecimal values for the page numbers in our SplitPage Transaction.  If you put the Hex value 99 in you will get the decimal value 153.  If you put the Hex value 9b in you will get decimal value 155.  And there we go confirmation that we have split from Page 153 to 155.

The operation itself was at the Root level of the index, and depending on your index size and the split that occurs, the logged operation can change..  I'm simplifying the process quite a bit, but from this example that we had a SplitPage operation and through the description we could validate what page split from which.


These are logged operations that we don't need to single out, but I wanted to cover them in case you were curious.  These operations mean our table is growing, data is being added, and you can track that many many better ways than scouring the transaction log for events and descriptions.

Good luck finding Mid Page Splits and as always thanks for stopping by and reading.

Thanks,

Brad

7 comments:

  1. Very nice. Page split information has eluded me for a while now and this is quite clear with good examples.

    ReplyDelete
  2. I really appreciate it, Thank You for reading! This was a lot of fun to dive into and figure out. I almost stopped at the Mid Page Splits, but had a lot of fun finding the End Page Splits as well.

    ReplyDelete
  3. Nice writeup. Good illustration of the issue. Be careful using fn_dblog though. Using it adds a non-trivial amount of overhead to any busy system. Adding a call to it in a scheduled job should only be done after careful consideration, and for the overwhelming majority of systems one could argue never at all.

    In SQL Server 2012 we have a lightweight way to track page splits using Extended Events. The technique actually keys off the same transaction log entries you're reading directly from the transaction log, except with Extended Events we can inspect them as (or before) they are written so the overhead is minimal. If you think mid-page splits are an issue in your 2008 R2 (or earlier) database a better option than using fn_dblog in my opinion is to indirectly track the results of page splits by trending fragmentation over time in your indexes and adjusting your fill factor up or down as needed.

    ReplyDelete
  4. Another approach to this is to use the undocumented function fn_dump_dblog() to read the transaction log backups. This can be used to offload the processing to another SQL Server unstance that has access to the log backups. The results can be dumped to a table for trending purposes which is very helpful in analysis

    ReplyDelete
  5. bassplayerdoc - be careful with fn_dump_dblog(), as it can open a scheduler that never dies. (Paul Randall updated his blog entry that mentions it with a warning). Very glad we can do that with XE these days.

    ReplyDelete
  6. Agree with Bourgon if you haven't read Paul's blog essentially it is an undocumented function and leaves and leaves threads open per backups scanned, you can face thread starvation on a machine if you use it. So you can use it, but you would want to make sure it was on a Machine you could reboot.

    ReplyDelete
  7. Hi,
    It's very nice article which has given me a good understanding about the page/splits which was really wonderful.
    One thing to ask is "when i ran the above first query it is taking me the time and as well the CPU has suddenly increased to 80%.... can you please tell me how should i proceed if i plan this to be scheduled in the job!!!

    ReplyDelete