Tales From The Field Weekly Wrap Up for the Week of 04-03-2023 & Easter

Hello Dear Reader!  Hopefully you had a great weekend.  This weekend was filled with gatherings.  On Saturday we gathered with friends and had a really nice Game Night.... where no games were played and it was just perfect.  

It had been so long since we all gathered that we ate taco's, made some movie plans, watched hockey, and just generally enjoyed one another's company.  The food theme was tacos, my compliments to the many chef's.  Mere descriptions alone will not do it justice but there was homemade taco meat, refried beans, melted cheese, homemade rice & black beans, fresh guacamole, and some of the best chocolate chip cookies ever.

The following day was Easter.  My parents joined Sue, the kiddos, and myself.  Even though the kids are 19, 18, & 13 years old respectively.  Yet, they are never too old for an Easter basket!  When I was over in Wales & London for SQLBits I love to get candy for Easter.  There are so many amazing options to choose from and the fact that it is just slightly different, but oh so delicious, adds an element of wonder and amazement.

Some of my favorite Dad things to do are when we prepare for Christmas or Easter.  I also love some of the little things.  When the kids were small I loved making lunches. Making sure to add cinnamon apple sauce, snack cakes, some type of gummi snacks, making sure they each had their own individualized sandwiches. Serenity's favorite was a peanut butter and sliced bananawith no crust, I prefer the crust on mine.  All of those small things, that they are too big for now.

Lots of wonderful happy memories Dear Reader, the most important thing is to be present in the moment and love the people around you.  Alright, enough reminiscing.  Grab a Reese's Peanut Butter Cup shaped like an Easter Egg because it's time to bounce over to the re-cap!

WEDNESDAY 04-05-2023

On Wednesday we released our latests MS Tech Bits diving into the structures of Heaps and Clustered Indexes.  Why do we have the VS. on the slide?  The default table type in Oracle is a Heap, the default table type in SQL Server is a Clustered Index.  For years therese were the two most dominiant relational database systems and having different default recommendations creates confusions.

 

It's importiant to remember what we've learned from Ted Lasso.  "All people are different people", "Be a goldfish", also that tip about the peanut butter was pretty good and I would recommend the White Chocolatey Wonderful. I digress, all databases are different databases.  Some may be based off of ACID some are not, but indexing structures in SQL Server, PostgreSQL, MySQL, or Oracle are all things you need to understand.

It affect performance and every day functionality.  So we dive into the conceptual structures in this video, demos are coming this week!

TUESDAY 04-04-2023

On Tuesday we had our Azure Data Community Round Table!  The wonderful Daniel Taylor (Twitter | @DBABullDog) was in for Neeraj Jhaveri (Twitter | @Neeraj_Jhaveri), Josh Luedeman (Twitter | @JoshLuedeman), Andres Padillia (Twitter | @nodestreamio), and myself had a great day talking about the AMAZING content created by the Azure Data Community.

No time to waste, here is that wonderful content in the order it was covered on the show:

Dan

• DBA in training: SQL Server high availability options  by Pamela Mooney Twitter @PamelaMooney

Bradley

• T-SQL Tuesday #160: Round-Up  by Damien Jones Twitter  @amazonwebshark

Andres

• Edge AI: From Model Development to Deployment   by Raj Prasanna Ponnuraj

Josh

• Faster training and inference using the Azure Container for PyTorch in Azure ML  by Bea Stollnitz Twitter @beastollnitz

Dan

• [How-To] SSRS migration by Garry Bargsley Twitter @gbargsley

Bradley

• The Kingdom of AdventureWorks Calls for Aid - Asad Khan, Bob Ward, Sanjay Mishra & Buck Woody by  SQLBits,  Asad Khan, Bob Ward, Sanjay Mishra & Buck Woody Twitter @SQLBits, @Bob_Ward, @BuckWoodyMSFT, @AzureSQL 

Andres

• Create AI tracking Drone using DJI Tello  by Murtadha Bazli Tukimat  Twitter @RobotAndCode

Josh

• Single Server to Flexible Server Migration tool is GA in Azure Database for PostgreSQL  by Shriram Muthukrishnan  Twitter @azureDBpostgres

Dan

• How To Learn Microsoft Azure in 2022 by Thomas Maurer Twitter @ThomasMaurer

Bradley

• Power BI datasets CI/CD (the easy-ish way) by Stephanie Bruno Twitter @StephTBruno

Andres

• Amazing Invention-This Drone will change everything by Mark Rober  Twitter @MarkRober

Josh

• GitHub Codespaces for Students and Educators  by Nehemiah Emmanuel  Twitter @devgenix

WRAP IT UP.... AND SQL SATURDAY JACKSONVILLE!!!

This week we have the Community Round Table on Tuesday at 1 pm EST.  Join us to see the Azure Data Community blogs & videos we will be featuring.

On Wednesday we will have our MS Tech Bits, this will be the Demos that accompanied last weeks conceptual ideas about indexes!

SQL Saturday Jacksonville is just under 1 month away!! All of the Tales from the Field crew will be at the event.  Make sure to come to Daniel, Neeraj, & my pre-con A Full Day of DBA Fundamentals.  The full event schedule is LIVE!!

This will be a fun week, we hope you have a great one!  And as always Dear Reader, Thank you for stopping by.

Thanks,

Brad

SQL Live 360 Kicks off Today!

Hello Dear Reader!  Today in beautiful Orlando FL at the Loews Royal Pacific Resort the Live 360 Conference kicks off.  Live 360 combines 4 different conferences into one.  Visual Studio, SharePoint, SQL Server, and Cloud & Virtualization all have their own conferences.  I’m helping to kick off the SQL Live 360 by presenting not once, not twice, but three times today!  

Before we dive into the content I want to say a big Thank You to my wife Silva.  Every time I’m presenting at a conference she’s taking off work to be at home and manage the kiddos.  Without her none of this is possible for me.  Thanks Silva!

“So Balls”, you say, “What are you presenting on?”

Excellent question Dear Reader.  We will be kicking off the day with Trimming Indexes Getting Your Database in Shape, next up is Transparent Data Encryption Inside and Out in SQL Server 2012, and we end the day with the Page & Row Compression Deep Dive in SQL Server 2012.  Conferences are expensive and I want to make sure that you get the most out of your experience.  So to help you decide if you should be spending your time with me, I’m placing the decks and demos online now.  They are also live on the Resource Page.  Attendees should have a copy of all of this information on their conference CD, but just in case you didn’t find it here we go!

KICKING IT OFF WITH A BANG

If you’re in town for the conference I hope you get a chance to stop by.  Here are the abstracts for each presentation:

Trimming Indexes Getting Your Database In Shape

Indexes are a wonderful thing.  We should be using them, and we should be maintaining them.  But over time our production databases start to look a little pudgy around the mid-section.  Maybe they are a little bloated with Unused Indexes, maybe they have Duplicate Indexes, and possibly even Reverse Indexes.  The first step to fixing these problems it so see if you have them, and if you do the second is to set about fixing them. You could be costing yourself CPU cycles, I/Op’s, and space and never even know it.

Slide Deck, Demos

Transparent Data Encryption Inside and Out in SQL 2012

Security is a very important part of your job and in how data is utilized.  We have many tools to make data more secure, and starting in SQL 2008 we were able to add Transparent Data Encryption to that list.  Find out What it does and What it doesn’t do, How it effects Read-Only Filegroups, Performance, Compression (Backup and Row/Page), What the X.509 Encryption Standard is and Why you should be careful of what you store and where, and other Advance Features as well as some tips on how to manage it.

Slide Deck, Demos

Page & Row Compression Deep Dive with SQL Server 2012

Page and Row Compression are powerful new tools. Page & Row with SQL 2008 RTM, and Page & Row with Unicode Compression with SQL 2008 R2, and Spatial Types in SQL 2012.  We can turn it on, we can turn it off, but we want more!  What are Access Methods and how in the SQL Engine do they affect Compression?  What are the “Gotchas” of Page Compression?  How does Compression differ in the way it treats Non-Leaf & Leaf Level pages?  What additional functionality did we get in DBCC Page, DMV’s, Perfmon Counters, and Extended Events to support our usage of Compression?  How do complex query plans affect Compression?  Come find out!

Slide Deck, Demos

LUNCH!

That’s not all, on top of having 3 great sessions to choose from if you’re still not sick of me we’ve got a speaker round table on Wednesday where I’ll be hosting a table on SQL Server 2012 and new features.  While I may not have presented on them there are a lot of great 2012 topics like Columnstore Indexes, Always On Availability Groups, Always On Clustering Improvements, and more that we could chat about.

Great SQL People like MVP’s Jen Stirrup(@JenStirrup | blog), Grant Fritchey (@GFritchey | blog), Allen White (@SQLRunr | Blog),  Thomas Larock (@SQLRockstar | Blog), and William Pearson (@Bill_Pearson) will be there hosting different tables as well!

WRAP IT UP

Okay with that being said, it should be a fun filled week of SQL Learning.  There will be some wonderful night time activities as part of the conference as well. 

  I won’t make it to all of them, Wednesday night is the baby’s Christmas play at her day care can’t miss that!  For the rest of the week though I look forward to getting to see you all.

If you are in town I hope you get to stop by and say “Hi BALLS!”, and have a great time!

Thanks,

Brad

PASS DBA Virtual Chapter Deck and Demo's Live

Hello Dear Reader, I wanted to say a quick Thank You to the PASS DBA Virtual Chapter for having me  today.  My Deck and Demo's are up on the Resource Page.  Thank You to the 150 people who took the time to attend as well!  We couldn't do this without you.

Thanks Again,

Brad

PASS DBA Virtual Chapter Trimming Indexes, Getting Your Database in Shape

Hello Dear Reader!  Tomorrow at 12 noon eastern I’ll be presenting for the PASS DBA Virtual chapter, click here to sign up for the meeting.

If you aren’t a member of/or familiar with PASS it is the ProfessionalAssociation for SQL Server.  PASS put’s together great things for us throughout the year like SQL Saturdays, which are put together by local PASS User Groups (click here to find the one in your area) groups that meet FREE monthly and have presentations on different SQL topics, The PASS Summit (Largest SQL Server conference in the world!), 24 Hours of PASS (free 12 spans of great training), and the PASS Virtual Chapters.  Virtual Chapters range 16 different subjects and 3 different languages, soon to be four different languages!

Joining PASS doesn’t cost you a dime, and I don’t get a penny for it, but it opens the door to a large amount of free technical content and training.  If you are not familiar I’d encourage you to click on the above links and become familiar with PASS today!

“So Balls”, you say, “What is this presentation you’re doing?”

Glad you asked Dear Reader!  I’m presenting for the DBA Virtual Chapter, 1 of the 16, and my subject is Trimming Indexes, Getting Your Database in Shape.

I’LL TAKE THE #2 SUPER SIZED

Here’s the abstract and then we’ll talk a little more:

“Indexes are a wonderful thing. We should be using them, and we should be maintaining them. But over time our production databases start to look a little pudgy around the mid-section. Maybe they are a little bloated with Unused Indexes, maybe they have Duplicate Indexes, and possibly even Reverse Indexes. The first step to fixing these problems it so see if you have them and if you do the second is to set about fixing them. You could be costing yourself CPU cycles, I/Op's, and space and never even know it.”

If you’ve been a DBA for a while you will inevitably inherit a system where you find indexes being used in less than optimal ways.  A lot of this is created by turn over in a company, going with all of the suggestions from DTA (Database Tuning Advisor), or having too many cooks in the kitchen.

It is possible to get things like Reverse Indexes, Duplicate Indexes, and unused Indexes.  You may be asking,  “What do those terms mean?  What secret ninja SQL Language are you speaking?  I know Clustered and Non-Clustered, but what-in-the-sam-hell is a REVERSE index!?”

It’s alright Dear Reader, no new secret terms.  A Duplicate index is just an Index where the physical structure exists more than once on a table.  Take the following Table:

CREATE TABLE Students(

          studentID int identity(1,1) primary key clustered

          ,ssn char(9)

          ,firstName varchar(50)

          ,middleInitial char(1)

          ,lastName varchar(100)

          ,gender char(1)

          )

If we created a Non-Clustered Index on the SSN column and we called it nclx_Students_SSN, and then someone else made a Non-Clustered Index on the SSN column and called it nclx_Students_SSN2 we would have a duplicated index.

“But Balls”, you say, “I would never do that!”

Of course not, you wouldn’t ever do that on purpose.   As code gets migrated from Dev to Production perhaps the Developer or Jr DBA adds an index that they didn’t realize you already has in place.  Or maybe you get a query plan with a “Missing Index hint” in Dev, only that index had been created as an urgent Production change, and never got implemented in Dev.  Migration comes around and as long as the names are different, WHAMO, you have two Non-Clustered Indexes on your SSN Column.

This example might not seem that bad, but imagine a 50 row table with a duplicate Non-Clustered Index on 5 rows, 10 rows, or 15 rows.  That’s a lot of extra data having to be persisted to disk and maintained.

"I wish my abs..I mean... databases where in shape"

Using the previous table let’s know make a Reverse Index.  We’ll create a Covering Non-Clustered index for a stored procedure that requires the SSN, FirstName, and LastName fields.  Some farther down the road you’ve left that company and a new developer is writing a different block of code and a different stored procedure and they create their Non-Clustered Index on LastName, FirstName, and SSN.

Then you have Unused Indexes.  These are the indexes that it seemed like a good idea to build, but nobody is using them.  In some places you have code that gets retired, but we still need the database structures,  in the Data Modeling phase Indexes were designed that were not used, or Database Tuning Advisor recommended it and it just wasn’t used.

Finding these are important because we are maintaining them, but the slackers do not contribute to our query performance. 

WRAP IT UP

So our goal for the hour will be to discuss Indexes make sure that we have a good foundation in them and what they store so we can understand why these 3 types of indexes are bad, and then use some scripts and DMV’s to identify them.

I'm also doing this presentation in a much longer format for SQL Live 360 in December of this year, as well as a couple more presentations.  Click on this link to check out Live 360!

I hope you’ll get a chance to stop by and join us!

Thanks Again,

Brad

Let's Talk About Indexes: An Introduction

A very good friend of mine, whose got this kick-ass job working for the President J, wrote me a question about indexes.  And I thought how better than a phone call and a follow up blog talking about indexes in general.  The question centered on the uses of Clustered Indexes vs. Non-Clustered Indexes and the use for each.

CLUSTERED vs NON-CLUSTERED

The good news is this really isn't a VS kind of thing.  Clustered Indexes and Non-Clustered Indexes are both indexes.  They are both made of of an internal B-Tree structure.  

When you think of a B-Tree, think of your standard Hierarchy.  There are 3 named levels to a B-Tree structure.  The Root Level which is our top level and contains values that direct a path towards the next level, the Intermediate level and just like the Root Page points to the next level of Data, and finally the Leaf Page.

One of difference between a Clustered Index and a Non-Clustered Index is that a Clustered Index physically sorts the Data, based off of the Clustered Index.  Another difference is you can only have one Clustered Index per Table, but you can have multiple Non-Clustered Indexes.  The Clustered Index is most commonly thought of as a Primary Key, (the Clustered Index doesn't have to be the Primary Key, but that could get a little confusing so we'll save that for another day).

So we'll say that our B-Tree up above is using the letters of the Alphabet as it's Clustered Index.  I'll issue a quick T-SQL Select to get our letter.

SELECT

    *

FROM dbo.alphabet

WHERE

    letter='G'

If I wanted to go get the letter G, at the Root Level I deduce that G comes after A and before J so I would take a pointer and go to the Intermediate level that contained A, C, F, and H, From there I would further deduce that G is after F and before H, and I would travel down to our Leaf Page where I would find the letter G.

*IF your query's first Search Argument is the Clustered Index, and your range of values are narrow, then you will use your Index structure in a manner called a Seek.  If you have to read the entire table to get the contents of your information back, it is called Scan.  Scan's have their place and are not necessarily a bad thing.  But it is entirely dependent on the amount of data being returned, and the route taken to search for it.

One of the best examples I can think of to describe what an Index does, and further illuminate our B-Tree Structure, and show how  Clustered Indexes and  Non-Clustered Indexes  work together is the good old Phone Book.  If I told you to find Bradley Ball's address in the phone book you've got 2 ways of finding my name.

1. Turn through the book 1 page at a time until you get to Bradley Ball, and then read his address.

2. Flip to the B's (our Clustered Index) and scan until you get to Ball.  Now Scan Ball for the first name of Bradley (our Non-Clustered Index), and find the address.  You just performed a Seek, using the search arguments of Last Name and First Name to find Address.  

So now let's toss out a little vocabulary, to help further this discussion along.

HEAP- A heap is a table that has no clustered index.  Data is stored on a head in the order that it is inserted.  This means it is easily fragmented, and could require scanning as selecting data would require a full scan of the heap in order to find it.

CLUSTERED INDEX- A Clustered Index is a Unique value, that determines the physical sort order of the data as it is stored on a page.  The leaf level of a Clustered Index is the data of the table.  Clustered indexes have a limited key size of 900 bytes.

NON-CLUSTERED INDEX- A Non-Clustered Index is an index placed on a row in a Table, the leaf level of a Non-Clustered Index contains the data that makes up the Non-Clustered Index, as well as a physical pointer to the leaf level of the Clustered Index.  You can have many Non-Clustered Indexes on a Clustered Index Table, or a Heap.  Non-Clustered Indexes have the same 900 byte limitation and are limited to 16 columns, you can get around this using Include Columns.

"So Balls", you say, "Indexes sound great, we should put them on EVERYTHING! Right?"

I'm Glad you asked Dear Reader, and the response, regarding OLTP systems, is NO.

BALANCING ACT

Every Index that you place on a table has a cost of additional overhead.  You want to use them to speed up your database, but if you're not careful you could grind it to a halt.  

If you have a table with a Clustered Index, when you insert and update pages, and GROW the data, you are increasing your over head, slightly and it's worth it, to make sure that your Root and Intermediate level pages have the correct values to point to your leaf pages.

When you add a Non-Clustered Index, every time you insert a value for that Index you write to your table AND your Non-Clustered Index.  So if you Add another, and another, and another, then suddenly every time you write 1 record to your table, you could have many, many, many writes to keep all of your Indexes updated.

One of the first things I look at when I get a call about a "performance" problem on a database is to look at the indexes.  I've had instances where I have found tables with more indexes than columns.

COVERING INDEXES

The final high level topic that I want to talk about is Covering Indexes.  A Covering Index is an index that satisfies all of the return requirements of a query.  You could take all of the fields that you want to return, limit 16, in a query and if you add them to a single Non-Clustered Index you can do so.

One way to make a covering index is to add each column to the Non-Clustered index, another is to add additional columns as Include Columns.  Include columns allow you to get around the 900 byte limitation of an index, because Include columns are not stored as key values.  There are some data types that are not allowed to participate in Included Columns, Text and Image datatypes.  You can have 1023 Included columns on an index.

I just want to point out thought that, 1023 should not be a goal.  Every value you add, will add additional overhead to your leaf level Non-Clustered index.  Be sure to take baselines before and after, in order to know if your Index changes are having a positive affect on your query performance.




Thanks,

Brad