Where?

Quest Connect is an online series of webcasts on various topics. Some are pre-recorded (like the session that I did.. more on that later…). They have web casts on a variety of topics, not just SQL. You can look at the on demand topic listing here to see all the chats that will be available on demand. There will also be several live web casts with Q&A, etc. You can see those here.

What?

Live SQL Webcast Monday 10/@10:00AM EDT

There is one Live Webcast with Q&A on SQL Server (“A Sysadmins guide to SQL Server“) at 10:00AM EDT. The hosts will be such experts as Brent Ozar (Blog/Twitter), Thomas LaRock (Blog/Twitter) and Tim Ford (Blog/Twitter). This will be great for those reluctant DBAs (you know, you’re a server admin or windows admin, you help out on the SAN, you have to manage SQL.. you do it all but you are scared every second).

Pre-recorded sessions (including one with me)

These will be available from 4AM on 10/21 –> 4AM on 10/22 (times are in EDT)

You can see the full catalog of sessions for all subject areas here. The ones specific to SQL Server are:

• “Top 10 SQL Server Administration Mistakes” – I’ll talk in more depth about this one below.
• “Understanding Execution Plans” – With Grant Frtichey (Blog/Twitter) They aren’t that scary. Watch a scary DBA walk you through how easy they can be if you just know what to look for!
• “The top 5 things you can do with Litespeed for SQL Server to be a backup and recovery hero” – A title after my own heart: it’s long…  Quest puts these things on for us for free so it’s only natural to have some talks about how their products can help. Litespeed is a great tool (not paid to say it, don’t use it currently but have in other environments). Learn about this great backup tool and I am sure that Brent will give some good backup/recovery advice at the same time. Brent Ozar and Jason Hall will present this one.

Why did they ask me to help?

I guess I’ve blogged about a few worst practices (Shrinking anyone?, Shoot From the hip much anyone?) so I was asked to help participate in the “Top 10 SQL Server Administration Mistakes” chat. This was a fun recording and is a great conversation with a few folks much smarter than me – Kevin Kline (Blog/Twitter), Colin Stasiuk (Blog/Twitter) and Jason Strate (Blog/Twitter) and myself about mistakes we have done, seen and had to help fix in our careers.

We’ll walk through a PowerPoint of 10 mistakes that DBAs often make when working with SQL Server. They range from issues like considering disk options based on space needs only, shoddy troubleshooting, not embracing change control (maybe embrace is too strong a word ) to bad thinking with backup and recovery.

It was an informative talk, I learned a few things on the talk myself and you’ll even hear an epiphany moment where three of us picked up a great tip around change control/rollback steps. A real “Why didn’t I think of that…” moment.

I hope you register for Quest Connect and check out all of the videos and I hope you check out the conversation that Kevin, Colin, Jason and myself had around DBA “stuff”, might help you out, might start a conversation.

Resources for the presentation I was a part of:

First of all: Comments. If you have any questions on the parts of the conversation I was in, you can hit me up on twitter but I don’t get there much on most weekdays. Leave a comment on this post if it is a question that others might benefit from or add insight into. Send me an e-mail if you want to talk out of the comment thread instead.

I hope you check out the blogs of all of those who participated, I gave links above. You can also go to SQLServerPedia where all of our blogs are syndicated and search blogs and SQL wiki articles on a topic you want to know more on.

If I remember correctly, I mentioned the SQL Server 2000 I/O Best Practices document. It is not 100% applicable to 2005/2008 but the basic concepts largely remain true and this is a good resource.But instead of linking you there, check out this document from Microsoft which is more recent and it references that SQL 2000 document as a resource at the bottom.

I also suggest visiting the SQLCAT team website with their best practices, great resource. If you don’t own the Inside SQL Server 2005 series, you should. Microsoft SQL Server 2008 Internals is also a great resource. These books go into a lot more depth on a lot of the areas we talked about.

There are also some blog topics on my blog that are related to what we talked about:

• Advice to beginning DBAs wondering, “Where do I start?!?!”
• Shrinking – several posts on the topic.
• Troubleshooting Methodology Woes (I also wrote an article for SQL Server Central on this topic)
• Empirical Evidence
• Documentation (or how I learned to stop procrastinating and love tolerate a chore necessary task)
• A little Paranoia and Control Freak attitude can help a DBA
• Checklists, Recipes and Algorithms – Learning about these great tools from other professions (Pilots, Chefs and Doctors) 
• How do I find all of my SQL Servers? – Maybe after listening to us you’ll want to spend more time with the instances at your network. Check this link out to learn about a tool that I use to find all the SQL Servers in my network.
• Questions to ask a software vendor – Working with a new vendor with their own database? Here are some questions I ask them.

Thanks for listening, thanks for stopping by to check out the blog. I hope you visit the other guy’s sites also. Lots of information and like I said, SQLServerPedia is a great place to find info. There are a lot of great bloggers there. I also have a lot of bloggers I read in my blogroll.

So I wrote a lot of posts around shrinking (Here, here, here). Some of my posts were turned into a wiki article on SQLServerPedia. I thought I was done posting about this but one more.

I talked about why we shouldn’t do it on a regular basis. Why it should really be reserved for use in dev or extenuating circumstances due to a lack of proper planning. I also talked about why it is bad to let your databases grow in small chunks.

I can’t stress it enough… Right sizing your database is critical. This requires PLANNING. If you don’t plan for performance you will be killed by performance in no time. If your company or your project manager says something like, “We don’t have time to plan for that” then you will pay for it later at least 75% of the time and you will pay more than you would have up front.

The below picture is from a system somewhere that I got involved in after it had been “alive” (persistent vegetative state…) for a couple years. It was set to do a DB shrink each night after a transaction log truncate (Again –> Don’t do this! If you want to recover to a point in time, use FULL Recovery mode and TAKE LOG BACKUPS at a frequency that manages your recovery needs (first) and your transaction log growth (second). This was done on each database on the instance.

Lots of Red and Lots of Fragments

This is with the windows defragment tool. A quick reminder: Blue (there are about 5 or 6 skinny blue lines in the above picture) indicates contiguous space, no physical file fragmentation. Red indicates fragments. No, I didn’t play with the colors. Most other drives remind me of Christmas also.

What does this mean? This means that I/O operations against this drive have an unfair advantage working against them. Reads will not be as well performing as they should. (For instance copying an LDF file off of this – data file drive – to move the .ldf to a log file drive took 4 minutes. No other activity, file was 1GB..)

How did it get this way? Every night the files on this data file drive were shrunk as part of a shrinkdb operation. Every night after this shrink an ETL process loads it. Files then grow in chunks during the ETL (and some of them grow during the day with use, like the report server temp database). These growths and shrinks lead to file fragmentation.

How long did it take? Much like my current status on the scale (14.6 stone when I should be 12.5.. I’ll let you bing, google or ask wolfram alpha if you care) this didn’t happen overnight. It took a period of neglect. Neglect including:

• The Shrinks/growths
• Never planning for the expected space and proper sizing in the first place
• Allocating the bare minimum to the drive (88% full right now) and adding more space from the SAN as “needed”.
• Never running a defragment process during maintenance.

What do we do? Well for the short term we are going to grow the databases to an appropriate amount (hopefully getting more space allocated before all of this obviously), back them up, turn off the SQL Server services, remove any files that shouldn’t be on this drive and defragment the volume.

What do we do in the future? The shrinking operations were killed awhile ago (except for one that was hidden in an ETL process, since removed), I will make sure that a process is put in place to check for fragmentation more frequently. I will keep fighting the fight to properly size a database from the beginning.

Not My Job

Nearly no other phrase can irk me quite like that. You don’t even have to say it, you can just experience it. Now I am guilty here also. Where this case happened, the databases are managed from SQL out by the DBA, the servers/storage/etc are managed by the storage and server team. Put trust in that team and never really looked at fragmentation. OOPS. Now it isn’t my job and normally I go out of my way to worry about things that could affect databases I care about, for some reason I never checked fragmentation here.

When talking to the sysadmin about I/O issues, I started talking about a desire to split some of the report server databases onto their own drives (large reports running, seeing a lot of blocking with the built in report server procs that write/read to the databases) and the reaction was very quickly “No, you have high fragmentation.. let me show you” and then we opened disk management in window and looked at the fragmentation. My reaction was guilt and shame for not checking. Then my second reaction was (internally) “How did you know this and not say anything?!” The conversation basically revealed that this was a known issue for some time. Now before the DBA role was filled there was no one else looking at database performance on the applications team. How did this sysadmin expect project managers and developers without much systems/DBA knowledge play Carnac and figure out the fragmentation?

The attitude that killed here? “I just create the disks and give them to the people who request it, I give them the space they asked for and that’s it.”

You can’t do that. If you want to be a good DBA you need to look above and below your part of the food chain and if you see an area that needs improvement, SPEAK UP. Same goes for sysadmins and storage teams. You are one company with the same customers and the same goal of happy customers.

Reminder –> Are you looking at your physical file fragmentation levels? I am sure you are a great DBA and update your stats, rebuild indexes when appropriate. Do you look at your I/O subsystem? Do you check your fragmentation? Do you get blinded by the simple in search for a complex situation? I do at times. Check your fragmentation levels and run some benchmarks. Fix this headache before it becomes a resume altering situation.