Can’t Troubleshoot? Don’t Apply!

If you can’t troubleshoot, you can’t work with me (if I can control it anyway). When I am interviewing you I am judging your troubleshooting skills as much as your SQL Server knowledge – if not more depending on the role. I’ve blogged a bit about troubleshooting (Methodology, Trusting Empirical Evidence) and wrote a SQL Server Central Article on it.

A couple weeks ago I almost threw out my shoulder patting myself on the back after replacing the starter in my truck. Up to that point the most complicated thing I’ve done to a vehicle is changed a battery, tire or fuse. I’ve changed the oil once or twice (in my lawn tractor…)  I won’t spike the football ;-) by talking too much about the fix here but I wanted to talk about the steps that brought me to the realization that I was going to be changing into my dirty clothes and lying down under my truck for a couple hours. Those steps followed the methodology I discussed before and effective troubleshooting has been on my mind as I prepare for my SQL Rally presentation on Professional Development (Iceberg, Dead Ahead!)

The Journey:

I Identified The Problem 

The Results of My Troubleshooting - A Bad Starter Gone

Out with the old..

I’ve had some intermittent issues with starting but jump starting, waiting or jiggling wires (not a proper methodology but a good quick step each time I had an issue) solved it each time. Well the day it looked serious I had the whole family in the truck (even moved the infant car seat over from minivan) for a ride to pick up flowers to plant. My first reaction was problem solving (rushed problem solving). That looked like sighing, rushing, jump starting, jiggling, tightening battery terminals, etc. It was messy and shotgun like. My wife gave me that look and I knew to stop and move everyone to the van for our trip.

This let me troubleshoot a bit more slowly and carefully later in the day. Here’s what I did:

  • Think – How does a truck start? The 100 level of it (where I am) is -  the key turns and, if the battery is good & transmission is in Park or Neutral (and truck knows it), electricity flows through the battery to a starter. That starter spins fast and starts the engine. So electrical flow is critical, some switches and relays are critical. Fuel helps, I hear.
  • Rule Outs – I ruled out the issues in order of “obvious problems” and easier to fix:
    • Fuel – Engine wasn’t even trying to turn over, just clicking. Starter wasn’t spinning the flywheel.
    • Battery – Tested, no issues. Terminals looked fine, connection tight.
    • Fuses – Fine
    • Relays – Starter relay had same number as some others. Swapped them, same sounds.
    • Wires/Cables – Connection from battery to starter was good
    • Key – I was pretty sure there was no chip in mine, but used a spare to be sure.
  • Analyzed Evidence -
    • I had ruled out many components. Really all that seemed left was the starter. Thought through rule outs and couldn’t rule it out without changing.
    • Probable cause – I was thinking starter now.
  • Verified – I am not mechanically inclined s0 I verified my direction
    • Internet – Found a few trustworthy sites describing starter issues. Symptoms matched, rule out steps matched and sites agreed. Still, it’s the internet…
    • Parts Store – Paid a visit and told them what I had done and what I was thinking. They agreed and added “starter problems almost always start intermittently, probably was better in winter because of how the metal moves”
  • Decided – I was going to sink the money on a starter. If it worked, it was likely it. If it didn’t I’d return/resell and call a “consultant” (tow truck to garage)

Researched Solution

  • Level of effort – Could I do this myself? By all accounts, yes, if you can follow directions, wires and spin a ratchet.
  • Instruction – Looked for some quick training. I wanted to know some basics (What does a starter look like for instance… what tools do I need.. What precautions should I take, etc.)
  • Tools – Verified I had the right tools (Socket set really)

Decided To Fix

  • Critical – I wanted to get this running, needed to for work.
  • Had a backup plan  – Tow truck, AAA, Wife’s family members who are mechanically inclined
  • Go/No-Go – Couldn’t see any reason to not try. I Didn’t even have to lift truck and worry about it slipping off jack/jack stands. We were a Go.

Fixed

  • Got Dirty – Spent a lot of time under the truck finding exact location of starter.
  • Traced wire to starter – Did that by following the starter wire from battery I had read about and tested earlier.
  • Found bolts, found ratchet, made them meet
  • Brought it down – It was difficult to access.. I had to take fender off, an unexpected diversion with components I didn’t know how to work with (little plastic pop screws, broke a few before I figured it out)
  • Replaced It – Disconnected old one (pictured in this post), hooked up the new one and put it in the right spot.
  • Hooked up battery – (I had disconnected it before starting – made sure I was not working on live system… Always bad when you mess up in live)
  • Tested it a few times and ways – Mainly because I liked the sound of my truck starting and the look of my blackened, greasy hands turning the key ;-)
  • SHIP IT! Problem solved.

Lessons For Next Time

I asked myself about what I could do differently next time -

  • Could have a few more tools for next time
  • Should have researched proper way to remove/replace fender fastener clips

So?

That’s really it. All of those steps are how we can solve any problem. When I ask that unrelated problem question or open ended “how would you deal with complaints of a slow SQL backed app?” I am looking for something logical, step based and calm. Something along the lines of these steps generally -

  1. Gather Initial Information
  2. Keep Thought Involved
  3. Rule In/Rule Out Causes Methodically
  4. Verify Your Understanding and Assumptions
  5. Propose Solution(s) Accordingly
  6. Carefully Test Proposed Solution  (if you can)
  7. Implement Fix
  8. Test and Verify Fix
  9. Learn for Next Time

And do all of that while remaining calm and keeping a picture on the big picture.

Share

Tags: , , ,

13 Comments

Leave a comment
  1. Jonathan Kehayias May 16, 2011 at 13:40 #

    Usually when the starter is the problem having someone turn the key in the ignition while you tap on the starter body with a hammer can validate that it is the source of the problem. The tapping with the hammer will usually work to free it up internally letting it turn over which proves the issue is inside of the starter itself. I always give the starter a final tap test before concluding it is the source of the problem.

    • Mike Walsh May 16, 2011 at 13:43 #

      I should have mentioned that. I have had someone tap the starter before ( someone who knows cars) and it didn’t make a difference. I think it was beyond it’s last leg. Even noticed it sounded different when starting with the new starter.

  2. Josh Feierman May 17, 2011 at 07:03 #

    Great post Mike. I don’t understand why people seem so reluctant to take this methodical, planned approach to troubleshoting, as opposed to the flail your arms, try everthing at once and shriek various curses method. Next time someone pulls that with me I’m pointing them here.

    • Mike Walsh May 17, 2011 at 09:17 #

      Thanks for the comment, Josh.

      I am not sure why it isn’t always followed. I can say earlier in my career and even still sometimes with household issues/challenges, my first reaction may not always be the calm, logical, step by step approach to a problem. I think it takes work to get to that mindset. Some times and with some people that is a lot more work but it takes work either way.

  3. CameronMergel May 17, 2011 at 09:15 #

    Congrats on taking the initiative and fixing your truck. I am not that brave.

    Good read and reminder that there is an art of troubleshooting. Best piece advice “Don’t panic and stay calm”. :)

    • Mike Walsh May 17, 2011 at 09:19 #

      Hey Cameron!

      I went through the pros and cons a few times before deciding on the “do it yourself” route. Seemed to be a fairly easy project and I could save myself the hassle of AAA tow to a garage, paying any difference and then paying the garage fee for the part, their markup and their time. I’ve been on a DIY kick lately. I find the more I’m willing to do myself, the more tools my wife lets me buy ;-)

  4. Karen Lopez May 19, 2011 at 10:25 #

    I’m always shocked when I run into people in IT who have almost no diagnostic skills. Even when I train them to go through a process that rules out certain possible conditions they still just don’t get it. I’ve often wondered if this sort if logic/cause/effect sense is a trait we are born with.

    I don’t know how one survives in IT without these sorts of skills, even if they have to fake them. :) We can all get better at what we do by following your steps here.

    • Mike Walsh May 19, 2011 at 10:32 #

      Thanks for the comment, Karen. I often run into people without those skills. I wish I could say it was rare, but it has to be at least monthly, if not weekly. It isn’t just technology, though – I’ve seen that skill lacking in other professions. I’ve even seen it spending time on ambulances, though those people never lasted (the practitioners, that is.)

      I go back and forth in my mind on if this is a nature or nurture trait. I look at myself and I can say I am a better diagnostician now than I was 12 years ago when I started in technology. I look around some household tasks and see that sometimes my first approach (depending on my interest in the task, perhaps?) sometimes is bordering on the “flail method”. So I think the traits/skills/”tools” can be learned but still you have to want to use them. So maybe it is a bit of both?

      • Mike McCallum December 1, 2012 at 07:10 #

        Great post.

        Two useful tips regarding troubleshooting that I picked up the hard way over the years:

        1. When testing theories, only make one change at a time. Otherwise, if the issue is resolved you won’t know which change solved it.

        2. Keep a log of the steps you are taking. It is easy to forget the exact steps you took. If this happens, even though you solved the issue, you may not truly understand what fixed it.

        These may seem quite obvious but I’ve been surprised that many people don’t follow them.

Trackbacks/Pingbacks

  1. Avoid Using Those Troubleshooting Skills | SQL Server Blog by Mike Walsh - May 17, 2011

    [...] Learn From Mistakes, Life Lessons, Professional Development TweetWhen I re-read yesterday’s blog post on troubleshooting steps, I felt guilty. Instantly. Why? Because I had failed to mention something to you in that post and [...]

  2. SQL University – Troubleshooting Week | Straight Path Solutions, a SQL Server Consultancy - October 4, 2011

    [...] If You Can’t Troubleshoot – Don’t Apply –> I relate the experience of finding and repairing a broken starter to IT troubleshooting skills. Seriously, if you can’t troubleshoot, don’t apply to work with me. Read on to see why. [...]

  3. 6 Reasons You Won't Get Hired For A Technical Job | Straight Path Solutions, a SQL Server Consultancy - January 31, 2012

    [...] me you had a passion to learn, could explain your thoughts, use logic and apply common sense to troubleshooting I may consider you over someone with a bit more skill but less of these traits. In fact, I’ll [...]

  4. SQL Server Join Syntax - It's Changed! | Straight Path Solutions, a SQL Server Consultancy - November 26, 2012

    [...] big fan of accurate and informed troubleshooting when your data is on the line. I blog a lot about troubleshooting and this post talks about some steps. One of them is to research the solution and understand [...]

Leave a Reply