On Feedback… And a Plea To Event Organizers

I really like presenting. I’ve blogged about some tips for others thinking about starting before. In those posts, I share how I started out terrified of speaking in public. One of the reasons I am enjoying it to date? Good feedback. Don’t read that as “positive” (though there has been more of that than I’ve expected!), but helpful feedback.

I’ve been meaning to write this post for awhile now (since helping plan and execute SQL Saturday 71). Seeing some recent frustrated posts from two individuals I respect. Two SQL Server MVPs who both speak and blog regularly. One was Aaron Bertrand, the other Allan Hirt. Their posts were reviewing feedback from SQL Bits (Aaron’s | Allan’s).

A common theme emerged – “If you are going to fault me in the points, TELL ME WHY!” . To that point, I say – Right On! But… I wonder is it the attendee’s fault or is it the Organizer’s fault? Now I don’t know the SQL Bits questions, I know the ones used at the PASS events. I know the ones used at most events. For those?

It IS The Event Organizer’s Fault!

I really hate the questions PASS asks. They are a bunch of 1-5 scaled questions that aren’t necessarily that helpful for a speaker alone. They allow space for comments but they don’t ask any open ended questions designed to solicit feedback.

Numerical ratings are tough. On one hand they are important so you can have one scale to rank speakers by. They are good for people who want to see where on the poor, fair,good, better,best scale they are (either for a quick ego stroke, confirmation to keep speaking or a not so subtle way to suggest another approach). On other hand – They are meaningless…

If someone comes to one of my sessions and gives me a 2 on a 1-5 scale or a 3/4 on a 1-10 scale, that doesn’t tell me why (unless it was a specific question like “please rank the speaker’s knowledge on the subject” but even there it doesn’t tell me what it was that caused the ranking) the ranking was so low. It doesn’t tell me what I can do to improve.

The line those two blog posts took was a good one and a good reminder for us all as session/training attendees – explain your scores (good and bad). But… I think the organizers can do more.

I’m not saying that my questions at SQL Saturday 71 were the right questions. I am not saying that I’m an expert on soliciting feedback, but I know people need to be prodded. So I have a plea to all you event organizers out there -

Ask Better Questions!

Yes. Still ask your numeric ratings questions, just replace some of them with questions that require writing. I know, not everyone will fill them out and just circle numbers. Those people will always exist and some people will just be critical and leave no comments,it’s just how some operate. But… A lot of people will provide useful feedback.

With that disclaimer that I’m not saying this is THE way, maybe looking at the questions asked (and the rationale behind them) at an event I helped organize will let the SQL community at large think of better questions.

What did I ask on the SQL Saturday 71 Feedback?

“How would you rate the overall quality of this session?” -  The only numerical rating. I could have asked more on a scale but this is the one that is good to see on a scale, the rest of the questions on typical session feedback aren’t so useful on a scale only. Even this one alone isn’t terribly useful if you want to get better.

“What is the best idea from this session that you plan to use?” - It’s why we speak. We want to help people learn a skill, behavior or grow in some way. As a speaker, this lets me see if the goals I had when setting out on this presentation were met. It also lets me see if there is a trend towards one point and maybe I can split that off into its own talk.

“What Could be done to improve the overall quality of score?” - There is probably a better way to ask this. Maybe even adding a question (our session feedback sheets were half sheets, to save on budget and paper) about “what idea will you probably never use?” But anyway, back to this question. This is gold for me as a speaker. There may be some tough things to read here. That is fine! I want to be better each time I speak. I want to get my ideas out better each time. This question allows an otherwise “polite” person to have an excuse to be constructive. It is begging them to write something other than “Great job!” or “Nice” which are useless comments. As you see in my feedback review from my SQL Saturday 71 talk, I learned a few things from answers here.

“Overall Comments” – I’ve heard that some people like to be verbose ;-) . This allows them to be. Maybe something else was on their mind that wasn’t covered by a question.

My Point

Just a few questions and the answers to them do a lot more to make me a better speaker than 6 scaled questions with optional comments that never seem to really provide much constructive feedback.  Please start asking questions like that. In the meantime, I’ll enjoy reading Aaron and Allan’s posts ;-)

Updated on 6/15/11 – Apparently this is Buck’s point also: Buck Woody had much the same thoughts here almost a month after I wrote this. He isn’t one of the two readers to this blog, so he came to his thoughts independent of my own journey. He also shared a great sample evaluation he’d like to see asked in his blog post on session/training feedback. Check it out!

Share

Tags: , , ,

6 Comments

Leave a comment
  1. WayneS May 17, 2011 at 12:10 #

    Excellent! At SQL Rally, it seemed like the evaluation forms were encouraged to give everyone an “Excellent” / 5 score on all the topics. I like this approach much better. (May I use it for the presentations I give?)

    • Mike Walsh May 17, 2011 at 12:15 #

      Hey Wayne -

      I actually didn’t get a chance to see the Rally form. I was only there 1 day, spoke in two of the slots and didn’t stay in a session long enough to evaluate one for the few in between. Since I really want to give my professional development talk at the Summit also, maybe I should be thankful that they are encouraging all 5s from your perspective ;-) Helps my chances of higher rankings influencing Summit session preferences ;-)

      You can definitely use it. In fact, use the idea and improve on it for me! I am sure there are better questions to ask that get to the same point.

  2. Steve Jones May 17, 2011 at 12:19 #

    Good post, and you make some good points. I’ve asked in the past to incorporate technology. Allow someone to send a text/email in to a specified address with comments, not ratings. Ultimately I think that having different ways to easily send feedback is good.

    However for people that like the session, it’s nice to have a quick way to say “good session, do it again” without typing that. So the rating makes sense there.

  3. Jack D Corbett May 17, 2011 at 14:33 #

    I’d definitely like to see better feedback for the speakers. A difference between SQLSaturday and other events, is that, in most cases the SQLSaturday organizers aren’t concerned with the speaker ratings, so those feedback forms can have more open-ended questions. For SQLRally and Summit, the speaker ratings are as much for the organizers as the speakers (I can’t speak for SQLBits), thus they need to be able to be compared.

    I’ve discussed this with Andy Warren several times, and we really haven’t come up with a good way to drive comments. We definitely want to add some text to say, “If you rank this below average please include a comment”.

    • Mike Walsh May 17, 2011 at 14:40 #

      Hey Jack -

      I agree with you. I was on the speaker review team for the 2011 Summit. I am just finishing up ranking speakers (not abstracts, just the speakers) and looking at the past feedback helps. But the ratings are still for two purposes, even at a Rally or SQL PASS – For the event to rank speakers (I mentioned that in the post, in fact) and still for the speakers to do both.

      I am not saying we eliminate all numerical ratings. I am saying we reduce them, analyze the ones that maybe don’t make sense and replace that space with some questions that actually give good information. Yeah, unfortunately we don’t have enough time when reviewing a speaker to read the answers to each question but the speakers can get the answers and improve as a result. Something that should also be a goal of PASS. We want to have great events that everyone wants to attend. Improving our speakers is a good way to make sure that happens.

Leave a Reply