CMP SERIES
CERTIFICATION MADE POSSIBLE
Eighty-two percent of respondents said they review the
data from event surveys and look for meaningful findings — which Thalheimer found troubling. “I think this
kind of non-specific review enables bias, doesn’t rely on
standards, and is non-directive about what to do,” he said.
And while 61 percent provide speakers with feedback,
nearly 40 percent don’t. Another revealing result: Fewer
than than half ( 42 percent) use the data to determine
if the organization has met its predetermined meeting
goals, which, Thalheimer said, indicates that “most
would seem not to have predetermined meeting goals.”
14 After you collect your conference feedback responses,
what do you do with the data? Select from the items below
those actions you routinely take as part of your post-
meeting review process. (Select as many as are true.)
820+180= 82% We review the data, looking
for meaningful findings.
700+300= 71% We use data to select session
topics for future meetings.
610+390= 61% We provide our speakers with
feedback.
520+480= 52% We use data to determine
the mix of session types for
future meetings.
460+540= 46% We create a report for our
senior stakeholders.
450+550= 45% We use data to select speakers
for future meetings.
420+580= 42% We determine if we’ve
met our predetermined
meeting goals.
310+690= 31% We rate our speakers.
230+770= 23% We use data to market our
services/products to poten-
tial vendors.
140+860= 14% We use data to create
educational programs to
help our speakers improve.
40+960= 4% Other
030+970= 3% We often do not look at the data.
What’s Not to Like About
Likert Scales?
“There is a ton of room for improvement” in the typical
conference feedback form, according to learning expert Will
Thalheimer, Ph. D., especially those using the popular Likert
scale. Favored by 87 percent of Convene’s survey respondents,
Likert-like scales provide a continuum of answer options, such
as “strongly disagree” to “strongly agree,” or 1 to 5.
But Thalheimer’s research indicates that people seem to prefer
questions that have more discernible answer choices as opposed to general Likert options. “I think when people see the
answer choices are more concrete, they know that the questions are better,” he said. “You see a typical smile sheet, and
people circle the same numbers all the way down. They’re not
really thinking and then they’re not really giving you good data.”
Thalheimer offered this question as an example: “How effective was the session in helping you comprehend the concepts
taught?” Respondents, he said, will have an easier time choosing between specifically worded options, such as:
› I still have significant confusion and/or blind spots.
› I gained a basic familiarity with the concepts taught.
› I gained a solid understanding of the concepts taught.
› I gained a deep and comprehensive understanding of the
concepts taught.
When survey respondents are only given the option to choose
along the Likert continuum, “there’s a little bit of fuzzy thinking
going on,” Thalheimer said. Providing specific answer choices
lends clarity to their thinking — and also sends the message
that their responses are more likely to be taken seriously.
Moreover, the results from this kind of answer bank offer better
insight to conference stakeholders. “When you get the results
back,” Thalheimer said, “you can see where you stand and say,
‘Okay, if basic familiarity was what we wanted and we got it,
that’s good. If that’s not enough — we really wanted people to
have a solid understanding — then how can we get there?’”
Although the Likert scale’s numeric tabulations make it easy
to rate sessions, Thalheimer questions the value of those
numbers. “If you get a 4. 1 between ‘strongly agree’ and ‘agree,’
you don’t know what to do with that, really,” he said. “Is it
good? Is it bad? If you have questions for which the answer
choice is distinguished better, you can actually decide on a set
of [session] standards before you get the data.”
Concrete answer choices can also help speakers. By offering
them a chance to review the survey questions and response
options well ahead of time, Thalheimer said, you’ll help give
them a better understanding of the outcomes you’re seeking
— and therefore prepare session content that is more aligned
with learners’ needs.
