Did it work
The room and the day

Did the training work? Finding out afterward, and changing the session

What a feedback form measures, what it does not, and the one measure of performance that is worth more than all of it. Then the change list, written while you still remember the minutes.

Start here Essentials

Decide what would count as evidence before the day, watch one thing, and write the change list while you still remember the minutes.

  1. Write down what you would accept as evidence it worked. Before the day, not after (section 1).
  2. Decide the one performance you will observe and record. One, chosen in advance (section 2).
  3. Write three feedback questions and label each with what it measures. Reaction, or something else (section 3).
  4. Fill in the after sheet within a day. While the minutes are still in your head (section 4).

Minimum time: about 20 minutes before the day, 25 minutes after. Done when: the sheets in After the workshop: what happened and what to change hold your planned and actual minutes per segment, one recorded observation of performance, and at least one change written for next time.

1. What a feedback form measures Understand

Almost every session ends with a form asking how it went, and almost everybody treats the answers as a verdict on whether it worked. The evidence on what those answers predict is clear enough to change how you read them.

The Quality Improvement Center for Workforce Development, a federally funded center, published an umbrella summary on trainee reactions in February 2022, drawing on meta-analytic evidence. What it reports:

  • Reaction surveys are the most commonly used type of training evaluation, and there are no universal or even widely used measures, so questions are written by each organization and what is measured varies a great deal.
  • Reactions are strongly connected to post-training motivation and moderately connected to self-efficacy, even controlling for pre-training levels.
  • Reactions are only modestly related to immediate knowledge outcomes and unrelated to delayed knowledge, and the connection with skill development is unknown.
  • Utility reactions have a modest connection to training transfer; affective reactions are not related to transfer.
  • Reactions are more positive when the trainer uses communication that creates connection, such as eye contact, moving around the room, using names, personal examples and humor; when there is more human interaction; and when trainees believe their employer values them.
Limits

It would be easy to read this as "do not ask people what they thought". That is the wrong conclusion. Reactions measure something real and worth having: motivation and self-efficacy in their own right, and the summary itself says they are useful for process evaluation. The error is treating them as a measure of learning or of skill, which the evidence does not support. Ask them, and label the answer correctly. Note also that this is a summary of meta-analytic evidence from workplace training research; it is not a measurement of your Saturday class.

The last bullet is the uncomfortable one. Warmth, movement, humor and using people's names raise reaction scores. They are good things to do and they do not tell you whether anybody can perform the skill. A session that scores 4.8 out of 5 and leaves nobody able to do the thing is a possible outcome, and so is the reverse: a demanding session where people struggled and learned, scoring lower.

2. One observation of performance

So what to measure instead. The cheapest honest answer is one observation, chosen before the day and recorded.

Pick the objective that matters most and decide what you will watch for and how you will record it. Three forms:

  • A count from a show-me. Everybody holds up their work and you count how many have the feature. "Seven of ten had the seam underneath." It takes fifteen seconds and it is a number.
  • A count from a final check. The last check of the session, which Checks for understanding: finding out whether it landed builds, is already a count; write it down rather than letting it evaporate.
  • A note of what you actually saw. Not a judgment. "Four people could not get the cartridge out without help; three of them had the wrong size wrench." That sentence is worth more than any score, because it names a cause you can act on.

The "what they could do" sheet in After the workshop: what happened and what to change takes one row per objective, with how you checked, how many could do it, out of how many, and what you saw. It uses a convention worth knowing: a blank in the count column reads as unknown rather than as zero, and the totals row says how many objectives you left unanswered. That matters because the honest answer for most sessions is that you checked one or two objectives and not the others, and a sheet that let you write zero would turn "I did not look" into "nobody could".

3. Writing three questions and labeling each

Three questions is usually the right number for a form people will actually fill in. The discipline is to label each one with what it measures, for yourself, in the sheet.

QuestionWhat it measuresWhat it does not tell you
How useful was this session? 1 to 5A utility reaction. Modestly connected to transferWhether anyone learned anything
How likely are you to try this in the next week? 1 to 5Post-training motivation, which reactions predict stronglyWhether they can do it, or whether they will
How confident do you feel doing this alone? 1 to 5Self-efficacy, which reactions are moderately connected toAbility. Confidence and competence come apart in both directions
What is the one thing you would change about the session?Process information for you, which is what reactions are good forAnything about outcomes
What was the muddiest point?Where your teaching was unclear, in their wordsWhether the clear parts landed
Write down the first thing you will do with thisAn intention, and a useful closing activity in its own rightWhether they did it

Two of these are worth more than a scale for a short practical session. "What is the one thing you would change" gives you a change list. And the Minute Paper's second question, from the University of Connecticut's list of 50 classroom assessment techniques, is the strongest single question in the subject: what important question remains unanswered? It costs a minute at the end and produces a list of what to fix, written by the people who were there.

The QIC-WD summary gives one piece of advice worth following literally: decide the purpose of the survey before writing its questions. A form written without a purpose becomes a habit, and a habit nobody reads.

4. The after sheet, filled within a day

The single highest-value thing in this chapter is filling in the "what happened" sheet within a day of teaching.

It takes one row per segment from your run sheet, with planned minutes and actual minutes side by side. Both columns total, so the gap between the plan and the room is visible at the foot. That gap is the buffer figure that Workshop timing: minutes on paper and minutes in the room declines to give you, derived from your own session rather than from a convention.

Why within a day. The memory of how it felt lasts for weeks; the memory of where the minutes went is gone by Monday. The feeling is the part you least need, because it is the part most influenced by whether people were warm to you. The minutes and the specific moments are what change the next version.

Three things to write while they are fresh, beyond the minutes:

  • The question you could not answer. And whether you sent the answer round.
  • The step where more than one person got stuck. With what they did instead of the right thing, which goes straight into the errors sheet in Demonstration and practice sheets.
  • What you cut, and whether anybody noticed. Usually nobody does, which is information about what belongs in the session.

Then the changes sheet: one row per change, with why, and where in the plan it goes. The "where" column matters because a change with no place in the plan is a feeling, and it will not survive the next time you are busy.

5. The four levels, as their owner states them Go deeper

Somebody will mention Kirkpatrick levels. The model is owned and published by Kirkpatrick Partners, and their page states the four levels as follows:

  1. Reaction. The degree to which the audience finds the experience favorable, engaging, relevant and supportive.
  2. Learning. The degree to which they acquire the intended knowledge, skills, attitude, confidence and commitment.
  3. Behavior. The degree to which they perform the critical behaviors in their environment.
  4. Results. The degree to which targeted organizational outcomes occur.

The same page states that whenever applying the model it is critical to start with Level 4, Results, and describes the model as cyclical rather than linear.

Convention Two honest observations about using it for a two-hour workshop. The instruction to start at Level 4 assumes an organizational outcome somebody is accountable for, which many of the sessions this manual is written for do not have: a bread class has no targeted organizational outcome, and pretending otherwise produces a made-up one. And this is a definition published by the model's owner, which is the right source for what the model says and not an independent assessment of whether the model predicts anything.

What is useful for a session like yours is the distinction between the levels, particularly between 1 and 2. Most sessions measure Level 1 and report it as if it were Level 2. Measuring one observation of performance, as section 2 asks, is a small piece of Level 2, and it is far more than most sessions collect.

6. Following up weeks later Go deeper

Almost nobody does this, and it is where the interesting information is. The reaction evidence says reactions are unrelated to delayed knowledge, which means the thing a form at the end cannot tell you is exactly the thing you most want to know: did any of it survive the week?

Two cheap versions:

  • One question, three weeks later. "Have you done it since? Yes or no, one word is fine." Sent to the same channel the pre-session question went to. The reply rate will be low and the answers are real.
  • Ask for a photograph. For anything that produces an object, "send me a picture of your first one at home" produces evidence, and it produces it from people who are pleased, which is a bias worth naming.

Record whatever comes back in the "what they could do" sheet as a separate row, so that it sits beside what you observed on the day. The gap between the two is the retention problem that Showing someone how, so they can see it describes from the Peyton trial, where skills retention was equally low for both teaching methods. It is a real problem, it is not solved by teaching harder, and what tends to help is the thing people take home and what they do in the first week.

7. Revising from records rather than from the loudest comment Go deeper

After a session, one comment will stick in your head. It will usually be a negative one, and it will usually be from a confident person. Revising the session around it is the most common way a good session gets worse.

Three rules that help:

  • Count before you change. One person said the demonstration was too fast. How many people got it wrong at the check? If the count was fine, the comment is about that person; if the count was low, it is about the demonstration.
  • Prefer what you saw to what you were told. Your note that four people used the wrong wrench is evidence. A comment that "the tools were confusing" is a hypothesis about it.
  • Change one thing at a time. If you change the demonstration, the practice and the handout at once, the next session tells you nothing about which change helped.

By the third or fourth run of a session you have something most people teaching do not have: a record of planned and actual minutes, a record of what people could do, and a list of changes with their reasons. That is what makes the difference between teaching a session four times and teaching it better four times.

8. See it See it

Fictional example

Dale's fourth run, and what he changed. Eight people, ninety minutes, eight stations, no helper.

His form, as it used to be: five questions on a 1 to 5 scale. Average 4.6. He had been running the session on that number for three sessions.

What he recorded instead, on the fourth run: a count from the final show-me. Six of eight got the cartridge out and back in without help. Two needed help, both at the same step.

Known: eight people, the count of six out of eight, and his minutes: the second practice block ran nineteen minutes against a planned fifteen.

What he saw: both of the two who needed help had the wrench at the wrong angle, which he had not seen in previous sessions because he had not been counting.

His three questions: how likely are you to try this at home this week (labeled: motivation); what is the one thing you would change (labeled: process information for me); what question remains unanswered (labeled: what to fix).

What came back: motivation high, one person wanted more time, and three people asked the same unanswered question about older faucets.

Assumed, and whose assumption it is: Dale assumes the wrench angle is a demonstration problem rather than a tool problem. That is his assumption, and the way he tests it is to change one thing: next time he demonstrates that step from the side and counts again. If the count improves, the assumption held.

What he wrote in the changes sheet: one row. "Demonstrate the wrench angle from the side. Why: two of eight had it wrong and I only saw it because I counted. Where: demonstration breakdown, step 4."

Open, and who settles it: whether to add a segment on older faucets, which would cost five minutes he does not have. His own next run settles it, and the question is on the not-today list until then.

9. Practice Practice

Minimum: one observation chosen in advance and recorded, and the minutes written down within a day. Time: about 20 minutes before, 25 minutes after. Materials: the sheets in After the workshop: what happened and what to change.

  1. Write your evidence sentence before the day. "I will believe this worked if ___ out of ___ can ___." Put it in the "what they could do" sheet as a row with the count blank.
  2. Decide how you will record the count. A show-me, the final check, or a note. Write it in the "how you checked" column now, so that you do not have to invent it at the end.
  3. Write three questions and label each. What it measures, in the label, for your own reading of the answers.
  4. Within a day, fill the minutes. One row per segment, planned and actual, and look at the total gap.
  5. Write at least one change, with its why and its place. One row in the changes sheet. One change is worth more than ten, because you will actually make it.
Go deeper

Three weeks after the session, send one question to the same channel you used before it: "Have you done it since?" Record the answers as a second row in the "what they could do" sheet, labeled as the follow-up, and leave the count blank if too few people reply to call it a count. The gap between what people could do on the day and what they have done since is the most useful number in this manual, and almost nobody collects it.

10. Your session Your session

11. Check your work Check your work

Criteria

  • Did you write what would count as evidence before the day rather than after?
  • Is there one recorded observation of performance, with a count or a specific note?
  • Is each feedback question labeled with what it measures?
  • Are planned and actual minutes recorded per segment, within a day?
  • Is there at least one change written with a why and a place in the plan?

Worked example

Dale wrote his evidence sentence before the session, recorded a count of six out of eight, and noticed a cause he had missed for three runs because he had not been counting. His questions are labeled, which changes how he reads the high motivation score: as motivation rather than as proof the session worked. His minutes are in and the gap is four minutes on one block. His change sheet has one row with a why and a place. Five criteria met. What he has not done is the follow-up three weeks later, which is where the retention question lives.

Common mistakes

  • Reading a high score as proof it worked. Reactions predict motivation and self-efficacy, and are unrelated to delayed knowledge.
  • Deciding the evidence after the session. You will choose the evidence that is available, which is the form.
  • Writing zero where you did not check. Blank reads as unknown, and the difference matters.
  • Filling the sheet a week later. The minutes are gone and the feeling is all that is left.
  • Changing five things at once. The next run then tells you nothing about which change helped.
  • Revising around the loudest comment. Count first; one person's experience may not be the room's.

What is still unresolved

This chapter cannot tell you whether a single session changes what people do at home or at work, and neither can the sources behind it: the reaction evidence is explicit that the link to skill is unknown, and the retention finding in Showing someone how, so they can see it is sobering. What the session controls is what happens in the room and what leaves it, which is why Slides, handouts and materials: what you give them and when treats the job aid as the deliverable. The changes you write here go back into Structuring a session: the run sheet and Workshop timing: minutes on paper and minutes in the room for the next run.

12. Knowledge check and scenario

What do reaction surveys predict, and what do they not?
They are strongly connected to post-training motivation and moderately connected to self-efficacy, even controlling for pre-training levels. They are only modestly related to immediate knowledge, unrelated to delayed knowledge, and their connection with skill development is unknown. Utility reactions have a modest connection to transfer; affective reactions do not. So ask them, and label the answer as motivation rather than as learning.
Why does the "what they could do" sheet treat a blank as unknown rather than zero?
Because most sessions check one or two objectives and not the others, and writing zero would turn "I did not look" into "nobody could". The totals row reports how many objectives were left unanswered, which is itself a useful number: it tells you how much of your own session you have evidence about.
Why fill the minutes in within a day?
Because the memory of where the minutes went is gone by Monday while the memory of how it felt lasts for weeks, and the feeling is the part most influenced by whether people were warm to you. The minutes are what change the next version, and the gap between planned and actual totals is the buffer figure for the next run, derived from your own session rather than from a convention.
(Go deeper) What does the Kirkpatrick page itself say about where to start?
It states that whenever applying the model it is critical to start with Level 4, Results, and describes the model as cyclical rather than linear. That instruction assumes an organizational outcome somebody is accountable for, which many community and volunteer sessions do not have, and inventing one produces a measurement of nothing.
Your feedback forms average 4.7 out of 5 and your final show-me count was four of twelve. What do you conclude?
This is the reading the evidence does not support. A high reaction score is strongly connected to motivation, not to whether anyone acquired the skill, and the count is the measure closer to the thing you taught. It is possible the check was badly written, which you can test by asking it differently next time, and four of twelve is a result to investigate rather than to explain away.
Both numbers are real and they measure different things. High motivation is worth having, and the count says most people left unable to do it. The two places to look first are the demonstration, where a count below 35 percent often points at a sightline or a merged step, and the practice arithmetic, which may be giving people one truncated attempt.
The form is measuring something genuine and the error was in how it was being read. Keep it, label the questions, and add the count beside it. A form plus one observation is much stronger than either alone, because the two together distinguish a session people liked from a session that taught them something.

Checklist

Records your learning only; it is not a qualification.