Training is one of the most common answers to a performance problem, and one of the least often evaluated. Organizations count courses and hours, but what matters is whether people can do the work, whether they do it, and whether the business result improves.
This guide covers how to check that training is the right fix, how to define and track competency, how to evaluate training with the Kirkpatrick four levels, and how to estimate return on investment honestly. A worked example follows an inspection training program.
Before You Start
Why Training Effectiveness Matters
Training Is a Large Investment
Time away from work, trainers, and materials add up. Without evaluation, an organization cannot tell what works.
Attending Is Not Learning
A completed course shows attendance. It does not show that anyone can do the task or that the task is now done differently.
Knowledge Fades Without Reinforcement
People forget a large part of what they learn if they do not use it. Follow-up and coaching decide whether training sticks.
Competence Is a Quality System Requirement
Many standards and customers expect an organization to define required competence and keep evidence that people have it.
Start With the Need, Not the Course
Before designing training, confirm that it is the right fix. If people are not doing something correctly, the cause may be unclear procedures, missing tools, poor feedback, or conflicting incentives, and training will not fix those. A short analysis asks four questions:
- What result do we need? State it in operational terms, such as fewer inspection escapes.
- What must people do differently? Define the observable behavior.
- Do they lack knowledge or skill, or is something else in the way? Use observation, a 5 Whys or a skills assessment.
- If it is training, what is the smallest effective intervention? Prefer job aids, on-the-job coaching, and standard work over long classroom courses where possible.
Define and Track Competency
A competency framework lists the knowledge, skills, and behaviors a role needs, with levels that describe what each looks like. A skills matrix then shows who can do what, so the organization can see gaps and single points of dependence.
| Level | Description |
|---|---|
| 0 | No knowledge of the task |
| 1 | Learning: can do it with supervision |
| 2 | Competent: can work alone to standard |
| 3 | Proficient: handles exceptions and problems |
| 4 | Expert: can teach others |
Use the Skills Gap Analyzer to compare people with required levels and spot coverage risks, and keep records in the Skills/Training Matrix Template.
Evaluating Training: The Four Levels
Donald Kirkpatrick's four-level model, developed from the 1950s and updated by Kirkpatrick Partners, is the most widely used framework for evaluating training. Jack Phillips later added a fifth level, return on investment.
| Level | Question | Typical evidence |
|---|---|---|
| 1 Reaction | Did participants find it relevant and useful? | Feedback forms, comments |
| 2 Learning | Did they gain knowledge, skill, or confidence? | Pre and post tests, practical demonstration |
| 3 Behavior | Do they apply it on the job? | Observation, audits, manager feedback after 30 to 90 days |
| 4 Results | Did the business result improve? | Defects, cycle time, safety, customer measures |
| 5 ROI (Phillips) | Do the benefits exceed the cost? | Monetized benefits compared with program cost |
Worked Example: Inspection Training
A plant trains 30 inspectors in a revised visual inspection procedure, because too many defects reach customers. The numbers are illustrative.
| Item | Value |
|---|---|
| Training cost (30 people, materials, trainer, wages during class) | $18,000 |
| Escapes to customers before training | 14 per quarter |
| Escapes 90 days after training | 8 per quarter |
| Reduction | 6 per quarter, 24 per year |
| Assumed cost per escape (returns, rework, sorting) | $2,500 |
| Annual benefit | 24 × $2,500 = $60,000 |
| Net benefit | $60,000 − $18,000 = $42,000 |
| ROI (net benefit / cost) | 42,000 / 18,000 = 233% |
Before claiming the full $60,000, the team asks whether anything else changed, such as a new gauge, a different supplier lot, or lower volumes. It also checks whether the behavior gain lasts. Follow-up audits at 90 and 180 days show whether the procedure is still followed, and coaching for the supervisors of the inspectors reinforces it. If part of the improvement came from another cause, the ROI is lower than 233%, so many organizations report a range or apply a conservative adjustment.
See the Training Reinforcement Guide for what happens after class, and the Train-the-Trainer Guide for building internal trainers.
Defining Levels of Competence
Saying that someone is “trained” is not the same as saying that they are competent. Competence is shown by doing the task to standard. A simple scale of levels, applied to each skill in a matrix, makes the difference visible.
Define each skill and level by observable evidence. “Independent” for a torque check could mean three consecutive observed performances to standard, with correct recording. The same rule should apply to everyone.
Decide who assesses. Assessors should be competent themselves, and trained in observing and recording. Where safety or quality depends on the skill, use a second person to verify.
Keep the matrix current. Colored squares of a skills matrix show coverage at a glance: which stations have enough people at level three, which skills depend on one person, and who is ready to move up.
Making Training Stick: Transfer and Reinforcement
Much training is forgotten within weeks if nothing supports it. Whether it changes behavior at work depends more on what happens before and after the session than on the session itself.
| Before | During | After |
|---|---|---|
| Agree the job-related goal with the trainee and manager; clarify how it will be used | Demonstrate, then have people practice with feedback on real tasks | Manager observes and coaches within days; job aids available at the workstation |
| Check prior knowledge to avoid wasted time | Use realistic scenarios and equipment | Follow up at 30, 60, and 90 days with observation and short refreshers |
Managers matter most. A trainee whose manager discusses the goals beforehand, observes the new skill, and gives feedback afterwards is far more likely to use it. Brief managers on what was taught and what to look for.
Evaluate at the right level. The four levels of the Kirkpatrick model, reaction, learning, behavior, and results, need different evidence. Check behavior through observation, and results through measures such as defects or rework on the trained task. Link evaluation to the original need.
Retire training that does not help. If observed behavior does not change, find out why. The cause may be unclear standards, poor tools, or conflicting incentives, and more training will not fix those.
Pitfalls. Counting hours of training as the measure; classroom-only methods for practical skills; no time to practice; ignoring language and literacy needs; and records that cannot show who is qualified for what. See the Train-the-Trainer Guide, the Training Reinforcement Guide, and the Skills Training Matrix Template. Figures in the examples are illustrative.
Self-Assessment Questions
- Did we confirm that training is the right solution to the performance problem?
- Do we define required competence per role, and record who has it?
- Do we evaluate beyond satisfaction, including learning and on-the-job behavior?
- Is there follow-up and coaching after training?
- Do we account for other causes before claiming the business result and ROI?
Common Mistakes
Counting Completions
Course completion is a compliance record, not evidence of competence. Add practical checks and observation.
Training for a Non-Training Problem
If the procedure is unclear or the tool is missing, training will not fix it.
Stopping at Level 1
Happy participants do not guarantee changed behavior. Plan for Levels 2 to 4 before the course starts.
Claiming All the Credit
Business results have many causes. Be honest about what else changed.
Competency and Training Effectiveness: Frequently Asked Questions
What are the Kirkpatrick four levels of training evaluation?
They are Level 1 Reaction (did participants find it relevant), Level 2 Learning (did they gain knowledge or skill), Level 3 Behavior (do they apply it on the job), and Level 4 Results (did the business outcome improve). Jack Phillips added a fifth level, return on investment, which compares monetized benefits with program cost.
How do we measure whether training changed behavior?
Observe the work or audit results 30 to 90 days after training, ask managers for feedback, and compare against the standard the training taught. Behavior change depends on reinforcement, so follow-up coaching and supervisor support are part of the measurement plan, not an extra.
How do we calculate the ROI of training?
Convert the measured benefit, such as fewer defects, into money using a stated cost per event, subtract the program cost, and divide the net benefit by the cost. Be honest about other causes of the improvement, and consider reporting a range rather than a single figure.
Sources and Further Reading
- Donald L. Kirkpatrick and James D. Kirkpatrick, Evaluating Training Programs: The Four Levels.
- Jack J. Phillips, Return on Investment in Training and Performance Improvement Programs.
- Robert F. Mager and Peter Pipe, Analyzing Performance Problems.
- ASQ Certified Manager of Quality/Organizational Excellence Body of Knowledge, sections on training and development.