How Good Is Your Team at Visual Threat Detection?
In this post, we've updated how we describe our approach to evaluating visual threat detection skills, incorporating a clearer evaluation framework and current data. This post was originally published in 2019 and has been substantially revised and republished in 2026.
Training needs to be more than a lecture or watching someone else complete a task. Effective training gives personnel a chance to practice a skill, and just as importantly, a way to measure whether they can actually perform it. That's where evaluation comes in.
There's a real difference between a trainee saying "I feel like my skill improved" and being able to say "my skill improved by 71%." A simple post-training self-report, where trainees rate their own skill improvement, only gets you the first kind of statement. It measures what trainees think they learned rather than what they actually learned.
A better approach is to test knowledge or skill directly, both before and after training, so improvement can be measured rather than assumed. This distinction matters more than it might seem: a program that presents information but never assesses whether trainees can actually perform a skill is arguably not training at all, just a presentation. We touch on this distinction between presenting information, educating, and true training in more depth in our post on instructional design.
This is where a widely used framework from training evaluation, the Kirkpatrick Model, is useful. It separates evaluation into levels: Level 1 measures reaction, what trainees thought of the training. Level 2 measures learning, what trainees actually know or can do differently afterward. Level 3 measures behavior, whether trainees actually apply what they learned on the job. Each level tells you something different, and relying on only one gives an incomplete picture.
Pre and Post-Tests
Knowledge is relatively easy to test: true-false or multiple-choice questions on course material work well, especially in an online format. Testing is harder in instructor-led courses, where time and class size are real constraints, but that doesn't mean it isn't worth doing. In our online programs, this knowledge testing is a formal requirement, not just an exercise: trainees must maintain a 70% average across exams in our Threat Awareness for Security Professionals program, and 80% in our Threat Awareness for Law Enforcement program, to complete the program.
The more valuable evaluation is testing skill improvement directly. Can trainees perform the task they were trained on better after the training than before? A pre-test is what lets you attribute any improvement to the training itself, rather than assuming it. If personnel already knew what they were doing going in, or didn't improve at all based on pre and post-test results, that's important information before spending more on that training again.
Skills Testing in in Structured Observation
A skills test needs to be a valid representation of the actual skill being trained, and testing observation-based skills can be harder, and more time-intensive, than testing most other law enforcement or security tasks.
For our instructor-led and online courses alike, we use video-based observation tasks that take 25 to 30 minutes to complete. Trainees are asked to identify a potential person of interest who may pose a threat to themselves or others, watching a video and answering a structured set of questions about what they observe. This task is tested more formally in the online format, where it's built directly into the platform, but the same underlying skills test is used in person as well.
Take a look at the video clip of a shopping area below.
Visual Threat Detection Video Clip
Trainees watch a 3 to 4-minute video like this one, at normal speed, and answer questions covering three things:
The pattern of consistent behavior in the location.
The extent to which specific individuals' behavior deviates from that pattern, and how suspicious those deviations appear.
Whether any of those individuals should be identified as a person of interest requiring further observation.
Rather than a single before-and-after comparison, our current programs test trainees on this skill at multiple points throughout the course, letting us track improvement as it happens rather than relying on one pre-test and one post-test. These practical exams are also reinforcing; they help trainees better understand the techniques while they're being tested on them.
In our Threat Awareness for Law Enforcement and Threat Awareness for Security Professionals programs, we repeatedly test trainees on these skills through a series of practice exercises and practical exams. You can read more about our structured observation framework elsewhere on our blog, or explore our training directly.
What Trainees Report: Level 1 Reaction Data
Pre and post-testing tells us what trainees learned. It doesn't tell us how they felt about the training itself, which is its own useful signal, particularly for gauging whether a program is engaging and relevant enough that personnel take it seriously.
Based on end-of-program surveys of participants in our Threat Awareness for Security Professionals program, trainees agreed that the training left them better prepared to use visual behavior to identify a suspicious person (98.8%) and gave them new tools to help articulate their observations (98.1%). They also agreed it would assist them in their job (98.8%) and addressed the material in a way that stimulated critical thinking (98.6%). Participants in our Threat Awareness for Law Enforcement program report similar reactions in their own words, describing the training as giving them, for the first time, a way to articulate the "why" behind what they'd already been noticing on the job for years.
Reaction data like this is worth taking seriously, but it's still Level 1. It tells you whether trainees liked the training and believed it would help them. It doesn't, on its own, tell you whether they actually apply it once they're back on the job.
Some Test Results: Level 2 Evidence
In an earlier formal study, we conducted true pre and post-testing, not just testing throughout the course, with military personnel, police officers, and police recruits, specifically to measure this kind of improvement. In a sample of 40 police recruits, 10% successfully identified a person of interest on the pre-test. After training, 71% of that same group succeeded on the post-test.
Even among post-test participants who didn't correctly identify the person of interest, we saw a clear improvement in their ability to rule out individuals who weren't a threat and focus their attention on those displaying genuinely suspicious behavior.
That kind of improvement shows observation is a learned skill, one that improves measurably with a systematic approach and real practice, not something people either have or don't. This is Level 2 evidence: it shows what trainees actually learned to do, not just what they thought of the course.
From Learning to Application: Level 3 Evidence
The real test of any training is whether it changes what people actually do. We explored this directly in a follow-up study of 143 law enforcement and security professionals who completed our Threat Awareness in Practice refresher training, discussed in more detail in Recognizing and Responding to Threats in the Real World. That group reported improvements in their ability to identify suspicious persons (97.2%), identify threatening behavior (97.9%), and better articulate and explain their decision-making (96.5%) well after their initial training, with 97.2% saying the techniques had genuinely helped them in their security or law enforcement mission.
We've also seen this play out in a specific, real case. In Structured Observation in Action: Identifying a Concealed Weapon at a Protest, a trainee applied exactly this kind of structured observation, months after training, to identify a concealed weapon in a crowd of several hundred people. That's Level 3 evidence in its most concrete form: not a survey response, but an account of the training being used, successfully, in the field.
Taken together, Level 1, 2, and 3 evidence tell a fuller story than any one of them alone. Trainees report feeling better prepared. Testing shows they actually are more skilled afterward. And real-world cases show that skills hold up months later, under real conditions.
Wrapping It Up
Training is an investment of both money and time. Evaluating that training is how you make sure it's actually working, and how you protect that investment going forward. Our Threat Observation, Threat Awareness for Law Enforcement, and Threat Awareness for Security Professionals programs are all built around this kind of rigorous evaluation.
If you want to talk about training evaluation and making sure your personnel are learning what they need to, we'd like to hear from you.
-
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating Training Programs: The Four Levels (3rd ed.). Berrett-Koehler Publishers.