What "Works in Practice" Actually Means: The Four Data Points DOJ Monitors Are Looking For

This piece is for the CCO sitting across from that reality asking a specific, practical question: What does "works in practice" actually look like as a document I can hand to a monitor? The answer is more specific than most compliance professionals realize — and more demanding than most current programs can deliver.

When the DOJ updated its Evaluation of Corporate Compliance Programs in 2024, it reinforced a standard that has been quietly evolving across enforcement actions for the better part of a decade. The three-part framework — is the program well-designed, is it adequately resourced, and does it work in practice — is familiar to most compliance officers. The third criterion is where the conversation usually stalls.

"Works in practice" sounds like a qualitative judgment. It is not. Across consent decrees, deferred prosecution agreements, and enforcement negotiations of the past several years, a pattern has emerged in what monitors actually request when they are evaluating whether a compliance program produces real behavioral outcomes.

Most organizations can produce none of them on demand. Some can produce one. The gap between what regulators are looking for and what compliance programs are designed to generate is the single most significant unaddressed risk in corporate governance today.

Most organizations can produce none of them on demand. Some can produce one. The gap between what regulators are looking for and what compliance programs are designed to generate is the single most significant unaddressed risk in corporate governance today.

This is the most fundamental misunderstanding in how compliance programs approach remediation documentation. A culture assessment conducted once, at a single point in time, is legally almost useless. It tells a monitor where you are. It does not tell them where you were, what changed, or whether you caused the change.

What monitors are looking for is trajectory — measurable improvement across a sustained period that began before the incident or enforcement action, not after. The before/after structure matters enormously. A compliance program that suddenly produces excellent culture scores six months after a consent decree looks different in a negotiation than one that shows consistent upward movement beginning eighteen months before the decree was signed.

The evidentiary standard here is a dimensional score tracked quarterly across at minimum four data points, anchored to a documented baseline, with a margin of error disclosure. Each dimension must be tied to a validated behavioral construct — not a proprietary label that a monitor cannot independently evaluate.

The four dimensions that map most directly to what monitors are examining are psychological safety, organizational belonging, organizational trust, and feedback openness. These are not arbitrary categories. They are each anchored to peer-reviewed research with decades of validation behind them: Edmondson's psychological safety framework (1999), Mayer, Davis and Schoorman's organizational trust model (1995), Allen and Kern's belonging research (2017), and London and Smither's feedback openness construct (2002). When you present dimensional scores in a remediation report, those academic anchors are what make the numbers defensible rather than self-serving.

Without them, opposing counsel will argue — correctly — that your scores are arbitrary.

This is the dimension that separates organizations with genuine compliance infrastructure from those with compliance theater. Pre-incident behavioral evidence is documentation that demonstrates the organization was actively monitoring culture health before an incident occurred — and that employees were demonstrably safer speaking up, raising concerns, and flagging ethical ambiguity than they were in whatever prior environment gave rise to the enforcement action.

The challenge is obvious: most organizations only start measuring culture after something goes wrong. At that point, the data they generate is post-incident remediation documentation, which carries weight but is not the same as demonstrating that the organization had systems in place before the problem emerged.

For organizations that are currently in remediation, this means the clock matters. Begin generating behavioral evidence now, document your baseline carefully, and understand that the trajectory you are building today becomes your most important pre-incident evidence if a future issue arises. The documentation you create during voluntary monitoring is often more valuable than any single score. For a deeper look at what evidence of remediation requires, see our dedicated analysis.

Psychological safety data is particularly significant here. An organization that can demonstrate — through aggregated behavioral indicators, not surveys — that employees felt genuinely safe raising concerns before an incident occurred has a materially different legal posture than one relying solely on the existence of a hotline.

This category is the most legally dangerous one to get wrong, and the most commonly misunderstood. Any early warning system — any mechanism that identifies potential culture or conduct risk before it escalates — creates legal notice. Notice means the organization knew, or should have known, that a risk existed. Notice without a documented response is not neutral. It is affirmatively damaging.

The remedy is not to avoid early warning systems — they are valuable and regulators view their presence favorably as evidence of proactive program design. The remedy is to treat every alert as a legal record that requires a documented resolution chain. For every risk signal identified: what was the specific trigger, which dimension of organizational health was affected, when was HR or compliance leadership notified, what action was taken, and what was the documented outcome?

A complete alert-to-resolution log, maintained contemporaneously and available for monitor review, converts an early warning system from a liability into one of the strongest pieces of evidence in a remediation package. Seven alerts, all formally resolved, with documented HR action and outcome verification, is a more compelling demonstration of a functioning compliance program than zero alerts — which simply signals that the monitoring system is not sensitive enough to detect risk.

The question a monitor asks when reviewing an early warning log is not "did you have problems?" It is "when you identified a problem, what did you do about it?"

The fourth data category is about the integrity of the evidence itself. As behavioral assessment becomes more common in compliance — and it will, because regulators are demanding behavioral outcomes — the methodologies used to generate that evidence will face the same scrutiny applied to any expert testimony in a legal proceeding.

Three questions will be asked of any behavioral assessment methodology presented in a remediation context. Organizations and their vendors should be prepared to answer all three before the data ever appears in a compliance filing.

First: what validated constructs underlie your measurements? "Proprietary methodology" is not an answer. It is an invitation to challenge. The specific dimensions being measured must be anchored to peer-reviewed research with academic consensus behind it. If they cannot be, they cannot be reliably defended.

Second: what is the benchmark, and how was it established? A score of 78 out of 100 means nothing without context. Against what norm? Compared to which organizations? A score anchored to an industry benchmark database — with the methodology for constructing that benchmark available for review — is defensible. A score that exists only in reference to the client's own prior data is far weaker.

Third: is the methodology available for independent audit? The answer must be yes. Not because independent auditors will necessarily be appointed — but because the willingness to subject a methodology to scrutiny is itself evidence that the methodology has nothing to hide. Organizations that have built compliance programs around unauditable black-box assessments will face an increasingly difficult time as regulatory expectations continue to evolve toward outcome-based evaluation.

Assembled correctly, these four categories of evidence produce a documentation package that a CCO can hand to legal counsel with confidence — not certainty that every regulatory requirement has been met, which is a legal determination that belongs to counsel, but confidence that the organization has done what it can be expected to do to demonstrate genuine remediation.

The package has a specific structure. It opens with a data governance statement — confirming that any behavioral data collected was gathered with appropriate employee disclosure and is presented in aggregated, anonymized form. It proceeds to dimensional trend data across a meaningful time period, with baseline documentation, benchmark reference, and margin of error disclosure. It includes a complete alert resolution log if any early warning signals were generated during the period.

It maps documented evidence to specific regulatory frameworks — the ECCP "works in practice" standard, EEOC psychological safety thresholds, state-level DEI reporting mandates — with careful, precise language that describes what has been generated rather than making legal conclusions about what has been satisfied. And it closes with full methodology disclosure, including the academic anchors for each dimension and a statement of availability for independent review.

What this package is not: a legal filing, a finding of regulatory compliance, a guarantee against enforcement action, or a replacement for qualified legal counsel. The determination of whether any specific regulatory requirement has been met rests with attorneys, not with the documentation system that generated the evidence. What it is: the most complete behavioral evidence package currently available to a CCO seeking to demonstrate, in concrete and auditable terms, that their compliance program produces real behavioral outcomes — which is precisely what the DOJ's "works in practice" standard is asking for.

The compliance programs that will define the next era of governance leadership are not being built around policies or training modules. They are being built around evidence — continuous, behavioral, trajectory-based evidence that begins before an incident, responds visibly to risk signals, and can withstand the scrutiny of a monitor, a plaintiff's attorney, or a federal judge.

This requires a different kind of infrastructure than most organizations currently have. It requires assessment methodologies anchored to validated research. It requires monitoring systems that generate audit trails, not just alerts. It requires documentation practices that treat every data point as a potential exhibit rather than an internal management tool.

The organizations that invest in this infrastructure now — before they need it — will have a material advantage in any future enforcement negotiation, regulatory examination, or litigation context. They will also, and this is worth stating plainly, be building genuinely healthier workplaces. The metrics that matter most to a DOJ monitor — psychological safety, organizational trust, belonging, feedback openness — are the same metrics that predict employee retention, reduce misconduct escalation, and create the conditions under which people do their best work. This alignment is at the heart of the Human Risk Governance Framework.

The compliance case and the culture case are the same case.

The Emerald EI Framework described in this article — including the four evidentiary categories, the dimensional construct anchors, the benchmark floor methodology, and the alert resolution protocol — is operationalized in the Culture Pulse Remediation Report™, generated by the Kaya Platform™. The report was developed by working backward from the questions a DOJ monitor and a defense attorney would ask, stress-testing every element against the scrutiny it would face in a legal or regulatory proceeding.

The platform does not determine whether any regulatory requirement has been met. That determination belongs to legal counsel. What it does is generate the behavioral evidence package that makes that determination possible — documentation that a CCO can hand to their attorney, a board, or a monitor with confidence that it reflects genuine organizational investment in behavioral outcomes, not procedural compliance theater.

This is the second article in Emerald EI Academy's governance intelligence series. The first piece — "The Human Risk Layer: Why Governance, Risk & Compliance Must Evolve Beyond Policies" — examines why compliance failures are fundamentally behavioral failures and why the regulatory landscape has shifted decisively toward outcome-based evaluation.