Your Hiring Funnel Got Worse. Every Job Family Improved.

The chart that punishes improvement

Suppose a team recruits for two job families. In period one, it reviews 90 candidates for an easier-to-fill family and advances 45; for a harder-to-fill family it reviews 10 and advances one. The overall advancement rate is 46/100 = 46%. In period two, the easy family has just 10 reviews and six advancements, while the hard family has 90 reviews and 18 advancements. Both families improved: 50% to 60% and 10% to 20%. Yet the company-wide rate falls to 24%.

The calculation is not a bug. It is a change in composition. This is the kind of reversal described by Simpson’s paradox: subgroup trends and pooled trends can point in opposite directions. The research explains the statistical mechanism; the recruiting application here is an operational adaptation, not a published industry benchmark. Research on aggregation. ^{↗ ↗}

The three denominators that must not be blended

Before debating conversion, write down the unit of analysis. A candidate may have several applications. An application may enter several hiring stages. A requisition can be paused, reopened or reassigned. Mixing them can create false movement even when job mix stays constant. ^↗

A practical event ledger needs a stable application ID, requisition ID, job family, cohort-entry date, stage-entry timestamp and decision status. Do not quietly treat pending applications as rejected. For a fixed entry cohort, either wait until the decision window matures or show a separate pending category. ATS documentation distinguishes candidates from applications for exactly this kind of reporting discipline. Reporting units. ^{↗ ↗}

A mix-adjusted comparison without statistical theater

Use a simple fixed-weight comparison. Take the job-family shares from the baseline period and apply them to both periods’ within-family rates. In the worked example, baseline weights are 90% easy and 10% hard. Period-one standardized advancement is 0.9×50% + 0.1×10% = 46%. Period-two standardized advancement is 0.9×60% + 0.1×20% = 56%. That is a ten-percentage-point improvement at a constant mix, while the actual blended figure falls by 22 points. ^↗

This is a diagnostic decomposition, not a causal estimate. It does not control for candidate quality, different interview criteria or seasonality. It answers a narrower, useful question: how much of the headline could be explained by composition? ^↗

A decision table a VP can actually use

Publish both the observed total and the like-for-like view. Suppressing the observed total would hide the real workload and demand change; publishing only the total would obscure process performance. ^↗

A seven-field export that survives an audit

Ask analytics for application ID, job-family ID, cohort month, entered-stage date, decision date, current stage and decision outcome. Keep the export version and an explanation of how duplicate applications, withdrawals, reopens and internal transfers are treated. Freeze definitions before comparing periods. ^↗

Then choose a meaningful segmentation level. Job family is often more stable than individual requisition, but a family that combines internship and senior roles can still conceal a mix shift. Split only where the process genuinely differs; excessive slicing creates tiny denominators and noisy percentages. A practical minimum is to display counts alongside every rate and flag very small groups instead of pretending they are precise. ^↗

The counterintuitive KPI: stable weights, unstable reality

Fixed weights can improve comparability while hiding the fact that the organization now does mostly difficult searches. That is why the right report has two lenses: observed workload-weighted outcomes for capacity planning, and standardized rates for process diagnosis. Neither should replace the other. ^{↗ ↗}

A third lens can examine stage age. If 60% of applications are still awaiting a decision, an advancement rate is a provisional snapshot, not a final outcome. The professional move is to make uncertainty visible before leaders demand an explanation for a misleading trend. ^↗

The two-week verification sprint

Day 1–2: reconcile candidate and application counts; identify stage events and paused requisitions. Day 3–4: create stable job-family cohorts and a pending category. Day 5: calculate within-family and observed rates. Day 6: standardize to the baseline family mix. Day 7: sample ten decisions per family for process drift. Week 2: publish the paired metrics with the denominator dictionary and a short exception log. ^↗

If the standardized improvement disappears when you segment by seniority or location, record that result. The goal is not to rescue a flattering narrative. It is to learn which operational decision should change. ^↗

The sentence to put in the board deck

Use wording such as: “Observed advancement fell from 46% to 24% as hard-to-fill searches grew from 10% to 90% of reviews. Within both families, advancement improved; at the prior mix, the comparable rate would be 56%. This does not establish that hiring quality improved.” ^↗

That sentence is longer than “conversion down 22 points,” but it prevents a team from being punished for taking on more complex work. It also gives finance a better capacity story. Use the calculator below with aggregate synthetic counts; never use it to rank individual candidates. ^↗

Run the operational check

Use synthetic or aggregated inputs only. This local tool does not make individual hiring decisions.

Implementation and evidence guardrails

Before rollout, appoint an owner, define the decision the metric will inform, and record the unit of analysis. Check examples from at least two job families, keep a record of contradictory evidence and use accessible candidate communications. Never use these local calculators to make automated adverse hiring decisions. Re-evaluate after two process cycles; if the observed result contradicts the hypothesis, update the workflow rather than the narrative. ^↗

The decision worth changing this week

Choose one reversible process change, define success and potential harm, and retain the evidence that could prove the idea wrong. Good recruiting operations are measurable without pretending that every candidate can be reduced to a score. ^{↗ ↗}