Microsoft Entra Conditional Access: A Practical Guide and Conditional Access Best Practices cover building policies. This article covers something different and just as important: reviewing a Conditional Access policy set that already exists, using tools like Entra's built-in What If tool — often built up over years, by different administrators, with no single person holding the full picture — for gaps, conflicts, and quiet drift.
Why existing policy sets need periodic review, not just initial design
A Conditional Access policy set rarely stays exactly as originally designed. New applications get onboarded with ad hoc exclusions "to get it working." A former administrator's temporary troubleshooting exception never gets removed. Policies designed independently, at different times, can interact in ways no single policy author intended. A structured review methodology catches this drift before an audit, an incident, or a new administrator discovers it the hard way.
A review methodology
| Check | What to Look For |
|---|---|
| Emergency access | At least one break-glass account, excluded from all policies, monitored, and recently tested |
| Legacy authentication | Confirm the block policy is still active and not weakened by a newer, conflicting exclusion |
| MFA coverage | Every user group and application actually covered — not assumed covered by an old baseline policy |
| Report-only policies | Any policy stuck in Report-only long-term — either enforce it deliberately or remove it, don't leave it indefinite |
| Named locations | Still accurate — office moves, VPN provider changes, and closed sites can leave stale IP ranges in policy |
| Exclusions | Every exclusion has a documented reason and an owner — an undocumented exclusion is an unreviewable standing exception |
| Session controls | Sign-in frequency and app-enforced restrictions still match current risk tolerance for sensitive applications |
| Policy conflicts | Multiple policies combining into an unintended requirement, or one policy silently negating another's intent |
Every exclusion is a standing exception
An exclusion added months or years ago to unblock a specific troubleshooting scenario, with no note explaining why, persists silently: it does not surface in any policy summary, and it outlives every administrator who inherits the tenant. Microsoft's own Conditional Access gap analyzer workbook is built around exactly this class of gap — it highlights sign-ins with no Conditional Access policy applied so that, in Microsoft's words, you can "ensure that there are no users, applications, or locations that were unintentionally excluded from Conditional Access policies." Treat every exclusion as a standing exception requiring the same justification-and-review discipline as any other access exception — see the same principle applied to standing privileged access in Privileged Identity Management and Privileged Access.
Detecting policy conflicts
Because Conditional Access combines every applicable policy's grant controls (see Conditional Access Evaluation Flow — "most restrictive wins"), a conflict isn't always a hard error; it's often a silent, unintended combination. Two practical detection approaches:
- Simulate specific sign-in scenarios (a specific user, application, and context) using the "What If" tool available in the Conditional Access admin experience, comparing the actual combined result against what was intended.
- Review policies grouped by the users/applications they target, not just individually — a conflict is easiest to spot when looking at everything that applies to one specific access scenario at once, rather than reading each policy in isolation.
Emergency access verification
A review isn't complete without actually confirming break-glass access works — not just that the account exists and is excluded from policy on paper.
- Confirm the account is excluded from every current policy, not just the ones that existed when it was set up.
- Confirm credentials are current, accessible to authorized personnel, and stored per Microsoft's emergency access account guidance and the practices described in Privileged Identity Management and Privileged Access.
- Test sign-in periodically (not during an actual emergency) to confirm the account functions as expected.
- Confirm monitoring/alerting on the account's activity is still active and routes to the right person.
Operational review cadence
| Trigger | What to Review |
|---|---|
| New application onboarded | Confirm it's covered by baseline policy or has an intentional, documented policy |
| Staff change (admin departure) | Review any exclusions or break-glass access tied to that person |
| Quarterly | Full exclusion list review — confirm every exclusion still has a valid reason |
| Annually | Full policy set review using the methodology above, including emergency access testing |
Common mistakes
- Reviewing policies individually rather than by scenario, missing conflicts that only appear when multiple policies apply together.
- Never testing break-glass access outside of an actual emergency, discovering a problem with it for the first time when it's actually needed.
- Treating Report-only mode as a permanent state for a policy, rather than a temporary testing phase with a defined decision point.
- No owner assigned to exclusions, making it unclear who can authorize removing one during a later review.
FAQ
How long should a policy stay in Report-only mode before a decision is made? A few days to two weeks is typically enough to observe real-world impact for most policies — leaving a policy in Report-only indefinitely without a decision defeats its purpose as a testing phase, not a permanent state.
Should every exclusion be treated as a finding? Not automatically — a documented, justified, owned exclusion (like the mandatory break-glass exclusion) is expected and correct. The finding is specifically an undocumented or unowned exclusion with no clear current justification.
Is the "What If" tool sufficient for detecting all policy conflicts? It's a strong tool for testing specific scenarios but doesn't automatically enumerate every possible conflict across a large policy set — a structured, scenario-grouped manual review remains necessary for comprehensive coverage.