Conditional Access assessment: prove the access boundary
A Conditional Access assessment checks whether the right identities can reach the right Microsoft 365 resources under the conditions your organisation intends. A useful assessment combines policy review, a documented test matrix, carefully authorised sign-in checks and evidence of the resulting decisions. Counting enabled policies is not enough: a policy can exist while an important user, application or access route falls outside its coverage.
This guide gives tenant owners and security teams a practical way to prepare that assessment. It focuses on test design, evidence and acceptance criteria rather than a universal policy template. The examples are hypothetical and must be adapted to your licences, applications, device estate and operational requirements. They are not findings from a customer tenant, and no production policy changes are required merely to follow the planning steps.
Start with a business statement that can be tested: for example, finance administrators should use approved authentication from managed workstations; external collaborators should reach only their approved resources; ordinary staff should retain their agreed mobile working experience. Translate each statement into named identities, resources and conditions. Our Microsoft 365 security audit reviews these boundaries alongside email, permissions and device controls, so a successful sign-in is not mistaken for proof that the entire environment is secure.
Define the outcome before reviewing the configuration
Ask the service owner to describe permitted access in ordinary language before opening the policy list. Which people need access? Which information or administrative operations matter? Which devices and authentication methods are acceptable? What must remain available during an identity incident? Record any disagreement now. If IT and finance hold different expectations for remote access, a technically accurate configuration export will not resolve the underlying requirement.
Give each outcome an identifier and an accountable owner. A statement such as “protect privileged access” is too broad to assess consistently. A stronger statement identifies the administrator population, the administrative resources, the permitted working devices and the expected treatment of exceptions. Preserve the distinction between an agreed requirement, a proposed improvement and a mandatory contractual obligation. An assessor should not silently turn a preference into a compliance finding.
Define success and failure before the test runs. If an unmanaged browser is intended to be denied, denial is a successful control test. If a managed employee device is intended to work, denial is an availability defect even when it looks restrictive. Both results belong in the assessment. This keeps the report focused on reliable access decisions rather than rewarding the largest number of blocked attempts.
| Requirement | Positive check | Negative check |
|---|---|---|
| Privileged administration | Approved administrator completes an agreed administrative read operation | Equivalent test account cannot use an explicitly disallowed access context |
| Employee collaboration | Approved employee opens a synthetic document through the supported client | A disallowed context cannot reach that same protected resource |
| External collaboration | Approved guest reaches the designated test site | Guest cannot reach a separately restricted test site |
| Recovery access | Designated recovery procedure remains usable under agreed conditions | Ordinary accounts cannot acquire recovery privileges |
Build an inventory that explains coverage
Collect policy identifiers, names, states, assignments, exclusions, conditions, grant controls and session controls. Retain the collection time and tenant identifier with the export. A screenshot of a policy name is weak evidence because the name can imply protection that its settings do not provide. Keep an unmodified source export and create a separate working copy for analysis; redact account details before sharing extracts outside the assessment team.
Resolve referenced groups and applications rather than leaving opaque identifiers in the working inventory. Record how membership is maintained and who can change it. A well designed policy may become ineffective if the population it protects is maintained manually and new starters never enter that group. Conversely, a broad group change may affect unrelated users. Ask how joiners, movers, administrators and guests enter and leave the intended scope.
Include disabled and report-only policies in the review, clearly separated from enforced controls. They can explain a planned migration, an abandoned experiment or an assumption that never became operational. Do not describe them as active protection. For every apparent gap, check whether another enforced policy covers the same requirement before raising a finding. Policy count and naming conventions are useful maintenance signals, not evidence of effective protection by themselves.

Construct a risk-based Conditional Access test matrix
A full combination of every identity, device, application and network can become impractical. Build a deliberate sample around meaningful boundaries instead. Select representative ordinary users, privileged users, guests and any documented exception population. Choose the applications that hold sensitive data or support essential operations. Add client and device combinations that reflect how those people actually work, including legitimate arrangements that differ from the standard managed Windows laptop.
Each row should have one explicit purpose. Start with a known-good case, then change one condition where feasible. This helps distinguish an intended access decision from an unrelated client problem. A denied sign-in from an unconfigured test device tells you little if the account also lacks access to the target application. Validate prerequisites first and preserve enough context for another authorised reviewer to repeat the same case.
Give higher priority to boundaries with greater impact: administrative access, sensitive exports, external collaboration and exceptions. Document the combinations you did not test and why. A representative sample is acceptable when its rationale is visible; describing a small sample as complete tenant coverage is not. If a business-critical client cannot be exercised safely, mark that case as unverified and propose a supervised follow-up rather than assuming it behaves like a browser.
| Case field | What to record |
|---|---|
| Identity | Test account identifier, role and relevant group membership |
| Resource | Specific target application or resource and intended operation |
| Client and device | Client type, platform, registration and relevant compliance state |
| Network context | Agreed source context and any relevant named-location membership |
| Expected outcome | Allow, deny or require an additional agreed control |
| Evidence | UTC time, sign-in reference, applied policies and observed resource result |
| Disposition | Pass, fail, inconclusive or not tested; owner and next action |

Use simulation, observation and live checks for different questions
Microsoft’s What If tool estimates policy effects for specified sign-in conditions. It is useful for exploring cases before a supervised test, but it does not exercise application service dependencies. Treat its output as simulation evidence, not proof that an end-to-end Teams or other business workflow will function. Record the supplied inputs with the result so an omitted condition does not become an invisible assumption.
Use simulation to narrow the questions that need real observation. For example, compare an ordinary test user with an approved exception account under the same proposed context. If the difference is unexpected, investigate assignments before touching production. Preserve the policy state at the time of the simulation. A later result against a changed configuration cannot explain an earlier access event without that history.
Next, review existing sign-in evidence within the authorised assessment scope. Finally, where necessary, perform controlled sign-ins using designated accounts and synthetic resources. These are different evidence layers: a policy export shows configuration, simulation shows an estimated decision, and a controlled check shows observed behaviour in one context. A strong conclusion explains which layers support it and which remain unavailable.

Interpret report-only results without overstating protection
Microsoft documents report-only mode as evaluation without enforcement for supported policy scenarios. Its results can help estimate the effect of a proposed change. A report-only failure is not proof that the observed user was blocked; a report-only success is not proof that every other applicable policy allowed access. Keep the proposed policy’s result separate from the overall sign-in outcome.
For an assessment, ask the owner why a policy remains in this state and what evidence is needed before promotion. A policy under active evaluation should have an owner, a planned observation period and a decision date. One left indefinitely without review is a governance issue, but its severity depends on whether enforced controls already meet the requirement. Avoid equating every report-only policy with an exploitable security gap.
Do not turn an existing enforced policy into report-only merely to gather comparison data. Plan a separate proposed change through the normal approval process. Even observation requires care: Microsoft notes that some device-compliance report-only scenarios can produce certificate-selection prompts. Agree the test population and support arrangements with the tenant owner rather than assuming that all non-enforcing changes are operationally invisible.
Correlate the sign-in with the resource operation
Use a consistent evidence record for every controlled check. Capture the UTC start time, designated account, target resource, client, expected outcome and the operation attempted. Then locate the corresponding sign-in event and preserve its relevant policy results. Microsoft’s Conditional Access troubleshooting guidance describes using event details and correlation information to investigate unexpected outcomes. A matching timestamp alone is insufficient when several client requests occur together.
Record what the user could actually do after authentication. Opening a landing page, reading a protected document and completing a privileged operation are different observations. Use a harmless agreed operation that represents the requirement, such as reading a synthetic file in a dedicated test library. Avoid exporting real customer data to demonstrate that a policy boundary matters. If the application reports an error, determine whether access control or an unrelated dependency caused it.
Separate raw evidence from interpretation. Preserve the relevant event identifier and a restricted source record, then place a redacted explanation in the report. Include enough information to reproduce the reasoning without exposing tokens, cookies, personal correspondence or unnecessary employee identifiers. If logs are unavailable for the test window, state the limitation. Do not fill the gap with an assertion that the policy must have applied because its settings look correct.

Test exclusions as access paths with owners
An exclusion is not automatically a defect. It may support recovery, a migration or a carefully constrained business dependency. The assessment question is whether its purpose, population, compensating controls and review date are explicit. Compare the documented reason with actual membership. A temporary exception used by many ordinary accounts deserves a different conclusion from a tightly controlled recovery arrangement.
For each significant exception, identify the protected requirement it bypasses and the remaining controls that still apply. Use simulation and approved test accounts to establish the difference rather than attempting to place a real employee into an excluded group. Ask who can approve membership changes, how changes are detected and what happens when the original justification expires. An exception register that nobody operates is not a control.
Our separate Conditional Access exclusions review addresses the inventory in more detail. In the assessment matrix, make exceptions visible as their own cases. This prevents them disappearing into an average result such as “most staff are protected” and gives the service owner a concrete choice: retain with evidence, narrow, replace or remove through an approved change.
Distinguish device assumptions from verified device conditions
Ask the endpoint team how a device reaches the state expected by the access policy. A policy that relies on managed or compliant devices needs an operational process behind that classification. Select test devices whose relevant states are known at the time of testing. Record the source of that knowledge and any recent changes, rather than labelling a laptop compliant because someone recognises its asset tag.
Include permitted nonstandard working patterns in the sample: mobile access, contractor equipment, supported macOS clients or browser-only arrangements where these exist. Do not assume that a successful Windows test represents every platform. Equally, do not introduce unsupported devices solely to create a failure. The test should answer whether the organisation’s documented operating model is implemented, with proposed improvements clearly identified as recommendations.
When a device-related result is unexpected, involve both the identity and endpoint owners. Record whether the discrepancy concerns the access policy, the device’s reported state, an assignment, an onboarding process or missing evidence. This keeps remediation actionable. For the endpoint side of the assessment, the Intune audit guide explains why device configuration and the evidence of its application need separate attention.
Include session context and client dependencies in the plan
A fresh browser sign-in and an already established session are not interchangeable test conditions. Record how the test started, whether the client had an existing session and what the assessor did to establish a clean comparison. Do not interpret the absence of a visible prompt as proof that no control was satisfied. Conversely, a prompt does not establish that the intended resource boundary is protected.
Agree which session behaviours are actually in scope. For example, the organisation may want to understand the experience after a planned policy change or a supervised access removal. Such tests need separate approval, expected timing and a clear rollback plan. Avoid disrupting ordinary employee sessions to make the assessment appear comprehensive. Use designated accounts and describe precisely what was observed, including the elapsed time and client context.
For applications with several dependencies, test the agreed business operation rather than stopping at the application launcher. A collaboration client may need several resources to complete an ordinary task. If only the initial sign-in was tested, say so. This is especially important when a proposed policy appears safe in simulation but the user journey later fails at a resource that the original test did not include.
Check the recovery design before proposing stronger enforcement
Any recommendation that could change administrative access must account for recovery. Ask the tenant owner to demonstrate the documented recovery process under controlled conditions and explain how the designated credentials are protected and monitored. The assessor should not receive unnecessary recovery secrets. Evidence can include a witnessed procedure, authorised ownership records and a supervised check with sensitive details withheld from the report.
Keep the recovery case distinct from ordinary administration. Its purpose is to preserve a viable response to a failure of the normal access path, not to offer convenient daily access. Record the people authorised to invoke it, how invocation is detected and what review follows. Assess whether the proposed change would accidentally include or disable this route, and require the owner to resolve that risk before deployment.
An assessment can identify a missing or unverified recovery arrangement without performing a destructive lockout experiment. Do not deliberately break production access to prove the point. Use a documented review and safe rehearsal, then state what remains untested. The practical objective is a change plan that improves protection while leaving the organisation able to recover from mistakes and dependencies outside its direct control.

Write findings that an administrator can act on
A useful finding starts with the requirement and affected population, then explains the observed difference. Avoid titles such as “Conditional Access misconfigured” without identifying the boundary. A better hypothetical title is “Approved contractor population omitted from the sensitive-resource access requirement.” The body should identify the supporting evidence, tested scope, business consequence, limitations and proposed correction.
Do not claim a breach from a configuration weakness alone. If the assessment observed unintended access to a synthetic resource, describe that observation. If it found a plausible gap without permission to validate it, say that it remains unconfirmed. Severity should reflect the resource, privilege, exposure and existing controls, with confidence stated separately. This produces a more credible prioritisation than treating every deviation as critical.
| Finding component | Example of useful content |
|---|---|
| Expected boundary | Named contractor group should meet the approved device requirement for a specific resource |
| Observed difference | Designated test identity reached the synthetic resource under a disallowed device context |
| Evidence | Policy export reference, membership snapshot, sign-in event and resource-operation record |
| Limits | One client and resource tested; other combinations remain unverified |
| Remediation | Owner reviews the assignment and dependent workflows before an approved change |
| Closure | Repeat the negative case and the legitimate positive case against the changed configuration |
Include operational consequences in the recommendation. Correcting an assignment may require endpoint onboarding, user communication or an application owner’s acceptance. Name those dependencies so the ticket does not bounce between teams. A recommendation that says only “enable the policy” leaves the difficult work undefined and encourages hurried changes that the assessment should help prevent.
Turn the assessment into a staged remediation plan
Group recommendations by the access boundary they improve, not just by policy name. Several findings may depend on the same device onboarding work; one policy change may affect several business services. Identify these relationships before assigning deadlines. Keep urgent exposure reduction separate from longer-term simplification, and explain any temporary measures with an owner and expiry decision.
For each proposed change, record the baseline, intended difference, pilot population, validation cases, rollback authority and evidence needed for approval. Use a representative pilot that includes legitimate edge cases rather than only the IT team’s standard laptops. Ask service owners to confirm their critical workflow checks. A technically successful deployment can still be incomplete if a required client or guest journey was never exercised.
Retesting should repeat both the original failing case and neighbouring legitimate cases. Closing an unintended route while breaking normal access is not complete remediation. Preserve the changed configuration reference with the new evidence and update the exception register. If the organisation accepts a remaining risk instead, record that decision explicitly rather than marking the technical finding fixed.

What to request in the assessment deliverables
Request a concise executive explanation supported by a technical evidence pack. The executive section should explain which access boundaries were assessed, which material gaps were confirmed and which decisions require management support. The technical pack should contain the inventory, sampling rationale, test matrix, redacted evidence, findings and retest criteria. Keep sensitive raw exports in an agreed restricted location rather than embedding everything in an email attachment.
Ask the provider to separate completed tests from deferred checks and unavailable evidence. A report with a transparent coverage table is more useful than a percentage score without a denominator. It should also distinguish tenant configuration review from active testing, endpoint review and incident investigation. These activities overlap, but one does not automatically include the others.
Before accepting the report, select one finding and trace it from requirement to configuration, observation, recommendation and closure condition. If that chain is clear, your team can use the report to manage change. If it depends on unexplained screenshots or generic advice, request clarification while the assessment context is still fresh.
Prepare your Microsoft 365 Conditional Access assessment
Prepare the tenant context, relevant licence information, identity populations, sensitive resources, supported client types and known exceptions. Describe the business operations that must remain available and nominate identity, endpoint and application owners. You do not need to send passwords, session tokens or full employee exports through an enquiry form. Start with scope and objectives; agree secure evidence exchange and access arrangements before collection.
Use our Microsoft 365 audit scoping tool to outline the environment, or discuss a Conditional Access assessment with Atlant Security. We can help turn a broad request to “check MFA” into a defined review of identities, resources, exceptions and observed access decisions. The resulting scope should say what will be verified, what will remain outside the assessment and what evidence your team will receive.
This guide was reviewed on 25 October 2026. Microsoft interfaces, features and entitlements can change; confirm the applicable product documentation and tenant capabilities when planning the engagement. The test matrices and evidence structures above are practical assessment recommendations, not a Microsoft certification or a guarantee that every possible access path has been examined.
Published by Atlant Security. Sources, editorial policy and corrections.

