Independent Microsoft 365 security assessment.Atlant Security
365/AuditBY ATLANT SECURITY
Build your scope Audit brief builder

Identity security

Conditional Access Assessment: A Test Matrix That Proves Coverage

Build a Microsoft 365 Conditional Access assessment with a practical test matrix, sign-in evidence, exception checks and safe remediation acceptance criteria.

Discuss your requirements
Enterprise identity checkpoints represented by glass gateways and a workplace device

Conditional Access assessment: prove the access boundary

A Conditional Access assessment checks whether the right identities can reach the right Microsoft 365 resources under the conditions your organisation intends. A useful assessment combines policy review, a documented test matrix, carefully authorised sign-in checks and evidence of the resulting decisions. Counting enabled policies is not enough: a policy can exist while an important user, application or access route falls outside its coverage.

This guide gives tenant owners and security teams a practical way to prepare that assessment. It focuses on test design, evidence and acceptance criteria rather than a universal policy template. The examples are hypothetical and must be adapted to your licences, applications, device estate and operational requirements. They are not findings from a customer tenant, and no production policy changes are required merely to follow the planning steps.

Start with a business statement that can be tested: for example, finance administrators should use approved authentication from managed workstations; external collaborators should reach only their approved resources; ordinary staff should retain their agreed mobile working experience. Translate each statement into named identities, resources and conditions. Our Microsoft 365 security audit reviews these boundaries alongside email, permissions and device controls, so a successful sign-in is not mistaken for proof that the entire environment is secure.

Define the outcome before reviewing the configuration

Ask the service owner to describe permitted access in ordinary language before opening the policy list. Which people need access? Which information or administrative operations matter? Which devices and authentication methods are acceptable? What must remain available during an identity incident? Record any disagreement now. If IT and finance hold different expectations for remote access, a technically accurate configuration export will not resolve the underlying requirement.

Give each outcome an identifier and an accountable owner. A statement such as “protect privileged access” is too broad to assess consistently. A stronger statement identifies the administrator population, the administrative resources, the permitted working devices and the expected treatment of exceptions. Preserve the distinction between an agreed requirement, a proposed improvement and a mandatory contractual obligation. An assessor should not silently turn a preference into a compliance finding.

Define success and failure before the test runs. If an unmanaged browser is intended to be denied, denial is a successful control test. If a managed employee device is intended to work, denial is an availability defect even when it looks restrictive. Both results belong in the assessment. This keeps the report focused on reliable access decisions rather than rewarding the largest number of blocked attempts.

RequirementPositive checkNegative check
Privileged administrationApproved administrator completes an agreed administrative read operationEquivalent test account cannot use an explicitly disallowed access context
Employee collaborationApproved employee opens a synthetic document through the supported clientA disallowed context cannot reach that same protected resource
External collaborationApproved guest reaches the designated test siteGuest cannot reach a separately restricted test site
Recovery accessDesignated recovery procedure remains usable under agreed conditionsOrdinary accounts cannot acquire recovery privileges

Build an inventory that explains coverage

Collect policy identifiers, names, states, assignments, exclusions, conditions, grant controls and session controls. Retain the collection time and tenant identifier with the export. A screenshot of a policy name is weak evidence because the name can imply protection that its settings do not provide. Keep an unmodified source export and create a separate working copy for analysis; redact account details before sharing extracts outside the assessment team.

Resolve referenced groups and applications rather than leaving opaque identifiers in the working inventory. Record how membership is maintained and who can change it. A well designed policy may become ineffective if the population it protects is maintained manually and new starters never enter that group. Conversely, a broad group change may affect unrelated users. Ask how joiners, movers, administrators and guests enter and leave the intended scope.

Include disabled and report-only policies in the review, clearly separated from enforced controls. They can explain a planned migration, an abandoned experiment or an assumption that never became operational. Do not describe them as active protection. For every apparent gap, check whether another enforced policy covers the same requirement before raising a finding. Policy count and naming conventions are useful maintenance signals, not evidence of effective protection by themselves.

Policy inventory represented by organised identity cards and translucent sheets
Resolve policy references into people, resources and accountable owners. Conceptual illustration, not a tenant screenshot or client finding.

Construct a risk-based Conditional Access test matrix

A full combination of every identity, device, application and network can become impractical. Build a deliberate sample around meaningful boundaries instead. Select representative ordinary users, privileged users, guests and any documented exception population. Choose the applications that hold sensitive data or support essential operations. Add client and device combinations that reflect how those people actually work, including legitimate arrangements that differ from the standard managed Windows laptop.

Each row should have one explicit purpose. Start with a known-good case, then change one condition where feasible. This helps distinguish an intended access decision from an unrelated client problem. A denied sign-in from an unconfigured test device tells you little if the account also lacks access to the target application. Validate prerequisites first and preserve enough context for another authorised reviewer to repeat the same case.

Give higher priority to boundaries with greater impact: administrative access, sensitive exports, external collaboration and exceptions. Document the combinations you did not test and why. A representative sample is acceptable when its rationale is visible; describing a small sample as complete tenant coverage is not. If a business-critical client cannot be exercised safely, mark that case as unverified and propose a supervised follow-up rather than assuming it behaves like a browser.

Case fieldWhat to record
IdentityTest account identifier, role and relevant group membership
ResourceSpecific target application or resource and intended operation
Client and deviceClient type, platform, registration and relevant compliance state
Network contextAgreed source context and any relevant named-location membership
Expected outcomeAllow, deny or require an additional agreed control
EvidenceUTC time, sign-in reference, applied policies and observed resource result
DispositionPass, fail, inconclusive or not tested; owner and next action
Different workplace devices approaching separate controlled glass access gates
Test both intended access and intended denial. Conceptual illustration, not a tenant screenshot or client finding.

Use simulation, observation and live checks for different questions

Microsoft’s What If tool estimates policy effects for specified sign-in conditions. It is useful for exploring cases before a supervised test, but it does not exercise application service dependencies. Treat its output as simulation evidence, not proof that an end-to-end Teams or other business workflow will function. Record the supplied inputs with the result so an omitted condition does not become an invisible assumption.

Use simulation to narrow the questions that need real observation. For example, compare an ordinary test user with an approved exception account under the same proposed context. If the difference is unexpected, investigate assignments before touching production. Preserve the policy state at the time of the simulation. A later result against a changed configuration cannot explain an earlier access event without that history.

Next, review existing sign-in evidence within the authorised assessment scope. Finally, where necessary, perform controlled sign-ins using designated accounts and synthetic resources. These are different evidence layers: a policy export shows configuration, simulation shows an estimated decision, and a controlled check shows observed behaviour in one context. A strong conclusion explains which layers support it and which remain unavailable.

A clean architectural model beside a real office environment
A simulated decision and an observed business workflow answer different questions. Conceptual illustration, not a tenant screenshot or client finding.

Interpret report-only results without overstating protection

Microsoft documents report-only mode as evaluation without enforcement for supported policy scenarios. Its results can help estimate the effect of a proposed change. A report-only failure is not proof that the observed user was blocked; a report-only success is not proof that every other applicable policy allowed access. Keep the proposed policy’s result separate from the overall sign-in outcome.

For an assessment, ask the owner why a policy remains in this state and what evidence is needed before promotion. A policy under active evaluation should have an owner, a planned observation period and a decision date. One left indefinitely without review is a governance issue, but its severity depends on whether enforced controls already meet the requirement. Avoid equating every report-only policy with an exploitable security gap.

Do not turn an existing enforced policy into report-only merely to gather comparison data. Plan a separate proposed change through the normal approval process. Even observation requires care: Microsoft notes that some device-compliance report-only scenarios can produce certificate-selection prompts. Agree the test population and support arrangements with the tenant owner rather than assuming that all non-enforcing changes are operationally invisible.

Correlate the sign-in with the resource operation

Use a consistent evidence record for every controlled check. Capture the UTC start time, designated account, target resource, client, expected outcome and the operation attempted. Then locate the corresponding sign-in event and preserve its relevant policy results. Microsoft’s Conditional Access troubleshooting guidance describes using event details and correlation information to investigate unexpected outcomes. A matching timestamp alone is insufficient when several client requests occur together.

Record what the user could actually do after authentication. Opening a landing page, reading a protected document and completing a privileged operation are different observations. Use a harmless agreed operation that represents the requirement, such as reading a synthetic file in a dedicated test library. Avoid exporting real customer data to demonstrate that a policy boundary matters. If the application reports an error, determine whether access control or an unrelated dependency caused it.

Separate raw evidence from interpretation. Preserve the relevant event identifier and a restricted source record, then place a redacted explanation in the report. Include enough information to reproduce the reasoning without exposing tokens, cookies, personal correspondence or unnecessary employee identifiers. If logs are unavailable for the test window, state the limitation. Do not fill the gap with an assertion that the policy must have applied because its settings look correct.

Magnifying glass over a sequence of event cards beside a clock
Tie each conclusion to a specific access attempt and a harmless resource operation. Conceptual illustration, not a tenant screenshot or client finding.

Test exclusions as access paths with owners

An exclusion is not automatically a defect. It may support recovery, a migration or a carefully constrained business dependency. The assessment question is whether its purpose, population, compensating controls and review date are explicit. Compare the documented reason with actual membership. A temporary exception used by many ordinary accounts deserves a different conclusion from a tightly controlled recovery arrangement.

For each significant exception, identify the protected requirement it bypasses and the remaining controls that still apply. Use simulation and approved test accounts to establish the difference rather than attempting to place a real employee into an excluded group. Ask who can approve membership changes, how changes are detected and what happens when the original justification expires. An exception register that nobody operates is not a control.

Our separate Conditional Access exclusions review addresses the inventory in more detail. In the assessment matrix, make exceptions visible as their own cases. This prevents them disappearing into an average result such as “most staff are protected” and gives the service owner a concrete choice: retain with evidence, narrow, replace or remove through an approved change.

Distinguish device assumptions from verified device conditions

Ask the endpoint team how a device reaches the state expected by the access policy. A policy that relies on managed or compliant devices needs an operational process behind that classification. Select test devices whose relevant states are known at the time of testing. Record the source of that knowledge and any recent changes, rather than labelling a laptop compliant because someone recognises its asset tag.

Include permitted nonstandard working patterns in the sample: mobile access, contractor equipment, supported macOS clients or browser-only arrangements where these exist. Do not assume that a successful Windows test represents every platform. Equally, do not introduce unsupported devices solely to create a failure. The test should answer whether the organisation’s documented operating model is implemented, with proposed improvements clearly identified as recommendations.

When a device-related result is unexpected, involve both the identity and endpoint owners. Record whether the discrepancy concerns the access policy, the device’s reported state, an assignment, an onboarding process or missing evidence. This keeps remediation actionable. For the endpoint side of the assessment, the Intune audit guide explains why device configuration and the evidence of its application need separate attention.

Include session context and client dependencies in the plan

A fresh browser sign-in and an already established session are not interchangeable test conditions. Record how the test started, whether the client had an existing session and what the assessor did to establish a clean comparison. Do not interpret the absence of a visible prompt as proof that no control was satisfied. Conversely, a prompt does not establish that the intended resource boundary is protected.

Agree which session behaviours are actually in scope. For example, the organisation may want to understand the experience after a planned policy change or a supervised access removal. Such tests need separate approval, expected timing and a clear rollback plan. Avoid disrupting ordinary employee sessions to make the assessment appear comprehensive. Use designated accounts and describe precisely what was observed, including the elapsed time and client context.

For applications with several dependencies, test the agreed business operation rather than stopping at the application launcher. A collaboration client may need several resources to complete an ordinary task. If only the initial sign-in was tested, say so. This is especially important when a proposed policy appears safe in simulation but the user journey later fails at a resource that the original test did not include.

Check the recovery design before proposing stronger enforcement

Any recommendation that could change administrative access must account for recovery. Ask the tenant owner to demonstrate the documented recovery process under controlled conditions and explain how the designated credentials are protected and monitored. The assessor should not receive unnecessary recovery secrets. Evidence can include a witnessed procedure, authorised ownership records and a supervised check with sensitive details withheld from the report.

Keep the recovery case distinct from ordinary administration. Its purpose is to preserve a viable response to a failure of the normal access path, not to offer convenient daily access. Record the people authorised to invoke it, how invocation is detected and what review follows. Assess whether the proposed change would accidentally include or disable this route, and require the owner to resolve that risk before deployment.

An assessment can identify a missing or unverified recovery arrangement without performing a destructive lockout experiment. Do not deliberately break production access to prove the point. Use a documented review and safe rehearsal, then state what remains untested. The practical objective is a change plan that improves protection while leaving the organisation able to recover from mistakes and dependencies outside its direct control.

Protected emergency route with sealed credentials and two physical keys
Recovery access needs a controlled procedure, not an untested assumption. Conceptual illustration, not a tenant screenshot or client finding.

Write findings that an administrator can act on

A useful finding starts with the requirement and affected population, then explains the observed difference. Avoid titles such as “Conditional Access misconfigured” without identifying the boundary. A better hypothetical title is “Approved contractor population omitted from the sensitive-resource access requirement.” The body should identify the supporting evidence, tested scope, business consequence, limitations and proposed correction.

Do not claim a breach from a configuration weakness alone. If the assessment observed unintended access to a synthetic resource, describe that observation. If it found a plausible gap without permission to validate it, say that it remains unconfirmed. Severity should reflect the resource, privilege, exposure and existing controls, with confidence stated separately. This produces a more credible prioritisation than treating every deviation as critical.

Finding componentExample of useful content
Expected boundaryNamed contractor group should meet the approved device requirement for a specific resource
Observed differenceDesignated test identity reached the synthetic resource under a disallowed device context
EvidencePolicy export reference, membership snapshot, sign-in event and resource-operation record
LimitsOne client and resource tested; other combinations remain unverified
RemediationOwner reviews the assignment and dependent workflows before an approved change
ClosureRepeat the negative case and the legitimate positive case against the changed configuration

Include operational consequences in the recommendation. Correcting an assignment may require endpoint onboarding, user communication or an application owner’s acceptance. Name those dependencies so the ticket does not bounce between teams. A recommendation that says only “enable the policy” leaves the difficult work undefined and encourages hurried changes that the assessment should help prevent.

Turn the assessment into a staged remediation plan

Group recommendations by the access boundary they improve, not just by policy name. Several findings may depend on the same device onboarding work; one policy change may affect several business services. Identify these relationships before assigning deadlines. Keep urgent exposure reduction separate from longer-term simplification, and explain any temporary measures with an owner and expiry decision.

For each proposed change, record the baseline, intended difference, pilot population, validation cases, rollback authority and evidence needed for approval. Use a representative pilot that includes legitimate edge cases rather than only the IT team’s standard laptops. Ask service owners to confirm their critical workflow checks. A technically successful deployment can still be incomplete if a required client or guest journey was never exercised.

Retesting should repeat both the original failing case and neighbouring legitimate cases. Closing an unintended route while breaking normal access is not complete remediation. Preserve the changed configuration reference with the new evidence and update the exception register. If the organisation accepts a remaining risk instead, record that decision explicitly rather than marking the technical finding fixed.

Progressively illuminated glass panels representing staged change and validation
Promote changes with evidence, rollback ownership and legitimate-access checks. Conceptual illustration, not a tenant screenshot or client finding.

What to request in the assessment deliverables

Request a concise executive explanation supported by a technical evidence pack. The executive section should explain which access boundaries were assessed, which material gaps were confirmed and which decisions require management support. The technical pack should contain the inventory, sampling rationale, test matrix, redacted evidence, findings and retest criteria. Keep sensitive raw exports in an agreed restricted location rather than embedding everything in an email attachment.

Ask the provider to separate completed tests from deferred checks and unavailable evidence. A report with a transparent coverage table is more useful than a percentage score without a denominator. It should also distinguish tenant configuration review from active testing, endpoint review and incident investigation. These activities overlap, but one does not automatically include the others.

Before accepting the report, select one finding and trace it from requirement to configuration, observation, recommendation and closure condition. If that chain is clear, your team can use the report to manage change. If it depends on unexplained screenshots or generic advice, request clarification while the assessment context is still fresh.

Prepare your Microsoft 365 Conditional Access assessment

Prepare the tenant context, relevant licence information, identity populations, sensitive resources, supported client types and known exceptions. Describe the business operations that must remain available and nominate identity, endpoint and application owners. You do not need to send passwords, session tokens or full employee exports through an enquiry form. Start with scope and objectives; agree secure evidence exchange and access arrangements before collection.

Use our Microsoft 365 audit scoping tool to outline the environment, or discuss a Conditional Access assessment with Atlant Security. We can help turn a broad request to “check MFA” into a defined review of identities, resources, exceptions and observed access decisions. The resulting scope should say what will be verified, what will remain outside the assessment and what evidence your team will receive.

This guide was reviewed on 25 October 2026. Microsoft interfaces, features and entitlements can change; confirm the applicable product documentation and tenant capabilities when planning the engagement. The test matrices and evidence structures above are practical assessment recommendations, not a Microsoft certification or a guarantee that every possible access path has been examined.

Published by Atlant Security. Sources, editorial policy and corrections.

PUT THE GUIDANCE TO WORK

Choose your next step.

LET’S START A CONVERSATION

Define the scope.
Take the next step.

Your audit objectives, control boundaries and evidence period. A useful starting point for your assessment.

Discuss your requirements