Set up a response rule in CyberSentry
Open CyberSentry > Response Rules, look at an existing rule's shape and the always-on Confidence Engine, then build a new rule end to end: one trigger, two actions, a client scope and priority, a Test against history attempt, and a save that lands it Enabled in Monitor mode.
What you will have
- See the Response Rules list on a tenant that already ships 15 seeded rules, and the always-on Confidence Engine row above them.
- Open an existing rule to see the shape every rule shares: an ordered TRIGGERS sequence, an ordered ACTIONS list, and a SETTINGS & SAFETY card.
- Open the Confidence Engine's own drill-in page and see its Factor Weights and Confidence Bands sitting on their seeded defaults.
- Build a new rule: name it, set one trigger step, and add a Notify action and an Open ticket action.
- Scope the rule to one client by hand and give it a priority.
- Try Test against history and see the honest result on a tenant whose clients have no connected Microsoft 365 tenant.
- Save the rule, meet the save-time break-glass warning, and see the rule Enabled in Monitor mode back in the table.
Why it works this way
A new rule defaults to Disabled. Monitor evaluates every match live and records what the rule would have done - nothing is locked, revoked, notified, or ticketed. Only a deliberate switch to Live lets it act for real, which is why this walkthrough stops at Monitor rather than arming the rule.
The match window (60 minutes by default) only matters once a rule has more than one trigger step: it is how far apart those steps can happen and still count as one sequence. This rule has a single step, so the window never comes into play for it.
The auto-action cap (5 accounts/hour by default here) limits the most distinct accounts a Live rule can act on in a rolling hour, so a misfiring rule cannot lock out an entire client at once. It only does anything once a rule is actually Live.
Never act on break-glass / service accounts stays checked by default so a locking rule can never touch a client's designated emergency-admin accounts. That checkbox is a different thing from the save-time warning this walkthrough hit: a warning that a scoped client, Bluebird Dental, has no break-glass account configured on its own edit screen at all.
Never act on our own (MSP) access also stays checked by default, so the rule skips your own team's logins into the client rather than ever acting on a tech's own access.
Stop processing further rules on match, left off here, would make a rule that fires skip lower-priority rules for that same account on that run only; it never touches other accounts.
Neither this rule nor any other response rule sends a real notification or opens a real ticket while it sits at Disabled. An alert only ever gets a real ticket or notification from one of two doors: a rule armed past Disabled with the matching action, or the Confidence Engine promoted to Live.
Steps
Open CyberSentry > Response Rules.
The list already carried 15 seeded rules on this trial tenant - detector-parity rules mirroring CyberSentry's built-in detectors, and further single- and multi-stage templates, every one shipped in Monitor. Above the table, one row is not a rule you can delete: Confidence Engine, marked Auto, showing Monitor and "Always-on scoring - 0 recorded events" with its own On switch. It is the fleet-wide, always-on detector; the rules below it are the specific IF/THEN overrides you write on top of it.
Open CyberSentry > Response Rules. Open an existing rule and look at its TRIGGERS card.
Opening the seeded MFA method changed rule showed the shape every rule shares. TRIGGERS holds an ordered sequence of steps that must happen in order, within the match window; this rule has one step, Event type is MFA method changed, and a standing caution box explains that a single-step rule fires on every match with no corroboration - fine for a rare, strong signal, but likely to flood you for a broad one, in which case a second step (like a following inbox-rule or MFA change) makes it confirm before firing.
Open an existing rule and look at its TRIGGERS card. Look at the same rule's ACTIONS card.
ACTIONS runs top to bottom, in this order, the tail end of the rule: this one had Raise alert (title "MFA method changed", severity medium) then Notify (tier Escalation). Add action builds an action list by hand; four seeded bundles sit right below it, Contain Account, Escalate to Team, Watch, and Contain Compromised Token, for composing a common set in one click instead of one action at a time. Stop if an action fails, checked here, halts the rest of the list the moment one action errors.
Look at the same rule's ACTIONS card. Look at the same rule's SETTINGS & SAFETY card.
Every rule carries the same safety card: Match window (60 minutes here) and Priority (100) on top, Applies to (All clients or Manual) with an Auto-action cap next to it, then three checkboxes - Stop processing further rules on match, Never act on break-glass / service accounts, and Never act on our own (MSP) access - the last two checked, which is the shipped default. Below this card, Recent Activity and Test against history round out the page.
Look at the same rule's SETTINGS & SAFETY card. Open the Confidence Engine's own page.
Clicking into the Confidence Engine row (or Back to Response Rules from here) opens its own screen: a Promote to Live button, then Auto Response Thresholds - the glass box for how a signal becomes a confidence percentage and what that percentage fires. Confidence Bands (confidence -> what fires, evaluated highest-first) and Factor Weights (the weights that add up to that percentage) both read "No bands/factors configured - the seeded defaults are in effect," since nothing has been customized on this tenant. Would-Have-Fired Observations further down had nothing to show yet either.
Open the Confidence Engine's own page. Back on Response Rules, click New Rule and name it Northwind test rule.
A new rule opens on Disabled - "This rule is off - it doesn't run or record anything" - with Monitor and Live as the other two choices. An Ask Elise panel sits alongside the form, offering to draft a rule from a plain-English description instead of building it field by field; this walkthrough builds it by hand. Typing into Rule name was the only field filled in at this point.
Back on Response Rules, click New Rule and name it Northwind test rule. Set Step 1's event type to Admin role granted.
The Event type dropdown lists the events CyberSentry watches for - sign-ins, token redemptions, inbox rules, OAuth consents, MFA changes, admin role grants, password changes - each paired with is / is not / is any of / is none of and a value picker. Admin role granted is exactly the kind of rare, strong signal the single-step caution box calls out as safe to fire on with no second confirming step.
Set Step 1's event type to Admin role granted. Add a Notify action and an Open ticket action.
Add action twice built the list one action at a time: Notify with its Notification tier set to Escalation, then Open ticket with its Ticket board left on Inherit default (CyberSentry settings). These are two of the same effects the seeded bundles compose for you - the bundles just save the trip through Add action when a common combination like Contain Account or Escalate to Team fits.
Add a Notify action and an Open ticket action. Scope the rule to Bluebird Dental, set its priority to 50, and open the client list.
Applies to switched from All clients to Manual, which reveals a Clients picker. Add client opens a searchable list of every client CyberSentry watches, with your own organization in it too, named Northwind Managed Services (your organization) rather than left looking like a client. Typing Bluebird found and added Bluebird Dental, which is why it shows as a chip and is no longer offered in the list. Priority changed from the default 100 to 50. Match window stayed at 60 and Auto-action cap at 5. The open list covers the three guard checkboxes, whose full labels are in step 4; their states are unchanged here, Stop processing further rules on match off and both break-glass and MSP-access on.
Scope the rule to Bluebird Dental, set its priority to 50, and open the client list. Try Test against history against Bluebird Dental, the way to test a rule before enabling it.
Picking Bluebird Dental, leaving the window at 7 days, and pressing Run test is meant to dry-run the rule over that client's recent activity and preview its blast radius - which accounts it would have acted on, and what the guards would have blocked - with nothing locked, revoked, or notified for real. This is the way to size the auto-action cap and catch surprises before ever moving a rule to Live. The Client list here offers only what the rule is scoped to, so Bluebird Dental was the one choice on it.
Note: This trial tenant's clients have no connected Microsoft 365 tenant, so every client here reads this same message. On a client with a live Microsoft 365 connection this panel instead lists the accounts the rule would have acted on and what the guards would have blocked.Try Test against history against Bluebird Dental, the way to test a rule before enabling it. Click Save, and meet the break-glass warning.
Saving opened a "Clients without a break-glass account" dialog: "1 client(s) in this rule's scope have no break-glass / emergency-admin account configured. If this rule auto-acts, it could lock their administrator and leave no way back in. Set a break-glass account on each client's edit screen first," naming Bluebird Dental, with Save anyway and Cancel. Save anyway went ahead and saved the rule as scoped.
Warning: This is a save-time check on the client, not the rule's own break-glass checkbox: it fires because Bluebird Dental itself has no break-glass account set on its edit screen, regardless of how this rule's guards are configured.Click Save, and meet the break-glass warning. See the saved rule back in the Response Rules table.
After picking Monitor and saving past the same break-glass warning, Northwind test rule sat in the table alongside the seeded rules: Identity (M365), Single-step, Monitor (dry run), Enabled, scope 1 clients, priority 50 - matching everything set on the way in.
See the saved rule back in the Response Rules table.
Other ways to do this
Start from a seeded rule
Open any seeded rule on the list - Admin role granted (detector parity), for instance - and use its TRIGGERS, ACTIONS, and SETTINGS & SAFETY as a filled-in example before typing into a blank New Rule form.
Ask Elise
On the New Rule page, describe the rule in plain English in the Ask Elise panel and press Draft with Elise. The draft comes back inert - nothing is enabled or armed - so it still needs review and its own Save.
If it did not work
- If Test against history reads 'This client has no connected Microsoft 365 tenant to replay,' the picked client has no live Microsoft 365 connection; a trial tenant with only synthetic clients will show this for every client in the picker.
- If saving warns about clients without a break-glass account, either open the named client's edit screen and set one first, or accept the risk with Save anyway.
- If you want to see this rule (or any rule) move past Monitor, start with CyberSentry at a glance's Response Rules section: it explains what Monitor, Live, and the Confidence Engine's own Auto row each do. This walkthrough stops at Monitor, which is where every new rule starts and stays until you deliberately arm it.
Questions this page answers
How do the WHEN steps and THEN effects work?
WHEN is an ordered sequence: Step 1 must happen, then Step 2, and so on, all within the match window and all for the SAME account. Each step is one or more conditions (field → operator → value). THEN is an ordered list of effects that run top to bottom when the sequence matches. Order matters on both sides - the steps are a timeline, and the effects run in the order you list them.
What are effects, and what are the seeded templates?
Effects are the THEN side - ordered bundles of actions (revoke sessions, lock account, open ticket, notify). Four templates ship seeded: Contain Account, Escalate to Team, Watch, and Contain Compromised Token (which follows Microsoft's token-theft containment order: revoke → disable → ticket → notify). You can clone and edit them, or compose your own from the action palette. Note: you cannot revoke a single LIVE access token - the closest is revoking sessions plus CAE.
How do I limit a rule to certain clients?
Set "Applies to" to "Manual" and pick the clients you want. The list also has your own organization, shown as "(your organization)". It is there because CyberSentry watches your tenant too, so you can scope a rule to it like any client. "All clients" runs the rule everywhere CyberSentry watches. Scope also filters the backtest list, so you can only test a rule against a client it would run on.
What does priority do?
When several rules could match the same event, lower priority numbers run first. Combined with "stop processing further rules on match", priority lets you put a decisive containment rule ahead of softer watch/notify rules.
What is the match window?
How far apart the steps can be in time and still count as one sequence. With a 60-minute window, "10 failed logins" in Step 1 and "a foreign success" in Step 2 only match if they happen within 60 minutes of each other. A wider window catches slower attacks but can be noisier.
What is the "auto-action cap (accounts / hour)"?
A safety limit: the most DISTINCT accounts a Live rule will act on in a rolling one-hour window. If a rule misfires (say a bad detector run), the cap stops it from locking your entire client at once, once it has acted on that many accounts in the hour, further matches are blocked and recorded instead. Leave it blank for no cap. It only applies in Live mode; Monitor never acts, so the cap does nothing there.
What does "exclude break-glass accounts" do?
When on (the default), the rule will never act on the client's designated emergency-admin ("break-glass") accounts, their MSP service accounts, or accounts on the location-exception list. This is the guarantee that a locking rule can never lock you out of a tenant during an incident. Break-glass accounts are set on the client's edit screen. Keep this on unless you have a very specific reason not to.
What does "exclude MSP access" do?
When on (the default), the rule skips your own team's access to the client - logins from your techs (via GDAP/B2B) or your service accounts. It stops a rule from acting on you when you are legitimately working in the client's tenant.
What does "stop processing further rules on match" do?
Rules run in priority order. When a rule with this on fires for an account, lower-priority rules skip that same account for this run - useful when a broad "contain" rule should short-circuit narrower ones. It is per-account, so other accounts still flow through every rule.
What does "Ask Elise" do?
Describe the rule you want in plain English and Elise drafts it for you to review - grounded in the real fields/effects and Microsoft's token-theft playbook, so it only uses things that exist and it flags anything it could not map. Every draft comes back inert (Monitor + disabled + guards on); Elise never saves or arms a rule - you review, tweak, and save it yourself.
How do I test a rule before enabling it?
Use "Test against history" in the builder: pick a client and a window (7/14/30 days) and Elise replays that client's real recent activity through the rule - without locking, revoking, or notifying anything. You get the blast radius (which accounts it would have acted on) and what the guards would have blocked, so you can size the auto-action cap and catch surprises before arming.
Was this helpful?