Table of Contents
Summary
The Automation Orchestrator for AutoAssess is a unified hub for configuring and running AI-powered automation across your vendor assessment lifecycle. Instead of manually initiating assessments, gathering documents, analyzing vendor security posture, and compiling summaries one step at a time, you can automate some or all of that work.
The Orchestrator is built around four specialized AI agents, each handling one phase of the assessment process:
- Initiator – Automatically starts vendor assessments based on the schedule and risk criteria you set
- Collector – Gathers security documents and evidence from Trust Centers, the web, existing vendor sources and the ability to request questionnaires and documents of your choice
- Analyst – Runs AI analysis on collected sources to generate risk scoring and summaries (vendor summary and SOC summary)
- Reporter – Compiles an executive summary and notifies specified recipients
Together, these four agents make up AutoAssess — the first workflow built on the Orchestrator. The Orchestrator itself is designed to scale beyond assessments, with additional agents planned for other Whistic workflows over time.
You can turn on individual agents to automate specific steps while keeping manual control over the rest, or enable all four together with Full AutoAssess for end-to-end automation. You're always able to pause and step in manually at any point, even within a fully automated workflow.
With Full AutoAssess enabled, the humans still make the risk decision and are the ones who close the assessment, the agents simply do the work. Giving you time savings, allowing you to scale without headcount and providing the ability to run your risk program, on your terms.
🚀 Getting Started
Requirements to Access & Enable:
- Whistic AI must be enabled on your account
- Assess (VRM) must be enabled on your account
- You must have an Admin role to configure the Orchestrator
🔧 How to Set Up the Orchestrator
Step 1: Enable Whistic AI & Assess
Whistic AI and Assess (VRM) must both be turned on for your account before the Orchestrator is available. If you're not sure whether these are enabled, check with your account admin.
Step 2: Navigate to the Orchestrator
- From your Dashboard, go to Resources (or Assess)
- Select Automation Orchestrator
Step 3: Choose Individual Agents or Full AutoAssess
Before configuring any agent settings, decide how much of the process you want to automate. You'll set this up as you go — each agent can be turned on independently, but it helps to know your target approach going in.
- Individual Agents – Turn on one, two, or three agents and keep manual control over the rest. This is a good way to ease into automation.
- Full AutoAssess – Turn on all four agents together for complete end-to-end automation, from initiating the assessment through final report generation.
When all four agents are enabled, here's how an assessment flows:
- The Initiator detects a vendor due for assessment and creates the assessment
- The Collector gathers sources from Trust Centers, the web, and existing documents
- The Analyst runs AI analysis and generates a summary with confidence scoring
- The Reporter compiles an executive summary and routes it to your approvers
You can configure pause points at any step if you want a person to review before moving forward — or let it run end-to-end and simply review the executive summary.
Good to know:
- With Full AutoAssess enabled, the agents do the work, but people still make the risk decision and close out the assessment.
Step 4: Configure the General Automation Settings
Before setting up individual agents, configure the general settings that apply across all of them.
- From the Orchestrator, select Settings on the upper right
- Under Who gets notified about agent activity?, click Manage Recipients to choose which team members receive in-app and/or email notifications about agent activity
- Under Notification delivery, choose:
- Delivery type – how often notifications are sent:
- Timing – the time of day notifications go out
- Optionally, check Don't send notification if there was no activity to skip notifications on days with nothing to report
- Under Which vendors belong in this group?, set the criteria for which vendors are included in this automation group (for example, by Billing Country, Business Unit, Criticality, Inherent Risk or a data point of your choice)
- Click Save
Good to know:
- These notification settings apply globally, going forward, to all agents — not just one.
- Leaving the vendor criteria empty means no vendors will be auto-assessed.
- If a vendor matching your criteria doesn't have Smart Search enabled, Whistic will automatically enable it for that vendor when you save. This may take a few minutes to process.
- If a vendor already has an individual Assessment Automation schedule set up, adding it to this group may result in duplicate assessment requests. Review or cancel that vendor's individual schedule from its details page to avoid overlap.
Step 5: Configure the Initiator Agent
The Initiator automatically starts vendor assessments based on the schedule and criteria you define.
- Click the pencil icon to begin configuring the Initiator settings
- Set your assessment cadence (e.g., every 3, 6, 12, 18, or 24 months)
- You will see a preview of the vendors who are scheduled to be initiated based off the general settings that includes your criteria
- Save your settings with two options:
- Save & Start Later - Saving but not enabling the agent (more work to be done)
- Save & Start Now - Saves and enables the agent (go live)
Good to know:
- If a new vendor is pushed into Whistic through an integration, API or intake form and meets your configured criteria, the Initiator will automatically start an assessment for it.
- This agent also reviews existing vendors and will trigger the assessment based off the previous assessment completed
- If you manually start an assessment for a vendor that already has one scheduled through automation, Whistic will recognize this and reschedule the next automated assessment instead of creating a duplicate.
- You can view scheduled, completed, and canceled assessments for this agent in its activity log.
Step 6: Configure the Collector Agent
The Collector automatically gathers vendor documents and evidence so you always have current information for assessments.
- Click the pencil icon to begin configuring the Collector settings
- Choose a setup option:
- Use Default Settings – Quick setup which includes: Trust Center collection, and existing documents in the vendor document repository are trusted up to 12 months old.
- Configure Custom Settings – Manually adjust all settings yourself.
- If using Custom Settings, configure:
-
Trust Center Collection – Turn on/off automatic discovery and capture of vendor Trust Center sources
- Pause assessment and notify if no Trust Center URL found
- Pause assessment and notify if no sources found by Trust Center Capture
- Existing Document Age Threshold – How old an existing document can be and still be used (e.g., up to 6 months, up to 2 years)
-
Request Sources – Select any questionnaires or documents to request from the vendor, along with a due date
- Specify who receives the request: All vendor contacts or Primary vendor contacts
- Automatically cancel unreturned requests after due date
-
Trust Center Collection – Turn on/off automatic discovery and capture of vendor Trust Center sources
- Save your settings with two options:
- Save & Start Later - Saving but not enabling the agent (more work to be done)
- Save & Start Now - Saves and enables the agent (go live)
Good to know:
- If you start with Default Settings and later change any individual setting, your configuration automatically switches to Custom — nothing is lost in the process.
- You can choose to have Whistic pause and notify you if a Trust Center URL or usable sources can't be found, so you can step in manually.
- You can turn on automatic cancellation of any outstanding vendor requests once the due date passes.
Step 7: Configure the Analyst Agent
The Analyst runs AI analysis on the sources the Collector gathers, generating a risk score and confidence rating.
- Click the pencil icon to begin configuring the Analyst settings
- Select Generate Vendor Summary
- Choose whether to include Web Sources (publicly available information) in the analysis, in addition to vendor-provided documents
- Choose the assessment framework you want the AI to evaluate against
-
Manual Review Thresholds - Optionally pause the Analyst for manual review when output quality falls below set thresholds.
- Minimum compliance score - Pause and notify if Vendor Summary compliance score falls below this threshold.
- Maximum unknowns - Pause and notify if Vendor Summary has too many Unknown Answer questions.
Good to know:
- Each Vendor Summary finding includes the evidence source used, a confidence score, and the reasoning behind it.
- By default, analysis is limited to documents the vendor has provided. Web sources are only included if you turn that setting on.
- SOC 2 Summary generation through the Analyst agent is coming soon.
Step 9: Configure the Reporter Agent
The Reporter compiles the final executive summary and routes it for approval.
- Click the pencil icon to begin configuring the Reporter settings
- Enable the Create Executive Summary
- Set your approvers for final sign-off (coming soon)
Good to know:
- Approvers can Approve, Approve with Conditions, or Reject the assessment.
- Once an assessment is approved, it's finalized, and the next assessment is automatically scheduled based on your Initiator settings.
📋 Reviewing and Managing Automated Assessments
- Each agent has its own activity log showing completed, scheduled, and canceled actions, along with a link to view each vendor directly.
- Each enabled agent displays who turned it on and the date it was enabled.
- If an agent pauses and you complete a step manually, advancing the assessment to the next step will hand it back off to the next agent automatically.
Activity Log (Example)
Enablement Tracking
❓ FAQ
Can I choose which vendors go through automation, or does it run on everything?
Yes. The Initiator agent is configurable — you set the criteria (for example, only High Risk vendors, or only Low Risk vendors), and only vendors matching that criteria will be automated.
If we push new vendors into Whistic through an integration, will the Orchestrator pick those up?
Yes. When a new vendor is added through an integration, the Initiator evaluates it against your configured criteria. If it matches, an assessment is automatically started. If not, it's left for manual handling.
Can we use our own risk definitions instead of Whistic's built-in inherent risk levels?
Yes, you can manage risk levels and data classifications in Assess > Assess Settings > Program Automation.
Can we set different assessment cadences for different risk levels?
Not currently. Each agent is configured for a single vendor grouping at this time. Support for different configurations by vendor segment is planned for a future release, so stay tuned!
Can the AI tell me whether a vendor is good to go or not?
Not at this time. The Analyst agent provides risk scoring, confidence ratings, and detailed findings to support your decision, but final approval decisions are made by your configured approvers.
What happens if the AI finds ambiguous or conflicting information in the sources?
The Vendor Summary marks each item as compliant, non-compliant, or unknown, along with a confidence score. You can configure a pause point to review findings before the assessment advances.
How accurate is the AI analysis?
Whistic's AI operates only within the sources you specify (plus web sources, if enabled), performs multi-pass analysis, and shows the evidence and reasoning behind every finding. Results are highly accurate when backed by strong source documentation from the start.
Can I step in manually at any point during an automated workflow?
Yes. You can pause automation and take over manually at any step, working in tandem with the AI agents. Once you complete that step and advance the workflow, the next agent picks up automatically.
Do I need a separate license to use the Orchestrator?
The Orchestrator requires Whistic AI and Assess (VRM) to be enabled on your account. Reach out to your account team if you're unsure what's included in your plan.