This applies to outbound AI SDR pilots on email and LinkedIn for B2B teams with a CRM and at least one person who can review messages daily. A small audience may not produce enough held meetings for a clear result. That is an insufficient-evidence outcome, not a pass.
What Is An AI SDR Pilot And What Should It Decide
Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, naming unclear business value among the causes, so an AI SDR pilot has to end in one measurable decision. The pilot itself is a time-boxed test of one workflow on one audience.
Gartner framed this as a prediction, not a measured outcome. It listed poor data quality, inadequate risk controls and escalating costs alongside unclear value. A pilot avoids that stall by naming its decision in advance. At the end, the team scales the workflow, changes it or stops it.
The thing being tested is the workflow, not the vendor. AI SDR tools such as AiSDR and Artisan research prospects, write messages and send them. The pilot asks whether that chain produces more held, qualified meetings than the team's current approach on the same kind of prospect. That question needs a fair comparison group, which is why the pilot is built around a holdout.
By the end, the pilot should prove three things. The AI SDR produced held, qualified meetings at a cost the team accepts. It did so without a compliance or deliverability incident. And the team could review and own its output with the people it already has.
How To Launch An AI SDR Pilot Step By Step
Launching an AI SDR pilot takes seven moves in order, from a written charter and a narrow audience to a holdout group, deliverability checks, review permissions, a fixed live window and a go/no-go decision agreed before the first send. Each move below points to the section that covers it.
- Write the pilot charter and get it signed off (this section)
- Choose one narrow audience and suppress customers, open deals and do-not-contact records (narrow audience)
- Draw a random holdout from the same list and record the baseline (baseline and holdout)
- Clear Gmail sender rules and CAN-SPAM duties (deliverability and compliance)
- Set the workflow, review permissions and reply ownership in each tool (workflow and permissions)
- Fix the live window and its review stages (run length)
- Make the go, iterate, no-go or insufficient-evidence call on written thresholds (metrics and criteria)
The charter fits on one page and carries six items:
- The one question: Does this AI SDR workflow produce more held, qualified meetings per contacted account than the current workflow?
- The owner: One named person signs off the launch, reads the weekly scorecard and makes the final call.
- The offer: One offer, written once. The AI sales email brief sets out the buyer problem, proof and ask the messages draw on.
- Allowed AI actions: What the AI SDR may do alone, such as researching leads, drafting messages and sending approved sequences.
- Actions needing human review: Every outgoing message in the first stretch, every reply that is not a clear out-of-office or unsubscribe, and any change of audience.
- Mandatory gates and pass thresholds: Identity and authentication, suppression, sending limits, permissions and reply ownership, plus the numbers that decide go or no-go, all signed off before launch.
Write the review rule into the charter in each vendor's own terms. Artisan can hold every message for approval, while AiSDR approves sample messages before launch and then sends without per-message approval. A charter that says "human in the loop" without naming the setting leaves the most important control undefined.
Choose A Narrow Audience For An AI SDR Pilot
A narrow AI SDR pilot audience is one segment, one territory, one offer and one owner, with customers, open deals and do-not-contact records suppressed before launch, because Artisan excludes only the organization's global do-not-contact list by default and AiSDR suppresses only the HubSpot lists you connect.
Size the audience from two limits. The first is review capacity, the number of messages one reviewer can read properly each day. The second is what the sending setup can carry without pushing the spam rate up. Both groups, pilot and holdout, come out of this one list, so it needs to be big enough to split.
Build the suppression before the first sync:
- Existing customers: Put them in a HubSpot company list and connect it as a suppression list.
- Open opportunities: Collect contacts and companies on open deals in a list of their own.
- Active conversations: Anyone a rep is already talking to goes on the same list.
- Do-not-contact records: Keep the organization's master list current, since Artisan applies its global DNC list by default.
AiSDR matches HubSpot contact or company suppression lists by email or domain. Artisan can also skip leads contacted by any campaign within a set number of days, which stops the pilot from re-mailing people another Artisan campaign just reached.
Vendors' default exclusions do not know who your customers are or which deals are live. Only your CRM does, so the suppression lists have to come from it.
Before buying new contacts for the pilot, enrich the CRM records you already hold. Records with a current title, company and email make a cleaner list, and they already carry the owner and deal history the suppression lists depend on.
Set The AI SDR Pilot Baseline With A Holdout Group
A trustworthy AI SDR pilot baseline comes from a random holdout drawn from the same audience list, which HubSpot can create as a static segment by number or percentage of records on Marketing Hub Professional or Enterprise. The holdout shows what the current workflow produces on comparable prospects over the same weeks.
A before-and-after comparison is not a test. Last quarter's numbers come from different prospects, a different offer and different market conditions, so any gap between them and the pilot cannot be pinned on the AI SDR. A random split from one list removes those differences.
Build the holdout in HubSpot in four steps:
- Create the pilot audience as one segment, with suppression already applied.
- Create a random sample from that segment by a number of records or a percentage. HubSpot saves the result as a new static segment.
- Name that static segment as the holdout, and build the pilot group from the original segment without it.
- Check access. The sample feature needs Marketing Hub Professional or Enterprise and Edit Segment permissions, and splitting one segment into several percentage-based segments is Enterprise only.
The holdout keeps the current workflow. If human SDRs work the segment today, they keep working the holdout the same way. If an existing sequence runs today, the holdout stays on it. The holdout is never left idle, because then the pilot only proves that contacting people beats not contacting them.
If the team cannot run a holdout, a historical baseline from previous months is the fallback. Label it as historical in every report, and treat a result built on it as weaker evidence than a random split.
Clear AI SDR Deliverability And Compliance Before Launch
Gmail's sender guidelines tell senders to keep the spam rate reported in Postmaster Tools below 0.10% and to avoid ever reaching 0.30%, and an AI SDR that ramps volume on new mailboxes can cross that line before the pilot books a single meeting. The same guidelines set authentication rules. All senders need SPF or DKIM, TLS and RFC 5322 formatting. Senders of more than 5,000 messages a day to personal Gmail accounts also need SPF and DKIM, DMARC and one-click unsubscribe for marketing and subscribed messages.
The law sets the second gate. The FTC's CAN-SPAM compliance guide says the law makes no exception for business-to-business email, so an AI SDR's cold emails carry the same duties as any commercial message. The guide lists these duties:
- Header information that is not false or misleading
- Subject lines that do not deceive
- Identification of the message as an ad
- A valid physical postal address
- A clear way to opt out, with opt-outs honoured within 10 business days
Each separate email in violation can draw a penalty of up to $53,088, a figure the FTC updated for inflation in January 2024.
Before launch, confirm each gate on the pilot's own sending setup:
- Postmaster Tools is set up for the sending domain, so the spam rate is visible from the first day
- SPF, DKIM and DMARC are in place for every pilot mailbox
- Every template carries the postal address and a clear opt-out line
- An opt-out from any channel reaches the AI SDR's suppression list well inside 10 business days
The pilot needs proof that each gate passes before the first send, not a full campaign build. Readers who need the full setup, from mailbox authentication to list verification, can follow the playbook on setting up your first cold email campaign.
This section is general information, not legal advice.
Build The AI SDR Workflow And Set Permissions
The review control differs by AI SDR vendor, and that difference changes how the pilot is staffed, because Artisan can hold every message Ava generates for approval while AiSDR previews sample messages before launch and then sends without per-message approval. Map who or what acts at each step before choosing a setting.
The AI SDR workflow runs as one chain:
- List in: RevOps loads the pilot segment, with suppression applied.
- Research and write: The AI SDR researches each lead and drafts the messages.
- Review: A named reviewer approves, edits or rejects messages under the review rule.
- Send: The AI SDR sends inside its sending window and daily cap.
- Reply handling: A named person owns every reply, or the AI SDR replies under written rules.
- CRM log: Contacts, activities and meetings sync to HubSpot, tagged pilot or holdout.
- Meeting held: The rep runs the meeting and sets its outcome in HubSpot.
- Review of the call: The owner checks recorded calls for real pain and the right persona.
"Human in the loop" means different things per vendor. Here is what each tool's review control actually allows:
Human review control by tool
| Tool | Review control | What it means |
|---|---|---|
| AiSDR | Sample preview before launch | No per-message approval |
| Artisan | Manual approval of every message | Autopilot replies optional |
| HubSpot prospecting agent | Review emails or Send without review | Native option inside the CRM |
| Airtable | Record review interface | Reviewer works flagged items |
| Gong | Consent profile set by Tech admin | Applies to all calls |
Start the live window with per-message approval wherever the tool offers it. In Artisan, turn manual approval on. In HubSpot's prospecting agent, choose Review emails. AiSDR has no per-message switch, so review its Test messaging samples closely and read a sample of live sends each day in HubSpot Activities.
Write the rule for loosening review before launch. It names the run of approved messages and the rejection rate that let the reviewer move from full review to sampled review, and the incident that sends review straight back to full. Without that rule, review quietly fades as the reviewer gets busy.
Readers still choosing a vendor can start with the AI SDR platforms comparison, then bring the chosen tool back to this workflow.
AiSDR In A Pilot
AiSDR suits a pilot where the team approves the messaging approach once, through sample emails and LinkedIn messages generated with Test messaging, and then accepts that each live message is personalised per lead at send time.
Role in the pilot: AiSDR is the AI SDR itself. It researches leads, writes the sequence and sends email and LinkedIn messages, so it carries the pilot group's whole outbound workload.
Human review control: Before launch, the Test messaging button on the campaign or messaging page generates sample emails and LinkedIn messages from the settings, persona and selected leads. The team approves the sequence structure and tone. After that, no manual approval is needed for each outgoing message.
CRM data flow: The HubSpot sync creates and updates contacts and logs activities and deal info, with email and LinkedIn events in HubSpot Activities. HubSpot lists can be assigned to campaigns, so new list members join automatically. Synced leads arrive with no owner and the source "Integration, AiSDR".
Limit to plan around: Only meetings booked through HubSpot meeting links are tracked. Route every pilot booking through a HubSpot meeting link, or the meeting never reaches the scorecard.
Cost driver: Self-serve use is charged per person prospected and charged daily. Team plans are priced on volume on quarterly contracts. Paid add-ons cover a warmed-up mailbox, a per-campaign managed service and a fully managed service. The pricing widget shows the plans.
AiSDR pricing
| Plan | Price | Billing | Notes |
|---|---|---|---|
| Fully managed service (add-on) | $2,500 | per month | |
| Pay as you go (self-serve) | $1.50 | usage-based | per contact |
| Per-campaign managed service (add-on) | $149 | ||
| Team plans | Custom pricing | per contact | |
| Warmed-up mailbox (add-on) | $33 | one-time |
Limitation: Live emails are similar to the approved samples but not identical, because AiSDR personalises each message per lead at send time. A sample that passed review does not guarantee every send will.
Artisan In A Pilot
Artisan suits a pilot that needs a human gate on every message, since its campaign settings can make each message Ava generates wait for approval in the Tasks page before it goes out.
Role in the pilot: Artisan's AI SDR, Ava, prospects and writes outreach for the pilot group, with the reviewer standing between Ava and every send.
Human review control: With manual approval on, every message Ava generates waits for approval before it goes out. A separate setting, Autopilot replies, lets Ava respond to leads on her own. Keep it off until reply ownership has been tested.
CRM data flow: The "Use CRM owners as senders" setting requires a connected CRM, so connect HubSpot before launch if reps' names should appear as senders.
Limit to plan around: Review tasks usually reach the Tasks page within minutes to a few hours, depending on queue size, and can be filtered by campaign. Staff a fixed daily review slot. The "Maximum new prospects to contact per day" setting caps outreach and sets the pilot's throughput, and a sending window controls when Ava sends and can skip dates.
Cost driver: Plans are sized to lead volume, mailboxes and dialer seats. Artisan lists its Team plan at about 2,500 leads contacted a month and Scale at about 6,000, and an AI dialer add-on is priced per seat. Scale adds a dedicated CSM, CRM setup and priority support. The pricing widget shows Artisan's plans.
Artisan pricing
| Plan | Price | Billing | Notes |
|---|---|---|---|
| Enterprise | Custom pricing | ||
| Scale | Custom pricing | ||
| Team | Custom pricing |
Limitation: Forwarding of positive and neutral replies to the sender's mailbox does not work while Autopilot replies is on. A team that lets Ava reply loses the inbox view a rep would normally rely on.
HubSpot As The Pilot System Of Record
HubSpot is the pilot's system of record, holding the random holdout segment, the AI SDR's synced contacts and activity, and the meeting outcomes that show whether a booked meeting was held.
Role in the pilot: HubSpot is where the pilot is judged. Both groups, every logged activity and every meeting outcome live in one place, so the scorecard is built from one source.
Human review control: HubSpot's prospecting agent has two modes, Review emails, which approves each message before it goes out, and Send without review. It is a native option for a team that prefers to pilot inside the CRM. The agent uses HubSpot Credits when it generates personalised outreach, and credit-using actions cannot run in sandbox accounts, so a sandbox cannot be used to rehearse it.
CRM data flow: A random sample from the pilot segment, by number or percentage of records, becomes a new static segment that serves as the holdout.
Limit to plan around: HubSpot allows up to 30 custom meeting outcomes of 50 characters each, and an outcome cannot be edited after saving. Name outcomes such as "Held, qualified" and "No show" carefully before launch, since deleting a custom outcome later can cause analytics issues.
Cost driver: The random sample feature needs Marketing Hub Professional or Enterprise, custom meeting outcomes need Sales Hub or Service Hub Professional or Enterprise, and the prospecting agent spends HubSpot Credits.
Limitation: Splitting one segment into several percentage-based segments is Marketing Hub Enterprise only, so a team on Professional builds one holdout sample at a time.
Airtable As The Pilot Log And Review Queue
Airtable works as the pilot's log, holding the incident list, the weekly scorecard and a record review interface where one reviewer works through flagged messages from a single page.
Role in the pilot: Airtable holds what the CRM is poor at holding, the reviewer's decisions, the incidents and the weekly numbers the owner signs off. A base with three tables covers it:
- Flagged messages, with the message text, lead, tool, flag reason and reviewer decision
- Incidents, with the date, type, affected records and fix
- Weekly metrics, with one row per week for each group, pilot and holdout
Human review control: The record review layout lets a reviewer work through many records from a single landing page. Reviewers can edit a field only if it is set to Editable, and buttons on the record detail can capture decisions such as approve or reject. The layout is available on all plan types.
Limit to plan around: Automations run within a monthly limit per workspace that rises by plan, with Team at 25,000 runs and Business at 100,000, resetting on the first of the month. Failed and successful attempts both count.
Cost driver: The plan tier drives cost, since automation runs scale with it. Only Creator or Owner users can create or edit automations, so the person who builds the log needs one of those roles. The pricing widget shows the plans.
Airtable pricing
| Plan | Price | Billing | Notes |
|---|---|---|---|
| Team |
|
| per seat |
| Business |
|
| per seat |
| Enterprise Scale | Custom pricing | per seat | |
| Free | Custom pricing | per seat |
Limitation: Failed automation attempts consume runs just as successful ones do, so a misconfigured automation that fails on every trigger burns runs without doing any work. Turned-off automations still count toward the per-base automation limit.
Gong For Reviewing Held Meetings
Gong answers the question reply rates cannot, whether the meetings an AI SDR booked became real sales conversations, by recording held calls and exporting each call's spotlight and link to HubSpot.
Role in the pilot: Gong is the quality check on held meetings. The owner listens for confirmed pain and the right persona, in pilot and holdout calls alike.
Human review control: The Tech admin sets a consent profile that applies to all calls. It can show a consent page before joining, send an email 10 minutes before the meeting and play an audio announcement that the call is recorded.
CRM data flow: Gong exports meetings, calls, emails and LinkedIn messages to HubSpot, each with the call name, participants, a link to the call in Gong, the call spotlight and action items. The transcript is not exported, and an export can take up to six hours.
Limit to plan around: Question-based smart trackers, the default builder since September 2025, need no minimum call volume, and up to 20 can be published at a time. Build trackers such as "pain confirmed" and "wrong persona" before the first held meeting.
Limitation: A meeting invite exports only if at least one external participant matches a HubSpot record. Using All calls or All emails alongside another app that exports to HubSpot can also create duplicates, which matters when the AI SDR already logs activity there.
How Long Should An AI SDR Pilot Run
An AI SDR pilot should run for a fixed live window that starts once setup and launch gates are complete and lasts long enough for both the pilot group and the holdout to book, hold and log meetings. Count from the day the gates pass, not from contract signature.
Size the window from two numbers the team can see before launch:
- Send capacity: How many new prospects the tool contacts per day. Artisan caps this with its maximum new prospects per day setting, so the pilot group's size divided by that cap gives the days needed just to reach everyone.
- Meeting lag: How long a booked meeting takes to be held and logged. Gong's export to HubSpot can take up to six hours, and reps set HubSpot meeting outcomes themselves, so the last meetings need time to land.
Is 30 days enough? Only if the whole audience is reached and its meetings are held and logged inside that time. The calendar length matters less than whether both groups finished the cycle from first touch to logged outcome.
The live window follows a fixed shape:
Shape of the live window
| Stage | Review | What happens |
|---|---|---|
| First week | Full review |
|
| Middle weeks | Sampled review | Loosen only under the written rule |
| Mid-point check | Gates only | The five charter gates |
| Final week | Sampled review | Log outcomes and decide |
The mid-point check looks at gates only, never at the scorecard, so nobody adjusts the test halfway on a partial result. A gate failure at that point stops the pilot, whatever the meetings look like.
Extending the window to rescue a weak result is a no-go in disguise. If the window ends with too few held meetings to judge, record insufficient evidence and decide whether a new, larger pilot is worth running.
AI SDR Pilot Metrics And Go/No-Go Criteria
The go/no-go call for an AI SDR pilot rests on held, qualified meetings and the pipeline they create in the pilot group against the holdout, judged on thresholds written down before launch, with reply rate kept as a diagnostic rather than the verdict. Gates come first, then the scorecard.
AI SDR pilot metrics
| Metric | Type | Compare |
|---|---|---|
| Held qualified meetings per contacted account | Primary | Pilot vs holdout |
| Pipeline per contacted account | Primary | Pilot vs holdout |
| Positive reply rate | Secondary | Pilot vs holdout |
| Review time per message | Secondary | Pilot only |
| Review rejection rate | Secondary | Pilot only |
| Incidents | Secondary | Pilot only |
| Cost per held qualified meeting | Secondary | Pilot vs holdout |
Booked is not held. A booked meeting that never happens creates no pipeline, so the primary metric counts only meetings the rep marks as held and qualified. Two settings make that count possible:
- Booking route: AiSDR's HubSpot sync tracks only meetings booked through HubSpot meeting links, so every pilot booking goes through one.
- Meeting outcomes: HubSpot custom meeting outcomes record what happened, so create outcomes such as "Held, qualified" and "No show" before launch.
Reply rate stays a diagnostic, because high cold email reply rates can hide poor pipeline when many replies are negative or never turn into meetings. Use it to spot a weak offer early, not to decide the pilot.
The five mandatory gates named in the charter pass before the scorecard counts:
- Identity and authentication: SPF, DKIM and DMARC on every pilot mailbox, honest headers, and the postal address and opt-out line in every template, with zero compliance incidents.
- Suppression: No customer, open deal or do-not-contact record contacted.
- Sending limits: The daily cap held, and the Postmaster Tools spam rate stayed below Gmail's 0.10% guideline.
- Permissions: The AI SDR acted only within its allowed actions and the written review rule.
- Reply ownership: Every reply handled by its named owner or written rule.
Then the scorecard decides one of four outcomes:
- Go: The gates passed and the pilot group beat the holdout on the primary metrics by the written threshold. Scale to the next segment.
- Iterate: The gates passed and the result fell short of the threshold with a clear cause, such as the offer or the audience. Change one thing and run a new pilot.
- No-go: A gate failed, or the pilot group did not beat the holdout. Stop.
- Insufficient evidence: Too few held meetings in either group to judge. Record it as such, never as a pass.
Write every threshold before launch and have the owner sign it. Gartner's warning about generative AI projects abandoned after proof of concept applies here. A pilot whose pass mark is set after the results arrive has no measurable decision to make.
The Real AI SDR Pilot Cost Beyond The Vendor Meter
The real cost of an AI SDR pilot is the vendor's meter plus everything around it, which for AiSDR means people prospected plus paid mailbox and managed-service add-ons, and for Artisan a plan sized to lead volume, mailboxes and dialer seats. Plan prices sit in the pricing widgets above.
The pricing basis hides the real cost, because each tool in the stack bills on something different:
- AiSDR: Charged per person prospected on self-serve, with paid add-ons for a warmed-up mailbox, a per-campaign managed service and a fully managed service.
- Artisan: A plan sized to lead volume, mailboxes and dialer seats, with an AI dialer add-on priced per seat.
- HubSpot: The holdout sample needs Marketing Hub Professional or Enterprise, custom meeting outcomes need Sales Hub or Service Hub Professional or Enterprise, and the prospecting agent spends HubSpot Credits.
- Airtable: Automation runs scale with the plan, and only Creator or Owner users can build automations.
The hidden cost is the reviewer. Per-message approval in the first week means a named person reads every send, and sampled review after that still takes time each day. Log review time per message in the Airtable scorecard so it enters the total.
Compute cost per held qualified meeting for both groups. For the pilot, add the vendor bill, add-ons, mailboxes, CRM tier upgrades, credits and reviewer hours, then divide by held qualified meetings. For the holdout, use the cost of the current workflow over the same window and divide by its held qualified meetings.
Common AI SDR Pilot Mistakes
A frequent AI SDR pilot mistake is measuring the wrong thing, such as replies or booked meetings that never happen, when AiSDR's HubSpot sync tracks only meetings booked through HubSpot meeting links. Several others follow the same pattern, a setting assumed instead of read.
- Trusting approved samples: AiSDR's live emails are similar to its Test messaging samples but not identical. Fix it by reading a daily sample of live sends in HubSpot Activities.
- Syncing with the All option: AiSDR's All sync option syncs leads even when they are on a suppression list. Fix it by avoiding that option and confirming the suppression lists before launch.
- Counting booked meetings: A booked meeting that never happens adds no pipeline. Fix it by scoring only meetings marked held and qualified in HubSpot.
- Ramping volume too fast: Raising daily volume on new mailboxes can push the spam rate past Gmail's 0.10% guideline. Fix it by watching Postmaster Tools daily and holding volume flat at the first rise.
- Switching on Autopilot replies early: Artisan's Autopilot replies lets Ava answer leads alone and stops reply forwarding to the sender's mailbox. Fix it by keeping it off until a named owner has handled replies through the first stretch.
- Judging on a historical baseline: Last quarter's numbers come from different prospects and conditions. Fix it with a random holdout drawn from the same list.
Frequently Asked Questions
How many prospects should an AI SDR pilot include?
An AI SDR pilot should include as many prospects as one reviewer can check and the sending setup can carry, split randomly into a pilot group and a holdout drawn from the same list. Artisan's daily cap on new prospects sets how fast the pilot group is reached, so size the list to finish inside the window.
Can I run an AI SDR pilot without a holdout group?
You can, but the result is weaker evidence. Without a holdout, the only comparison is a historical baseline, which mixes in different prospects, offers and conditions. HubSpot can create a random holdout as a static segment from the pilot segment on Marketing Hub Professional or Enterprise, so teams on those tiers have little reason to skip it.
Should the AI SDR reply to leads on its own during a pilot?
Not at the start. Keep replies with a named owner until reply ownership has been tested, then decide. In Artisan, Autopilot replies lets Ava respond on her own, but forwarding of positive and neutral replies to the sender's mailbox stops while it is on, so the rep loses that inbox view.
Does CAN-SPAM apply to B2B cold email from an AI SDR?
Yes. The FTC's CAN-SPAM compliance guide says the law makes no exception for business-to-business email, and a business stays responsible for messages others send on its behalf. Accurate headers, honest subject lines, a postal address and opt-outs honoured within 10 business days all apply. This is general information, not legal advice.
What happens after a no-go decision?
A no-go ends the pilot, and the AI SDR stops sending to the audience. Record which gate failed or how far the pilot fell short of the holdout, keep the suppression lists, and close the Airtable log. A new pilot needs a new charter, a changed workflow and fresh thresholds, never the old window extended.
Sources
Sources


