Experiments (A/B Tests)
An A/B test, called an experiment in Perspective, asks one question: does a changed version of your funnel convert better than the one you have now? Perspective keeps your funnel as it is (the original) and makes a copy with one idea changed (the variant). When the test runs, each visitor is sent to one of the two, following a split such as 50/50. After enough visitors have come through, you compare the share of visitors who converted on each side, the conversion rate. The version with the higher conversion rate wins, as long as the gap is unlikely to be luck.
:::note Availability Experiments are available on AI/HTML funnels for accounts with A/B testing enabled. The MCP server follows the same setting as the Perspective app: if your account does not have it, these tools are not listed. Classic (non-HTML) funnels cannot be tested. :::
Key ideas
- One test at a time. A funnel has at most one current test, which is either
draft(set up, not live yet) orrunning(live, visitors are being split). Finished tests stay in the history asconcluded(ended with a significant result) oraborted(ended without one), and never change again. - Two versions, one funnel. The original and the variant are two pages behind the same funnel URL. You edit either one with
update_funnelby naming theversion. - Publishing is the switch. Nothing reaches a visitor until the funnel is published. That is true for starting a test, and also for changes made to a test that is already running.
The lifecycle
- Create the test with
create_experiment. You give it a name, a hypothesis ("a shorter form will convert better") and, optionally, a split. The test is saved as adraft, and Perspective's AI writes the variant from the hypothesis. The original is not touched. Pollget_funnel_job_statuswith the returnedjobIduntil the variant is written. - Edit either version with
update_funnel, passingversion: "original"orversion: "variant". While a test exists, every edit has to say which version it is for. - Change the split or the name with
update_experiment, for example from 50/50 to 70/30. - Start the test with
publish_funnel. Publishing a funnel with a draft test puts both versions live and starts splitting visitors. The first call asks for confirmation instead of publishing (see Starting a test). - Read the results with
get_experiment_resultswhile the test runs. - End the test with
end_experimentand keep one version. Every visitor then sees the kept version, and the split stops. A test that never started can be deleted withdiscard_experimentinstead.
get_experiment and list_experiments tell you, at any step, whether a test exists and what state it is in.
:::warning Changes go live on publish
While a test is running, editing either version (update_funnel) or changing the split (update_experiment) is saved at once, but visitors do not see it yet. They keep getting the old content and the old split until the funnel is published again with publish_funnel.
Think of it like a printed menu: you can rewrite the menu in the kitchen, but guests only see the change once you print and hand out the new copies. Publishing is the printing step.
On a draft test there is nothing to republish yet, but keep in mind that the next publish of the funnel, whatever it is for, starts the test. :::
Tools
| Tool | What it does | Key inputs | Confirm with the user first |
|---|---|---|---|
get_experiment | Read the funnel's current test, or a past one by id | funnelId, experimentId? | No |
list_experiments | List the funnel's tests, newest first | funnelId, status? | No |
create_experiment | Set up a draft test and have the AI write the variant | funnelId, title, hypothesis, weights? | Yes |
update_experiment | Rename the current test or change its split | funnelId, title?, weights? | No (republishing a running test does) |
get_experiment_results | How each version is doing, and whether there is an answer yet | funnelId, experimentId? | No |
end_experiment | End a running test and keep one version | funnelId, winner | Yes, including which version to keep |
discard_experiment | Delete a test that has not started, with its variant | funnelId | Yes |
Two funnel tools take part in the test flow too. update_funnel edits one version through its version input, and publish_funnel starts or refreshes the test. Both are documented on the Funnels page.
get_experiment
Reads a funnel's A/B test. Without experimentId it returns the current test (draft or running), or data: null with a message when the funnel has none. Having no test is a normal result, not an error. Call it before editing or publishing a funnel to find out whether a test exists.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel (from list_funnels or create_funnel) |
experimentId | string | No | ID of a specific test (from list_experiments), in any status. Omit for the current test |
Returns { data: { id, title, status, startedAt?, endedAt?, winnerArmId?, arms }, message? }. Each item in arms is one version of the page:
| Field | Description |
|---|---|
id | ID of the arm; winnerArmId refers to it |
version | original or variant |
weight | Percent of the funnel's visitors sent to this version. On a running test this is the saved split, which reaches visitors at the next publish |
draftId | ID of the draft holding this version's page |
role | control or challenger; the same information as version |
list_experiments
Lists a funnel's tests, newest first: the current one and the past ones. Each item is a summary without arms; call get_experiment with its id for the full test. Without status, an empty data array means the funnel has no saved tests (a discarded draft is deleted, so it is not listed); with it, only that no saved test has those statuses.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
status | string[] | No | Only return tests in these statuses, e.g. ["concluded", "aborted"] for past tests. Omit for every status |
Returns { data: [{ id, title, status, startedAt?, endedAt?, winnerArmId? }] }.
create_experiment
Sets up an A/B test on an AI/HTML funnel, with the variant written by AI from a hypothesis. The test is created as a draft and the AI starts writing the variant right away. The original is not changed, and nothing is live yet: no visitor is split and the published funnel stays as it is.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
title | string | Yes | Name of the test, up to 40 characters |
hypothesis | string | Yes | What the variant should change and why, in plain words, e.g. "A shorter form with only name and email will convert better than the current five fields." The AI writes the variant from this, so be specific |
weights | { original, variant } | No | Traffic split as whole percents from 5 to 95 that add up to 100, e.g. { "original": 70, "variant": 30 }. Omit for 50/50 |
Returns { data, jobId?, message }. data is the new test in the same shape as get_experiment. Poll get_funnel_job_status with jobId until it reports completed; its summary describes what the variant changes.
If the AI could not start writing the variant, the test is still created, but jobId is missing and the variant is an exact copy of the original. To retry, call update_funnel with version: "variant" and a prompt that restates the whole hypothesis. To start over instead, discard the test and create it again. When a plan limit stopped the AI, for example because the account has run out of AI credits, message says what lifts the limit; retry once it is lifted.
Errors (nothing is created):
FUNNEL_GENERATION_IN_PROGRESS: an AI generation is still running on the funnel. Wait for it withget_funnel_job_status.EXPERIMENT_ALREADY_EXISTS: the funnel already has a draft or running test. Look at it withget_experimentand change it withupdate_experiment.
update_experiment
Renames the funnel's current test or changes how traffic is split between the two versions. Works on a draft or running test; a test that has ended cannot be changed. Pass at least one of title and weights.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
title | string | No | New name, up to 40 characters |
weights | { original, variant } | No | New split, whole percents from 5 to 95 that add up to 100 |
Returns { data, message? }: data is the test after the change, and message, present when the split changed, says when the new split reaches visitors. On a running test, a new split applies only after the next publish_funnel (see Changes go live on publish); on a draft test it applies when the test starts.
Errors: NOTHING_TO_UPDATE when neither field is passed. NO_CURRENT_EXPERIMENT when the funnel has no draft or running test.
Starting a test
There is no separate "start" tool. A draft test starts the next time the funnel is published, and publish_funnel guards this with a confirmation step:
- The first
publish_funnelcall on a funnel with a draft test does not publish. It fails withEXPERIMENT_START_NOT_CONFIRMEDand returns the test's name and split. - Confirm with the user that they want to start the test: both versions go live and visitors start being split. Agreeing to "publish my funnel" is not the same as agreeing to start a test.
- Call
publish_funnelagain with the same inputs plusconfirmStartExperiment: true.
After a publish, the response's message says what happened to the test:
- Started: the test is live and visitors are split.
- Refreshed: the test was already running; the latest edits and split now reach visitors.
- Not started: the user's role does not allow starting tests. The funnel is still published, showing only the original, and the test stays a draft.
- Failed: the funnel is published, but starting or refreshing the test failed. A draft test stays a draft, and a running test keeps serving each version as of the previous publish. Publishing again retries it.
Publishing a funnel with a draft test always tries to start the test. To publish it without the test, discard the test first.
get_experiment_results
Shows how each version is doing and whether the test has an answer yet. Without experimentId it reads the current test; with one, any past test.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
experimentId | string | No | ID of a past or current test (from list_experiments). Omit for the current test |
Returns { data, message }. message is a one-sentence plain-language reading of the numbers, such as "Not enough data yet: 42 % of the ~3,100 visitors this test needs." data is null when the funnel has no test or the test has not started.
data contains:
experiment:id,title,status(running,concludedoraborted).range:fromandto, the period the numbers cover (from the test start to its end, or to now).arms: one entry per version, the original first:id,version,sessions(visitors),conversions,conversionRate(0 to 1) andweight.significance:nullwhile a version has had no visitors. Otherwise:
| Field | Meaning |
|---|---|
progress | How much of the needed data the test has collected, from 0 to 1 |
isSignificant | true when the gap between the versions is unlikely to be chance (95 % level) |
leadingVersion | The version with the higher conversion rate, or null if they are equal |
absoluteLift | Variant rate minus original rate, e.g. 0.02 = 2 percentage points |
relativeLift | absoluteLift divided by the original's rate, e.g. 0.25 = 25 % better. null while the original has no conversions |
visitorsNeeded | Estimated total visitors (both versions) needed for a significant result, if the current gap holds. null while both convert equally |
:::info progress is data collected, not the chance of winning
Picture flipping two coins to see which one lands heads more often. After 10 flips each, a 6-to-4 gap could easily be luck. After 10,000 flips, the same gap is very unlikely to be luck. progress tells you how far along those flips you are.
So 60 % progress does not mean a 60 % chance that the variant wins. It means the test has 60 % of the data it needs to tell the versions apart. The Perspective app shows this number as "Confidence". :::
Two more rules for reading results:
progressreaches 1 exactly whenisSignificantturnstrue.- Significant only means the gap is unlikely to be chance, not that the variant is better. The variant has won only when
isSignificantistrueandleadingVersionisvariant. If the original leads, the change made things worse.
end_experiment
Ends a running test and keeps one of its two versions. Every visitor then sees the kept version, the other one stops being served, and traffic is no longer split. Ending is final: a test cannot be restarted. To test again, create a new one.
Before calling it, show the user the latest results and let them choose which version to keep. If the chosen version is not significantly ahead, the test can still be ended, but the numbers do not prove that version is better.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
winner | "original" | "variant" | Yes | The version to keep |
Returns { data, message }. data is the ended test: status is concluded when the kept version's result was significant, aborted when it was not, and winnerArmId is the kept arm. message says what the funnel serves now:
- Live funnel: every visitor sees the kept version. The split can take up to a minute to stop everywhere.
- Offline funnel: nothing is republished. The kept version goes live at the next
publish_funnel. - Promotion failed: the test has ended, but putting the kept version live failed, so visitors may still be split. Publishing the funnel again fixes it.
Errors (nothing is changed):
NO_RUNNING_EXPERIMENT: the funnel has no running test, or it has already ended.EXPERIMENT_NOT_STARTED: the test is still a draft. Usediscard_experimentinstead.EXPERIMENT_RESULTS_UNAVAILABLE: the results could not be read. Try again shortly.
discard_experiment
Deletes a test that has not started (draft). The test and its variant, with every edit made to it, are permanently deleted. The original is untouched, so the funnel has no test afterwards and a new one can be created right away.
| Name | Type | Required | Description |
|---|---|---|---|
funnelId | string | Yes | ID of the AI/HTML funnel |
Returns { message }.
Errors: NO_DRAFT_EXPERIMENT when the funnel has no test that has not started. A running test cannot be discarded; end it with end_experiment, which keeps one version.
Errors shared by every tool
FUNNEL_NOT_FOUND: the funnel does not exist, is not accessible to you, or is a classic funnel. Check the id withlist_funnels; a funnel withisHtmlFunnel: falseis classic and cannot be tested.EXPERIMENT_NOT_FOUND(tools that takeexperimentId): that test is not on this funnel.list_experimentsshows the valid ids.
Related
- Tools: Funnels —
update_funnelwithversion,publish_funnelwithconfirmStartExperiment - Scopes: Tool Scopes — which scopes include the experiment tools
- Guide: Metrics & analytics — the
abTestchart filter for per-page conversion by version