Manage - Winning Variant

Roles
Resource Kinds
List Resources
Examples
Get YAML for a single resource
Examples
Create or update resource
Examples
Experimentation Convenience Methods
Create Experiment
Add Variants
Add Cohort
Set Experiment Status
Update Experiment
Get Current Cohort
Describe Experiment YAML

The Winning Variant Native App allows users to manage experimentation resources directly inside of Snowflake by use of stored procedures created during installation.

Roles

A user must be assigned of the following application roles to execute any of the procedures described on this page:

admin
editor

Resource Kinds

All references to kind below must be one of:

lab
subjecttype
experiment

Read more about Resource Kinds.

List Resources

Each of the following “list” commands returns a table with the following columns:

Column	Description	Data Type
`kind`	The resource kind.	`VARCHAR`
`id`	ID of the resource.	`VARCHAR`
`name`	Name of the resource.	`VARCHAR`
`status`	Resource status	`VARCHAR`
`description`	Resource description	`VARCHAR`
`resource_version`	Current resource version	`INT`

USE <application_name>;

CALL experimentation.list_resources('<kind>' [, <status>, ...]);

Returns a list of all resources of the given kind. status must be one of ResourceStatus.

Examples

-- List all
USE <application_name>;

CALL experimentation.list_resources('lab');

-- List active
CALL experimentation.list_resources('lab', ['active']);

Get YAML for a single resource

USE <application_name>;

CALL experimentation.get_resource_yaml('<kind>', '<id>');

Returns a table with the following columns:

Column	Description	Data Type
`kind`	The resource kind.	`VARCHAR`
`id`	ID of the resource.	`VARCHAR`
`yaml`	The full configuration (YAML) for a single resource.	`VARCHAR`

Examples

USE <application_name>;

CALL experimentation.get_resource_yaml('experiment', 'my-experiment');

Create or update resource

USE <application_name>;

CALL experimentation.apply_resource('<yaml>');

If the resource with the id specified exists, it gets updated, otherwise it will be created.

Examples

USE <application_name>;

-- Create a new lab with id `marketing`
CALL experimentation.apply_resource($$schemaVersion: 1
kind: lab
metadata:
  id: marketing
  status: active
  name: Marketing
  description: All marketing and top-of-funnel experiments.
$$);

-- Update the name of the lab we just created
CALL experimentation.apply_resource($$schemaVersion: 1
kind: lab
metadata:
  id: marketing
  status: active
  name: Marketing Organization
  description: All marketing and top-of-funnel experiments.
$$);

Experimentation Convenience Methods

In addition to the above methods using raw YAML, you may also create/manage experiments with the following purpose-built convenience methods. All methods also have a *_preview variant is available that returns the YAML that would be applied without actually creating the experiment:

CALL experimentation.create_experiment_preview(
  'homepage-cta-test', 'Homepage CTA Test', NULL, NULL, NULL, 'USER',
  '[{"id":"control","name":"Blue Button","isControl":true}]', NULL
);

Create Experiment

Creates a new experiment with the following inputs. The experiment is created in draft status.

Parameter	Description	Data Type	Example
`id`	Unique identifier for the experiment. Alphanumeric, hyphens, underscores, dots only.	`VARCHAR`	`'homepage-cta-test'`
`name`	Human-readable name for the experiment.	`VARCHAR`	`'Homepage CTA Test'`
`description`	Optional description of the experiment. Pass `NULL` to omit.	`VARCHAR`	`'Testing button color on homepage'`
`parent_lab`	Optional ID of the parent lab. Pass `NULL` to use the DEFAULT lab. Must exist if provided.	`VARCHAR`	`'growth-lab'`
`hypothesis`	Optional hypothesis for the experiment. Pass `NULL` to omit.	`VARCHAR`	`'A green CTA will increase conversions by 10%'`
`subject_type`	ID of the subject type. Must already exist.	`VARCHAR`	`'USER'`
`variants`	JSON array of variant objects. Each must have `id`, `name`, and `isControl`. Optional `description`. At most 1 control.	`VARCHAR` (JSON)	`'[{"id":"control","name":"Blue Button","isControl":true},{"id":"green","name":"Green Button","isControl":false}]'`
`initial_cohort`	JSON array of cohort variant entries with `variant` and `split`. Splits must sum to 1.0. Requires `variants`. Pass `NULL` to omit.	`VARCHAR` (JSON)	`'[{"variant":"control","split":0.5},{"variant":"green","split":0.5}]'`

Returns: Table with columns operation, kind, id, name, status, description.

CALL experimentation.create_experiment(
  'homepage-cta-test',
  'Homepage CTA Test',
  'Testing button color on homepage',
  NULL,
  'A green CTA will increase conversions by 10%',
  'USER',
  '[{"id":"control","name":"Blue Button","isControl":true},{"id":"green","name":"Green Button","isControl":false}]',
  '[{"variant":"control","split":0.5},{"variant":"green","split":0.5}]'
);

Add Variants

Adds one or more new variants to an existing experiment. Variant IDs must not conflict with existing variants. At most 1 control variant is allowed across all variants on the experiment.

Parameter	Description	Data Type	Example
`experiment_id`	ID of the experiment to add variants to. Must exist.	`VARCHAR`	`'homepage-cta-test'`
`variants`	JSON array of variant objects. Each must have `id`, `name`, and `isControl`. Optional `description`.	`VARCHAR` (JSON)	`'[{"id":"red","name":"Red Button","isControl":false,"description":"Red variant"}]'`

Returns: Table with columns operation, kind, id, name, status, description.

CALL experimentation.add_variants(
  'homepage-cta-test',
  '[{"id":"red","name":"Red Button","isControl":false,"description":"A red CTA button"}]'
);

Add Cohort

Adds a new cohort to an existing experiment. The cohort index is automatically assigned as the next sequential value. The experiment must already have variants defined.

Parameter	Description	Data Type	Example
`experiment_id`	ID of the experiment. Must exist and have variants defined.	`VARCHAR`	`'homepage-cta-test'`
`cohort_variants`	JSON array of cohort entries. Each must have `variant` (must be an existing variant on the experiment) and `split` (0.001–1.0). Splits must sum to 1.0. At most 1 control variant.	`VARCHAR` (JSON)	`'[{"variant":"control","split":0.34},{"variant":"green","split":0.33},{"variant":"red","split":0.33}]'`

Returns: Table with columns operation, kind, id, name, status, description.

CALL experimentation.add_cohort(
  'homepage-cta-test',
  '[{"variant":"control","split":0.34},{"variant":"green","split":0.33},{"variant":"red","split":0.33}]'
);

Set Experiment Status

Sets the status of an existing experiment.

Parameter	Description	Data Type	Example
`experiment_id`	ID of the experiment. Must exist.	`VARCHAR`	`'homepage-cta-test'`
`status`	New status. Must be one of: `active`, `archived`, `draft`, `ended`, `winner_declared`.	`VARCHAR`	`'active'`

Returns: Table with columns operation, kind, id, name, status, description.

CALL experimentation.set_experiment_status('homepage-cta-test', 'active');

Update Experiment

Updates one or more properties of an existing experiment. Pass NULL for any field that should not change. At least one field must be provided.

Parameter	Description	Data Type	Example
`experiment_id`	ID of the experiment. Must exist.	`VARCHAR`	`'homepage-cta-test'`
`name`	New name. Pass `NULL` to keep current.	`VARCHAR`	`'Updated CTA Test'`
`description`	New description. Pass `NULL` to keep current.	`VARCHAR`	`'Revised experiment description'`
`status`	New status. Must be one of: `active`, `archived`, `draft`, `ended`, `winner_declared`. Pass `NULL` to keep current.	`VARCHAR`	`'ended'`
`hypothesis`	New hypothesis. Pass `NULL` to keep current.	`VARCHAR`	`'Updated hypothesis'`
`ended_reason`	Reason the experiment ended. Only valid when status is `ended`. Must be one of: `no_longer_needed`, `no_stat_sig`, `other_reason`, `success`, `tech_issue`.	`VARCHAR`	`'success'`
`winning_variant`	ID of the winning variant. Only valid when status is `winner_declared`. Must be a variant on the experiment.	`VARCHAR`	`'green'`

Returns: Table with columns operation, kind, id, name, status, description.

-- Update just the name and hypothesis
CALL experimentation.update_experiment(
  'homepage-cta-test', 'New Name', NULL, NULL, 'New hypothesis', NULL, NULL
);

-- Declare a winner
CALL experimentation.update_experiment(
  'homepage-cta-test', NULL, NULL, 'winner_declared', NULL, NULL, 'green'
);

-- End an experiment with a reason
CALL experimentation.update_experiment(
  'homepage-cta-test', NULL, NULL, 'ended', NULL, 'no_stat_sig', NULL
);

Get Current Cohort

Returns the current (highest-indexed) cohort for an experiment, with variant details.

Parameter	Description	Data Type	Example
`experiment_id`	ID of the experiment. Must exist and have at least one cohort.	`VARCHAR`	`'homepage-cta-test'`

Returns: Table with columns cohort_index (NUMBER), variant (VARCHAR), name (VARCHAR), description (VARCHAR), is_control (BOOLEAN), split (FLOAT).

CALL experimentation.get_current_cohort('homepage-cta-test');

Describe Experiment YAML

Given experiment YAML, returns a markdown-formatted description of the experiment including metadata, variants, and cohort details.

Parameter	Description	Data Type	Example
`yaml`	Full experiment YAML string.	`VARCHAR`	`(output from get_resource_yaml)`

Returns: Table with a single markdown column containing the formatted description.

-- First get the YAML, then describe it
CALL experimentation.get_resource_yaml('EXPERIMENT', 'homepage-cta-test');

-- Use the YAML output as input:
CALL experimentation.describe_experiment_yaml('<yaml string from above>');