Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.winningvariant.com/llms.txt

Use this file to discover all available pages before exploring further.

The Winning Variant Native App allows users to manage experimentation resources directly inside of Snowflake by use of stored procedures created during installation.

​
Roles

A user must be assigned of the following application roles to execute any of the procedures described on this page:
  • admin
  • editor

​
Resource Kinds

All references to kind below must be one of:
  • lab
  • subjecttype
  • experiment
Read more about Resource Kinds.

​
List Resources

Each of the following “list” commands returns a table with the following columns:
ColumnDescriptionData Type
kindThe resource kind.VARCHAR
idID of the resource.VARCHAR
nameName of the resource.VARCHAR
statusResource statusVARCHAR
descriptionResource descriptionVARCHAR
resource_versionCurrent resource versionINT
USE <application_name>;

CALL experimentation.list_resources('<kind>' [, <status>, ...]);
Returns a list of all resources of the given kind. status must be one of ResourceStatus.

​
Examples

-- List all
USE <application_name>;

CALL experimentation.list_resources('lab');

-- List active
CALL experimentation.list_resources('lab', ['active']);

​
Get YAML for a single resource

USE <application_name>;

CALL experimentation.get_resource_yaml('<kind>', '<id>');
Returns a table with the following columns:
ColumnDescriptionData Type
kindThe resource kind.VARCHAR
idID of the resource.VARCHAR
yamlThe full configuration (YAML) for a single resource.VARCHAR

​
Examples

USE <application_name>;

CALL experimentation.get_resource_yaml('experiment', 'my-experiment');

​
Create or update resource

USE <application_name>;

CALL experimentation.apply_resource('<yaml>');
If the resource with the id specified exists, it gets updated, otherwise it will be created.

​
Examples

USE <application_name>;

-- Create a new lab with id `marketing`
CALL experimentation.apply_resource($$schemaVersion: 1
kind: lab
metadata:
  id: marketing
  status: active
  name: Marketing
  description: All marketing and top-of-funnel experiments.
$$);

-- Update the name of the lab we just created
CALL experimentation.apply_resource($$schemaVersion: 1
kind: lab
metadata:
  id: marketing
  status: active
  name: Marketing Organization
  description: All marketing and top-of-funnel experiments.
$$);

​
Experimentation Convenience Methods

In addition to the above methods using raw YAML, you may also create/manage experiments with the following purpose-built convenience methods. All methods also have a *_preview variant is available that returns the YAML that would be applied without actually creating the experiment: 
CALL experimentation.create_experiment_preview(
  'homepage-cta-test', 'Homepage CTA Test', NULL, NULL, NULL, 'USER',
  '[{"id":"control","name":"Blue Button","isControl":true}]', NULL
);

​
Create Experiment

Creates a new experiment with the following inputs. The experiment is created in draft status.
ParameterDescriptionData TypeExample
idUnique identifier for the experiment. Alphanumeric, hyphens, underscores, dots only.VARCHAR'homepage-cta-test'
nameHuman-readable name for the experiment.VARCHAR'Homepage CTA Test'
descriptionOptional description of the experiment. Pass NULL to omit.VARCHAR'Testing button color on homepage'
parent_labOptional ID of the parent lab. Pass NULL to use the DEFAULT lab. Must exist if provided.VARCHAR'growth-lab'
hypothesisOptional hypothesis for the experiment. Pass NULL to omit.VARCHAR'A green CTA will increase conversions by 10%'
subject_typeID of the subject type. Must already exist.VARCHAR'USER'
variantsJSON array of variant objects. Each must have id, name, and isControl. Optional description. At most 1 control.VARCHAR (JSON)'[{"id":"control","name":"Blue Button","isControl":true},{"id":"green","name":"Green Button","isControl":false}]'
initial_cohortJSON array of cohort variant entries with variant and split. Splits must sum to 1.0. Requires variants. Pass NULL to omit.VARCHAR (JSON)'[{"variant":"control","split":0.5},{"variant":"green","split":0.5}]'
Returns: Table with columns operation, kind, id, name, status, description. 
CALL experimentation.create_experiment(
  'homepage-cta-test',
  'Homepage CTA Test',
  'Testing button color on homepage',
  NULL,
  'A green CTA will increase conversions by 10%',
  'USER',
  '[{"id":"control","name":"Blue Button","isControl":true},{"id":"green","name":"Green Button","isControl":false}]',
  '[{"variant":"control","split":0.5},{"variant":"green","split":0.5}]'
);

​
Add Variants

Adds one or more new variants to an existing experiment. Variant IDs must not conflict with existing variants. At most 1 control variant is allowed across all variants on the experiment.
ParameterDescriptionData TypeExample
experiment_idID of the experiment to add variants to. Must exist.VARCHAR'homepage-cta-test'
variantsJSON array of variant objects. Each must have id, name, and isControl. Optional description.VARCHAR (JSON)'[{"id":"red","name":"Red Button","isControl":false,"description":"Red variant"}]'
Returns: Table with columns operation, kind, id, name, status, description. 
CALL experimentation.add_variants(
  'homepage-cta-test',
  '[{"id":"red","name":"Red Button","isControl":false,"description":"A red CTA button"}]'
);

​
Add Cohort

Adds a new cohort to an existing experiment. The cohort index is automatically assigned as the next sequential value. The experiment must already have variants defined.
ParameterDescriptionData TypeExample
experiment_idID of the experiment. Must exist and have variants defined.VARCHAR'homepage-cta-test'
cohort_variantsJSON array of cohort entries. Each must have variant (must be an existing variant on the experiment) and split (0.001–1.0). Splits must sum to 1.0. At most 1 control variant.VARCHAR (JSON)'[{"variant":"control","split":0.34},{"variant":"green","split":0.33},{"variant":"red","split":0.33}]'
Returns: Table with columns operation, kind, id, name, status, description. 
CALL experimentation.add_cohort(
  'homepage-cta-test',
  '[{"variant":"control","split":0.34},{"variant":"green","split":0.33},{"variant":"red","split":0.33}]'
);

​
Set Experiment Status

Sets the status of an existing experiment.
ParameterDescriptionData TypeExample
experiment_idID of the experiment. Must exist.VARCHAR'homepage-cta-test'
statusNew status. Must be one of: active, archived, draft, ended, winner_declared.VARCHAR'active'
Returns: Table with columns operation, kind, id, name, status, description. 
CALL experimentation.set_experiment_status('homepage-cta-test', 'active');

​
Update Experiment

Updates one or more properties of an existing experiment. Pass NULL for any field that should not change. At least one field must be provided.
ParameterDescriptionData TypeExample
experiment_idID of the experiment. Must exist.VARCHAR'homepage-cta-test'
nameNew name. Pass NULL to keep current.VARCHAR'Updated CTA Test'
descriptionNew description. Pass NULL to keep current.VARCHAR'Revised experiment description'
statusNew status. Must be one of: active, archived, draft, ended, winner_declared. Pass NULL to keep current.VARCHAR'ended'
hypothesisNew hypothesis. Pass NULL to keep current.VARCHAR'Updated hypothesis'
ended_reasonReason the experiment ended. Only valid when status is ended. Must be one of: no_longer_needed, no_stat_sig, other_reason, success, tech_issue.VARCHAR'success'
winning_variantID of the winning variant. Only valid when status is winner_declared. Must be a variant on the experiment.VARCHAR'green'
Returns: Table with columns operation, kind, id, name, status, description. 
-- Update just the name and hypothesis
CALL experimentation.update_experiment(
  'homepage-cta-test', 'New Name', NULL, NULL, 'New hypothesis', NULL, NULL
);

-- Declare a winner
CALL experimentation.update_experiment(
  'homepage-cta-test', NULL, NULL, 'winner_declared', NULL, NULL, 'green'
);

-- End an experiment with a reason
CALL experimentation.update_experiment(
  'homepage-cta-test', NULL, NULL, 'ended', NULL, 'no_stat_sig', NULL
);

​
Get Current Cohort

Returns the current (highest-indexed) cohort for an experiment, with variant details.
ParameterDescriptionData TypeExample
experiment_idID of the experiment. Must exist and have at least one cohort.VARCHAR'homepage-cta-test'
Returns: Table with columns cohort_index (NUMBER), variant (VARCHAR), name (VARCHAR), description (VARCHAR), is_control (BOOLEAN), split (FLOAT). 
CALL experimentation.get_current_cohort('homepage-cta-test');

​
Describe Experiment YAML

Given experiment YAML, returns a markdown-formatted description of the experiment including metadata, variants, and cohort details.
ParameterDescriptionData TypeExample
yamlFull experiment YAML string.VARCHAR(output from get_resource_yaml)
Returns: Table with a single markdown column containing the formatted description. 
-- First get the YAML, then describe it
CALL experimentation.get_resource_yaml('EXPERIMENT', 'homepage-cta-test');

-- Use the YAML output as input:
CALL experimentation.describe_experiment_yaml('<yaml string from above>');