Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.winningvariant.com/llms.txt

Use this file to discover all available pages before exploring further.

Winning Variant is designed to be simple to get started with. While there are many ways to configure and manage the app or customize your experiments, you can get started creating your first experiment right away. The Snowflake Native App manages its own infrastructure internally, so there’s nothing else you need to do once it’s installed and running.

​
Create an A/B test (experiment)

To create your first experiment, simply pass the experiment configuration (via YAML) to a built-in stored procedure apply_resource. 
CALL experimentation.apply_resource($$schemaVersion: 1
kind: experiment
metadata:
  id: my-first-experiment
  status: active
  name: My First Experiment
spec:
  variants:
    - id: control
      isControl: true
      name: Control
    - id: treatment
      isControl: false
      name: Treatment
  cohorts:
    - index: 1
      variants:
      - variant: control
        split: 0.5
      - variant: treatment
        split: 0.5
$$);
This creates an experiment named “My First Experiment” (ID: my-first-experiment). It has two variants (control and treatment), each receiving 50% of traffic in a single cohort.

​
Make Assignments

Every experiment needs to have some identifier for which traffic is randomized. Typically this is a user ID, anonymous ID, session ID, etc. We call this a Subject. An assignment is the variant that a subject ID is assigned within a given experiment. Once assigned, a subject ID will be pinned to that variant forever and always. There are currently 3 ways to implement experiments and to get/set assignments:
  1. Directly in SQL
  2. Via the RESTful API
  3. Using the Python SDK
Between these 3 methods, it’s possible to implement any type of split test in any environment, inside or outside of Snowflake. For the sake of example, we’ll create an assignment using SQL: 
SELECT experimentation.create_assignment('user_123', 'my-first-experiment');
SELECT experimentation.create_assignment('user_456', 'my-first-experiment');
This creates an assignment for each of two users in our new experiment. If you call this same commands over and over, you’ll see that the same variant is returned each time for the respective users. See other implementation examples

​
View the data

The internal assignment data pipeline is typically delayed 30-60 minutes (this is configurable), so you’ll need to set a timer and check back. When you do, you can query start to query the raw data, which will become the foundation for your experimentation reporting.

​
Experiments

You can view the experiment we created: 
SELECT * FROM experimentation.experiments;
.. or the variants in the experiment: 
SELECT * FROM experimentation.variants;
… or the cohorts in the experiment: 
SELECT * FROM experimentation.cohorts;
Read more about the dimension tables available

​
Assignments

You can also view the assignments you created above: 
SELECT * FROM experimentation.assignments;
Read more about analyzing assignments

​
You did it!

That’s the basic setup and implementation of split tests using the Winning Variant Snowflake Native App. At this point, you have everything in place to run live experiments and analyize the results. You may be asking, What about the reports? Well, we intentionally do not include reporting in the product. Companies often have nuanced metrics they care about and shouldn’t be forced to consume overly-opinionated dashboards that they have to log into separately. Instead, we provide access to raw experimentation data and are available to help your team create a custom analytics pipeline to ultimately render experiment results in your existing BI tool (e.g., Snowflake Dashboards, Tableau, Sigma, etc).