Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: tracking #268

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions specification.json
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,13 @@
"RFC 2119 keyword": "MAY",
"children": []
},
{
"id": "Condition 2.7.1",
"machine_id": "condition_2_7_1",
"content": "The `provider` MAY define a function for tracking the occurrence of a particular user action or application state, with parameters `occurrence key` (string, required), `evaluation context` (optional) and `occurrence details` (optional) which returns nothing.",
"RFC 2119 keyword": "MAY",
"children": []
},
{
"id": "Requirement 3.1.1",
"machine_id": "requirement_3_1_1",
Expand Down Expand Up @@ -1044,6 +1051,64 @@
"content": "If the provider emits an event, the value of the client's `provider status` MUST be updated accordingly.",
"RFC 2119 keyword": "MUST",
"children": []
},
{
"id": "Condition 6.1.1",
"machine_id": "condition_6_1_1",
"content": "The implementation uses the dynamic-context paradigm.",
"RFC 2119 keyword": null,
"children": [
{
"id": "Conditional Requirement 6.1.1.1",
"machine_id": "conditional_requirement_6_1_1_1",
"content": "The `client` MUST define a function for tracking the occurrence of a particular action or application state, with parameters `occurrence key` (string, required), `evaluation context` (optional) and `occurrence details` (optional) which returns nothing.",
"RFC 2119 keyword": "MUST",
"children": []
}
]
},
{
"id": "Condition 6.1.2",
"machine_id": "condition_6_1_2",
"content": "The implementation uses the static-context paradigm.",
"RFC 2119 keyword": null,
"children": [
{
"id": "Conditional Requirement 6.1.2.1",
"machine_id": "conditional_requirement_6_1_2_1",
"content": "The `client` MUST define a function for tracking the occurrence of a particular action or application state, with parameters `occurrence key` (string, required) and `occurrence details` (optional) which returns nothing.",
"RFC 2119 keyword": "MUST",
"children": []
}
]
},
{
"id": "Requirement 6.1.3",
"machine_id": "requirement_6_1_3",
"content": "The evaluation context passed to the provider's track function MUST be merged in the order: API (global; lowest precedence) - transaction - client - invocation (highest precedence), with duplicate values being overwritten.",
"RFC 2119 keyword": "MUST",
"children": []
},
{
"id": "Requirement 6.1.4",
"machine_id": "requirement_6_1_4",
"content": "If the client's `track` function is called and the associated provider does not implement tracking, the client's `track` function MUST no-op.",
"RFC 2119 keyword": "MUST",
"children": []
},
{
"id": "Requirement 6.2.1",
"machine_id": "requirement_6_2_1",
"content": "The `occurrence details` structure MUST define an optional numeric `value`, associating a scalar quality with an `occurrence`.",
"RFC 2119 keyword": "MUST",
"children": []
},
{
"id": "Requirement 6.2.2",
"machine_id": "requirement_6_2_2",
"content": "The `occurrence details` MUST support the inclusion of custom fields, having keys of type `string`, and values of type `boolean | string | number | structure`.",
"RFC 2119 keyword": "MUST",
"children": []
}
]
}
5 changes: 5 additions & 0 deletions specification/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ This document defines some terms that are used across this specification.
- [Targeting Key](#targeting-key)
- [Fractional Evaluation](#fractional-evaluation)
- [Rule](#rule)
- [Tracking Occurrence](#tracking-occurrence)
- [SDK Paradigms](#sdk-paradigms)
- [Dynamic-Context Paradigm](#dynamic-context-paradigm)
- [Static-Context Paradigm](#static-context-paradigm)
Expand Down Expand Up @@ -195,6 +196,10 @@ Pseudorandomly resolve flag values using a context property, such as a targeting

A rule is some criteria that's used to determine which variant a particular context should be mapped to.

### Tracking Occurrence

A particular user action or application state representing a business objective or outcome, identified by a unique string, and recorded using the [tracking API](./sections/06-tracking.md).

## SDK Paradigms

Feature flag frameworks have SDKs which operate in two distinct paradigms: those designed for use with a single user client application (e.g. mobile phones, single-page web apps), and those designed for multi-user applications, such as web server applications. Some parts of the OpenFeature specification diverge depending on the paradigm.
Expand Down
31 changes: 31 additions & 0 deletions specification/sections/02-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,3 +256,34 @@ class MyProvider implements Provider {
Providers may maintain remote connections, timers, threads or other constructs that need to be appropriately disposed of.
Provider authors may implement a `shutdown` function to perform relevant clean-up actions.
Alternatively, implementations might leverage language idioms such as auto-disposable interfaces or some means of cancellation signal propagation to allow for graceful shutdown.

### 2.7. Tracking Support

[![experimental](https://img.shields.io/static/v1?label=Status&message=experimental&color=orange)](https://github.com/open-feature/spec/tree/main/specification#experimental)

Some flag management systems support tracking functionality, which can be used to associate feature flag evaluations with subsequent user actions or application state.

See [tracking](./06-tracking.md).

#### Condition 2.7.1

> The `provider` **MAY** define a function for tracking the occurrence of a particular user action or application state, with parameters `occurrence key` (string, required), `evaluation context` (optional) and `occurrence details` (optional) which returns nothing.

```java
class MyProvider implements Tracking {
//...

/**
* Record a tracking occurrence.
*/
void track(String occurrenceKey, EvaluationContext context, OccurrenceDetails details): void;

//...
}
```

The track function is a void function (function returning nothing).
The track function performs side effects required to record the `occurrence` in question, which may include network activity or other I/O; this I/O should not block the function call.
Providers should be careful to complete any communication or flush any relevant uncommitted tracking data before they shut down.

See [shutdown](#25-shutdown).
127 changes: 95 additions & 32 deletions specification/sections/06-tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,35 +10,98 @@ toc_max_heading_level: 4

## Overview

Experimentation is a primary use case for feature flags.
In practice, this often means flag variants are assigned to users at random or in accordance with a business rule, while the impact of the assigned variant on some business objective is measured.
Vendors and custom solutions often support a _tracking_ or _goal measuring_ API to facilitate the measurement of these business objectives.

### Goals

- Develop official terminology to support consistent implementation
- Specify a flexible API widely compatible with basic vendor functionality
- Define tracking event payload
- Define tracking event identifier
- Support A/B testing and experimentation use-cases
- Support client and server paradigms
- Provide recommendations around:
- Async vs sync
- Flushing mechanisms
- Event batching

### Non-goals

- Creating an experimentation platform
- Covering every user-tracking use case
- We will not define any data aggregation mechanisms
- We will not focus on "metrics", but instead, "facts"

### Design Principles

We value the following:

- Adherence to, and compatibility with OpenFeature semantics
- Maximum compatibility and ease-of-adoption for existing solutions
- Minimum traffic and payload size
- Ease-of-use for application authors, integrators, and provider authors (in that order)
The `tracking API` enables the association of feature flag evaluations with subsequent actions or application states, in order to facilitate experimentation, and analysis of the impact of feature flags on business objectives.

Combined with hooks which report feature flag evaluations to the analytics platform in question, tracking can allow for robust experimentation even for flag management systems that don't support tracking directly.

```mermaid
sequenceDiagram
Evaluation API->>+Tracking Hook: evaluate
Tracking Hook->>Analytics Platform: before
Tracking Hook->>Analytics Platform: after
Tracking Hook->>-Evaluation API: evaluate
Evaluation API->>Analytics Platform: track
```

### 6.1. Tracking API

#### Condition 6.1.1

> The implementation uses the dynamic-context paradigm.

see: [dynamic-context paradigm](../glossary.md#dynamic-context-paradigm)

##### Conditional Requirement 6.1.1.1

> The `client` **MUST** define a function for tracking the occurrence of a particular action or application state, with parameters `occurrence key` (string, required), `evaluation context` (optional) and `occurrence details` (optional) which returns nothing.

```java
// example tracking occurrence recording that a subject reached a page associated with a business goal
client.track("visited-promo-page", evaluationContext);
beeme1mr marked this conversation as resolved.
Show resolved Hide resolved

// example tracking occurrence recording that a subject performed an action associated with a business goal, with the occurrence details having a particular numeric value
client.track("clicked-checkout", evaluationContext, new OccurrenceDetails(99.77));

// example tracking occurrence recording that a subject performed an action associated with a business goal, with the occurrence details having a particular numeric value
client.track("clicked-checkout", evaluationContext, new OccurrenceDetails(99.77).add("currencyCode", "USD"));
```

See [evaluation context](../types.md#evaluation-context), [occurrence details](#62-occurrence-details).

#### Condition 6.1.2

[![experimental](https://img.shields.io/static/v1?label=Status&message=experimental&color=orange)](https://github.com/open-feature/spec/tree/main/specification#experimental)

> The implementation uses the static-context paradigm.

see: [static-context paradigm](../glossary.md#static-context-paradigm)

##### Conditional Requirement 6.1.2.1

> The `client` **MUST** define a function for tracking the occurrence of a particular action or application state, with parameters `occurrence key` (string, required) and `occurrence details` (optional) which returns nothing.

The track function is a void function (function returning nothing).
Though it may be associated with network activity or other I/O, it need not be awaited by application authors.

```java
// example tracking occurrence recording that a subject reached a page associated with a business goal
client.track("visited-promo-page");

// example tracking occurrence recording that a subject performed an action associated with a business goal, with the occurrence details having a particular numeric value
client.track("clicked-checkout", new OccurrenceDetails(99.77));

// example tracking occurrence recording that a subject performed an action associated with a business goal, with the occurrence details having a particular numeric and some additional details
client.track("clicked-checkout", new OccurrenceDetails(99.77).add("currencyCode", "USD"));
```

#### Requirement 6.1.3

> The evaluation context passed to the provider's track function **MUST** be merged in the order: API (global; lowest precedence) -> transaction -> client -> invocation (highest precedence), with duplicate values being overwritten.

The SDK passes a merged evaluation context to the provider's track function similarly to the manner it does in resolvers.

See: [context levels and merging](./03-evaluation-context.md#32-context-levels-and-merging).

#### Requirement 6.1.4

> If the client's `track` function is called and the associated provider does not implement tracking, the client's `track` function **MUST** no-op.

### 6.2. Occurrence Details

The `occurrence details` structure defines optional data pertinent to a particular `occurrence`.

#### Requirement 6.2.1

> The `occurrence details` structure **MUST** define an optional numeric `value`, associating a scalar quality with an `occurrence`.

`Value` is a well-defined field which some providers may map to equivalent numeric values in their API.

See [provider tracking support](./02-providers.md#27-tracking-support).

#### Requirement 6.2.2

> The `occurrence details` **MUST** support the inclusion of custom fields, having keys of type `string`, and values of type `boolean | string | number | structure`.

The `occurrence details` supports the addition of arbitrary fields, including nested objects, similar to the `evaluation context` and object-typed flag values.

See [structure](../types.md#structure), [evaluation context](.//03-evaluation-context.md).
8 changes: 8 additions & 0 deletions specification/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ An enumerated error code represented idiomatically in the implementation languag
| PROVIDER_FATAL | The provider has entered an irrecoverable error state. |
| GENERAL | The error was for a reason not enumerated above. |

### Evaluation Context

See [evaluation context](./sections/03-evaluation-context.md).

### Evaluation Options

A structure containing the following fields:
Expand Down Expand Up @@ -184,3 +188,7 @@ An enumeration of provider events.

A function or method which can be associated with a `provider event`, and runs when that event occurs.
It declares an `event details` parameter.

### Occurrence Details

See [occurrence details](./sections/06-tracking.md#62-occurrence-details).
Loading