feat: tracking

[toddbaert]

This PR is a first pass at a relatively simple tracking API. The main purpose is to close what is more or less the last gap between the OpenFeature API and that of most SDKs (many SDKs which we currently support have some kind of simple "track" method analogous to what's proposed here). Preceding discussion can be found here.

Some notable choices:

• I've used the term "occurrence" instead of "event" to minimize terminology overloading
• I've ordered the parameters for the tracking method occurrenceKey/context/occurrenceDetails.
• I've specified the track method should never block but instead queue any async work; we agreed there's no reason an application author would ever want to await a I/O associated with a single track call; most providers will batch these, I assume
• I've made value a "first class" property of the otherwise free-form OccurrenceDetails type, so it can be easily mapped into the equivalent field of various providers (something like targetingKey)

[kinyoklion]

I think that this details should be equivalent to the structure value available for contexts. Allowing not just a single-level map of raw types, but effectively any JSON representation. If it is just a flag map, then it places what seems to be an unnecessary constraint on less restrictive systems.

[toddbaert]

I'm in agreement with this proposal, with the caveat that we're OK with some additional complexity in provider implementation.

[kozikow]

It should probably be included in another proposal, but I think it would be also useful to include a concept of "occurence aggregation". At it's core it would be like "list of SQL queries on occurences".

E.g. see following example - this is a fake scenario built on flagd + openfeature - here track occurence is upvote or downvote. Explore "metrics" by flag, variant:

Here "aggregation" is "Sentiment Score": AVG(IF(occurence_key="feedback_up", 1, IF(occurence_key="feedback_down", -1, 0))) and other similar type of metrics.

Then various dashboarding providers can out of the box set up experiment evaluation dashboard. Here Bigquery + Looker Studio is used, but the same flow would be possible with open source Metabase for example.

[nicklasl]

It should probably be included in another proposal, but I think it would be also useful to include a concept of "tracking aggregation". At it's core it would be like "list of SQL queries on tracking events".
...
Then various dashboarding providers can out of the box set up experiment evaluation dashboard. Here Bigquery + Looker Studio is used, but the same flow would be possible with open source Metabase for example.

While I appreciate the idea of incorporating "tracking aggregation" as a way to streamline dashboard creation, I believe it's crucial to keep the specifications clean and neutral regarding the underlying technology for data storage, combination, and aggregation.
This approach ensures that our specifications remain versatile and adaptable, allowing various teams/vendors to implement them using the tools and technologies that best fit their needs and existing infrastructure.

For example, there should be nothing stopping me from building a tracking provider that just outputs to a file if that's what my business requires.

However, I am open to have some type of documentation of best practices or examples of use and maybe this is where such proposal belong?

[kozikow]

I think big use case for occurrences is joining occurrences with variants to see what "business impact" flag changes have.

The way I understand it, most people will be just "joining" this with what's proposed in Feature Flag Impression SemConv

People just need something to join both entries on. SemConv proposal only suggest logging "context id" without full context. I have been using targetingId for this use case so far. I am thinking - wouldn't it make sense to have track method schema match schema of SemConv?

I am not advocating for any particular solution, but for more alignment between those two proposals and enabling use case of joining variant impressions with occurrences.

[toddbaert]

I think big use case for occurrences is joining occurrences with variants to see what "business impact" flag changes have.

The way I understand it, most people will be just "joining" this with what's proposed in Feature Flag Impression SemConv

100%. This is how I envision things would work, and how most people will use it, I think.

People just need something to join both entries on. SemConv proposal only suggest logging "context id" without full context. I have been using targetingId for this use case so far. I am thinking - wouldn't it make sense to have track method schema match schema of SemConv?

I am not advocating for any particular solution, but for more alignment between those two proposals and enabling use case of joining variant impressions with occurrences.

I struggled a lot with this same question.

I agree that generally most people would probably join on the targetingKey but I'm not sure that would be the case 100% of the time. If we include the entire evaluation context in the track API, users have the ability to join on whatever attributes they want which are common to the telemetry data from a flag evaluation and from the track call (perhaps an attribute called email, for example). It's for this maximum flexibility I proposed the API take the evaluation context instead of the targetingKey only.

Additionally, taking the entire evaluationContext is more consistent with existing functions, keeping the API a bit more isomorphic.

All that said, I understand why taking the targetingKey directly in the track method is appealing. Like I said, I struggled with this question for a while, and I'm open to other perspectives.

cc @beeme1mr @dyladan @nicklasl

[nicklasl]

💯 percent agree with what @toddbaert wrote on the flexibility aspect and I would like to add something:
Passing the whole evaluation context in the track API allows the tracking functionality to stand on its own legs as we (the OpenFeature API) provide enough data to the tracking providers to be enable slicing and dicing the produced tracking data by imaginative dimensions that could be deduced in the evaluation context.

Additionally:
We are talking about joining data between the moment of evaluation (I think of this as the moment of exposure) and the moment of event tracking. The evaluation context may have changed between these two moments in time, it can also be that these two moments are captured in services with different evaluation context.
The question is; what context is important to track?
If I could only choose one context to store, I would keep the context of the tracking event but of course, having both of these stored would for sure be the best. So I'm arguing that from a business perspective the context data from the event is more important than the context data from the evaluation.

Now there may be some nuances in the above reasoning between static- and dynamic context paradigms, especially when it comes to the moment of exposure in the static paradigm. I would argue that for the static paradigm it is not the flag evaluation (when the client bulk fetches the flags from the backend) but rather the flag access that should be counted as the moment of exposure.

[roncohen]

I see the spec below requires the ability to set both a Value + custom fields. Once we figured out the implementation details for OccurrenceDetails it would be nice to include an example that shows how to do both.

[toddbaert]

Good point, I've added this:

new OccurrenceDetails(99.77).add("currencyCode", "USD")

Which is probably how it will end up looking if implemented in Java, using our existing Structure objects.