From 32ab69a9208c2de12383db98e0ded53e010a1d1f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 6 Sep 2021 21:50:03 -0600 Subject: [PATCH 01/34] Polls (mk II) --- proposals/3381-polls.md | 316 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 316 insertions(+) create mode 100644 proposals/3381-polls.md diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md new file mode 100644 index 0000000000..7863f5b37c --- /dev/null +++ b/proposals/3381-polls.md @@ -0,0 +1,316 @@ +# MSC3381: Chat Polls + +Polls are additive functionality in a room, allowing someone to pose a question and others to answer +until the poll is closed. In chat, these are typically used for quick questionares such as what to +have for lunch or when the next office party should be, not elections or anything needing truly +secret ballot. + +[MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) does introduce a different way of doing +polls (originally related to inline widgets, but diverged into `m.room.message`-based design). That +MSC's approach is discussed at length in the alternatives section for why it is inferior. + +## Proposal + +Polls are to be handled completely client-side and encrypted when possible in a given room. They are +simply started by sending an appropriate event, responded to with more events, and closed (eventually) +by the sender. Servers have no added requirements in order to support this MSC: they simply need to +be able to send arbitrary event types, which they already should be capable of. + +The events in this MSC make heavy use of [MSC1767: Extensible Events](https://github.com/matrix-org/matrix-doc/pull/1767). + +A poll can be started by sending an `m.poll` room event, similar to the following: + +```json5 +{ + "type": "m.poll", + "sender": "@alice:example.org", + "content": { + "m.poll": { + "question": { + "m.text": "What should we order for the party?" + }, + "kind": "m.open", + "answers": [ + { "m.text": "Pizza πŸ•" }, + { "m.text": "Poutine 🍟" }, + { "m.text": "Italian 🍝" }, + { "m.text": "Wings πŸ”₯" } + ] + }, + "m.message": [ + { + "mimetype": "text/plain", + "body": "What should we order for the party?\n1. Pizza πŸ•\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings πŸ”₯" + }, + { + "mimetype": "text/html", + "body": "What should we order for the party?
  1. 1. Pizza πŸ•
  2. 2. Poutine 🍟
  3. 3. Italian 🍝
  4. 4. Wings πŸ”₯
" + } + ] + }, + // other fields that aren't relevant here +} +``` + +As mentioned above, this is already making use of Extensible Events: The fallback for clients which don't +know how to render polls is to just post the message to the chat. Some of the properties also make use of +extensible events within them, such as the `question` and the elements of `answers`: these are effectively +`m.message` events (under the Extensible Events structure), which means they're required to have a plain +text component to them. HTML is allowed, though clients are generally encouraged to rely on the plain text +representation for an unbiased rendering. Meme value of HTML might be desirable to some clients, however. + +The `kind` refers to whether the poll is "secret" or "open". Secret polls reveal the results after the poll +has closed while open polls show the results at any time (or, if the client prefers, immediately after the +user has voted). These translate to `m.secret` and `m.open` under this MSC, though custom values using the +standardized naming convention are supported. Unknown values are to be treated as `m.secret` for maximum +compatibility with theoretical values. More specific detail as to the difference between open and secret +polls comes up later in this MSC. + +There is no limit to the number of `answers`, though more than 20 is considered bad form. Clients should +truncate the list at no less than 20. Similarly, there is no minimum though a poll of zero or one options +is fairly useless - clients should render polls with less than 2 options as invalid or otherwise unvotable. +Most polls are expected to have 2-8 options. + +The `m.message` fallback should be representative of the poll, but is not required and has no mandatory +format. Clients are encouraged to be inspired by the example above when sending poll events. + +To respond to a poll, the following event is sent: + +```json5 +{ + "type": "m.poll.response", + "sender": "@bob:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "event_id": "$poll" + }, + "m.poll.response": { + "answer": 2 // index of the answers array selected (zero-indexed) + } + }, + // other fields that aren't relevant here +} +``` + +Like `m.poll`, this `m.poll.response` event supports Extensible Events. However, it is strongly discouraged +for clients to include renderable types like `m.text` and `m.message` which could impact the usability of +the room (particularly for large rooms with lots of responses). The relationship is a normal MSC2675 reference +relationship, avoiding conflicts with message reactions described by [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677). + +**XXX**: It is almost certainly ideal if the server can aggregate the poll responses for us, but MSC2677 +crushes the source event type out of the equation, considering only the `key`. If MSC2677 were to consider +aggregating/grouping by event type and then by `key`, we could maintain the deliberate feature of being able +to react to polls while also aggregating poll responses. + +Users can vote multiple times, however only the user's most recent vote (by timestamp) shall be considered +by the client when calculating results. Votes are accepted until the poll is closed (again, by timestamp). + +The `answer` field is the zero-indexed position from the original `answers` array. Out of range or otherwise +invalid values must be considered a spoiled vote by a client. Spoiled votes are also how a user can "un-vote" +from a poll - redacting the vote event would cause the vote to become spoiled. + +Only the poll creator can close a poll. It is done as follows: +```json5 +{ + "type": "m.poll.end", + "sender": "@bob:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "event_id": "$poll" + }, + "m.poll.end": {}, + "m.text": "The poll has ended. Top answer: Poutine 🍟" + }, + // other fields that aren't relevant here +} +``` + +Once again, Extensible Events make an appearance here. There's nothing in particular metadata wise that +needs to appear in the `m.poll.end` property of `content`, though it is included for future capability. The +backup `m.text` representation is for fallback purposes and is completely optional with no strict format +requirements: the example above is just that, an example of what a client *could* do. Clients should be +careful to include a "top answer" in the end event as server lag might allow a few more responses to get +through while the closure is sent. Votes sent on or before the end event's timestamp are valid votes - all +others must be disregarded by clients. + +**Rationale**: Although clock drift is possible, as is clock manipulation, it is not anticipated that +polls will be closed while they are still receiving high traffic. There are some cases where clients might +apply local timers to auto-close polls, though these are typically used in extremely high traffic cases +such as Twitch-style audience polls - rejecting even 100 responses is unlikely to significantly affect +the results. Further, if a server were to manipulate its clock so that poll responses are sent after the +poll was closed, but timestamped for when it was open, the server is violating a social contract and likely +will be facing a ban from the room. This MSC does not propose a mitigation strategy beyond telling people +not to ruin the fun. Also, don't use polls for things that are important. + +Clients should disable voting interactions with polls once they are closed. Events which claim to close +the poll from senders other than the creator are to be treated as invalid and thus ignored. + +### Open polls + +These are most similar to what is seen on Twitch and often Twitter: members of the room are able to see +the results and vote accordingly. Clients are welcome to hide the poll results until after the user has +voted to avoid biasing the user. + +Once the poll ends, the results are shown regardless. + +### Secret polls + +With these polls, members of the room cannot see the results of a poll until the poll ends, regardless +of whether or not they've voted. This is enforced visually and not by the protocol given the votes +are sent to the room for local tallying - this is considered a social issue rather than a technical one. +Don't go spoiling the results if the sender didn't intend for you to see them. + +The poll results should additionally be hidden from the poll creator until the poll is closed by that +creator. + +## Potential issues + +As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this +isn't a huge deal, but it can be considered an issue with secret polls. This MSC strongly considers the +problem a social one: users who are looking to "cheat" at the results are unlikely to engage with the +poll in a productive way in the first place. And, of course, polls should never be used for something +important like electing a new leader for a country. + +Poll responses are also de-anonymized by nature of having the sender attached to a response. Clients +are strongly encouraged to demonstrate anonymization by not showing who voted for who, but might want +to warn/hint at the user that their vote is not anonymous. For example, saying "22 total responses, +including from TravisR, Matthew, and Alice" before the user votes. + +Limiting polls to client-side enforcement could be problematic if the MSC was interested in reliable +or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. +Bolt-on support for signing, verification, validity, etc can be accomplished as well in the future. + +The fallback support relies on clients already knowing about extensible events, which might not be +the case. Bridges (as of writing) do not have support for extensible events, for example, which can +mean that polls are lost in transit. This is perceived to be a similar amount of data loss when a Matrix +user reacts to an IRC user's message: the IRC user has no idea what happened on Matrix. Bridges, and +other clients, can trivially add message parsing support as described by extensible events to work +around this. The recommendations of this MSC specifically avoid the vote spam from being bridged, but +the start of poll and end of poll (results) would be bridged. There's an argument to be made for +surrounding conversation context being enough to communicate the results without extensible events, +though this is slightly less reliable. + +Though more important for Extensible Events, clients might get confused about what they should do +with the `m.message` parts of the events. For absolute clarity: if a client has support for polls, +it can outright ignore any irrelevant data from the events such as the message fallback or other +representations that senders stick onto the event (like thumbnails, captions, attachments, etc). + +## Alternatives + +The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) +which describes not only polls but also inline widgets. The poll implementation in MSC2192 is primarily +based around `m.room.message` events, using `msgtype` to differentiate between the different states. As +[a thread](https://github.com/matrix-org/matrix-doc/pull/2192/files#r514497274) notes on the MSC, this +is an awful experience on clients which do not support polls properly, leaving an irritating amount of +contextless messages in the timeline. Though not directly mentioned on that thread, polls also cannot be +closed under that MSC which leads to people picking options hours or even days after the poll has "ended". +This MSC instead proposed to only supply fallback on the start and end of a poll, leading to enough context +for unsupporting clients without flooding the room with messages. + +Originally, MSC2192 was intended to propose polls as a sort of widget with access to timeline events +and other important information, however the widget infrastructure is just not ready for this sort of +thing to exist. First, we'd need to be able to send events to the widget which reference itself (for +counting votes), and allow the widget to self-close if needed. This is surprisingly difficult when widgets +can be "popped out" or have a link clicked in lieu of rendering (for desktop clients): there's no +communication channel back to the client to get the information back and forth. Some of this can be solved +with scoped access tokens for widgets, though at the time of writing those are a long ways out. In the +end, it's simply more effective to use Extensible Events and Matrix directly rather than building out +the widgets infrastructure to cope - MSC2192 is a demonstration of this, considering it ended up taking +out all the widget aspects and replacing them with fields in the content. + +Finally, MSC2192 is simply inferior due to not being able to restrict who can post a poll. Responses +and closures can also be limited arbitrarily by room admins, so clients might want to check to make +sure that the sender has a good chance of being able to close the poll they're about to create just +to avoid future issues. + +## Security considerations + +As mentioned a multitude of times throughout this proposal, this MSC's approach is prone to disclosure +of votes and has a couple abuse vectors which make it not suitable for important or truly secret votes. +Do not use this functionality to vote for presidents. + +Clients should apply a large amount of validation to each field when interacting with polls. Event +bodies are already declared as completely untrusted, though not all clients apply a layer of validation. +In general, this MSC aims to try and show something of use to users so they can at least figure out +what the sender intended, though clients are also welcome to just hide invalid events/responses (with +the exception of spoiled votes: those are treated as "unvoting" or chosing nothing). Clients are +encouraged to try and fall back to something sensible, even if just an error message saying the poll +is invalid. + +## Unstable prefix + +While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place +of `m.`. Note that extensible events has a different unstable prefix for those fields. + +The 3 examples above can be rewritten as: + +```json5 +{ + "type": "org.matrix.msc3381.poll", + "sender": "@alice:example.org", + "content": { + "org.matrix.msc3381.poll": { + "question": { + "org.matrix.msc1767.text": "What should we order for the party?" + }, + "kind": "m.open", + "answers": [ + { "org.matrix.msc1767.text": "Pizza πŸ•" }, + { "org.matrix.msc1767.text": "Poutine 🍟" }, + { "org.matrix.msc1767.text": "Italian 🍝" }, + { "org.matrix.msc1767.text": "Wings πŸ”₯" } + ] + }, + "org.matrix.msc1767.message": [ + { + "mimetype": "text/plain", + "body": "What should we order for the party?\n1. Pizza πŸ•\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings πŸ”₯" + }, + { + "mimetype": "text/html", + "body": "What should we order for the party?
  1. 1. Pizza πŸ•
  2. 2. Poutine 🍟
  3. 3. Italian 🍝
  4. 4. Wings πŸ”₯
" + } + ] + }, + // other fields that aren't relevant here +} +``` + +```json5 +{ + "type": "org.matrix.msc3381.poll.response", + "sender": "@bob:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "event_id": "$poll" + }, + "org.matrix.msc3381.poll.response": { + "answer": 2 // index of the answers array selected (zero-indexed) + } + }, + // other fields that aren't relevant here +} +``` + +```json5 +{ + "type": "org.matrix.msc3381.poll.end", + "sender": "@bob:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "event_id": "$poll" + }, + "org.matrix.msc3381.poll.end": {}, + "org.matrix.msc1767.text": "The poll has ended. Top answer: Poutine 🍟" + }, + // other fields that aren't relevant here +} +``` + +Note that the extensible event fallbacks did not fall back to `m.room.message` in this MSC: this +is deliberate to ensure polls are treated as first-class citizens. Client authors not willing/able +to support polls are encouraged to instead support Extensible Events for better fallbacks. From a2baf26fcd412be00b902436b1b51fdedcabe6a0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 22:41:23 -0600 Subject: [PATCH 02/34] m.poll.start --- proposals/3381-polls.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 7863f5b37c..6540198f1c 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -18,14 +18,14 @@ be able to send arbitrary event types, which they already should be capable of. The events in this MSC make heavy use of [MSC1767: Extensible Events](https://github.com/matrix-org/matrix-doc/pull/1767). -A poll can be started by sending an `m.poll` room event, similar to the following: +A poll can be started by sending an `m.poll.start` room event, similar to the following: ```json5 { - "type": "m.poll", + "type": "m.poll.start", "sender": "@alice:example.org", "content": { - "m.poll": { + "m.poll.start": { "question": { "m.text": "What should we order for the party?" }, @@ -93,7 +93,7 @@ To respond to a poll, the following event is sent: } ``` -Like `m.poll`, this `m.poll.response` event supports Extensible Events. However, it is strongly discouraged +Like `m.poll.start`, this `m.poll.response` event supports Extensible Events. However, it is strongly discouraged for clients to include renderable types like `m.text` and `m.message` which could impact the usability of the room (particularly for large rooms with lots of responses). The relationship is a normal MSC2675 reference relationship, avoiding conflicts with message reactions described by [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677). From b9f1a72737347f93ef7924c088a2cb886c3e1d45 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 22:43:34 -0600 Subject: [PATCH 03/34] m.poll.* types --- proposals/3381-polls.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 6540198f1c..5f6ceca1bf 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -29,7 +29,7 @@ A poll can be started by sending an `m.poll.start` room event, similar to the fo "question": { "m.text": "What should we order for the party?" }, - "kind": "m.open", + "kind": "m.poll.open", "answers": [ { "m.text": "Pizza πŸ•" }, { "m.text": "Poutine 🍟" }, @@ -61,10 +61,10 @@ representation for an unbiased rendering. Meme value of HTML might be desirable The `kind` refers to whether the poll is "secret" or "open". Secret polls reveal the results after the poll has closed while open polls show the results at any time (or, if the client prefers, immediately after the -user has voted). These translate to `m.secret` and `m.open` under this MSC, though custom values using the -standardized naming convention are supported. Unknown values are to be treated as `m.secret` for maximum -compatibility with theoretical values. More specific detail as to the difference between open and secret -polls comes up later in this MSC. +user has voted). These translate to `m.poll.secret` and `m.poll.open` under this MSC, though custom values +using the standardized naming convention are supported. Unknown values are to be treated as `m.poll.secret` +for maximum compatibility with theoretical values. More specific detail as to the difference between open +and secret polls comes up later in this MSC. There is no limit to the number of `answers`, though more than 20 is considered bad form. Clients should truncate the list at no less than 20. Similarly, there is no minimum though a poll of zero or one options From 211f5918189f4cf8ab7eb90250a3ba57c8212ae4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 22:53:11 -0600 Subject: [PATCH 04/34] Allow multiple selections --- proposals/3381-polls.md | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 5f6ceca1bf..0ca256dca5 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -30,11 +30,12 @@ A poll can be started by sending an `m.poll.start` room event, similar to the fo "m.text": "What should we order for the party?" }, "kind": "m.poll.open", + "max_selections": 1, "answers": [ - { "m.text": "Pizza πŸ•" }, - { "m.text": "Poutine 🍟" }, - { "m.text": "Italian 🍝" }, - { "m.text": "Wings πŸ”₯" } + { "id": "pizza", "m.text": "Pizza πŸ•" }, + { "id": "poutine", "m.text": "Poutine 🍟" }, + { "id": "italian", "m.text": "Italian 🍝" }, + { "id": "wings", "m.text": "Wings πŸ”₯" } ] }, "m.message": [ @@ -69,7 +70,11 @@ and secret polls comes up later in this MSC. There is no limit to the number of `answers`, though more than 20 is considered bad form. Clients should truncate the list at no less than 20. Similarly, there is no minimum though a poll of zero or one options is fairly useless - clients should render polls with less than 2 options as invalid or otherwise unvotable. -Most polls are expected to have 2-8 options. +Most polls are expected to have 2-8 options. The answer `id` is an arbitrary string used within the polls +schemas. Clients should not attempt to parse or understand it. + +`max_selections` is optional and denotes the maximum number of responses a user is able to select. Users +can select fewer options, but not more. This defaults to `1`. Cannot be less than 1. The `m.message` fallback should be representative of the poll, but is not required and has no mandatory format. Clients are encouraged to be inspired by the example above when sending poll events. @@ -86,7 +91,9 @@ To respond to a poll, the following event is sent: "event_id": "$poll" }, "m.poll.response": { - "answer": 2 // index of the answers array selected (zero-indexed) + "answers": [ + "poutine", + ] } }, // other fields that aren't relevant here @@ -106,9 +113,11 @@ to react to polls while also aggregating poll responses. Users can vote multiple times, however only the user's most recent vote (by timestamp) shall be considered by the client when calculating results. Votes are accepted until the poll is closed (again, by timestamp). -The `answer` field is the zero-indexed position from the original `answers` array. Out of range or otherwise -invalid values must be considered a spoiled vote by a client. Spoiled votes are also how a user can "un-vote" -from a poll - redacting the vote event would cause the vote to become spoiled. +The `answers` array in the response is the user's selection(s) for the poll. Clients should only consider +the first `max_selections` worth of entries as valid: anything beyond that is simply ignored. The entries +are the `id` of each answer from the original poll start event. If *any* of the supplied answers is unknown, +or the field is otherwise invalid, then the user's vote is spoiled. Spoiled votes are also how users can +"un-vote" from a poll - redacting, or setting `answers` to an empty array, will spoil that user's vote. Only the poll creator can close a poll. It is done as follows: ```json5 From 851e7e42c2229a7533438d0efa1e31e2e002f7c2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 22:56:08 -0600 Subject: [PATCH 05/34] Give moderators access to closing polls --- proposals/3381-polls.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 0ca256dca5..92703aae0d 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -119,7 +119,13 @@ are the `id` of each answer from the original poll start event. If *any* of the or the field is otherwise invalid, then the user's vote is spoiled. Spoiled votes are also how users can "un-vote" from a poll - redacting, or setting `answers` to an empty array, will spoil that user's vote. -Only the poll creator can close a poll. It is done as follows: +Only the poll creator or anyone with a suitable power level for redactions can close the poll. The rationale +for using the redaction power level is to help aid moderation efforts: while moderators can just redact the +original poll and invalidate it entirely, they might prefer to just close it and leave it on the historical +record. + +Closing a poll is done as follows: + ```json5 { "type": "m.poll.end", From b350e4e5e8f942bdc26d2eaaa9b9bd3796b3ad23 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 23:05:51 -0600 Subject: [PATCH 06/34] Mention freeform edits --- proposals/3381-polls.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 92703aae0d..97e98625e8 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -254,6 +254,15 @@ the exception of spoiled votes: those are treated as "unvoting" or chosing nothi encouraged to try and fall back to something sensible, even if just an error message saying the poll is invalid. +## Future considerations + +Some aspects of polls are explicitly not covered by this MSC, and are intended for another future MSC +to solve: + +* Allowing voters/room members to add their own freeform options. The edits system doesn't prevent other + members from editing messages, though clients tend to reject edits which are not made by the original + author. Altering this rule to allow it on some polls could be useful in a future MSC. + ## Unstable prefix While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place From 17b10595e8b985018c577980e8c7c55852450abe Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 23:08:34 -0600 Subject: [PATCH 07/34] Mention how edits are a security issue --- proposals/3381-polls.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 97e98625e8..801e40db8b 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -254,6 +254,12 @@ the exception of spoiled votes: those are treated as "unvoting" or chosing nothi encouraged to try and fall back to something sensible, even if just an error message saying the poll is invalid. +Users should be wary of polls changing their question after they have voted. Considering polls can be +edited, responses might no longer be relevant. For example, if a poll was opened for "do you like +cupcakes?" and you select "yes", the question may very well become "should we outlaw cupcakes?" where +your "yes" no longer applies. This MSC considers this problem more of a social issue than a technical +one, and reminds the reader that polls should not be used for anything important/serious at the moment. + ## Future considerations Some aspects of polls are explicitly not covered by this MSC, and are intended for another future MSC From b8ceeb9bfbc6a30875fc76d7ff373d3e2917ab4d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 23:14:41 -0600 Subject: [PATCH 08/34] Enforce minimum and maximum length --- proposals/3381-polls.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 801e40db8b..575bb201bd 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -67,11 +67,9 @@ using the standardized naming convention are supported. Unknown values are to be for maximum compatibility with theoretical values. More specific detail as to the difference between open and secret polls comes up later in this MSC. -There is no limit to the number of `answers`, though more than 20 is considered bad form. Clients should -truncate the list at no less than 20. Similarly, there is no minimum though a poll of zero or one options -is fairly useless - clients should render polls with less than 2 options as invalid or otherwise unvotable. -Most polls are expected to have 2-8 options. The answer `id` is an arbitrary string used within the polls -schemas. Clients should not attempt to parse or understand it. +`answers` must be an array with at least 1 option and no more than 20. Lengths outside this range are invalid +and must not be rendered by clients. Most polls are expected to have 2-8 options. The answer `id` is an +arbitrary string used within the poll events. Clients should not attempt to parse or understand the `id`. `max_selections` is optional and denotes the maximum number of responses a user is able to select. Users can select fewer options, but not more. This defaults to `1`. Cannot be less than 1. From b4552662bf7d04e7bb9244b93fcbf9dca2a02e0e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 23:29:37 -0600 Subject: [PATCH 09/34] Rename open and secret --- proposals/3381-polls.md | 54 +++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 575bb201bd..15af154aaa 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -29,7 +29,7 @@ A poll can be started by sending an `m.poll.start` room event, similar to the fo "question": { "m.text": "What should we order for the party?" }, - "kind": "m.poll.open", + "kind": "m.poll.disclosed", "max_selections": 1, "answers": [ { "id": "pizza", "m.text": "Pizza πŸ•" }, @@ -60,12 +60,12 @@ extensible events within them, such as the `question` and the elements of `answe text component to them. HTML is allowed, though clients are generally encouraged to rely on the plain text representation for an unbiased rendering. Meme value of HTML might be desirable to some clients, however. -The `kind` refers to whether the poll is "secret" or "open". Secret polls reveal the results after the poll -has closed while open polls show the results at any time (or, if the client prefers, immediately after the -user has voted). These translate to `m.poll.secret` and `m.poll.open` under this MSC, though custom values -using the standardized naming convention are supported. Unknown values are to be treated as `m.poll.secret` -for maximum compatibility with theoretical values. More specific detail as to the difference between open -and secret polls comes up later in this MSC. +The `kind` refers to whether the poll's votes are disclosed while the poll is still open. `m.poll.undisclosed` +means the results are revealed once the poll is closed. `m.poll.disclosed` is the opposite: the votes are +visible up until and including when the poll is closed. Custom values are permitted using the standardized +naming convention are supported. Unknown values are to be treated as `m.poll.undisclosed` for maximum +compatibility with theoretical values. More specific detail as to the difference between two polls come up +later in this MSC. `answers` must be an array with at least 1 option and no more than 20. Lengths outside this range are invalid and must not be rendered by clients. Most polls are expected to have 2-8 options. The answer `id` is an @@ -160,36 +160,33 @@ not to ruin the fun. Also, don't use polls for things that are important. Clients should disable voting interactions with polls once they are closed. Events which claim to close the poll from senders other than the creator are to be treated as invalid and thus ignored. -### Open polls +### Disclosed versus undisclosed polls -These are most similar to what is seen on Twitch and often Twitter: members of the room are able to see -the results and vote accordingly. Clients are welcome to hide the poll results until after the user has -voted to avoid biasing the user. +Disclosed polls are most similar to what is seen on Twitch and often Twitter: members of the room are able +to see the results and vote accordingly. Clients are welcome to hide the poll results until after the user +has voted to avoid biasing the user. -Once the poll ends, the results are shown regardless. +Undisclosed polls do track who voted for what, though don't reveal the results until the poll has been +closed, even after a user has voted themselves. This is enforced visually and not by the protocol given +the votes are sent to the room for local tallying - this is considered more of a social trust issue than +a technical one. This MSC expects that rooms (and clients) won't spoil the results of an undisclosed poll +before it is closed. -### Secret polls - -With these polls, members of the room cannot see the results of a poll until the poll ends, regardless -of whether or not they've voted. This is enforced visually and not by the protocol given the votes -are sent to the room for local tallying - this is considered a social issue rather than a technical one. -Don't go spoiling the results if the sender didn't intend for you to see them. - -The poll results should additionally be hidden from the poll creator until the poll is closed by that -creator. +In either case, once the poll ends the results are shown regardless of kind. Clients might wish to avoid +disclosing who voted for what in an undisclosed poll, though this MSC leaves that at just a suggestion. ## Potential issues As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this -isn't a huge deal, but it can be considered an issue with secret polls. This MSC strongly considers the -problem a social one: users who are looking to "cheat" at the results are unlikely to engage with the +isn't a huge deal, but it can be considered an issue with undisclosed polls. This MSC strongly considers +the problem a social one: users who are looking to "cheat" at the results are unlikely to engage with the poll in a productive way in the first place. And, of course, polls should never be used for something important like electing a new leader for a country. Poll responses are also de-anonymized by nature of having the sender attached to a response. Clients -are strongly encouraged to demonstrate anonymization by not showing who voted for who, but might want -to warn/hint at the user that their vote is not anonymous. For example, saying "22 total responses, -including from TravisR, Matthew, and Alice" before the user votes. +are strongly encouraged to demonstrate anonymization by not showing who voted for who, but should consider +warning the user that their vote is not anonymous. For example, saying "22 total responses, including +from TravisR, Matthew, and Alice" before the user votes. Limiting polls to client-side enforcement could be problematic if the MSC was interested in reliable or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. @@ -267,6 +264,11 @@ to solve: members from editing messages, though clients tend to reject edits which are not made by the original author. Altering this rule to allow it on some polls could be useful in a future MSC. +* Verifiable or cryptographically secret polls. There is interest in a truly enforceable undisclosed poll + where even if the client wanted to it could not reveal the results before the poll is closed. Approaches + like [MSC3184](https://github.com/matrix-org/matrix-doc/pull/3184) or Public Key Infrastructure (PKI) + might be worthwhile to investigate in a future MSC. + ## Unstable prefix While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place From b5347dd2509245109497d0dd09d3ea6dc3007c58 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 11 Oct 2021 23:35:24 -0600 Subject: [PATCH 10/34] Mention message pinning --- proposals/3381-polls.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 15af154aaa..26019b28f7 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -269,6 +269,11 @@ to solve: like [MSC3184](https://github.com/matrix-org/matrix-doc/pull/3184) or Public Key Infrastructure (PKI) might be worthwhile to investigate in a future MSC. +## Other notes + +If a client/user wishes to make a poll statically visible, they should check out +[pinned messages](https://matrix.org/docs/spec/client_server/r0.6.1#m-room-pinned-events). + ## Unstable prefix While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place From d2a356b2f41b3e8db6e43120924d412eaeb9ea50 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 12 Oct 2021 15:29:54 -0600 Subject: [PATCH 11/34] Update unstable definition for polls --- proposals/3381-polls.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 26019b28f7..5b2ca7b8a3 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -283,19 +283,19 @@ The 3 examples above can be rewritten as: ```json5 { - "type": "org.matrix.msc3381.poll", + "type": "org.matrix.msc3381.poll.start", "sender": "@alice:example.org", "content": { - "org.matrix.msc3381.poll": { + "org.matrix.msc3381.poll.start": { "question": { "org.matrix.msc1767.text": "What should we order for the party?" }, - "kind": "m.open", + "kind": "org.matrix.msc3381.poll.disclosed", "answers": [ - { "org.matrix.msc1767.text": "Pizza πŸ•" }, - { "org.matrix.msc1767.text": "Poutine 🍟" }, - { "org.matrix.msc1767.text": "Italian 🍝" }, - { "org.matrix.msc1767.text": "Wings πŸ”₯" } + { "id": "pizza", "org.matrix.msc1767.text": "Pizza πŸ•" }, + { "id": "poutine", "org.matrix.msc1767.text": "Poutine 🍟" }, + { "id": "italian", "org.matrix.msc1767.text": "Italian 🍝" }, + { "id": "wings", "org.matrix.msc1767.text": "Wings πŸ”₯" } ] }, "org.matrix.msc1767.message": [ @@ -323,7 +323,7 @@ The 3 examples above can be rewritten as: "event_id": "$poll" }, "org.matrix.msc3381.poll.response": { - "answer": 2 // index of the answers array selected (zero-indexed) + "answers": ["italian"] } }, // other fields that aren't relevant here From 91494282a26af1bcf94dd29c51aef8274972229d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 23 Nov 2021 13:45:46 -0700 Subject: [PATCH 12/34] Rework responses into aggregations/annotations --- proposals/3381-polls.md | 138 ++++++++++++++++++++++++++++++++-------- 1 file changed, 111 insertions(+), 27 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 5b2ca7b8a3..ad25334f31 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -85,13 +85,12 @@ To respond to a poll, the following event is sent: "sender": "@bob:example.org", "content": { "m.relates_to": { - "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "rel_type": "m.annotation", // from MSC2677: https://github.com/matrix-org/matrix-doc/pull/2677 + "key": "poutine", "event_id": "$poll" }, "m.poll.response": { - "answers": [ - "poutine", - ] + // empty object: useful information is in the relationship `key`. } }, // other fields that aren't relevant here @@ -100,22 +99,40 @@ To respond to a poll, the following event is sent: Like `m.poll.start`, this `m.poll.response` event supports Extensible Events. However, it is strongly discouraged for clients to include renderable types like `m.text` and `m.message` which could impact the usability of -the room (particularly for large rooms with lots of responses). The relationship is a normal MSC2675 reference -relationship, avoiding conflicts with message reactions described by [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677). - -**XXX**: It is almost certainly ideal if the server can aggregate the poll responses for us, but MSC2677 -crushes the source event type out of the equation, considering only the `key`. If MSC2677 were to consider -aggregating/grouping by event type and then by `key`, we could maintain the deliberate feature of being able -to react to polls while also aggregating poll responses. - -Users can vote multiple times, however only the user's most recent vote (by timestamp) shall be considered -by the client when calculating results. Votes are accepted until the poll is closed (again, by timestamp). - -The `answers` array in the response is the user's selection(s) for the poll. Clients should only consider -the first `max_selections` worth of entries as valid: anything beyond that is simply ignored. The entries -are the `id` of each answer from the original poll start event. If *any* of the supplied answers is unknown, -or the field is otherwise invalid, then the user's vote is spoiled. Spoiled votes are also how users can -"un-vote" from a poll - redacting, or setting `answers` to an empty array, will spoil that user's vote. +the room (particularly for large rooms with lots of responses). + +Note that the response event annotates the poll start event, forming a relationship which can be aggregated +by the server under [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). The event also supports +only one answer: for polls which permit multiple responses, and where multiple responses are given, the +client would send multiple response events. If the user deselected (unvoted) an option, the corresponding +response event would be redacted to match [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677)'s +handling of "un-reacting" to an event. + +This MSC borrows a lot of the structure provided by reactions in [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) +but intentionally does not define how notifications (if desired) or other UX features work: this MSC +purely intends to use the server-side aggregation capability where possible. + +If a user sends multiple responses for the same answer, the user is considered to have voted for the answer +only once. They will need to redact/unvote *all* of those answers to have been fully considered as not voting +for that answer. + +Votes are accepted until the poll is closed according to timestamp: servers/clients which receive votes +which are timestamped before the close event's timestamp (or, when no close event has been sent) are valid. +Late votes should be ignored. Early votes (from before the start event) are considered to be valid for the +sake of handling clock drift as gracefully as possible. + +To enforce `max_selections`, distinct responses are ordered by timestamp and truncated at `max_selections`. +For example, if a user votes as `[A, B, A, A, D]` and `max_selections` is 2, then the valid votes would be +`[B, A]` (because `A` was voted for multiple times, the most recent being after `B` was voted for). `D` +would simply be ignored as it is out of range. If the user redacted *all* of their `A` votes, then it'd +be `[B, D]`. + +Responses with a non-sensical `key` (eg: not a valid answer) are simply ignored. This is primarily important +when using server-side endpoints for fetching all relations: some will be emoji or short text strings to +denote reactions. Those events can be implicitly ignored with sufficiently complex/unlikely answer IDs as +the client would automatically filter out `πŸ‘` reactions. This is particularly important in a world where +the event type for all associated events is `m.room.encrypted` rather than `m.reaction` or `m.poll.response` +from a server's perspective. Only the poll creator or anyone with a suitable power level for redactions can close the poll. The rationale for using the redaction power level is to help aid moderation efforts: while moderators can just redact the @@ -130,7 +147,7 @@ Closing a poll is done as follows: "sender": "@bob:example.org", "content": { "m.relates_to": { - "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 "event_id": "$poll" }, "m.poll.end": {}, @@ -175,6 +192,33 @@ before it is closed. In either case, once the poll ends the results are shown regardless of kind. Clients might wish to avoid disclosing who voted for what in an undisclosed poll, though this MSC leaves that at just a suggestion. +### Server behaviour + +Much of the handling for this proposal is covered by other MSCs already: + +* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how to aggregate the annotations + (poll responses), though notes that encrypted events can potentially be an issue. The aggregation is + also unaware of a stop time to honour the poll closure. MSC2677 also defines that users may only annotate + with a given key once, preventing most issues of users voting for the same answer multiple times. +* [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines the server-side aggregation approach + which can be useful to clients to determine which votes there are on an event. +* [MSC0001](https://github.com/matrix-org/matrix-doc/pull/0001) defines how clients can get all relations + for an event between point A and B (namely the poll start and close). + +No further behaviour is defined by this MSC: servers do not have to understand the rules of a poll in order +to support the client's implementation. They do however need to implement a lot of server-side handling for +the above MSCs. + +### Client behaviour + +Clients should rely on [MSC0001](https://github.com/matrix-org/matrix-doc/pull/0001) and +[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) for handling limited ("gappy") syncs. +Otherwise, it is anticipated that clients re-process polls entirely on their own to ensure accurate counts +with encrypted events (the response events might be encrypted, so the server-side aggregations endpoint +will be unaware of whether an event is a reaction, poll response, or some other random type). As mentioned +in the proposal text, clients should filter out non-sensical `key`s to automatically filter out reactions +and other non-poll-response types. + ## Potential issues As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this @@ -184,9 +228,9 @@ poll in a productive way in the first place. And, of course, polls should never important like electing a new leader for a country. Poll responses are also de-anonymized by nature of having the sender attached to a response. Clients -are strongly encouraged to demonstrate anonymization by not showing who voted for who, but should consider +are strongly encouraged to demonstrate anonymization by not showing who voted for what, but should consider warning the user that their vote is not anonymous. For example, saying "22 total responses, including -from TravisR, Matthew, and Alice" before the user votes. +from TravisR, Matthew, and Alice" before the user casts their own vote. Limiting polls to client-side enforcement could be problematic if the MSC was interested in reliable or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. @@ -279,6 +323,9 @@ If a client/user wishes to make a poll statically visible, they should check out While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place of `m.`. Note that extensible events has a different unstable prefix for those fields. +**Note**: Due to changes during the unstable period, poll responses are additionally annotated with a +`v2` to denote a change on November 21, 2021. For details, see below. + The 3 examples above can be rewritten as: ```json5 @@ -315,15 +362,16 @@ The 3 examples above can be rewritten as: ```json5 { - "type": "org.matrix.msc3381.poll.response", + "type": "org.matrix.msc3381.v2.poll.response", // note the v2 in the event type! "sender": "@bob:example.org", "content": { "m.relates_to": { - "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "rel_type": "m.annotation", + "key": "italian", "event_id": "$poll" }, "org.matrix.msc3381.poll.response": { - "answers": ["italian"] + // empty body } }, // other fields that aren't relevant here @@ -336,7 +384,7 @@ The 3 examples above can be rewritten as: "sender": "@bob:example.org", "content": { "m.relates_to": { - "rel_type": "m.reference", // from MSC2675: https://github.com/matrix-org/matrix-doc/pull/2675 + "rel_type": "m.reference", "event_id": "$poll" }, "org.matrix.msc3381.poll.end": {}, @@ -349,3 +397,39 @@ The 3 examples above can be rewritten as: Note that the extensible event fallbacks did not fall back to `m.room.message` in this MSC: this is deliberate to ensure polls are treated as first-class citizens. Client authors not willing/able to support polls are encouraged to instead support Extensible Events for better fallbacks. + +### Historical implementation: November 22, 2021 + +As of November 21, 2021 this proposal moved away from a global `m.reference` relationship to using +`m.annotation` on poll responses. The following documents the previous behaviour for implementations +which might run across the now-legacy event types/format. + +Unstable representation (never made it to stable): +```json5 +{ + "type": "org.matrix.msc3381.poll.response", // note the *lack* v2 in the event type! + "sender": "@bob:example.org", + "content": { + "m.relates_to": { + "rel_type": "m.reference", // this changed! + "event_id": "$poll" + }, + "org.matrix.msc3381.poll.response": { + "answers": ["italian"] // answers are recorded here! + } + }, + // other fields that aren't relevant here +} +``` + +The processing rules for this kind of response event were: + +1. Only the latest response event is considered. All others are ignored. +2. `answers` is truncated to `max_selections` - the remainder are ignored. +3. Users can un-vote by casting a ballot of `[]` or redacting all of their response events. + +All other rules (particularly related to late responses) remain the same. + +Clients can theoretically rely on the server-side relations endpoint for gappy syncs, though this +has not been fully verified. It is intended that implementations switch over to the proposal's new +text instead. From 73de2158fe9b073727d19afbb7fcc967c0ad5227 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 23 Nov 2021 13:47:36 -0700 Subject: [PATCH 13/34] Note how answer IDs should be unique --- proposals/3381-polls.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index ad25334f31..42d6fade51 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -251,6 +251,11 @@ with the `m.message` parts of the events. For absolute clarity: if a client has it can outright ignore any irrelevant data from the events such as the message fallback or other representations that senders stick onto the event (like thumbnails, captions, attachments, etc). +It's theoretically possible for a client which starts a poll to use answer IDs which conflict with +reactions. Clients are discouraged from doing this and should instead use strings which are unlikely +to be used in other annotations/reactions. For example, using `pollAnswer_${uuid}_${answerIndex}` as +a template. + ## Alternatives The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) From c46929e7d55cddefecd9b738fbe77c4137d3d53d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 23 Nov 2021 14:13:17 -0700 Subject: [PATCH 14/34] Mention MSC3523 --- proposals/3381-polls.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 42d6fade51..68afa29b56 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -202,7 +202,7 @@ Much of the handling for this proposal is covered by other MSCs already: with a given key once, preventing most issues of users voting for the same answer multiple times. * [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines the server-side aggregation approach which can be useful to clients to determine which votes there are on an event. -* [MSC0001](https://github.com/matrix-org/matrix-doc/pull/0001) defines how clients can get all relations +* [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) defines how clients can get all relations for an event between point A and B (namely the poll start and close). No further behaviour is defined by this MSC: servers do not have to understand the rules of a poll in order @@ -211,7 +211,7 @@ the above MSCs. ### Client behaviour -Clients should rely on [MSC0001](https://github.com/matrix-org/matrix-doc/pull/0001) and +Clients should rely on [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) and [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) for handling limited ("gappy") syncs. Otherwise, it is anticipated that clients re-process polls entirely on their own to ensure accurate counts with encrypted events (the response events might be encrypted, so the server-side aggregations endpoint @@ -219,6 +219,9 @@ will be unaware of whether an event is a reaction, poll response, or some other in the proposal text, clients should filter out non-sensical `key`s to automatically filter out reactions and other non-poll-response types. +For clarity: clients using [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) should use the +time-based shape of the endpoint, not the event ID shape, in order to honour the poll rules. + ## Potential issues As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this From 56f131a6541056d4738b346205a7d2df717d3939 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 24 Nov 2021 01:04:04 -0700 Subject: [PATCH 15/34] Revert change to annotations --- proposals/3381-polls.md | 149 +++++++++++++--------------------------- 1 file changed, 47 insertions(+), 102 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 68afa29b56..43df888654 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -84,13 +84,12 @@ To respond to a poll, the following event is sent: "type": "m.poll.response", "sender": "@bob:example.org", "content": { - "m.relates_to": { - "rel_type": "m.annotation", // from MSC2677: https://github.com/matrix-org/matrix-doc/pull/2677 - "key": "poutine", + "m.relates_to": { // from MSC2674: https://github.com/matrix-org/matrix-doc/pull/2674 + "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 "event_id": "$poll" }, "m.poll.response": { - // empty object: useful information is in the relationship `key`. + "answers": ["poutine"] } }, // other fields that aren't relevant here @@ -101,39 +100,25 @@ Like `m.poll.start`, this `m.poll.response` event supports Extensible Events. Ho for clients to include renderable types like `m.text` and `m.message` which could impact the usability of the room (particularly for large rooms with lots of responses). -Note that the response event annotates the poll start event, forming a relationship which can be aggregated -by the server under [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). The event also supports -only one answer: for polls which permit multiple responses, and where multiple responses are given, the -client would send multiple response events. If the user deselected (unvoted) an option, the corresponding -response event would be redacted to match [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677)'s -handling of "un-reacting" to an event. +The response event forms a reference relationship with the poll start event. This kind of relationship doesn't +easily allow for server-side aggregation, however the alternatives section goes into detail as to why this +isn't a requirement for Polls. -This MSC borrows a lot of the structure provided by reactions in [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) -but intentionally does not define how notifications (if desired) or other UX features work: this MSC -purely intends to use the server-side aggregation capability where possible. +Only a user's latest response event (by `origin_server_ts`) will be considered by clients. If that response +is after the poll has closed, the user is considered to have not voted. Votes are accepted until the poll +is closed (according to the `origin_server_ts` on the end/closure event). -If a user sends multiple responses for the same answer, the user is considered to have voted for the answer -only once. They will need to redact/unvote *all* of those answers to have been fully considered as not voting -for that answer. +The `answers` array in the response is the user's selection(s) for the poll. The array length is truncated +to `max_selections` length during processing. The entries are the `id` of each answer from the original poll +start event. If *any* of the supplied answers is unknown, or the field is otherwise invalid, then the user's +vote is spoiled. Spoiled votes are also how users can "un-vote" from a poll - redacting, or setting `answers` +to an empty array, will spoil that user's vote. Votes are accepted until the poll is closed according to timestamp: servers/clients which receive votes which are timestamped before the close event's timestamp (or, when no close event has been sent) are valid. Late votes should be ignored. Early votes (from before the start event) are considered to be valid for the sake of handling clock drift as gracefully as possible. -To enforce `max_selections`, distinct responses are ordered by timestamp and truncated at `max_selections`. -For example, if a user votes as `[A, B, A, A, D]` and `max_selections` is 2, then the valid votes would be -`[B, A]` (because `A` was voted for multiple times, the most recent being after `B` was voted for). `D` -would simply be ignored as it is out of range. If the user redacted *all* of their `A` votes, then it'd -be `[B, D]`. - -Responses with a non-sensical `key` (eg: not a valid answer) are simply ignored. This is primarily important -when using server-side endpoints for fetching all relations: some will be emoji or short text strings to -denote reactions. Those events can be implicitly ignored with sufficiently complex/unlikely answer IDs as -the client would automatically filter out `πŸ‘` reactions. This is particularly important in a world where -the event type for all associated events is `m.room.encrypted` rather than `m.reaction` or `m.poll.response` -from a server's perspective. - Only the poll creator or anyone with a suitable power level for redactions can close the poll. The rationale for using the redaction power level is to help aid moderation efforts: while moderators can just redact the original poll and invalidate it entirely, they might prefer to just close it and leave it on the historical @@ -192,32 +177,12 @@ before it is closed. In either case, once the poll ends the results are shown regardless of kind. Clients might wish to avoid disclosing who voted for what in an undisclosed poll, though this MSC leaves that at just a suggestion. -### Server behaviour - -Much of the handling for this proposal is covered by other MSCs already: - -* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how to aggregate the annotations - (poll responses), though notes that encrypted events can potentially be an issue. The aggregation is - also unaware of a stop time to honour the poll closure. MSC2677 also defines that users may only annotate - with a given key once, preventing most issues of users voting for the same answer multiple times. -* [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines the server-side aggregation approach - which can be useful to clients to determine which votes there are on an event. -* [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) defines how clients can get all relations - for an event between point A and B (namely the poll start and close). - -No further behaviour is defined by this MSC: servers do not have to understand the rules of a poll in order -to support the client's implementation. They do however need to implement a lot of server-side handling for -the above MSCs. - -### Client behaviour +### Client implementation notes Clients should rely on [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) and -[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) for handling limited ("gappy") syncs. -Otherwise, it is anticipated that clients re-process polls entirely on their own to ensure accurate counts -with encrypted events (the response events might be encrypted, so the server-side aggregations endpoint -will be unaware of whether an event is a reaction, poll response, or some other random type). As mentioned -in the proposal text, clients should filter out non-sensical `key`s to automatically filter out reactions -and other non-poll-response types. +[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) for handling limited ("gappy") syncs. The +relations endpoint can give (paginated) information about which results have been selected and when the +poll has closed, overriding any stale local state the client might have. For clarity: clients using [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) should use the time-based shape of the endpoint, not the event ID shape, in order to honour the poll rules. @@ -254,11 +219,6 @@ with the `m.message` parts of the events. For absolute clarity: if a client has it can outright ignore any irrelevant data from the events such as the message fallback or other representations that senders stick onto the event (like thumbnails, captions, attachments, etc). -It's theoretically possible for a client which starts a poll to use answer IDs which conflict with -reactions. Clients are discouraged from doing this and should instead use strings which are unlikely -to be used in other annotations/reactions. For example, using `pollAnswer_${uuid}_${answerIndex}` as -a template. - ## Alternatives The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) @@ -287,6 +247,31 @@ and closures can also be limited arbitrarily by room admins, so clients might wa sure that the sender has a good chance of being able to close the poll they're about to create just to avoid future issues. +### Aggregations instead of references? + +A brief moment in this MSC's history described an approach which used aggregations (annotations/reactions) +instead of the proposed reference relationships, though this had immediate concerns of being too +complicated for practical use. + +While it is beneficial for votes to be quickly tallied by the server, the client still needs to do +post-processing on the data from the server in order to accurately represent the valid votes. The +server should not be made aware of the poll rules as it can lead to over-dependence on the server, +potentially causing excessive network requests from clients. + +As such, the reference relationship is maintained by this proposal in order to remain consistent with +how the poll close event is sent: instead of clients having to process two paginated requests they can +use a single request to get the same information, but in a more valuable form. + +For completeness, the approach of aggregations-based responses is summarized as: + +* `m.annotation` `rel_type` +* `key` is an answer ID +* Multiple response events for multi-select polls. Only the most recent duplicate is considered valid. +* Unvoting is done through redaction. + +Additional concerns are how the client needs to ensure that the answer IDs won't collide with a reaction +or other annotation, adding additional complexity in the form of magic strings. + ## Security considerations As mentioned a multitude of times throughout this proposal, this MSC's approach is prone to disclosure @@ -331,9 +316,6 @@ If a client/user wishes to make a poll statically visible, they should check out While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place of `m.`. Note that extensible events has a different unstable prefix for those fields. -**Note**: Due to changes during the unstable period, poll responses are additionally annotated with a -`v2` to denote a change on November 21, 2021. For details, see below. - The 3 examples above can be rewritten as: ```json5 @@ -370,16 +352,15 @@ The 3 examples above can be rewritten as: ```json5 { - "type": "org.matrix.msc3381.v2.poll.response", // note the v2 in the event type! + "type": "org.matrix.msc3381.poll.response", "sender": "@bob:example.org", "content": { - "m.relates_to": { - "rel_type": "m.annotation", - "key": "italian", + "m.relates_to": { // from MSC2674: https://github.com/matrix-org/matrix-doc/pull/2674 + "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 "event_id": "$poll" }, "org.matrix.msc3381.poll.response": { - // empty body + "answers": ["poutine"] } }, // other fields that aren't relevant here @@ -405,39 +386,3 @@ The 3 examples above can be rewritten as: Note that the extensible event fallbacks did not fall back to `m.room.message` in this MSC: this is deliberate to ensure polls are treated as first-class citizens. Client authors not willing/able to support polls are encouraged to instead support Extensible Events for better fallbacks. - -### Historical implementation: November 22, 2021 - -As of November 21, 2021 this proposal moved away from a global `m.reference` relationship to using -`m.annotation` on poll responses. The following documents the previous behaviour for implementations -which might run across the now-legacy event types/format. - -Unstable representation (never made it to stable): -```json5 -{ - "type": "org.matrix.msc3381.poll.response", // note the *lack* v2 in the event type! - "sender": "@bob:example.org", - "content": { - "m.relates_to": { - "rel_type": "m.reference", // this changed! - "event_id": "$poll" - }, - "org.matrix.msc3381.poll.response": { - "answers": ["italian"] // answers are recorded here! - } - }, - // other fields that aren't relevant here -} -``` - -The processing rules for this kind of response event were: - -1. Only the latest response event is considered. All others are ignored. -2. `answers` is truncated to `max_selections` - the remainder are ignored. -3. Users can un-vote by casting a ballot of `[]` or redacting all of their response events. - -All other rules (particularly related to late responses) remain the same. - -Clients can theoretically rely on the server-side relations endpoint for gappy syncs, though this -has not been fully verified. It is intended that implementations switch over to the proposal's new -text instead. From 7f5f111494841ee76d6cf9cbbe3c15662504fd1e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 24 Nov 2021 01:10:07 -0700 Subject: [PATCH 16/34] Mention how to handle invalid closure events --- proposals/3381-polls.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 43df888654..fb684998de 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -124,6 +124,8 @@ for using the redaction power level is to help aid moderation efforts: while mod original poll and invalidate it entirely, they might prefer to just close it and leave it on the historical record. +Closure events which are sent by users without appropriate permission are ignored. + Closing a poll is done as follows: ```json5 From a0b9cf288ca929dc6bb09df1dfc0bf5fe7c2238b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 24 Nov 2021 01:13:18 -0700 Subject: [PATCH 17/34] Only count the first closure --- proposals/3381-polls.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index fb684998de..c414e67c9c 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -124,7 +124,8 @@ for using the redaction power level is to help aid moderation efforts: while mod original poll and invalidate it entirely, they might prefer to just close it and leave it on the historical record. -Closure events which are sent by users without appropriate permission are ignored. +Closure events which are sent by users without appropriate permission are ignored. A poll is considered +closed once the first valid closure event is received - repeated closures are ignored. Closing a poll is done as follows: From 513cd2e9218c5d37aca2c50fae765308bbd2076a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 3 Jan 2022 10:15:22 -0700 Subject: [PATCH 18/34] Add push rules --- proposals/3381-polls.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index c414e67c9c..0c2bf492a5 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -190,6 +190,42 @@ poll has closed, overriding any stale local state the client might have. For clarity: clients using [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) should use the time-based shape of the endpoint, not the event ID shape, in order to honour the poll rules. +### Notifications + +In order to have polls behave similar to message events, the following underride push rules are defined: + +```json +{ + "rule_id": ".m.rule.polls_one_to_one", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "room_member_count", "is": "2"}, + {"kind": "event_match", "key": "type", "pattern": "m.poll.start"} + ], + "actions": [ + "notify", + {"set_tweak": "sound", "value": "default"} + ] +} +``` + +```json +{ + "rule_id": ".m.rule.polls", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "event_match", "key": "type", "pattern": "m.poll.start"} + ], + "actions": [ + "notify" + ] +} +``` + +When clients modify the related `m.room.message` rules they should also modify these rules. + ## Potential issues As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this From c4a1e9ca70241bd753bbdd5d9157dbf62b117970 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 3 Jan 2022 10:17:21 -0700 Subject: [PATCH 19/34] spelling --- proposals/3381-polls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 0c2bf492a5..7ebdfc1ec5 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -321,7 +321,7 @@ Clients should apply a large amount of validation to each field when interacting bodies are already declared as completely untrusted, though not all clients apply a layer of validation. In general, this MSC aims to try and show something of use to users so they can at least figure out what the sender intended, though clients are also welcome to just hide invalid events/responses (with -the exception of spoiled votes: those are treated as "unvoting" or chosing nothing). Clients are +the exception of spoiled votes: those are treated as "unvoting" or choosing nothing). Clients are encouraged to try and fall back to something sensible, even if just an error message saying the poll is invalid. From 162f76bd6aef0e2541552ccca47aa4bb448cd868 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 3 Jan 2022 10:30:56 -0700 Subject: [PATCH 20/34] Clarify the extensible events structure --- proposals/3381-polls.md | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 7ebdfc1ec5..3648594887 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -55,10 +55,36 @@ A poll can be started by sending an `m.poll.start` room event, similar to the fo As mentioned above, this is already making use of Extensible Events: The fallback for clients which don't know how to render polls is to just post the message to the chat. Some of the properties also make use of -extensible events within them, such as the `question` and the elements of `answers`: these are effectively -`m.message` events (under the Extensible Events structure), which means they're required to have a plain -text component to them. HTML is allowed, though clients are generally encouraged to rely on the plain text -representation for an unbiased rendering. Meme value of HTML might be desirable to some clients, however. +extensible events within them, such as the `question` and the elements of `answers`: these are essentially +nested events themselves. For example, the following can represent the same `question`: + +```json +{ + "question": { + "m.text": "How are you?" + } +} +``` +```json5 +{ + "question": { + // a plaintext format is always required + "m.text": "How are you?", + "m.html": "How are you?" + } +} +``` +```json5 +{ + "question": { + "m.message": [ + // a plaintext format is always required + {"body": "How are you?", "mimetype": "text/plain"}, + {"body": "How are you?", "mimetype": "text/html"}, + ] + } +} +``` The `kind` refers to whether the poll's votes are disclosed while the poll is still open. `m.poll.undisclosed` means the results are revealed once the poll is closed. `m.poll.disclosed` is the opposite: the votes are From 22f1c27c339386b0f0b03d78f0d69241d54c13cb Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 14 Jan 2022 14:25:37 -0700 Subject: [PATCH 21/34] Fix wording of answer handling in start event --- proposals/3381-polls.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 3648594887..6e77e726c3 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -93,9 +93,10 @@ naming convention are supported. Unknown values are to be treated as `m.poll.und compatibility with theoretical values. More specific detail as to the difference between two polls come up later in this MSC. -`answers` must be an array with at least 1 option and no more than 20. Lengths outside this range are invalid -and must not be rendered by clients. Most polls are expected to have 2-8 options. The answer `id` is an -arbitrary string used within the poll events. Clients should not attempt to parse or understand the `id`. +`answers` must be an array with at least 1 option and are truncated to 20 options. Polls with fewer than 1 +option should not rendered, and only the first 20 options are considered for rendering. Most polls are +expected to have 2-8 options. The answer `id` is an arbitrary string used within the poll events. Clients +should not attempt to parse or understand the `id`. `max_selections` is optional and denotes the maximum number of responses a user is able to select. Users can select fewer options, but not more. This defaults to `1`. Cannot be less than 1. From 95fdc44b904d2b4d2f227db99050e539e43f3509 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 14 Jan 2022 14:37:04 -0700 Subject: [PATCH 22/34] Clarify how the push rules work, and add one for ending polls --- proposals/3381-polls.md | 56 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 4 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 6e77e726c3..be61adb7cc 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -219,11 +219,12 @@ time-based shape of the endpoint, not the event ID shape, in order to honour the ### Notifications -In order to have polls behave similar to message events, the following underride push rules are defined: +In order to have polls behave similar to message events, the following underride push rules are defined. +Note that the push rules are mirrored from those available to `m.room.message` events. ```json { - "rule_id": ".m.rule.polls_one_to_one", + "rule_id": ".m.rule.poll_start_one_to_one", "default": true, "enabled": true, "conditions": [ @@ -239,7 +240,7 @@ In order to have polls behave similar to message events, the following underride ```json { - "rule_id": ".m.rule.polls", + "rule_id": ".m.rule.poll_start", "default": true, "enabled": true, "conditions": [ @@ -251,7 +252,49 @@ In order to have polls behave similar to message events, the following underride } ``` -When clients modify the related `m.room.message` rules they should also modify these rules. +```json +{ + "rule_id": ".m.rule.poll_end_one_to_one", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "room_member_count", "is": "2"}, + {"kind": "event_match", "key": "type", "pattern": "m.poll.end"} + ], + "actions": [ + "notify", + {"set_tweak": "sound", "value": "default"} + ] +} +``` + +```json +{ + "rule_id": ".m.rule.poll_end", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "event_match", "key": "type", "pattern": "m.poll.end"} + ], + "actions": [ + "notify" + ] +} +``` + +Servers should keep these rules in sync with the `m.room.message` rules they are based upon. For +example, if the `m.room.message` rule gets muted in a room then the associated rules for polls would +also get muted. Similarly, if either of the two poll rules were to be muted in a room then the other +poll rule and the `m.room.message` rule would be muted as well. + +Clients are expected to not require any specific change in order to support these rules. Their user +experience typically already considers an entry for "messages in the room", which is what a typical +user would expect to control notifications caused by polls. + +The server-side syncing of the rules additionally means that clients won't have to manually add support +for the new rules. Servers as part of implementation will update and incorporate the rules on behalf +of the users and simply send them down `/sync` per normal - clients which parse the push rules manually +shouldn't have to do anything as the rule will execute normally. ## Potential issues @@ -285,6 +328,11 @@ with the `m.message` parts of the events. For absolute clarity: if a client has it can outright ignore any irrelevant data from the events such as the message fallback or other representations that senders stick onto the event (like thumbnails, captions, attachments, etc). +The push rules for this feature are complex and not ideal. The author believes that it solves a short +term need while other MSCs work on improving the notifications system. Most importantly, the author +believes future MSCs which aim to fix notifications for extensible events in general will be a more +preferred approach over this MSC's (hopefully) short-term solution. + ## Alternatives The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) From 078fd3c4f21288d0a289e8e8ef563b8e8ace3cd1 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sat, 12 Nov 2022 17:39:55 -0700 Subject: [PATCH 23/34] WIP rewrite of polls to consider content blocks system --- proposals/3381-polls.md | 62 ++++++++++++++++++++++------------------- 1 file changed, 34 insertions(+), 28 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index be61adb7cc..e58096a02d 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -11,48 +11,54 @@ MSC's approach is discussed at length in the alternatives section for why it is ## Proposal -Polls are to be handled completely client-side and encrypted when possible in a given room. They are -simply started by sending an appropriate event, responded to with more events, and closed (eventually) -by the sender. Servers have no added requirements in order to support this MSC: they simply need to -be able to send arbitrary event types, which they already should be capable of. +Polls are intended to be handled completely client-side and encrypted when possible in a given room. +They are started by sending an event, responded to using events, and closed using more events - all +without involving the server (beyond being the natural vessel for sending the events). Other MSCs +related to polls might require changes from servers, however this MSC is intentionally scoped so that +it does not need server-side involvement. -The events in this MSC make heavy use of [MSC1767: Extensible Events](https://github.com/matrix-org/matrix-doc/pull/1767). +The events in this MSC make use of the following functionality: -A poll can be started by sending an `m.poll.start` room event, similar to the following: +* [MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767) (extensible events & `m.markup`) +* [Event relationships](https://spec.matrix.org/v1.4/client-server-api/#forming-relationships-between-events) +* [Reference relations](https://github.com/matrix-org/matrix-spec/pull/1206) (**TODO:** Link to final spec) + +To start a poll, a user sends an `m.poll` event into the room. An example being: ```json5 { - "type": "m.poll.start", + "type": "m.poll", "sender": "@alice:example.org", "content": { - "m.poll.start": { - "question": { - "m.text": "What should we order for the party?" - }, - "kind": "m.poll.disclosed", - "max_selections": 1, - "answers": [ - { "id": "pizza", "m.text": "Pizza πŸ•" }, - { "id": "poutine", "m.text": "Poutine 🍟" }, - { "id": "italian", "m.text": "Italian 🍝" }, - { "id": "wings", "m.text": "Wings πŸ”₯" } - ] - }, - "m.message": [ + "m.markup": [ + // Markup is used as a fallback for text-only clients which don't understand polls. Specific formatting is + // not specified, however something like the following is likely best. { "mimetype": "text/plain", "body": "What should we order for the party?\n1. Pizza πŸ•\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings πŸ”₯" - }, - { - "mimetype": "text/html", - "body": "What should we order for the party?
  1. 1. Pizza πŸ•
  2. 2. Poutine 🍟
  3. 3. Italian 🍝
  4. 4. Wings πŸ”₯
" } - ] - }, - // other fields that aren't relevant here + ], + "m.poll": { + "kind": "m.disclosed", + "max_selections": 1, + "question": { + "m.markup": [{"body": "What should we order for the party?"}] + }, + "answers": [ + {"m.id": "pizza", "m.markup": [{"body": "Pizza πŸ•"}]}, + {"m.id": "poutine", "m.markup": [{"body": "Poutine 🍟"}]}, + {"m.id": "italian", "m.markup": [{"body": "Italian 🍝"}]}, + {"m.id": "wings", "m.markup": [{"body": "Wings πŸ”₯"}]}, + ] + } + } } ``` +!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! +!!!! TODO: EDIT BEYOND THIS LINE !!!! +!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + As mentioned above, this is already making use of Extensible Events: The fallback for clients which don't know how to render polls is to just post the message to the chat. Some of the properties also make use of extensible events within them, such as the `question` and the elements of `answers`: these are essentially From 1af936555aad25d67ecbf47f1d0b59b4b1cce0a3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 14 Nov 2022 15:49:44 -0700 Subject: [PATCH 24/34] Finish the polls rewrite --- proposals/3381-polls.md | 333 +++++++++++++++++----------------------- 1 file changed, 137 insertions(+), 196 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index e58096a02d..7d6d294a1c 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -21,13 +21,13 @@ The events in this MSC make use of the following functionality: * [MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767) (extensible events & `m.markup`) * [Event relationships](https://spec.matrix.org/v1.4/client-server-api/#forming-relationships-between-events) -* [Reference relations](https://github.com/matrix-org/matrix-spec/pull/1206) (**TODO:** Link to final spec) +* [Reference relations](https://github.com/matrix-org/matrix-spec/pull/1206) (**TODO:** Link to final spec here & below) -To start a poll, a user sends an `m.poll` event into the room. An example being: +To start a poll, a user sends an `m.poll.start` event into the room. An example being: ```json5 { - "type": "m.poll", + "type": "m.poll.start", "sender": "@alice:example.org", "content": { "m.markup": [ @@ -55,136 +55,133 @@ To start a poll, a user sends an `m.poll` event into the room. An example being: } ``` -!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -!!!! TODO: EDIT BEYOND THIS LINE !!!! -!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! +With consideration for extensible events, a new `m.poll` content block is defined: -As mentioned above, this is already making use of Extensible Events: The fallback for clients which don't -know how to render polls is to just post the message to the chat. Some of the properties also make use of -extensible events within them, such as the `question` and the elements of `answers`: these are essentially -nested events themselves. For example, the following can represent the same `question`: +* `kind` - An optional namespaced string to represent a poll's general approach. Currently specified + values being `m.disclosed` and `m.undisclosed`. Clients which don't understand the `kind` should + assume `m.undisclosed` for maximum compatibility. The definitions for these values are specified + later in this proposal. +* `max_selections` - An optional integer to represent how many answers the user is allowed to select + from the poll. Must be greater than or equal to `1`, and defaults to `1`. +* `question` - A required object to represent the question being posed by the poll. Takes an `m.markup` + content block within. More blocks might be added in the future. Clients should treat this similar + to how they would an `m.message` event. +* `answers` - Array of options users can select. Each entry is an object with an `m.markup` content + block, similar to `question`, and an opaque string field `m.id` for use in response events. More + blocks might be added in the future. Clients should treat each entry similar to how they would an + `m.message` event. The array is truncated to 20 maximum options. -```json -{ - "question": { - "m.text": "How are you?" - } -} -``` -```json5 -{ - "question": { - // a plaintext format is always required - "m.text": "How are you?", - "m.html": "How are you?" - } -} -``` -```json5 -{ - "question": { - "m.message": [ - // a plaintext format is always required - {"body": "How are you?", "mimetype": "text/plain"}, - {"body": "How are you?", "mimetype": "text/html"}, - ] - } -} -``` +Together with content blocks from other proposals, an `m.poll.start` is described as: -The `kind` refers to whether the poll's votes are disclosed while the poll is still open. `m.poll.undisclosed` -means the results are revealed once the poll is closed. `m.poll.disclosed` is the opposite: the votes are -visible up until and including when the poll is closed. Custom values are permitted using the standardized -naming convention are supported. Unknown values are to be treated as `m.poll.undisclosed` for maximum -compatibility with theoretical values. More specific detail as to the difference between two polls come up -later in this MSC. +* **Required** - An `m.markup` block to act as a fallback for clients which can't process polls. +* **Required** - An `m.poll` block to describe the poll itself. Clients use this to show the poll. -`answers` must be an array with at least 1 option and are truncated to 20 options. Polls with fewer than 1 -option should not rendered, and only the first 20 options are considered for rendering. Most polls are -expected to have 2-8 options. The answer `id` is an arbitrary string used within the poll events. Clients -should not attempt to parse or understand the `id`. +The above describes the minimum requirements for sending an `m.poll.start` event. Senders can add additional +blocks, however as per the extensible events system, receivers which understand poll events should not +honour them. -`max_selections` is optional and denotes the maximum number of responses a user is able to select. Users -can select fewer options, but not more. This defaults to `1`. Cannot be less than 1. +If a client does not support rendering polls inline, the client would instead typically represent +the event as a plain text message. This would allow users of such clients to participate in the poll, +even if they can not vote properly on it (ie: by using text messages or reactions). -The `m.message` fallback should be representative of the poll, but is not required and has no mandatory -format. Clients are encouraged to be inspired by the example above when sending poll events. - -To respond to a poll, the following event is sent: +To respond or vote in a poll, a user sends an `m.poll.response` event into the room. An example being: ```json5 { "type": "m.poll.response", "sender": "@bob:example.org", "content": { - "m.relates_to": { // from MSC2674: https://github.com/matrix-org/matrix-doc/pull/2674 - "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 - "event_id": "$poll" + // Reference relationship formed per spec + // https://github.com/matrix-org/matrix-spec/pull/1206 + // TODO: Link to reference relationship spec + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$poll_start_event_id" }, - "m.poll.response": { - "answers": ["poutine"] - } - }, - // other fields that aren't relevant here + "m.selections": ["poutine"] + } } ``` -Like `m.poll.start`, this `m.poll.response` event supports Extensible Events. However, it is strongly discouraged -for clients to include renderable types like `m.text` and `m.message` which could impact the usability of -the room (particularly for large rooms with lots of responses). +With consideration for extensible events, a new `m.selections` content block is defined: -The response event forms a reference relationship with the poll start event. This kind of relationship doesn't -easily allow for server-side aggregation, however the alternatives section goes into detail as to why this -isn't a requirement for Polls. +* An array of string identifiers to denote a user's selection. Can be empty to denote "no selection". + Identifiers are determined by the surrounding event type context, if available. -Only a user's latest response event (by `origin_server_ts`) will be considered by clients. If that response -is after the poll has closed, the user is considered to have not voted. Votes are accepted until the poll -is closed (according to the `origin_server_ts` on the end/closure event). +Together with content blocks from other proposals, an `m.poll.response` is described as: -The `answers` array in the response is the user's selection(s) for the poll. The array length is truncated -to `max_selections` length during processing. The entries are the `id` of each answer from the original poll -start event. If *any* of the supplied answers is unknown, or the field is otherwise invalid, then the user's -vote is spoiled. Spoiled votes are also how users can "un-vote" from a poll - redacting, or setting `answers` -to an empty array, will spoil that user's vote. +* **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. +* **Required** - An `m.selections` block to list the user's preferred selections in the poll. Clients + must truncate this array to `max_selections` during processing. Each entry is the `m.id` of a poll + answer option from the poll start event. If *any* of the supplied answers is unknown, the sender's + vote is spoiled (as if they didn't make a selection). -Votes are accepted until the poll is closed according to timestamp: servers/clients which receive votes -which are timestamped before the close event's timestamp (or, when no close event has been sent) are valid. -Late votes should be ignored. Early votes (from before the start event) are considered to be valid for the -sake of handling clock drift as gracefully as possible. +The above describes the minimum requirements for sending an `m.poll.response` event. Senders can add +additional blocks, however as per the extensible events system, receivers which understand poll events +should not honour them. -Only the poll creator or anyone with a suitable power level for redactions can close the poll. The rationale -for using the redaction power level is to help aid moderation efforts: while moderators can just redact the -original poll and invalidate it entirely, they might prefer to just close it and leave it on the historical -record. +There is deliberately no textual or renderable fallback on poll responses: the intention is that clients +which don't understand how to process these events will hide/ignore them. -Closure events which are sent by users without appropriate permission are ignored. A poll is considered -closed once the first valid closure event is received - repeated closures are ignored. +Only a user's most recent vote (by `origin_server_ts`) is accepted, even if that event is invalid or +redacted. Votes with timestamps after the poll has closed are ignored, as if they never happened. -Closing a poll is done as follows: +To close a poll, a user sends an `m.poll.end` event into the room. An example being: ```json5 { "type": "m.poll.end", - "sender": "@bob:example.org", + "sender": "@alice:example.org", "content": { + // Reference relationship formed per spec + // https://github.com/matrix-org/matrix-spec/pull/1206 + // TODO: Link to reference relationship spec "m.relates_to": { - "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 - "event_id": "$poll" + "rel_type": "m.reference", + "event_id": "$poll_start_event_id" }, - "m.poll.end": {}, - "m.text": "The poll has ended. Top answer: Poutine 🍟" }, - // other fields that aren't relevant here + "m.markup": [{ + // Markup is used as a fallback for text-only clients which don't understand polls. Specific formatting is + // not specified, however something like the following is likely best. + "body": "The poll has closed. Top answer: Poutine 🍟" + }], + "m.poll.results": { // optional + "pizza": 5, + "poutine": 8, + "italian": 7, + "wings": 6 + } } ``` -Once again, Extensible Events make an appearance here. There's nothing in particular metadata wise that -needs to appear in the `m.poll.end` property of `content`, though it is included for future capability. The -backup `m.text` representation is for fallback purposes and is completely optional with no strict format -requirements: the example above is just that, an example of what a client *could* do. Clients should be -careful to include a "top answer" in the end event as server lag might allow a few more responses to get -through while the closure is sent. Votes sent on or before the end event's timestamp are valid votes - all -others must be disregarded by clients. +With consideration for extensible events, a new `m.poll.results` content block is defined: + +* A dictionary object keyed by answer ID (`m.id` from the poll start event) and value being the integer + number of votes for that option as seen by the sender's client. Note that these values might not be + accurate, however other clients can easily validate the counts by retrieving all relations from the + server. + * User IDs which voted for each option are deliberately not included for brevity: clients requiring + more information about the poll are required to gather the relations themselves. + +Together with content blocks from other proposals, an `m.poll.end` is described as: + +* **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. +* **Required** - An `m.markup` block to act as a fallback for clients which can't process polls. +* **Optional** - An `m.poll.results` block to show the sender's perspective of the vote results. + +The above describes the minimum requirements for sending an `m.poll.end` event. Senders can add additional +blocks, however as per the extensible events system, receivers which understand poll events should not +honour them. + +If a client does not support rendering polls (generally speaking), the client would instead typically +represent the poll start event as text (per above), and thus would likely do the same for the closure +event, keeping users in the loop with what is going on. + +If a `m.poll.end` event is received from someone other than the poll creator or user with permission to +redact other's messages in the room, the event must be ignored by clients due to being invalid. The +redaction power level is chosen to support moderation: while moderators can just remove the poll from the +timeline entirely, they may also wish to simply close it to keep context visible. **Rationale**: Although clock drift is possible, as is clock manipulation, it is not anticipated that polls will be closed while they are still receiving high traffic. There are some cases where clients might @@ -195,10 +192,32 @@ poll was closed, but timestamped for when it was open, the server is violating a will be facing a ban from the room. This MSC does not propose a mitigation strategy beyond telling people not to ruin the fun. Also, don't use polls for things that are important. -Clients should disable voting interactions with polls once they are closed. Events which claim to close -the poll from senders other than the creator are to be treated as invalid and thus ignored. +The `m.poll.end`'s `origin_server_ts` determines when the poll closes exactly: if no valid end event +is received, the poll is still open. If the poll is closed, only votes sent on or before that timestamp +are considered, even if those votes are from before the start event. This is to handle clock drift over +federation as gracefully as possible. + +Repeated end events are ignored - only the first (valid) closure event by `origin_server_ts` is counted. +Clients should disable voting interactions with polls once they are closed. + +### Poll kinds + +This proposal defines an `m.poll` content block with a `kind` field accepting namespaced strings, with +`m.disclosed` and `m.undisclosed` being mentioned (`m.undisclosed` being the default), however it does +not describe what these values represent. + +In short, `m.disclosed` means the votes for poll are shown to users while the poll is still open. An +`m.undisclosed` poll would only show results when the poll is closed. -### Disclosed versus undisclosed polls +**Note**: because poll responses are sent into the room, non-compliant clients or curious users could +tally up results regardless of the poll being explicitly disclosed or not. This proposal acknowledges +the issue, but does not fix it. + +Custom poll kinds are possible using the [standardized namespace grammar](https://spec.matrix.org/v1.4/appendices/#common-namespaced-identifier-grammar), +and clients which do not recognize the kind are to assume `m.undisclosed` for maximum compatibility +with other poll kinds. + +#### Disclosed versus undisclosed polls Disclosed polls are most similar to what is seen on Twitch and often Twitter: members of the room are able to see the results and vote accordingly. Clients are welcome to hide the poll results until after the user @@ -215,13 +234,12 @@ disclosing who voted for what in an undisclosed poll, though this MSC leaves tha ### Client implementation notes -Clients should rely on [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) and -[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) for handling limited ("gappy") syncs. The -relations endpoint can give (paginated) information about which results have been selected and when the -poll has closed, overriding any stale local state the client might have. +Clients can rely on the [`/relations`](https://spec.matrix.org/v1.4/client-server-api/#get_matrixclientv1roomsroomidrelationseventidreltype) +API to find votes which might have been received during limited ("gappy") syncs, or whenever they become +descynchronized and need to recalculate events. -For clarity: clients using [MSC3523](https://github.com/matrix-org/matrix-doc/pull/3523) should use the -time-based shape of the endpoint, not the event ID shape, in order to honour the poll rules. +This MSC does not describe an aggregation approach for poll events, hence the need for the client to retrieve +all referenced events rather than simply relying on bundles. ### Notifications @@ -319,21 +337,6 @@ Limiting polls to client-side enforcement could be problematic if the MSC was in or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. Bolt-on support for signing, verification, validity, etc can be accomplished as well in the future. -The fallback support relies on clients already knowing about extensible events, which might not be -the case. Bridges (as of writing) do not have support for extensible events, for example, which can -mean that polls are lost in transit. This is perceived to be a similar amount of data loss when a Matrix -user reacts to an IRC user's message: the IRC user has no idea what happened on Matrix. Bridges, and -other clients, can trivially add message parsing support as described by extensible events to work -around this. The recommendations of this MSC specifically avoid the vote spam from being bridged, but -the start of poll and end of poll (results) would be bridged. There's an argument to be made for -surrounding conversation context being enough to communicate the results without extensible events, -though this is slightly less reliable. - -Though more important for Extensible Events, clients might get confused about what they should do -with the `m.message` parts of the events. For absolute clarity: if a client has support for polls, -it can outright ignore any irrelevant data from the events such as the message fallback or other -representations that senders stick onto the event (like thumbnails, captions, attachments, etc). - The push rules for this feature are complex and not ideal. The author believes that it solves a short term need while other MSCs work on improving the notifications system. Most importantly, the author believes future MSCs which aim to fix notifications for extensible events in general will be a more @@ -348,7 +351,7 @@ based around `m.room.message` events, using `msgtype` to differentiate between t is an awful experience on clients which do not support polls properly, leaving an irritating amount of contextless messages in the timeline. Though not directly mentioned on that thread, polls also cannot be closed under that MSC which leads to people picking options hours or even days after the poll has "ended". -This MSC instead proposed to only supply fallback on the start and end of a poll, leading to enough context +This MSC instead proposes to only supply fallback on the start and end of a poll, leading to enough context for unsupporting clients without flooding the room with messages. Originally, MSC2192 was intended to propose polls as a sort of widget with access to timeline events @@ -429,80 +432,18 @@ to solve: ## Other notes If a client/user wishes to make a poll statically visible, they should check out -[pinned messages](https://matrix.org/docs/spec/client_server/r0.6.1#m-room-pinned-events). +[pinned messages](https://spec.matrix.org/v1.4/client-server-api/#mroompinned_events). ## Unstable prefix -While this MSC is not eligible for stable usage, the `org.matrix.msc3381.` prefix can be used in place -of `m.`. Note that extensible events has a different unstable prefix for those fields. - -The 3 examples above can be rewritten as: +While this MSC is not considered stable, implementations should use `org.matrix.msc3381.v2.*` as a prefix +in place of `m.*` throughout this proposal. Note that extensible events and content blocks might have their +own prefixing requirements. -```json5 -{ - "type": "org.matrix.msc3381.poll.start", - "sender": "@alice:example.org", - "content": { - "org.matrix.msc3381.poll.start": { - "question": { - "org.matrix.msc1767.text": "What should we order for the party?" - }, - "kind": "org.matrix.msc3381.poll.disclosed", - "answers": [ - { "id": "pizza", "org.matrix.msc1767.text": "Pizza πŸ•" }, - { "id": "poutine", "org.matrix.msc1767.text": "Poutine 🍟" }, - { "id": "italian", "org.matrix.msc1767.text": "Italian 🍝" }, - { "id": "wings", "org.matrix.msc1767.text": "Wings πŸ”₯" } - ] - }, - "org.matrix.msc1767.message": [ - { - "mimetype": "text/plain", - "body": "What should we order for the party?\n1. Pizza πŸ•\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings πŸ”₯" - }, - { - "mimetype": "text/html", - "body": "What should we order for the party?
  1. 1. Pizza πŸ•
  2. 2. Poutine 🍟
  3. 3. Italian 🍝
  4. 4. Wings πŸ”₯
" - } - ] - }, - // other fields that aren't relevant here -} -``` - -```json5 -{ - "type": "org.matrix.msc3381.poll.response", - "sender": "@bob:example.org", - "content": { - "m.relates_to": { // from MSC2674: https://github.com/matrix-org/matrix-doc/pull/2674 - "rel_type": "m.reference", // from MSC3267: https://github.com/matrix-org/matrix-doc/pull/3267 - "event_id": "$poll" - }, - "org.matrix.msc3381.poll.response": { - "answers": ["poutine"] - } - }, - // other fields that aren't relevant here -} -``` - -```json5 -{ - "type": "org.matrix.msc3381.poll.end", - "sender": "@bob:example.org", - "content": { - "m.relates_to": { - "rel_type": "m.reference", - "event_id": "$poll" - }, - "org.matrix.msc3381.poll.end": {}, - "org.matrix.msc1767.text": "The poll has ended. Top answer: Poutine 🍟" - }, - // other fields that aren't relevant here -} -``` +Normally extensible events would only be permitted in a specific room version, however as a known-lossy chat +feature, this proposal's events are permitted in any room version, provided they are of the unstable variety. +The stable event types must only be sent in a room version which supports extensible events. -Note that the extensible event fallbacks did not fall back to `m.room.message` in this MSC: this -is deliberate to ensure polls are treated as first-class citizens. Client authors not willing/able -to support polls are encouraged to instead support Extensible Events for better fallbacks. +Client implementations should note that a previous draft of this proposal had a different format and some of +those events might be found in the wild, hence the `v2` portion of the unstable prefix. Clients interested in +this older format should review older drafts of this proposal. From 169afe0bfa22f946dc4b3d1c1f6f01c9e4d1bff1 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 14 Nov 2022 15:59:21 -0700 Subject: [PATCH 25/34] Remove notifications --- proposals/3381-polls.md | 86 +---------------------------------------- 1 file changed, 2 insertions(+), 84 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 7d6d294a1c..5fed60ef69 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -241,85 +241,6 @@ descynchronized and need to recalculate events. This MSC does not describe an aggregation approach for poll events, hence the need for the client to retrieve all referenced events rather than simply relying on bundles. -### Notifications - -In order to have polls behave similar to message events, the following underride push rules are defined. -Note that the push rules are mirrored from those available to `m.room.message` events. - -```json -{ - "rule_id": ".m.rule.poll_start_one_to_one", - "default": true, - "enabled": true, - "conditions": [ - {"kind": "room_member_count", "is": "2"}, - {"kind": "event_match", "key": "type", "pattern": "m.poll.start"} - ], - "actions": [ - "notify", - {"set_tweak": "sound", "value": "default"} - ] -} -``` - -```json -{ - "rule_id": ".m.rule.poll_start", - "default": true, - "enabled": true, - "conditions": [ - {"kind": "event_match", "key": "type", "pattern": "m.poll.start"} - ], - "actions": [ - "notify" - ] -} -``` - -```json -{ - "rule_id": ".m.rule.poll_end_one_to_one", - "default": true, - "enabled": true, - "conditions": [ - {"kind": "room_member_count", "is": "2"}, - {"kind": "event_match", "key": "type", "pattern": "m.poll.end"} - ], - "actions": [ - "notify", - {"set_tweak": "sound", "value": "default"} - ] -} -``` - -```json -{ - "rule_id": ".m.rule.poll_end", - "default": true, - "enabled": true, - "conditions": [ - {"kind": "event_match", "key": "type", "pattern": "m.poll.end"} - ], - "actions": [ - "notify" - ] -} -``` - -Servers should keep these rules in sync with the `m.room.message` rules they are based upon. For -example, if the `m.room.message` rule gets muted in a room then the associated rules for polls would -also get muted. Similarly, if either of the two poll rules were to be muted in a room then the other -poll rule and the `m.room.message` rule would be muted as well. - -Clients are expected to not require any specific change in order to support these rules. Their user -experience typically already considers an entry for "messages in the room", which is what a typical -user would expect to control notifications caused by polls. - -The server-side syncing of the rules additionally means that clients won't have to manually add support -for the new rules. Servers as part of implementation will update and incorporate the rules on behalf -of the users and simply send them down `/sync` per normal - clients which parse the push rules manually -shouldn't have to do anything as the rule will execute normally. - ## Potential issues As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this @@ -337,11 +258,6 @@ Limiting polls to client-side enforcement could be problematic if the MSC was in or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. Bolt-on support for signing, verification, validity, etc can be accomplished as well in the future. -The push rules for this feature are complex and not ideal. The author believes that it solves a short -term need while other MSCs work on improving the notifications system. Most importantly, the author -believes future MSCs which aim to fix notifications for extensible events in general will be a more -preferred approach over this MSC's (hopefully) short-term solution. - ## Alternatives The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) @@ -434,6 +350,8 @@ to solve: If a client/user wishes to make a poll statically visible, they should check out [pinned messages](https://spec.matrix.org/v1.4/client-server-api/#mroompinned_events). +Notifications support for polls have been moved to [MSC3930](https://github.com/matrix-org/matrix-spec-proposals/pull/3930). + ## Unstable prefix While this MSC is not considered stable, implementations should use `org.matrix.msc3381.v2.*` as a prefix From cb0a9fe5b16e51395caa31a03da9f763f749d120 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 17 Apr 2023 21:32:56 -0600 Subject: [PATCH 26/34] Update links and MSC1767 behaviour --- proposals/3381-polls.md | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 5fed60ef69..a9be25a639 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -19,9 +19,9 @@ it does not need server-side involvement. The events in this MSC make use of the following functionality: -* [MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767) (extensible events & `m.markup`) -* [Event relationships](https://spec.matrix.org/v1.4/client-server-api/#forming-relationships-between-events) -* [Reference relations](https://github.com/matrix-org/matrix-spec/pull/1206) (**TODO:** Link to final spec here & below) +* [MSC1767](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/1767-extensible-events.md) (extensible events & `m.text`) +* [Event relationships](https://spec.matrix.org/v1.6/client-server-api/#forming-relationships-between-events) +* [Reference relations](https://spec.matrix.org/v1.6/client-server-api/#reference-relations) To start a poll, a user sends an `m.poll.start` event into the room. An example being: @@ -30,8 +30,8 @@ To start a poll, a user sends an `m.poll.start` event into the room. An example "type": "m.poll.start", "sender": "@alice:example.org", "content": { - "m.markup": [ - // Markup is used as a fallback for text-only clients which don't understand polls. Specific formatting is + "m.text": [ + // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is // not specified, however something like the following is likely best. { "mimetype": "text/plain", @@ -42,13 +42,13 @@ To start a poll, a user sends an `m.poll.start` event into the room. An example "kind": "m.disclosed", "max_selections": 1, "question": { - "m.markup": [{"body": "What should we order for the party?"}] + "m.text": [{"body": "What should we order for the party?"}] }, "answers": [ - {"m.id": "pizza", "m.markup": [{"body": "Pizza πŸ•"}]}, - {"m.id": "poutine", "m.markup": [{"body": "Poutine 🍟"}]}, - {"m.id": "italian", "m.markup": [{"body": "Italian 🍝"}]}, - {"m.id": "wings", "m.markup": [{"body": "Wings πŸ”₯"}]}, + {"m.id": "pizza", "m.text": [{"body": "Pizza πŸ•"}]}, + {"m.id": "poutine", "m.text": [{"body": "Poutine 🍟"}]}, + {"m.id": "italian", "m.text": [{"body": "Italian 🍝"}]}, + {"m.id": "wings", "m.text": [{"body": "Wings πŸ”₯"}]}, ] } } @@ -63,17 +63,17 @@ With consideration for extensible events, a new `m.poll` content block is define later in this proposal. * `max_selections` - An optional integer to represent how many answers the user is allowed to select from the poll. Must be greater than or equal to `1`, and defaults to `1`. -* `question` - A required object to represent the question being posed by the poll. Takes an `m.markup` +* `question` - A required object to represent the question being posed by the poll. Takes an `m.text` content block within. More blocks might be added in the future. Clients should treat this similar to how they would an `m.message` event. -* `answers` - Array of options users can select. Each entry is an object with an `m.markup` content +* `answers` - Array of options users can select. Each entry is an object with an `m.text` content block, similar to `question`, and an opaque string field `m.id` for use in response events. More blocks might be added in the future. Clients should treat each entry similar to how they would an `m.message` event. The array is truncated to 20 maximum options. Together with content blocks from other proposals, an `m.poll.start` is described as: -* **Required** - An `m.markup` block to act as a fallback for clients which can't process polls. +* **Required** - An `m.text` block to act as a fallback for clients which can't process polls. * **Required** - An `m.poll` block to describe the poll itself. Clients use this to show the poll. The above describes the minimum requirements for sending an `m.poll.start` event. Senders can add additional @@ -92,8 +92,7 @@ To respond or vote in a poll, a user sends an `m.poll.response` event into the r "sender": "@bob:example.org", "content": { // Reference relationship formed per spec - // https://github.com/matrix-org/matrix-spec/pull/1206 - // TODO: Link to reference relationship spec + // https://spec.matrix.org/v1.6/client-server-api/#reference-relations "m.relates_to": { "rel_type": "m.reference", "event_id": "$poll_start_event_id" @@ -134,15 +133,14 @@ To close a poll, a user sends an `m.poll.end` event into the room. An example be "sender": "@alice:example.org", "content": { // Reference relationship formed per spec - // https://github.com/matrix-org/matrix-spec/pull/1206 - // TODO: Link to reference relationship spec + // https://spec.matrix.org/v1.6/client-server-api/#reference-relations "m.relates_to": { "rel_type": "m.reference", "event_id": "$poll_start_event_id" }, }, - "m.markup": [{ - // Markup is used as a fallback for text-only clients which don't understand polls. Specific formatting is + "m.text": [{ + // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is // not specified, however something like the following is likely best. "body": "The poll has closed. Top answer: Poutine 🍟" }], @@ -167,7 +165,7 @@ With consideration for extensible events, a new `m.poll.results` content block i Together with content blocks from other proposals, an `m.poll.end` is described as: * **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. -* **Required** - An `m.markup` block to act as a fallback for clients which can't process polls. +* **Required** - An `m.text` block to act as a fallback for clients which can't process polls. * **Optional** - An `m.poll.results` block to show the sender's perspective of the vote results. The above describes the minimum requirements for sending an `m.poll.end` event. Senders can add additional From cd4cb41816189ec8c8f47281744441d745fe4873 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 17 Apr 2023 21:33:45 -0600 Subject: [PATCH 27/34] Remove excess artifact --- themes/docsy | 1 - 1 file changed, 1 deletion(-) delete mode 160000 themes/docsy diff --git a/themes/docsy b/themes/docsy deleted file mode 160000 index 5023a29145..0000000000 --- a/themes/docsy +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 5023a2914528e012ecf3ec85a56028c00ee97dd2 From df1926c748c331a7cb1f2361d282f0ee052539a0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 1 Jun 2023 13:50:45 -0600 Subject: [PATCH 28/34] Note UISI problems in potential issues section --- proposals/3381-polls.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index a9be25a639..efebcb60e5 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -256,6 +256,12 @@ Limiting polls to client-side enforcement could be problematic if the MSC was in or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. Bolt-on support for signing, verification, validity, etc can be accomplished as well in the future. +Due to the reference relationship between responses and the poll start event, it's possible that a +client facing an "unable to decrypt" error on the response won't know if it's a poll response specifically +or some other reference relationship. Clients are encouraged to tell users when there's a possibility +that not all responses are known, potentially impacting the results, such as where related events are +undecryptable. + ## Alternatives The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) From e0d9065883e085be5e10918a0ead0382f3cc378e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 1 Jun 2023 13:56:02 -0600 Subject: [PATCH 29/34] Note why MSC3523 is a bad idea --- proposals/3381-polls.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index efebcb60e5..1a0de5b182 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -234,7 +234,11 @@ disclosing who voted for what in an undisclosed poll, though this MSC leaves tha Clients can rely on the [`/relations`](https://spec.matrix.org/v1.4/client-server-api/#get_matrixclientv1roomsroomidrelationseventidreltype) API to find votes which might have been received during limited ("gappy") syncs, or whenever they become -descynchronized and need to recalculate events. +descynchronized and need to recalculate events. Ranged approaches, such as [MSC3523](https://github.com/matrix-org/matrix-spec-proposals/pull/3523), +are not suitable for this particular case because the gap between syncs might contain events which are not +revealed by the range. For example, if a remote server took an extra hour to send events and the receiving +client had a gappy sync over a span of 15 minutes: the client might not know that it needs to go back potentially +hours to see the missing event. This MSC does not describe an aggregation approach for poll events, hence the need for the client to retrieve all referenced events rather than simply relying on bundles. From 35ffaf97a8ee5ccc472dcaba7cfde360c74ff17f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 25 Jun 2023 10:11:37 -0600 Subject: [PATCH 30/34] Update proposals/3381-polls.md Co-authored-by: Maxime NATUREL <46314705+mnaturel@users.noreply.github.com> --- proposals/3381-polls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 1a0de5b182..c895202bbe 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -1,7 +1,7 @@ # MSC3381: Chat Polls Polls are additive functionality in a room, allowing someone to pose a question and others to answer -until the poll is closed. In chat, these are typically used for quick questionares such as what to +until the poll is closed. In chat, these are typically used for quick questionnaires such as what to have for lunch or when the next office party should be, not elections or anything needing truly secret ballot. From e1084e6484168d17f52e61e45e2d7d75962326db Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 3 Jul 2023 21:15:22 -0600 Subject: [PATCH 31/34] Update MSC --- proposals/3381-polls.md | 93 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 82 insertions(+), 11 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index c895202bbe..19838436c1 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -113,7 +113,8 @@ Together with content blocks from other proposals, an `m.poll.response` is descr * **Required** - An `m.selections` block to list the user's preferred selections in the poll. Clients must truncate this array to `max_selections` during processing. Each entry is the `m.id` of a poll answer option from the poll start event. If *any* of the supplied answers is unknown, the sender's - vote is spoiled (as if they didn't make a selection). + vote is spoiled (as if they didn't make a selection). If an entry is repeated after truncation, only + one of those entries counts as the sender's vote (each sender gets 1 vote). The above describes the minimum requirements for sending an `m.poll.response` event. Senders can add additional blocks, however as per the extensible events system, receivers which understand poll events @@ -123,7 +124,11 @@ There is deliberately no textual or renderable fallback on poll responses: the i which don't understand how to process these events will hide/ignore them. Only a user's most recent vote (by `origin_server_ts`) is accepted, even if that event is invalid or -redacted. Votes with timestamps after the poll has closed are ignored, as if they never happened. +redacted. Votes with timestamps after the poll has closed are ignored, as if they never happened. Note +that redaction currently removes the `m.relates_to` information from the event, causing the vote to be +detached from the poll. In this scenario, the user's vote is *reverted* to its previous state rather +than explicitly spoiled. To "unvote" or otherwise override the previous vote state, clients should send +a response with an empty `m.selections` array. To close a poll, a user sends an `m.poll.end` event into the room. An example being: @@ -360,16 +365,82 @@ If a client/user wishes to make a poll statically visible, they should check out Notifications support for polls have been moved to [MSC3930](https://github.com/matrix-org/matrix-spec-proposals/pull/3930). +Normally extensible events would only be permitted in a specific room version, however as a known-lossy chat +feature, this proposal's events are permitted in any room version. The stable event types should only be sent +in a room version which supports extensible events, however. + ## Unstable prefix -While this MSC is not considered stable, implementations should use `org.matrix.msc3381.v2.*` as a prefix -in place of `m.*` throughout this proposal. Note that extensible events and content blocks might have their -own prefixing requirements. +While this MSC is not considered stable, implementations should use `org.matrix.msc3381.*` as a namespace +instead of `m.*` throughout this proposal, with the added considerations below. Note that extensible events +and content blocks might have their own prefixing requirements. + +Unstable implementations should note that a previous draft is responsible for defining the event format/schema +for the unstable prefix. The differences are rooted in a change in MSC1767 (Extensible Events) where the approach +and naming of fields changed. The differences are: + +* For `m.poll.start` / `org.matrix.msc3381.poll.start`: + * `m.text` throughout becomes a single string, represented as `org.matrix.msc1767.text` + * `m.poll` becomes `org.matrix.msc3381.poll.start`, retaining all other fields as described. Note the `m.text` + under `question` and `answers`, and the `org.matrix.msc3381` prefix for `kind` enum values. +* For `m.poll.response` / `org.matrix.msc3381.poll.response`: + * `m.selections` becomes an `org.matrix.msc3381.poll.response` object with a single key `answers` being the + array of selections. + * `m.relates_to` is unchanged. +* For `m.poll.end` / `org.matrix.msc3381.poll.end`: + * `m.text` has the same change as `m.poll.start`. + * `m.poll.results` is removed. + * `org.matrix.msc3381.poll.end` is added as an empty object, and is required. + +Examples of unstable events are: + +```json +{ + "type": "org.matrix.msc3381.poll.start", + "content": { + "org.matrix.msc1767.text": "What should we order for the party?\n1. Pizza πŸ•\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings πŸ”₯", + "org.matrix.msc3381.poll.start": { + "kind": "org.matrix.msc3381.poll.disclosed", + "max_selections": 1, + "question": { + "org.matrix.msc1767.text": "What should we order for the party?", + }, + "answers": [ + {"id": "pizza", "org.matrix.msc1767.text": "Pizza πŸ•"}, + {"id": "poutine", "org.matrix.msc1767.text": "Poutine 🍟"}, + {"id": "italian", "org.matrix.msc1767.text": "Italian 🍝"}, + {"id": "wings", "org.matrix.msc1767.text": "Wings πŸ”₯"} + ] + } + } +} +``` -Normally extensible events would only be permitted in a specific room version, however as a known-lossy chat -feature, this proposal's events are permitted in any room version, provided they are of the unstable variety. -The stable event types must only be sent in a room version which supports extensible events. +```json +{ + "type": "org.matrix.msc3381.poll.response", + "content": { + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$fw8dod4VdLCkakmKiD6XiVj7-RrFir9Jwc9RW6llJhU" + }, + "org.matrix.msc3381.poll.response": { + "answers": ["pizza"] + } + } +} +``` -Client implementations should note that a previous draft of this proposal had a different format and some of -those events might be found in the wild, hence the `v2` portion of the unstable prefix. Clients interested in -this older format should review older drafts of this proposal. +```json +{ + "type": "org.matrix.msc3381.poll.end", + "content": { + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$fw8dod4VdLCkakmKiD6XiVj7-RrFir9Jwc9RW6llJhU" + }, + "org.matrix.msc1767.text": "The poll has ended. Top answer: Italian 🍝", + "org.matrix.msc3381.poll.end": {}, + } +} +``` From a707fc13aad324a69a0ac816d741d5c732fd4573 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 4 Jul 2023 09:45:28 -0600 Subject: [PATCH 32/34] Update proposals/3381-polls.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: KΓ©vin Commaille <76261501+zecakeh@users.noreply.github.com> --- proposals/3381-polls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index 19838436c1..ea84ef0fe4 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -382,7 +382,7 @@ and naming of fields changed. The differences are: * For `m.poll.start` / `org.matrix.msc3381.poll.start`: * `m.text` throughout becomes a single string, represented as `org.matrix.msc1767.text` * `m.poll` becomes `org.matrix.msc3381.poll.start`, retaining all other fields as described. Note the `m.text` - under `question` and `answers`, and the `org.matrix.msc3381` prefix for `kind` enum values. + under `question` and `answers`, and the `org.matrix.msc3381.poll` prefix for `kind` enum values. * For `m.poll.response` / `org.matrix.msc3381.poll.response`: * `m.selections` becomes an `org.matrix.msc3381.poll.response` object with a single key `answers` being the array of selections. From 231262056d5223ee8945db75dfae3b3f8c60a10a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 5 Dec 2023 15:03:13 -0700 Subject: [PATCH 33/34] Typo: event layering --- proposals/3381-polls.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index ea84ef0fe4..b4adc88370 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -143,17 +143,17 @@ To close a poll, a user sends an `m.poll.end` event into the room. An example be "rel_type": "m.reference", "event_id": "$poll_start_event_id" }, - }, - "m.text": [{ - // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is - // not specified, however something like the following is likely best. - "body": "The poll has closed. Top answer: Poutine 🍟" - }], - "m.poll.results": { // optional - "pizza": 5, - "poutine": 8, - "italian": 7, - "wings": 6 + "m.text": [{ + // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is + // not specified, however something like the following is likely best. + "body": "The poll has closed. Top answer: Poutine 🍟" + }], + "m.poll.results": { // optional + "pizza": 5, + "poutine": 8, + "italian": 7, + "wings": 6 + } } } ``` From 2ff98703398d1cc871147bb26683b573b0ce6a49 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 5 Dec 2023 15:22:01 -0700 Subject: [PATCH 34/34] Minor clarifications from non-blocking review feedback --- proposals/3381-polls.md | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md index b4adc88370..413b179f5d 100644 --- a/proposals/3381-polls.md +++ b/proposals/3381-polls.md @@ -71,6 +71,9 @@ With consideration for extensible events, a new `m.poll` content block is define blocks might be added in the future. Clients should treat each entry similar to how they would an `m.message` event. The array is truncated to 20 maximum options. + Note that arrays are inherently ordered. Clients *should* render options in the order presented in + the array - a future MSC may add a flag to permit rendering in a different or random order. + Together with content blocks from other proposals, an `m.poll.start` is described as: * **Required** - An `m.text` block to act as a fallback for clients which can't process polls. @@ -121,10 +124,13 @@ additional blocks, however as per the extensible events system, receivers which should not honour them. There is deliberately no textual or renderable fallback on poll responses: the intention is that clients -which don't understand how to process these events will hide/ignore them. +which don't understand how to process these events will hide/ignore them. This is to mirror what a +client which *does* support polls would do: they wouldn't render each vote as a new message, but would +aggregate them into a single result at the end of the poll. By not having a text fallback, the vote +is only revealed when the poll ends, which does have a text fallback. -Only a user's most recent vote (by `origin_server_ts`) is accepted, even if that event is invalid or -redacted. Votes with timestamps after the poll has closed are ignored, as if they never happened. Note +Only a user's most recent vote (by `origin_server_ts`) is accepted, even if that event is invalid. +Votes with timestamps after the poll has closed are ignored, as if they never happened. Note that redaction currently removes the `m.relates_to` information from the event, causing the vote to be detached from the poll. In this scenario, the user's vote is *reverted* to its previous state rather than explicitly spoiled. To "unvote" or otherwise override the previous vote state, clients should send @@ -171,7 +177,9 @@ Together with content blocks from other proposals, an `m.poll.end` is described * **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. * **Required** - An `m.text` block to act as a fallback for clients which can't process polls. -* **Optional** - An `m.poll.results` block to show the sender's perspective of the vote results. +* **Optional** - An `m.poll.results` block to show the sender's perspective of the vote results. This + should not be used as a trusted block, but rather as a placeholder while the client's local results + are tabulated. The above describes the minimum requirements for sending an `m.poll.end` event. Senders can add additional blocks, however as per the extensible events system, receivers which understand poll events should not @@ -382,7 +390,8 @@ and naming of fields changed. The differences are: * For `m.poll.start` / `org.matrix.msc3381.poll.start`: * `m.text` throughout becomes a single string, represented as `org.matrix.msc1767.text` * `m.poll` becomes `org.matrix.msc3381.poll.start`, retaining all other fields as described. Note the `m.text` - under `question` and `answers`, and the `org.matrix.msc3381.poll` prefix for `kind` enum values. + under `question` and `answers`, and the `org.matrix.msc3381.poll` prefix for `kind` enum values. `m.id` under + `answers` additionally becomes `id`, without prefix. * For `m.poll.response` / `org.matrix.msc3381.poll.response`: * `m.selections` becomes an `org.matrix.msc3381.poll.response` object with a single key `answers` being the array of selections. @@ -444,3 +453,15 @@ Examples of unstable events are: } } ``` + +### Implementation considerations + +Client authors should note that as a feature using the Extensible Events system, +usage of the *stable* event types in regular room versions is not permitted. As +of writing (December 2023), Extensible Events does not have a *stable* room version +which supports such events, therefore meaning that clients will have to use the +*unstable* event types if they intend to support polls in existing room versions. + +When Extensible Events as a system is released in a dedicated room version, clients +will be able to use the stable event types there. The unstable event types should +not be used in that dedicated room version.