Feature Request: Add content-based filtering for Gossip client rebroadcasts

[arilotter]

Feature Request: Content-based filtering for Gossip client rebroadcasts

Current Behavior

The Gossip client in Iroh's Node currently rebroadcasts all messages associated with a topic that the node is subscribed to, without any content-based filtering.

Proposed Enhancement

Add the ability to filter rebroadcasts based on message content, not just the topic. This would allow nodes to drop messages that are not relevant for some reason, reducing traffic that would be dropped anyways when read by well-behaved nodes.
This could reduce bandwidth for applications where messages can be incorrect.

Use Cases

1. Dropping malformed messages

   • A node could misbehave, either due to a bug or malicious behavior, and if their data doesn't match a checksum, it could be dropped instead of rebroadcasted.

2. Dropping stale messages after some expiry

   • This could be used to implement a time-to-live (TTL) mechanism for messages
   • It would ensure that only current information is circulated in the network, preventing stale data from needlessly using bandwidth.

3. Dropping messages from unauthenticated users

   • With some authentication system, a public key might no longer have permission to participate in a network after their broadcast has been sent. You could check if a user is still authenticated before rebroadcasting their message, reducing bandwidth and DoS potential from unauthenticated users.

Proposed Implementation

1. Extend the Gossip client API to accept a custom filtering function
2. Allow users to define their own filtering logic based on message content
3. Apply the filter before rebroadcasting any message

Current API

The current API unconditionally rebroadcasts any messages with the same gossip topic.

let (gossip_tx, gossip_rx) = node
    .gossip()
    .subscribe(topic, peer_ids)
    .await?;

Proposed API

Perhaps we could add filter options to the SubscribeOpts struct, letting consumers set a filter when calling subscribe_with_opts:

pub struct SubscribeOpts {
    pub bootstrap: BTreeSet<NodeId>,
    pub subscription_capacity: usize,
    pub filter: Option<Box<dyn Fn(&SubscribeResponse) -> bool + Send + Sync>>,
}

[matheus23]

pub struct SubscribeOpts {
    pub bootstrap: BTreeSet<NodeId>,
    pub subscription_capacity: usize,
    pub filter: Option<Box<dyn Fn(&SubscribeResponse) -> bool + Send + Sync>>,
}

I'd say it's not unlikely that the filter function will want to do asynchronous work. E.g. when you want to do a quick DB lookup.

The obvious way would be to just make filter a Fn that returns a BoxFuture, but another way would be to turn the interface inside out, in the external iterators vs. internal iterators kind of sense. I.e. when you receive a message, you get a continuation with it. So unless you call SubscribeResponse::continue_rebroadcast, it's essentially filtering out the message.

This way you have the context in which you accept subscriptions at your service (variables in scope, an async context, etc.). But it's also a way bigger change & needs some thinking in terms of what this means for the gossip's actor loop, and you can wreak so much more havoc this way.