Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi split operator #1309

Merged
merged 4 commits into from
Jul 10, 2023
Merged

Multi split operator #1309

merged 4 commits into from
Jul 10, 2023

Conversation

jponge
Copy link
Member

@jponge jponge commented Jun 22, 2023

Fixes #1278

@jponge jponge added enhancement New feature or request noteworthy-feature Noteworthy feature labels Jun 22, 2023
@jponge jponge added this to the 2.3.0 milestone Jun 22, 2023
@codecov
Copy link

codecov bot commented Jun 22, 2023

Codecov Report

Merging #1309 (f325e33) into main (a5a45be) will decrease coverage by 0.02%.
The diff coverage is 88.49%.

❗ Current head f325e33 differs from pull request most recent head 7f4f398. Consider uploading reports for the commit 7f4f398 to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #1309      +/-   ##
============================================
- Coverage     89.84%   89.83%   -0.02%     
- Complexity     3340     3357      +17     
============================================
  Files           458      459       +1     
  Lines         13215    13327     +112     
  Branches       1639     1656      +17     
============================================
+ Hits          11873    11972      +99     
- Misses          695      698       +3     
- Partials        647      657      +10     
Impacted Files Coverage Δ
...va/io/smallrye/mutiny/operators/AbstractMulti.java 100.00% <ø> (ø)
...ye/mutiny/operators/multi/split/MultiSplitter.java 88.07% <88.07%> (ø)
...tation/src/main/java/io/smallrye/mutiny/Multi.java 94.44% <100.00%> (+0.32%) ⬆️
...io/smallrye/mutiny/operators/uni/UniMemoizeOp.java 92.42% <100.00%> (+0.23%) ⬆️

... and 14 files with indirect coverage changes

@jponge jponge requested a review from cescoffier June 22, 2023 11:06
@jponge jponge marked this pull request as ready for review June 22, 2023 11:06
@cescoffier
Copy link
Contributor

@ozangunalp we need to discuss how reactive messaging can leverage this new feature.

@jponge
Copy link
Member Author

jponge commented Jun 22, 2023

Note that the back-pressure management is simple (1-by-1 requests as long as demand + quorum are met).

We could do something more clever by snapshotting the minimal demand among current subscribers, but that'd lead to more complex code, not counting what happens when a split subscriber leaves, etc.

@jponge jponge modified the milestones: 2.3.0, 2.4.0 Jun 22, 2023
@ozangunalp
Copy link
Collaborator

I am thinking that it'd be useful to expose the key type from MultiSplitter adding a method like public Class<K> getKeyType().

@ozangunalp
Copy link
Collaborator

On a different note, during my tests I find that request propagation is trailing by one. But I need to write a proper test to describe the issue.

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

I am thinking that it'd be useful to expose the key type from MultiSplitter adding a method like public Class<K> getKeyType().

Yes we can do that

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

@ozangunalp I've added an interface, it's similar to a grouped Multi

@ozangunalp
Copy link
Collaborator

@jponge That's also useful, but I meant to be able to get the key type (so the enum class of the key) from the MultiSplitter.
One use case I've is to be able to programmatically create split subscribers and bind them to outgoing channels in Reactive Messaging.

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

Ok I can add a method for that as well

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

@ozangunalp
Copy link
Collaborator

Sure but that requires having a key to begin with :)

To give a more visual example, I am willing to relax the single outgoing target requirement in Reactive Messsaging in order to enable the following usage :

    @ApplicationScoped
    public static class SplitterMulti {

        enum Caps {
            ALL_CAPS,
            ALL_LOW,
            MIXED
        }

        @Incoming("in")
        @Outgoing("sink1")
        @Outgoing("sink2")
        @Outgoing("sink3")
        public MultiSplitter<String, Caps> reshape(Multi<String> in) {
            return in.split(Caps.class, s -> {
                if (Objects.equals(s, s.toLowerCase())) {
                    return Caps.ALL_LOW;
                } else if (Objects.equals(s, s.toUpperCase())) {
                    return Caps.ALL_CAPS;
                } else {
                    return Caps.MIXED;
                }
            });
        }

    }

At wiring time the framework verifies the number of splits and the number of outgoing targets and wires them in the order of declaration, ALL_CAPS -> sink1, ALL_LOW -> sink2, MIXED -> sink3

@cescoffier wdyt ?

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

Ah ok, so you need that on MultiSplitter itself then, right?

In that case is having access to the key in the split any useful?

@ozangunalp
Copy link
Collaborator

ozangunalp commented Jul 3, 2023

Ah ok, so you need that on MultiSplitter itself then, right?

Yes!

In that case is having access to the key in the split any useful?

Not really sure, I only see that would be useful if you pass the SplitMulti around. But I can't think of any use.

@cescoffier
Copy link
Contributor

That's more or less what I had in mind. I'm a bit worried about the outgoing order (enum ordering can be funny).

@jponge
Copy link
Member Author

jponge commented Jul 3, 2023

There you go, I have:

  1. reverted the in-split key exposure,
  2. added a key type method to to MultiSplitter

@jponge
Copy link
Member Author

jponge commented Jul 10, 2023

@ozangunalp all good to merge or do you require more time for investigations on the RM side?

@ozangunalp
Copy link
Collaborator

@jponge Thanks, all good for me for Mutiny! I'd need more time to finish the multiple outgoings support in RM.

@jponge
Copy link
Member Author

jponge commented Jul 10, 2023

Thanks!

@jponge jponge merged commit 763cfec into main Jul 10, 2023
4 checks passed
@jponge jponge deleted the feat/multi-split branch July 10, 2023 15:12
@jponge jponge modified the milestones: 2.4.0, 2.4.0-RC1 Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request noteworthy-feature Noteworthy feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement split operator
3 participants