Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT][CELEBORN-1490][CIP-6] Support Flink hybrid shuffle integration with Apache Celeborn #2678

Closed
wants to merge 16 commits into from

Conversation

codenohup
Copy link
Contributor

@codenohup codenohup commented Aug 12, 2024

What changes were proposed in this pull request?

Support Flink hybrid shuffle integration with Apache Celeborn

Why are the changes needed?

Implements CIP-6

Does this PR introduce any user-facing change?

No

How was this patch tested?

IT


The authorship belong to me and @reswqa @TanYuxin-tyx

@mridulm
Copy link
Contributor

mridulm commented Aug 12, 2024

+CC @venkata91

Copy link

codecov bot commented Aug 14, 2024

Codecov Report

Attention: Patch coverage is 6.54206% with 100 lines in your changes missing coverage. Please review.

Project coverage is 33.17%. Comparing base (ea6617c) to head (7d0cd40).
Report is 24 commits behind head on main.

Files Patch % Lines
...a/org/apache/celeborn/common/meta/MapFileMeta.java 0.00% 43 Missing ⚠️
.../common/network/protocol/SubPartitionReadData.java 0.00% 18 Missing ⚠️
...org/apache/celeborn/common/util/PbSerDeUtils.scala 0.00% 15 Missing ⚠️
...born/common/protocol/message/ControlMessages.scala 0.00% 13 Missing ⚠️
...cala/org/apache/celeborn/common/CelebornConf.scala 0.00% 6 Missing ⚠️
...ache/celeborn/common/network/protocol/Message.java 57.15% 3 Missing ⚠️
...born/common/network/protocol/TransportMessage.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2678      +/-   ##
==========================================
- Coverage   39.83%   33.17%   -6.66%     
==========================================
  Files         239      311      +72     
  Lines       15026    18323    +3297     
  Branches     1362     1681     +319     
==========================================
+ Hits         5984     6076      +92     
- Misses       8711    11907    +3196     
- Partials      331      340       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codenohup
Copy link
Contributor Author

We need to split this PR into multiple smaller commits to facilitate the review process.
Therefore, we will close this PR now. 🤧

@codenohup codenohup closed this Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants