Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The number of messages from the dropped is not matched after publish. #1079

Open
snz2 opened this issue Jul 15, 2024 · 4 comments
Open

The number of messages from the dropped is not matched after publish. #1079

snz2 opened this issue Jul 15, 2024 · 4 comments

Comments

@snz2
Copy link

snz2 commented Jul 15, 2024

The number of messages from the dropped is not matched after publish.
The WS.SEND is a dropped message of the same number of dropped messages as the counter 2 of the return results, and Publish seems to be missing.

Testing is a method of send 100000 sync(not async) messages from another processor and testing a message that the server cannot handle.

uWebsockets.js version=20.43.0

  1. publish
    proc1 send message * 100,000 -> proc2(uWebsockets.js) publish -> proc1 check receive Count
    ->return true=100,000, dropped callback count 13,000, echo receive 28,000, proc2 drain 1(buffer length=0)

  2. ws.send
    proc1 send message * 100,000 -> proc2(uWebsockets.js) ws.send -> proc1 check receive Count
    ->return result 2=72,000, dropped callback count 72,000(same), echo receive 28,000, proc2 drain 1(buffer length=0)

*There are detailed numbers, but omitted

In addition to the same message, the number of eco -response has been checked, so it can be seen that Publish was not successful. For reference, the Drain callback came once(1) out of 100,000 times in both publish and ws.send.

Publish is automatically managed by the backpressure value, but it seems that the MISSED must have a message. It is a test that sends the same message as WS.SEND. If Publish is a design that doesn't care much about missing messages, it is inevitable. Since Publish is aimed at multiple topic users, it is a problem because it is not structurally like GetBufferedamount, so I want to create something like Topicws = {} that manages the topic separately. Please let me know if I missed. thank you
ps. I turned a language translator. Please understand

@uNetworkingAB
Copy link
Contributor

Can you make this into a integration test? Can run it in CI.

@uNetworkingAB
Copy link
Contributor

In other words; can you provide the code that you based your conclusion on?

@snz2
Copy link
Author

snz2 commented Jul 16, 2024

uWebsocket Server (proc2)

`
var droppedCnt=0, droppedCnt2=0, doneCnt=0;
var usp=uws.App().ws('/*', {
idleTimeout:0,
open:(ws)=>{
ws.subscribe('node');
},
message:async (ws, msg, isBinary)=>{
let data;
try{
data=JSON.parse(Buffer.from(msg).toString());
let res=await db.get(data.query);
// res is object {...}
// ws.send
res['uuid']=data.uuid;
let sendRes=ws.send(JSON.stringify(res), true, true);
if(sendRes==2)droppedCnt2++;

     // publish 
     // usp.publish('node', JSON.stringify(res), true, true);
     
     doneCnt++;
}catch(e){
    console.error('message', data, e);
}},
drain:(ws)=>{
    console.log('drain:', ws.getBufferedAmount().length)
},
dropped:(ws, message, isBinary)=>{
    droppedCnt++;
},
close:async (ws, code)=>{try{

}catch(e){

}}

})`

In Proc1, WS turns for loops and sends a 100,000 DB request message. recognize the request response in the WS message by putting UUID in the message.

Because I test like this
In WS.SEND(Proc2 uWebsockets server side), DROPP Callback(droppedCnt) and ws.Send's Result: The result 2(droppedCnt2) is exactly matched, so handling is possible.
Publish comes only 13,000 messages to DroppedCallBack(droppedCnt), and only 28,000 messages come to the WS side(proc 1).

doneCnt is well(doneCnt) -driven by 100,000 when doing send or publish. In other words, the DB query was no problem. no Crash, no error

Of course, I will check the buffer if I use WS.SEND in the production code, but the problem is that the publish is not controlled by the dropped.

@snz2
Copy link
Author

snz2 commented Jul 16, 2024

I don't think the code function is not good in the GitHub editor ... I can't modify it. Anyway, the server side code is conceptually everything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants