Notes on interleaved evaluations

[borkdude]

9:51 PM
borkdude Has self-hosted CLJS been tested to work correctly in the presence of concurrent evaluations? E.g. when you evaluate behind a click handler or setTimeOut and evaluate code which loads a couple of overlapping namespaces "at the same time"? I don't have a bug, I just want to know if anyone has some details on this. (edited)
11:25 PM
borkdude E.g. is this supposed to work/supported?
(js/setTimeout #(eval '(do (ns foo1) (defn foo []))) 1)
(js/setTimeout #(eval '(do (ns foo2) (defn foo []))) 1)
(edited)
11:30 PM
borkdude I would be ok if it's not, I'm just trying to get information
7:44 AM
thheller @borkdude there is no such thing as "at the same time". JS is single threaded so there is only ever one thing at a time happening
7:45 AM
ns is a special form so running it in a do is not really supported, although I'm no entirely sure how self-host handles that case (edited)
8:53 AM
borkdude @thheller I understand but async eval can be interleaved and this might mess things up when you have multiple evals running. Same reason why async mutex exists in JS, that’s where my question is coming from.
8:55 AM
I have a similar problem in SCI where I bind ns to a certain value and then pop the value in a finally promise function. But this could go wrong if your UI triggers multiple evals. So I wonder if self hosted has similar issues and if not, how that is handled.
8:55 AM
thheller again .. there is no such thing as multiple evals running. they are all in a queue in the event loop
8:55 AM
if you want more control over the queue make your own
8:56 AM
but if you use setTimeout like your example above they will execute in that order, it won't randomly switch (edited)
8:56 AM
but to be entirely sure you should use your own queue regardless
8:57 AM
borkdude So that’s what I’m asking. Since evals are handled using callbacks in self hosted, and you evaluate multiple expressions in one go, are they interleaved or executed automatically
8:57 AM
Atomically (ff ing phone)
8:58 AM
thheller no, the compiler doesn't manage queues for you. not all actions are async so most of the time things just execute directly (edited)
8:58 AM
it is async because things like ns need to do async IO. most other forms do not, really only ns and require (edited)
9:01 AM
borkdude Yeah, exactly the same for NBB. But because the ns form is async there, every top level form is executed async, one by one, chained in promises. Yielding one composite promise per eval. So if you trigger multiple evals the you might have that top level expressions are executed interleaved. But perhaps I should just not support that
9:01 AM
thheller IMHO there is no such thing as an "async mutex" in JS. what most people describe that as I would call a queue 😛
9:03 AM
borkdude Well sure. I have tried one such impl yesterday which imitated a lock. It almost worked. But nested load-strings didn’t. Because the inner one could not acquire the “lock”
9:04 AM
thheller don't build on promises that just creates nightmares
9:06 AM
you can't ever lock because of the single thread. the best you can do is queue. so I'd recommend staying from anything that pretends to emulate a lock and instead switch your program to be queue based from the start
9:08 AM
borkdude I quoted the word lock. I should just paste the example I used, I understand there is a single thread ;)
9:09 AM
thheller just saying .. there are many JS libs that pretend you can lock and stuff like that. just so your code looks more like other platforms where you can actually lock. I don't think that is a valid way to develop JS code
9:09 AM
borkdude So does self-hosted use a queue or should you use one yourself or doesn’t it have any of these issues?
9:09 AM
thheller no, build your own
9:10 AM
borkdude But it can have these issues? Yes or no?
9:10 AM
thheller yes, everything that needs to go async has these issues when also accepting inputs at the same time (edited)
9:10 AM
borkdude Right
9:13 AM
How would you handle nested evals with a queue?
9:13 AM
thheller in shadow-cljs I run each watch worker in a single thread and core.async channels to manage that. works well but isn't necessary, just an atom like the compiler env should be enough (edited)
9:15 AM
for me its just easier to think in channels with the downside of being a little harder to debug at times since some state is in go loops which are harder to inspect (edited)
9:16 AM
so if I was going to do it again I'd maybe just use a single atom
9:16 AM
so you put a ::work-queue in the compiler env and only ever append to that
9:17 AM
then after adding you check if work is currently pending, aka. a timeout pending. if so you do nothing, otherwise you set that timeout
9:17 AM
the timeout then picks the first item to work off and on each "eval" result you repeat the cycle
9:19 AM
since its all single thread you don't even need to worry about locks and stuff which you'd otherwise have
9:29 AM
borkdude @thheller Right, thanks
9:33 AM
I basically just wanted to verify if self-hosted had similar problems and if so, if it doesn't protect users from these problems, I probably won't either, but will just write some docs that users should use a mechanism (queue) to protect themselves. The use case for interleaved eval is probably niche anyway, but you never know. (edited)
9:35 AM
thheller what are you building now? sounded like something you were building for your own uses? 😛
9:35 AM
borkdude @thheller normally nbb is used quite similar to how CLJS works: namespaces at the top. one program execution started from the main entrypoint let's say
9:36 AM
but I'm exposing an API and you never know how people are going to use this API :)
9:37 AM
thheller then I'd probably manage that for them? its not that much work for you to switch your code to (swap! env your-eval code callback) instead of (your-eval code callback) 😛
9:38 AM
borkdude Doing too much can have disadvantages as well. I'll think more about it :)

[borkdude]

Interesting article: https://staltz.com/promises-are-not-neutral-enough.html

then() is a mix of map() and flatMap()

Thanks for sharing @littleli