<mdiers_>
Did i take the right executors there? Since the attached_executors are deprecated, which one should I use now?
<simbergm>
mdiers_: so... it seems like you only want to schedule threads on a particular worker thread (not a range), is that correct?
<simbergm>
in that case using scheduling hints is most likely easiest
<simbergm>
but as you've already seen things are a bit in flux
<simbergm>
so currently on master you can use either default_executor which has a constructor that takes a scheduling hint
<simbergm>
or do what you've done now with the attached_executor
<simbergm>
they're more or less equivalent in behaviour except that the attached executor allows you to specify a range of threads to schedule threads on
<simbergm>
after things have been a bit cleaned up the default executor will just be an alias to the parallel executor (which currently doesn't take a scheduling hint, but will)
<simbergm>
so usage of the default executor should remain more or less unchanged (it's just the implementation that will change)
<mdiers_>
i would like to work only numa safe, one work job per numa.
<simbergm>
the attached executor will also get a replacement but it'll be renamed (because the static_priority/local/etc. part of attached executor is a lie)
<simbergm>
one work job per numa? does that mean only one task per numa node?
<mdiers_>
one work job per numa, which can be processed in parallel on the numa node
<simbergm>
I see
<mdiers_>
see numa_hint_server:work_load
<simbergm>
for that case there is also a numa scheduling hint, but only the shared_priority_queue_scheduler actually takes that into account
<simbergm>
so I'd recommend you use block_executor
<simbergm>
it uses the attached executors as an implementation detail
<simbergm>
there's an example somewhere, let me dig it out
<mdiers_>
thanks, i will then adapt the example and test it. currently i get a random distribution which is not bound to the numa nodes.
<simbergm>
yeah, note that currently even if you specify that tasks should be scheduled on a specific thread or numa node, they may still get stolen to another worker thread
<mdiers_>
yeah, but all?
<simbergm>
if you really must have tasks stay on the worker thread the shared priority queue scheduler supports some experimental options to make sure they stay, but that's very much wip
<simbergm>
hard to say, they should not all be randomly distributed, but we may of course have a bug...
<simbergm>
if you have a small example that demonstrates it that would be helpful
<simbergm>
what is your system topology (multiple numa nodes, etc.)? what do you expect num_hint to be and what is it in your case? do you run this once on every numa node at the same time?
<simbergm>
you can try if `--hpx:numa-sensitive=2` does anything
<simbergm>
diehlpk_work: yep, ok, I'll do it this week
<Yorlik>
hkaiser: I think I'll just use a luastate pool and not make the states thread local. I'll just assign a state from the pool to a task until it's finished.
gonidelis has quit [Quit: Ping timeout (120 seconds)]
<zao>
hkaiser_: did you remember to say something to the SoC person from last night?
<hkaiser_>
zao: I was waiting for him to show up again
<zao>
Ack.
gonidelis has joined #ste||ar
hkaiser_ has quit [Ping timeout: 256 seconds]
nan has joined #ste||ar
nan is now known as Guest90784
gonidelis has quit [Ping timeout: 240 seconds]
hkaiser has joined #ste||ar
AndroUser has joined #ste||ar
AndroUser has quit [Client Quit]
<Yorlik>
hkaiser: I replaced the thread local Lua states with states pulled from a thread-safe thread_local pool (to reduce contention) and can now call all these functions which used to crash the Lua Engine. I still need to test it more and check performance but at first look the issue seems to be solved.
<hkaiser>
Yorlik: nice!
<hkaiser>
Yorlik: you need to make sure that the hpx threads (tasks) always return to the same lua state they were launched from
<hkaiser>
they should 'carry' the state around
<Yorlik>
The state is kept around until the function returns
<hkaiser>
that'snot what I meant
<Yorlik>
Its in a unique_ptr with a custom deleter which gives it back to the pool after it goes out of scope
<hkaiser>
the task needs to always return to its originating state
<Yorlik>
So the state is guaranteed to be the same - it's no longer thread_local.
<hkaiser>
ok, it's passed to the task, good
<Yorlik>
I just use get_luaengine inside my updater inside the parloop
<Yorlik>
Then the real updated in Lua gets called.
<Yorlik>
updater
<Yorlik>
Even if the task gets moved to another core - the luastate is given back after the lua function returns.
<Yorlik>
That was different wioth the thread local engines i used.
<Yorlik>
They just poofed aay on a task migration
hkaiser has quit [Quit: bye]
<diehlpk_work>
simbergm, Looks good
Guest90784 has quit [Remote host closed the connection]
nanmiao has joined #ste||ar
hkaiser has joined #ste||ar
diehlpk__ has quit [Ping timeout: 268 seconds]
hkaiser has quit [Ping timeout: 256 seconds]
nanmiao has quit [Remote host closed the connection]