<Yorlik>
How would I start an action and demand to start execution right away, like realtime priority? I could still wait for the future, since it's a long running action, but I don't want any delay in startup.
<Yorlik>
NVM - found it: hpx::async<agns::game::Controller::start_action>(hpx::launch::sync, gcId);
<Yorlik>
Argh - crap thats wrong ... fork maybe?
<jbjnr_>
Yorlik: fork will suspend the current task and immediately switch to the new one - but this would only work for a local action. If you wanted to do it remotely, you need to look at direct_Action
<Yorlik>
How would I start a task immediately but async - is there a way to control that ?
<jbjnr_>
fork policy should work, (I tend to use a high priority task and am prepared to wait)
<Yorlik>
But you said fork suspends the current task. i want it to be non blocking but guarentee to start immediately
<Yorlik>
Or asap
<jbjnr_>
"non blocking but guarentee to start immediately" - that's a contradiction unless you spawn a completely new worker thread to launch it on
<jbjnr_>
can you reserve a core for a special thread pool for these realtime tasks?
<jbjnr_>
(one at a time), or more threads in the pool if you need multiple
<jbjnr_>
cores^
<Yorlik>
I am starting this out of hpx_main and its essentially a task running for the lifetime of the program
<jbjnr_>
Then the best thing to do would be create aseparate thread pool with 1 core in it and only run that task on that pool
<Yorlik>
OK. Is there a way to iterate of my worker threads to initialize thread local objects?
<jbjnr_>
then normal tasks will use the 'default' thread pool and your special task will be in it's own sandbox
<Yorlik>
Makes sense. My default pool would have a lua engine in each OS thread. I'd like to initialize all of them at start. How could I iterate over them?
<Yorlik>
i just want to call a function at startup on each thread which created the static thread local lua engine
<jbjnr_>
yup. let me think a moment
<jbjnr_>
I'm looking for an example to help you
<Yorlik>
We don't keep state in the lua engines - they just run one off tasks that store data on the C++ side
<Yorlik>
So we can run any gameobject message handlers on any lua state
<jbjnr_>
if you are running this from main(), then just loop over the number of threads and async spawn a task for each lua engine/interpreter, if no other taks are running then each core will take one task automagically
<Yorlik>
That I could do.
<Yorlik>
I have a separate init function that would just do it
<jbjnr_>
to be 100% certain that nothing else runs there, create a pool name "lua" or something and allocate N cores to it, then start N lua tasks on that pol. All other taks will go on the "default" pool
<jbjnr_>
the default pool must have at least one core assigned to it
<Yorlik>
How would I pin a task to a thread?
<jbjnr_>
we plan to allow pools to coexist on the same cores (so on 1 4 core machine you could have a 4 core lua pool and a 1 core default pool and 1 core would have to run two worker threads, but we haven't enabled it yet)
<Yorlik>
That might for example make sense for pipelining scenarios
<jbjnr_>
step 1, create a lua pol (this example creates an MPI pool)
<jbjnr_>
step 2, create an executor that is bound to that pools
<jbjnr_>
step 3, async(executor, task)
<jbjnr_>
step 1 must be done at startup, steps 2,3 can be done any time
<Yorlik>
So I can pin to a pool, but not a thread in that pool? Would I need single thread pools then?
<jbjnr_>
you could create N thread pools each with one core, but if the task never suspends, one it starts runing it will stay on the core it started on, so it is sort of pinned by default.
<jbjnr_>
we do support launching on a single core within a pool, but there isn't an example and the API is 'in flux'
<jbjnr_>
so it would be better to use N pools like the first example I showed you
<Yorlik>
So I cannot iterate over the threads within a pool but just have several single threded pools I could then iterate over?
<jbjnr_>
(named_pool_executor test)
<jbjnr_>
yes.
<Yorlik>
Which would spool any scheduling ifc.
<Yorlik>
ifc=ofc
<Yorlik>
err -- spool = spoil
<jbjnr_>
it is possible to iterate over threads, but the API is a bit broken and you're better off using N pools and iterating over them with an executor for each
<Yorlik>
But what about my scheduling then?
<Yorlik>
I still would like to just give a tsk to the pool and not care where its running
<Yorlik>
I simply want a guaranteed initialization at start.
<jbjnr_>
but you said these lua tasks run forever, so no other tasks can run there anyway?
<Yorlik>
Nono
<Yorlik>
The lua tasks are short
<Yorlik>
But the task doing the main event loop runs forever
<Yorlik>
It does the task construction
<jbjnr_>
video chat?
<Yorlik>
Like batching 100 objects for update into a task
<Yorlik>
Sure
<jbjnr_>
appear.in?
<Yorlik>
Ya
<jbjnr_>
I'll just make a coffee. Started meeting in stellar room of appear. in, join when you like
<Yorlik>
OK
<jbjnr_>
sorry cut you off abruptly at the end then
<Yorlik>
NP - thanks for the time you took !
<Yorlik>
Starting talking about this project easily become a bit too open eneded ;)
rori has joined #ste||ar
<zao>
Yorlik: Can't scope-creep if you have no scope :P
<Yorlik>
If only my English would go far enough to understand that ...
<zao>
Scope-creep being the act of growing something well-defined and simple far past its intended boundaries.
<Yorlik>
Naw - it wasn't scope creep - it's just a big project. :)
<Yorlik>
Many aspects from different directions
<zao>
If you aim to do everything, nothing is out of bounds :)
<jbjnr_>
the changes you made a month or two ago to the schedulers for fast_idle and scheduler mode...
<jbjnr_>
every thread has it's own 'mode' flags, so they check things like stealing independently - is there a use case for this - it seems like all the threads in a pool are likely to be controlled the same - is this used currently?
<hkaiser>
this is to avoid false sharing
<jbjnr_>
but false sharing only matters if the cache line is modified - if they all share the flags and are not changing frequently - then it doesn't matter
<jbjnr_>
no?
<hkaiser>
jbjnr_: is it really just a problem when things are modified?
<jbjnr_>
well if 10 threads al use a var in a location, but it never changes value, then the cache isn't invalidated (as long as we use the cach_line_padding) so there's no need to reload the cache line etc etc
<hkaiser>
jbjnr_: wikipedia: "When a system participant attempts to periodically access data that will never be altered by another party, but those data share a cache block with data that are altered, the caching protocol may force the first participant to reload the whole unit despite a lack of logical necessity."
<jbjnr_>
it might get evicted from local cache - but that would happen anyway
<jbjnr_>
the key phrase is "tat are altered"
<jbjnr_>
^that
<jbjnr_>
if the flags are mostly consts, then it doens't matter
<hkaiser>
it says 'that never be altered'
<jbjnr_>
I don't think we need the duplication.
<jbjnr_>
bbiab
<hkaiser>
shrug, feel free to remove it if it has no perf effect
<jbjnr_>
as long as nothing that shares the cache line is changed, then there is no need for any core to reload the line, so I thinkit's enough to pad it to the line and that's that. If the flag is changed after a parallel_for loop, then it will trigger a reload on each thread, but that would have happened anyway
<jbjnr_>
I also noticed that you made a few tweaks to return values from wait_or_add_new and other functions. Could I ask you to do me a favour ...
<hkaiser>
I don't think I did those changes, at least can't remember that I did
<jbjnr_>
I'm still worried that I do the wrong thing in wait_or_add_new and I don't believe the comments around it are very clear, could you please have a look at the comment for wait_or_add new and reword it to make it a bit more clear what should happen. then send me the text and I'll add it to my next commit/PR?
<jbjnr_>
oh^ if it wasn't you then no matter, I'll make the changes and clean up sas and when ...
<hkaiser>
jbjnr_: I think the comment says it all, I'm not aware of any other things that should be done there
<jbjnr_>
should stealing be done inside wait_or_add_new, or done in get_next_thread
<jbjnr_>
(IYHO)
<hkaiser>
jbjnr_: we do steal from ready queues in get_next_thread, but steal from staged queues in wait_or_add_new
<hkaiser>
jbjnr_: but that's some heller has implemented, he will know best
<hkaiser>
something*
<hkaiser>
simbergm: I don't know how to make the transition to hpx::program_options smooth for the user
<hkaiser>
no idea how to make this a gradual process
<hkaiser>
if users have code that relies on boost::prgram_options it will simply fail to compile
<hkaiser>
I could let the user decide at configure time whether to use boost, but that wouldn
<hkaiser>
t enable us to issue any deprecation warnings
<jbjnr_>
hkaiser: ok thanks. That seems fine then. I will check csarefully whats in the other schedulers - I have a lot of conflicts that I thibnk I've messed up in my stuff and I need to spend some time double checking everything.
<hkaiser>
ok
<jbjnr_>
simbergm: merge that threads stuff asap to give me time to clean up my stuff rebased onto it.
<jbjnr_>
(when oyu're happy it doesn't break anything)
<hkaiser>
or we issue the deprecation warning at configure time - the user might not relate that to his compilation problems, though
<K-ballo>
templates..? duck typing?
<simbergm>
jbjnr_: gotcha, it was clean before my last rebase so hopefully I can merge it once pycicle is ready (tomorrow)
<K-ballo>
if it looks like a boost::program_options::description object, warn/reject?
<simbergm>
hkaiser: can we alias hpx::program_options to boost::program_options for one release? have it off in the next release, switch it to the "real" hpx::program_options the next one
<simbergm>
it should be compatible after all if we're copying it over
<simbergm>
(not sure if it's changed significantly between 1.61 and 1.71)
<jbjnr_>
hkaiser: or simbergm do any of the thread_xxx_executors work properly with the new thread pools if I want to put a thread on core 0, core 1, core 2, etc
<jbjnr_>
yorlik was asking about this and I want to make a demo that uses them if they are functional. I'm hoping they can convert their thread numbers into hints that are passed into the schedukers properly and I can disable stealing
<simbergm>
jbjnr_: afaik they support it as much as any other executor
<jbjnr_>
ok, which executor can I say, bind to thread 0 on this pool, thread 1 on the pool etc?
<simbergm>
I never remember how we pass schedule hints to async/apply but as long as the scheduler actually cares about hints, the executors will pass them along
<hkaiser>
simbergm: we can do that, but we can't warn the user
<simbergm>
jbjnr_: maybe it's not exposed... (I distinctly remember there was a way to do it, but I can't find an example now)
<simbergm>
hkaiser: can we not?
<jbjnr_>
we have a pool_executor, but I'm not aware of a 'core_executor'
<hkaiser>
simbergm: how can we?
<jbjnr_>
I would like to create one that works cleanly with the new RP stuff, but if one of the existing ones works, then it should be cleaned up and repurposed
<hkaiser>
simbergm: if the user include boost::po in his code and call HPX using those objects
<jbjnr_>
I'm not quite sure what the thread_pool_executor and os_executors are for
<hkaiser>
jbjnr_: construct a pool that uses some cores only
<simbergm>
hkaiser: right, I'm dumb... you're right
<K-ballo>
we can tell boost::po from hpx::po
<hkaiser>
simbergm: I would like to avoid replicating all of the init stuff for both libraries
<hkaiser>
K-ballo: how so?
<simbergm>
hkaiser: yep, that'd be a pain
<K-ballo>
subtypes?
<hkaiser>
derive our own types from the boost types, hmm, that might work
<K-ballo>
only as far as detecting those, the subtypes won't show in their chained call interfaces or anything
<hkaiser>
sure
<hkaiser>
I see what you mean
<hkaiser>
make the derived types implicitly constructable from the bases
<hkaiser>
we could attach the deprecation warnings to those constructors
<K-ballo>
even simpler
<hkaiser>
that would create some unneeded copies, but hey
aserio has joined #ste||ar
<jbjnr_>
did anyone answer my executor questions? I think they were lost in the other conversation
<jbjnr_>
ok. I see that's the one for the right scheduler. I forgot about that
<heller>
Yuo
<jbjnr_>
ok, so I should probably try to investiagte that code too and see if it can be massged into working with the RP and the thread hints and stuff
<heller>
And it's not that it doesn't work...
<heller>
Yes, that would be perfect
<Yorlik>
I'd sugesst to not delete too much stuff (yet). HPX isn't THAT famous yet, and you never know what crazy use cases people come up with. Someone had a reason to create that thing you want top delete in the first place ;).
<Yorlik>
IIRC you usually have people from science using, but now we are coming and trying tomake a game with it. New use cases coming up, like "Give me a way to cleanly initialize" or "Give me persistent AGAS IDs" ...
<zao>
I tried using HPX for applications once.
<zao>
Then I took an arrow to the kne.
<zao>
*knee
<Yorlik>
Better an arrow in the knee than an arrow in the know ;)
<Yorlik>
Created my first HPX components from Lua today :)
aserio has quit [Ping timeout: 264 seconds]
lsl88 has quit [Read error: Connection reset by peer]
aserio has joined #ste||ar
nikunj has joined #ste||ar
aserio has quit [Ping timeout: 248 seconds]
rori has quit [Quit: bye]
<nikunj>
hkaiser: yt?
Yorlik has quit [Quit: Leaving]
<nikunj>
is it possible to initialize a lambda capture in C++ 11?
<nikunj>
K-ballo: ^^
Yorlik has joined #ste||ar
<hkaiser>
nikunj
<hkaiser>
nikunj: we need to create that reproducibility appendix for the paper
<nikunj>
hkaiser: ohh
<nikunj>
which would mean giving the scripts right?
<diehlpk_work>
Here is a sample formm but have you ever locked in the submission system?
<diehlpk_work>
*logged
<diehlpk_work>
There you should see what kind of information they want to have
<nikunj>
can we submit this paper as student paper?
<diehlpk_work>
Just generate a submission for your workshop and upload what you have, you can update everything till the deadline
<nikunj>
I'm not sure about the first point, but I pass all other points
<nikunj>
diehlpk_work: I have not logged into the submission system
<diehlpk_work>
Please do to check what information is needed for your workshop
<nikunj>
let me create an account now
<diehlpk_work>
Gabriel and I had several issues to upload our papers
<nikunj>
ohh
<diehlpk_work>
I had to contact the support to get the SC paper uploaded
<diehlpk_work>
Therefore, you should upload in advance to have still enough time to contact them
<nikunj>
what should I use for Company/Institution section of contact info?
<nikunj>
IIT Roorkee or STE||AR GROUP?
<diehlpk_work>
First
<nikunj>
ok thanks
<nikunj>
diehlpk_work: Do I need to fill in STE||AR as 2nd Company/Institution?
<diehlpk_work>
No, as long you have it on the paper it is fine
<nikunj>
alright
<diehlpk_work>
As stellar group is not a company or a institute, it can lead to complications
<nikunj>
ohh, makes sense
<diehlpk_work>
Once, I had to provide a state and a country for stellar group
<nikunj>
hkaiser: I have successfully created an account. I will be able to submit the paper
<nikunj>
I will do the same tomorrow
<nikunj>
diehlpk_work: thanks a lot for the help!
<nikunj>
diehlpk_work: does my work come under student submission?
<nikunj>
I'm not sure of the first point which says: "It must be original work by the student, in which the student is the primary contributor (e.g., responsible for at least 50% of the work)."
<diehlpk_work>
I would not submit it as a student submission
<nikunj>
ohh any reasons?
<diehlpk_work>
I am not even sure what the difference is
<heller>
diehlpk_work: Betelgeuse, Orion
<diehlpk_work>
heller, Yeah, next time I will add it and see what happens
<diehlpk_work>
nikunj, discuss with hkaiser about the student submission
<hkaiser>
nikunj: submission should be possible again now, I pinged Keita
<nikunj>
hkaiser: submission as in?
<nikunj>
I didn't quite get what you mean to say
<nikunj>
hkaiser: I've made the graph fonts bigger and also made the graph bigger as well
<hkaiser>
nikunj: the paper submission site was closed this morning
<hkaiser>
now its open again, so you scan go ahead and submit a first version asap
<nikunj>
hkaiser: ohh
<nikunj>
I didn't know that
<nikunj>
hkaiser: I've also made the y axis to start from 0 for 1d stencil
bita has joined #ste||ar
<nikunj>
hkaiser: please go through the graphs when you have time. I'll submit the paper post that
aserio has joined #ste||ar
<nikunj>
hkaiser: also, I ported most of it to C++11. 2 errors still remain, first initializing within a lambda (which idk is possible with C++11) and std::index_sequence. If you have a way to get around, please let me know and I will make the change
hkaiser has quit [Quit: bye]
maxwellr96 has quit [Read error: Connection reset by peer]
hkaiser has joined #ste||ar
<hkaiser>
nikunj: so we should create that appendix, diehlpk_work has given you the template already, has he?
<diehlpk_work>
nikunj, Do you have to submit the form or the pdf document?
<diehlpk_work>
nikunj, Have you seen that there is a bash script you have to run and provide the output?