hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoC2018: https://wp.me/p4pxJf-k1
<hkaiser>
ok
<hkaiser>
would it make sense to simply integrate your stuff with hpx_init.a?
<nikunj>
most of my work will relate to hpx_main and hpx_wrap only
<nikunj>
now that we are defining according to variables, I think we can integrate it in hpx_init.a also
<hkaiser>
nod
<nikunj>
I can add various checks to prevent my code to work for hpx_init.hpp
<nikunj>
and at the same time work for hpx_main.hpp
<hkaiser>
cool
<nikunj>
I will work on it today. It should be done by today night itself.
<hkaiser>
:D
nikunj has quit [Quit: Bye]
diehlpk_mobile has joined #ste||ar
diehlpk_mobile has quit [Quit: Yaaic - Yet another Android IRC client - http://www.yaaic.org]
hkaiser has quit [Quit: bye]
K-ballo has quit [Quit: K-ballo]
nanashi55 has quit [Ping timeout: 276 seconds]
nanashi55 has joined #ste||ar
Anushi1998 has quit [Quit: Bye]
jaafar has joined #ste||ar
jaafar has quit [Ping timeout: 256 seconds]
quaz0r has quit [Ping timeout: 260 seconds]
nikunj has joined #ste||ar
<nikunj>
do we have a way to register a function on kernel thread to run on HPX thread instead?
<nikunj>
or a function running on a thread registered with the HPX runtime?
<nikunj>
by registering a function I mean that we can invoke all HPX functionality (and not limited API calls)
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
Anushi1998 has joined #ste||ar
Anushi1998 has quit [Read error: Connection reset by peer]
<M-ms>
jakub_golinowski: cool! would you like to answer? the second point is valid, that's why it's a backup... sounds like suspend/resume could be what he's hoping for
<M-ms>
maybe it would still be possible to have two HPX backends, just not automatically choosing the appropriate one
<M-ms>
the first point I'm not an expert, but it's definitely possible but not trivial
<M-ms>
I'm not sure if opencv matrices would have to be backed by partitioned_vector (or equivalent) but in principle parallel for loops are nice to distribute, it's just the data movement that costs time
<jakub_golinowski>
M-ms, I can answer to this following your suggestions as I am not familiar with distributing the workload over different workers. As for the second point what does he mean by static initialization?
jakub_golinowski has quit [Quit: Ex-Chat]
jakub_golinowski has joined #ste||ar
<M-ms>
jakub_golinowski: sure, as I said I'm not quite sure what would be required to have a distributed parallel for loop so don't write anything too definite ?
<M-ms>
as for the static initialization I assume he's talking about something like the static tbb::task_arena which represents a runtime/thread pool or something like that
<jakub_golinowski>
M-ms, I think that in principal it is possible but would require some more work, first to explore the necessary data structures and then to see if parallel_for can be used to do it somehow transparently
<M-ms>
jakub_golinowski: yep, sounds about right, but I think it's out of scope for this project
david_pf_ has joined #ste||ar
mcopik has quit [Ping timeout: 240 seconds]
hkaiser has joined #ste||ar
K-ballo has joined #ste||ar
<nikunj>
hkaiser: I was able to link libhpx_wrap with every executable and things look fine as of now. I was able to achieve it using weak symbols attribute. I added a weak variable in hpx_wrap.cpp and overrided it in hpx_main.hpp. This way any file including hpx_main will override it. At runtime I am simply checking for it's value accordingly with an if else statement, so if overriden then it behaves differently.
<nikunj>
hkaiser: I used this method as I could not bring about this behavior using pp constants
<hkaiser>
nikunj: that sounds perfectly fine
<hkaiser>
using pp constants was just a suggestion, if you need runtime support this is certainly not sufficient
<K-ballo>
there's a question on the slack channel
jakub_golinowski has quit [Ping timeout: 256 seconds]
quaz0r has joined #ste||ar
<hkaiser>
K-ballo: thanks
jakub_golinowski has joined #ste||ar
jakub_golinowski has quit [Ping timeout: 245 seconds]
david_pf_ has quit [Quit: david_pf_]
mcopik has joined #ste||ar
jakub_golinowski has joined #ste||ar
<nikunj>
hkaiser: integrated my code into hpx_init. Examples are running fine. Moving on to running tests on them
<hkaiser>
nikunj: nice!
<M-ms>
jakub_golinowski: is tomorrow around 3 okay for the call?
<jakub_golinowski>
M-ms, yest it is
<jakub_golinowski>
3 CEST, right?
<M-ms>
yes
mcopik has quit [Ping timeout: 255 seconds]
Anushi1998 has joined #ste||ar
<nikunj>
hkaiser: yt?
<hkaiser>
here
<nikunj>
hkaiser: Do we have an HPX function that registers other functions (registered with HPX thread but not running on it) to run on HPX thread?
<hkaiser>
nikunj: sure
<hkaiser>
do you need to wait for the function to run?
<nikunj>
yes
<nikunj>
I just want to register it so that when it runs, it runs on HPX thread
<nikunj>
if that's possible then Billy's idea of utilising init_seg will work
<hkaiser>
this synchronously runs F on an hpx thread
<nikunj>
doesn't run_as_hpx_thread starts running the function when it's called?
<hkaiser>
yes
<nikunj>
is there a way to to just register it?
<hkaiser>
nikunj: that's what I asked 'do you need to wait for the function to run'
<nikunj>
oh I took it differently
<nikunj>
I need to wait until the compiler calls the function to run
<nikunj>
I only want to register it so that it runs on HPX thread when compiler calls it
<hkaiser>
nikunj: please elaborate, I don't understand what you said
<nikunj>
ok, so I was thinking to use the implementation of init_globally and putting it in init_seg(lib). It will be last function to be called by CRT but it will called before any global object making it the sweet spot we desire.
<hkaiser>
ok
<nikunj>
The problem here though is that main function still runs on kernel thread
<hkaiser>
sure, so does the constructor of the global object in init_globally
<nikunj>
but we want all HPX functionality to be available directly from main
<hkaiser>
ok
<nikunj>
so we are limited to a few functions only. (using hpx::future will throw an error)
<hkaiser>
but if you just register a hpx thread you have no idea when it actually runs
<hkaiser>
and it will not run before hpx itself is up
<hkaiser>
(obviously)
<hkaiser>
so I still don't see what you want
<nikunj>
I will use the init_seg to initialize the hpx runtime system the way it is done in init_globally
<hkaiser>
ok
<nikunj>
so hpx_main will still be the entry point
<hkaiser>
ok
<hkaiser>
the hpx_main member in the global object?
<nikunj>
and since its implemented using start instead of init there is not wait
<hkaiser>
yes
<nikunj>
hpx_main is member of the struct manage_global_runtime
<hkaiser>
yes
<nikunj>
so now that the runtime system is initialized, main thread is registered with HPX runtime
<hkaiser>
yes
<nikunj>
but it does not run on one
<hkaiser>
right
<nikunj>
please correct me if I'm misunderstanding something here
<nikunj>
ok, so now I was thinking that I could register main in hpx_main
<nikunj>
to register main to run on HPX thread itslelf
<nikunj>
but using run_as_hpx_thread will fire up the main function
<hkaiser>
or just call it from hpx_main
<nikunj>
which I do not want to do
<nikunj>
Because it would mean that main will be called twice now
<nikunj>
once by me, and then by compiler
<hkaiser>
k, how would registering an hpx thread help?
<nikunj>
it will lead to undefined behaviors
<nikunj>
I want to register main on HPX thread. This way when main is called it runs on one. Any subsequent calls from main will also run on HPX threads now
<nikunj>
I do not think such functionality occurs as thread creation requires a function as well
<nikunj>
so I wanted to know if such functionality is present in HPX
<hkaiser>
brb, sorry
<nikunj>
ok
<hkaiser>
nikunj: you said that main will be called anyways, so how do you want to run main as an hpx thread without main being run twice?
<nikunj>
there were 2 ways that I thought of. First was to run it after all global objects, I'm still investigating that. Second, run it before all global objects and then register main on HPX thread in such a way that when main called by compiler runs on HPX thread (an impossible case, but I thought I should confirm it with you)
<nikunj>
what I mean in 2nd case is that we only register the function and not call it. Any call to the function from a thread registered with hpx runtime will run on HPX thread
<nikunj>
so compiler's call to main will also be called as HPX thread. Again, all this seems impossible but I had to make sure
jakub_golinowski has quit [Ping timeout: 256 seconds]
jakub_golinowski has quit [Ping timeout: 256 seconds]
nikunj has joined #ste||ar
<hkaiser>
nikunj: I don't think this is possible
<hkaiser>
and yes, thanks - I'll have a look at the PR
<nikunj>
hkaiser: that's what I thought as well. Ok, I'll look into the prospects of my first point
<hkaiser>
there might be a way
<hkaiser>
the difference of an hpx thread is that it has a non-zero thread local variable that points to a thread_self data structure
<hkaiser>
w might be able to convert any thread into an hpx thread if we can create an object that exposes the same interface as the thread-self type emulating its behavior