hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoC2018: https://wp.me/p4pxJf-k1
K-ballo has quit [Quit: K-ballo]
K-ballo has joined #ste||ar
kisaacs has joined #ste||ar
K-ballo has quit [Ping timeout: 276 seconds]
verganz has quit [Quit: Leaving.]
K-ballo has joined #ste||ar
kisaacs has quit [Ping timeout: 276 seconds]
kisaacs has joined #ste||ar
hkaiser has quit [Quit: bye]
kisaacs has quit [Ping timeout: 240 seconds]
eschnett has joined #ste||ar
kisaacs has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
diehlpk has joined #ste||ar
K-ballo has joined #ste||ar
kisaacs has quit [Ping timeout: 264 seconds]
diehlpk has quit [Ping timeout: 240 seconds]
kisaacs has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
kisaacs has quit [Ping timeout: 268 seconds]
kisaacs has joined #ste||ar
kisaacs has quit [Ping timeout: 260 seconds]
kisaacs has joined #ste||ar
parsa has joined #ste||ar
parsa[w] has quit [Read error: Connection reset by peer]
<7GHAATXF3>
hpx/master a13649e Mikael Simberg: Disable background thread when networking is off...
<7GHAATXF3>
hpx/master 4fb41fb Mikael Simberg: Force one locality in tests when HPX_WITH_NETWORKING=OFF and ignore relevant options in hpxrun.py...
<7GHAATXF3>
hpx/master 9a62c15 Mikael Simberg: Change two tests to work with networking off
<jbjnr>
hkaiser: I booked a room at the noah hotel. I'll definitely be wanting rides, thanks. I see that the train/bus is only about 15-20 mins though, so hopefully all is well.
<hkaiser>
jbjnr: sure, np
aserio has joined #ste||ar
eschnett has quit [Quit: eschnett]
mbremer has joined #ste||ar
hkaiser has quit [Ping timeout: 256 seconds]
eschnett has joined #ste||ar
kisaacs has joined #ste||ar
nanashi64 has joined #ste||ar
nanashi55 has quit [Ping timeout: 240 seconds]
nanashi64 is now known as nanashi55
hkaiser has joined #ste||ar
<mbremer>
@hkaiser: please let me know when you have time today, I'm relatively flexible. I'll try to log off irc if I can't make it. So we can also meet on short notice
<hkaiser>
mbremer: ok, thanks
stmatengss has joined #ste||ar
aserio1 has joined #ste||ar
<nikunj97>
hkaiser: did you tried running my experimental code?
aserio has quit [Ping timeout: 276 seconds]
aserio1 has quit [Ping timeout: 276 seconds]
diehlpk has quit [Ping timeout: 255 seconds]
aserio has joined #ste||ar
<hkaiser>
nikunj97: I have not - sorry :/
<hkaiser>
too many balls in the air - *sigh*
<nikunj97>
hkaiser: ok, I'd need help to move forward for linux based to improve upon that experiment. In the mean time, I'll learn more about msvc and how hooking actually works for windows
<hkaiser>
thanks
<nikunj97>
hkaiser: just an update of my gsoc project, I have coded the for linux based (it needs refinement though)
<nikunj97>
also for unix based since my solution is a linux/unix based one
<nikunj97>
I will read more about msvc in general now, how I can hook for windows based systems
<hkaiser>
nikunj97: we'll ask zao if your solution works on FreeBSD ;-)
<nikunj97>
hkaiser: in theory it should work, but FreeBSD has it's own ways of working :D
<zao>
A lot of the things that work on Linux tend to be due to glibc or kernel behaviour, and not posix or unix things.
<zao>
This time was RelWithDebInfo, previous one was Debug.
<zao>
Gonna build a Debug too, just to make sure.
<nikunj97>
zao: ya, please do that too
<nikunj97>
zao: I'm glad it is working :)
<nikunj97>
zao: the last gist gave me a heart attack :P
<zao>
Got to keep things exciting.
<nikunj97>
zao: sure :D
<nikunj97>
zao: should i ignore the last gist or should i look into it?
<nikunj97>
zao: coz i'd like to make sure that my solution overall works
<zao>
I'm suspecting that the previous failure was due to problems we had around release.
<zao>
But I'm still building a Debug HPX too.
<zao>
What CMAKE_BUILD_TYPE are you using, whatever is default?
<nikunj97>
zao: ya, debug should help clear things up
<nikunj97>
zao: I haven't checked on debug
Viraj has quit [Ping timeout: 260 seconds]
<nikunj97>
zao: I use release
<diehlpk>
jakub_golinowski, I reviewed your code and opened some issues
<jakub_golinowski>
diehlpk, thank you I got an email :D
<diehlpk>
Perfect. Please use #ref and #closes in your commits
<diehlpk>
You have to put the number of the issue with a space to these to commands
<jakub_golinowski>
so for example:
<jakub_golinowski>
git commit -m "Change all occruances of std::cout to hpx::cout within hpx runtime. #closes 2"
<jakub_golinowski>
??
<zao>
nikunj97: debug build of the current master also passes your evil test.
<nikunj97>
zao: that's really good to hear :)
<nikunj97>
zao: I'll ask hkaiser to help me refine it overall so that I could open a pull request for working code with linux/unix based system with gcc and clang
bibek has joined #ste||ar
<nikunj97>
zao: now I'll shutdown this pc and open up my windows boot. I'll try to make things work with windows as well now
<zao>
For your references, this is Debian 9.2 in a container, with its clang 3.8.1, HPX built with C++14 and Boost 1.65.1
<nikunj97>
zao: thanks for the info :)
nikunj97 has quit [Quit: Leaving]
kisaacs has quit [Ping timeout: 255 seconds]
nikunj has joined #ste||ar
<nikunj>
I'm back, running windows this time
<nikunj>
I want to code in visual studio for my windows section, which version of msvc should I prefer to code on?
<zao>
2015 will probably give you the least number of surprises, while 2017's regular releases may break HPX occasionally, particularly now with the new 15.7 release.
<zao>
If you're dumb like me and run 2017 Preview, well, then you have no-one else to blame but yourself :P
<nikunj>
I'll use 2015 in that case ;)
<zao>
HPX looks to support "the last two releases" or something like that.
<zao>
(might have the range wrong)
<jakub_golinowski>
diehlpk, did you see my question above?
<diehlpk>
jakub_golinowski, Was away. But yes, it looks good
<jakub_golinowski>
also I have the following use case
<diehlpk>
Now #2
<diehlpk>
*No
<diehlpk>
You have to use # and after this the number of the issue, sorry
<K-ballo>
"the last two releases" was a thing when it'd include 2017 and 2015... we weren't ready for the new release model
<zao>
"closes #9001" etc.
aserio1 has joined #ste||ar
<zao>
K-ballo: Indeed, we live in wondrous times.
<zao>
MS technically has a long-term-service version of MSVC 2017, but heaven knows how you get your hands on it.
<zao>
It's essentially a moving target now, and you can't reliably install any older non-LTSB version.
<diehlpk>
jakub_golinowski, git commit -m "Change all occruances of std::cout to hpx::cout within hpx runtime. closes #2" would be correct
<zao>
Not sure if the standalone build tools they distribute are versioned or "latest" either.
<diehlpk>
It will close the issue #2 for you with the commit
<diehlpk>
#refs #2 would show this commit in the comments of the issue
<diehlpk>
*refs #2
<jakub_golinowski>
diehlpk, I have refactored the code a bit and already fixed the std::cout incositency earlier, So can I now just bup the cmake required version and close two issues AND update code a bit?
aserio has quit [Ping timeout: 260 seconds]
aserio1 is now known as aserio
<diehlpk>
Yes
<diehlpk>
you can also use multiple closes
<jakub_golinowski>
zao, thank you very much for the link
<jakub_golinowski>
there is also example of multiple closes there.
<diehlpk>
You could do it like this
<diehlpk>
jakub_golinowski, I have to go
mbremer has joined #ste||ar
<jakub_golinowski>
diehlpk, OK I will finish my changes to the code and the commit the changes with closign issues
<diehlpk>
I will get an e-mail
<jakub_golinowski>
great
aserio has quit [Read error: Connection reset by peer]
aserio has joined #ste||ar
<jakub_golinowski>
hey, If I am scheduling a future a function that is manipulating the data passed by reference and there is no reterun value what is the best way to wait for the result?
<jakub_golinowski>
for now I just return boolean value and use future.get();
<simbergm_>
hey jakub_golinowski: async(void_returning_function) returns a future<void> and you can .wait on the future
<jakub_golinowski>
simbergm_, thank you
<jakub_golinowski>
but I think the problem is elswhere as well :/
<jakub_golinowski>
so I suspect that the problem is that I pass a future.get() as an argument to hpx::async(executor, &func, ..., future.get())
<jakub_golinowski>
when I run get() on this new future I get the error:
<K-ballo>
that makes the whole thread wait for `future` to be ready, is that what you want?
<jakub_golinowski>
"this future has no valid shared state"
<simbergm_>
that means you've already consumed the future
<simbergm_>
it's been moved somewhere else
<simbergm_>
and if you want to do async with a future like that you might want to look at dataflow
<simbergm_>
dataflow is a bit like async but can take futures as arguments and it will do the right thing for you
<jakub_golinowski>
oh I remember now, I was using dataflow for the matrix multiplication
<simbergm_>
no documentation though, unit tests are your friends :)
<jakub_golinowski>
so this is my code for reference:
<jakub_golinowski>
so for this use-case you suggest using dataflows?
jaafar has joined #ste||ar
kisaacs has joined #ste||ar
<jakub_golinowski>
K-ballo, so actually this is what I want (I think) - the function itself is a blocking call (to load the image) an it will be executed on a different thread pool
<simbergm_>
jakub_golinowski: yep, dataflow is perfect for that use case
<simbergm_>
alternatively a simple future.then(exec, show_image) for example
<simbergm_>
if you only have one argument to pass on to another task
<jakub_golinowski>
but i think it will not work
<jakub_golinowski>
this is how I wanted to start
<jakub_golinowski>
but from docs I understood that future.then adds the continuation without waiting for the resutl
<jakub_golinowski>
but show_image is dependent on the result of the future
<jakub_golinowski>
it needs the result of the future
<K-ballo>
the basic building block would be `future.then([](future f) { func(..., f.get()); })`
<K-ballo>
[some weak choices for naming there]
<simbergm_>
right, .then will return before the result is done, but the task with show_image will only be called once the future is ready
<aserio>
jakub_golinowski: you can think of a dataflow as a wait_all.then(...)
<aserio>
which is what simbergm_ just described
<jakub_golinowski>
OK, in this case future.then() is quite handy
<simbergm_>
there are more than way of writing equivalent code, which can be a bit frustrating in the beginning
<jakub_golinowski>
and just to clarify one thing that was said here
<jakub_golinowski>
K-ballo, said that a call like this will block hpx::async(opencv_executor, &show_image, f_image.get()); the entire *thread*
eschnett has joined #ste||ar
<simbergm_>
thread = HPX thread in this context
<jakub_golinowski>
ok, and in consequence the OS-thread that is running it
<simbergm_>
meaning if the future is not ready the HPX thread will suspend, and if there is other work to be done the scheduler will schedule that instead
<K-ballo>
no, just the HPX thread
<jakub_golinowski>
so it does not case the thing that we were discussing before that a blocking HPX thread blocks the entire OS-thread
<jakub_golinowski>
as in the case of system call
<K-ballo>
right, because HPX knows about hpx::future
<jakub_golinowski>
because if this is the case - why cannot we just wrap the system calls in the futures instead of using a separate thread pool?
<simbergm_>
because wrapping them in async/futures will just create another HPX thread which will also be blocked
<simbergm_>
(potentially block)
<simbergm_>
does that make sense? not sure if that clarified anything...
<jakub_golinowski>
but is it a problem since you said that: if the future is not ready the HPX thread will suspend, and if there is other work to be done the scheduler will schedule that instead
<jakub_golinowski>
You know what is my doubt?
<K-ballo>
whenever HPX is blocked HPX can choose to do something else instead, whenver the OS is blocked we are all blocked
<simbergm_>
I think you might be confusing a task and a future, a future only represents the result of a task
<jakub_golinowski>
ah damn
<simbergm_>
if the future is not ready HPX will do the right thing, if the task does something that blocks the OS thread things are bad
<jakub_golinowski>
ok - so actually in my example both things happen. Because I imagine that loading the image from file system is an OS blocking call
<simbergm_>
yep, correct, which is why you want to do it on a special pool, like your custom pool or could be the io pool
<jakub_golinowski>
thank you for your patience now I am back on track :)
<simbergm_>
or can be blocking again
<simbergm_>
good :)
kisaacs has quit [Ping timeout: 256 seconds]
<jakub_golinowski>
also since we are talking
<jakub_golinowski>
I have one more Q I put earlier on gdoc
<jakub_golinowski>
in what thread pool I am when I am in hpx_main() body?
<simbergm_>
in the default pool named "default"
<jakub_golinowski>
and this is the pool WorkerThreads - each having a PU under his control and scheduling HPX tasks?
<jakub_golinowski>
* pool OF WorkerThreads
<jakub_golinowski>
?
<simbergm_>
hmm, yes, but so are your custom pools
diehlpk has quit [Ping timeout: 260 seconds]
<jakub_golinowski>
yes, yes, that is right
<jakub_golinowski>
but I was wondering maybe there is yet another thread or sth for the hpx_main() body
<jakub_golinowski>
so hpx_main() is itself an HPX task?
<simbergm_>
nope, nothing special about it except the way it's added to the runtime (since it's added from outside the runtime)
<simbergm_>
so yes, it's an HPX task
<simbergm_>
jakub_golinowski: going to leave now, but feel free to leave more questions in the doc and I'll have a look at your repo tomorrow
simbergm_ has quit [Quit: WeeChat 2.1]
mcopik_ has quit [Ping timeout: 260 seconds]
mcopik has quit [Ping timeout: 264 seconds]
aserio has quit [Ping timeout: 260 seconds]
khuck has joined #ste||ar
<khuck>
hkaiser: did you fix the annotated functions already?
mbremer has quit [Quit: Page closed]
<hkaiser>
khuck: sorry, no
<khuck>
well, I just configured and built in Xcode, and they work now.
<khuck>
so I wonder if it is dependent on the CMAKE_BUILD_TYPE value.
<khuck>
hkaiser: are annotated_functions "disabled" in a Release configuration?
<hkaiser>
khuck: shrug, are they?
<hkaiser>
they depend on HPX_HAVE_THREAD_DESCRIPTION being defined
<khuck>
yes, but APEX forces that
<hkaiser>
which might be off for Release builds :/
<khuck>
I'll keep looking
mbremer has joined #ste||ar
<hkaiser>
hey mbremer
<hkaiser>
mbremer: I have 15 minutes to talk now
<mbremer>
Cool works for me
<mbremer>
let me run somewhere
<mbremer>
I'll ping you when I'm ready
<mbremer>
should be 5 minutes
tianyi has quit [Quit: Leaving]
mbremer has quit [Client Quit]
mbremer has joined #ste||ar
<mbremer>
@hkaiser: I'm ready on skype
<hkaiser>
sec
Vir- has joined #ste||ar
aserio has joined #ste||ar
Vir has quit [Ping timeout: 240 seconds]
mbremer has quit [Quit: Page closed]
aserio has quit [Quit: aserio]
aserio has joined #ste||ar
aserio has quit [Ping timeout: 255 seconds]
galabc has joined #ste||ar
<galabc>
Hi I have a question concerning the Rostam clusters, I am trying to find the path to the library HWLOC/1.11.3. This pages https://github.com/STEllAR-GROUP/hpx/wiki/Running-HPX-on-Rostam helped me find the location of the boost library but there is no information on the path to the HWLOC library
kisaacs has joined #ste||ar
aserio has joined #ste||ar
aserio has quit [Client Quit]
<galabc>
It seems I managed to build hpx on the cluster without specifying the path to hwloc in the cmake command.
<galabc>
Not really sure why it worked but it did :D
hkaiser has quit [Ping timeout: 256 seconds]
jakub_golinowski has quit [Quit: Ex-Chat]
Vir has joined #ste||ar
Vir- has quit [Ping timeout: 248 seconds]
<zao>
galabc: Are you familiar with module systems?
<zao>
Loading a module adjusts the environment in your session to include paths to the software you loaded, including things like include paths, library search paths, executable paths, pretty much anything you could envision.
<zao>
I'm not familiar with the Rostam/LSU setup personally, but you should be able to see the change in the environment prior and after running a `module load`.
<zao>
If it's fancy enough, `module show` might work too.