hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoD: https://developers.google.com/season-of-docs/
hkaiser has quit [Quit: bye]
<jbjnr__> parsa: no. that file was committed by mistake, it's the good version of some changes I was making that I could fall back onto if I needed to unroll them. It should be deleted
rori has joined #ste||ar
heller has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
heller has joined #ste||ar
<Yorlik> I'm probably missing something very basic here: Why does this not compile? https://gist.github.com/McKillroy/2abd36f947f8bc60e642483b0b6b9bd7
<jbjnr__> Yorlik: do you need to supply a (default) constructor with arg "hpx::id_type const& id"?
<jbjnr__> (I just looked atthe local_new test to see what was different)
<Yorlik> I didn't supply any constructor - would I have to?
<jbjnr__> try adding one, just in case
<Yorlik> OK - thanks !
<jbjnr__> hmmm. actually, there's a test without one
<Yorlik> I wonder if my cmake is false
<Yorlik> I updated the gist with the cmake I use - is something wrong with that ?
<Yorlik> I used add_executablke together with hpx_setup_target
<jbjnr__> component_base doesn't have a constructor that takes an id type
<jbjnr__> I added this
<jbjnr__> test(hpx::id_type const& id) : hpx::components::component_base<test>() {}
<jbjnr__> and it comppiles
<jbjnr__> but it probably isn't what you want
<zao> Is that the base you're supposed to use?
<jbjnr__> aha
<jbjnr__> local new doesn't need find_here
<Yorlik> It failed again for me
<jbjnr__> because it's local - and here is already here
<Yorlik> OK - lemme remove that
<jbjnr__> so drop the constructor I suggested and drop the find_here
<Yorlik> Still doesn't build: error LNK2019: unresolved external symbol "public: static int __cdecl hpx::traits::component_type_database<struct test,void>::get(void)" (?get@?$component_type_database@Utest@@X@traits@hpx@@SAHXZ) referenced in function "int __cdecl hpx::components::get_component_type<struct test>(void)" (??$get_component_type@Utest@@@components@hpx@@YAHXZ)
<Yorlik> Do you think my cmake is OK?
<jbjnr__> ok, that's link error, you need to make sure you use link to the component stufff
<Yorlik> What would I have to add ?
<Yorlik> in hpx_setup_target ?
<jbjnr__> typedef hpx::components::simple_component<test> server_type;
<jbjnr__> HPX_REGISTER_COMPONENT(server_type, test);
<jbjnr__> to fix the link errors
<jbjnr__> not sure if I used the right macros, but it fixed them for me
<Yorlik> Works!
<Yorlik> Now I only need to understand why
<Yorlik> :)
<jbjnr__> not sure if simple_component is correct for component_base
<Yorlik> Seriously - we need a good doc / tutorial for that.
<jbjnr__> it workls because you are declaring a templated component type, but you need to instantiate the members to make the clinker happy
<jbjnr__> ^linker
<jbjnr__> and the REGISTER... macro does that
<Yorlik> So - just using it from the local_new is not enough to instantiate it?
<jbjnr__> you might want typedef hpx::components::component instead of typedef hpx::components::simple_component
<jbjnr__> the local new invokes the construvctor, but the code wasn't linked in. The register macro trigger that
<Yorlik> why not just component base - as i used in the templates crtp?
<jbjnr__> I'm not sure, I never use these components
<jbjnr__> I looked at one of the examples to see what was used there
<Yorlik> I need documentation for all of that - this otherwise is just black magic.
<jbjnr__> ^correct
<jbjnr__> it's witchcraft
<jbjnr__> only a handful of us have any idea what's going on and if you're not a c++ black belt, it's s nightmare
<Yorlik> I really can't justify all the work just to get these basics done.
<jbjnr__> I just looked at your github profile. nice avatar - is your game on there anywhere?
<Yorlik> Seriously - hitting this type of walls at a beginner level is making me thing to just ditch HPC and rtoll my own task and parcel system.
<Yorlik> The GitHub is horribly outdated
<jbjnr__> "just to get these basics done" Once you've learned it once you can move on ...
<Yorlik> Our main repo is not open and on GitLab
<jbjnr__> aha
<Yorlik> We will spin off parts over time as open source as we see fit.
<Yorlik> This type of problems is really a major roadblock in using HPX - as promising and great it looks from its features.
<jbjnr__> agrred
<Yorlik> And the worst thing is - it's next to impüossible within reasonable time to figure it out for yourself.
<jbjnr__> feel free to contribute a blog article or docs that others can learn from when you solve annoying problems like this
<Yorlik> I have no issues to dig into source code and get my arms dirty.
<Yorlik> I have made smallish writeups for myself and also created some gists from them, but how would I write up anything meaning without more fundamental understanding?
<Yorlik> s/maeaning/meaningful/g
<jbjnr__> write up what you've got, then send it to one of us, or file an issue with 'improve the docs' and an article attached
<jbjnr__> then we can maybe tidy it up and get it in
<Yorlik> Stuff like this: https://gitlab.com/snippets/1821389
<jbjnr__> lol
<Yorlik> Ditch the last one -
<Yorlik> I don't really feel qualified to prodice something that would be worth a PR
hkaiser has joined #ste||ar
<jbjnr__> maybe not a PR, but create an issus with all those docs linked and maybe someone will turn it into a page in the docs
<hkaiser> rori: yt?
<jbjnr__> I can't find the hpx docs anywhere
<jbjnr__> why are they not linked from the wiki or the github main page
<jbjnr__> oops
<jbjnr__> found it
<jbjnr__> Yorlik: you're stuff would fit nicely into a compoentns section here https://stellar-group.github.io/hpx/docs/sphinx/latest/html/index.html
<Yorlik> That's the one I'm using.
<Yorlik> I'd be totally open to work with one of you guys for documentation - i just can't do it completely regularly, since my work with HPC stuff in the moment is very much on and off.
<Yorlik> HPX
<jbjnr__> file an issue anyway and maybe one of us will find time to add it - we have a summer of docs project running, maybe someone on that could chip in
<Yorlik> The point is not some recipes - we need a comprehensible architecture writeup together with the how to do it stuff.
<Yorlik> There are some starting points in the docs ofc, but that's not sufficient.
<Yorlik> We need the HOW? together with the WHY?
<jbjnr__> that's what someone like me or one of the others can add to your material
<Yorlik> The problem is, I cannot magically create material out of nothing - it only comes out of someone teaching me this stuff. So - every little bit I can produce comes from hogging someone elses time.
<Yorlik> These three links came from Hartmut explaining stuff here.
<Yorlik> To have something like this work someone has to bleed under my asking stupid questions .
<jbjnr__> Since I'm doing an HPX course in october, maybe I can work through your stuff and put together a proper tutorial for then.
<Yorlik> So - who's ready to suffer that? :)
<Yorlik> So - get down from the ivory tower and visit the kindergarden ! :D
<jbjnr__> we are missing a proper "distributed hpx" section from our course, so it's a thing worth doing
<hkaiser> Yorlik: we've got a GSoD student accepted, she will work on these things
<Yorlik> I have high hopes tbh
<jbjnr__> File that issue today with all those links in and start the ball rolling
<hkaiser> Yorlik: a good source of information are the tests, btw
<hkaiser> those are usually simple use cases
<Yorlik> OK.
<hkaiser> examples are simple enough mostly as well
<Yorlik> Concerning tests: Is it possible to use catch2 with hpx for testing?
<hkaiser> sure
<hkaiser> no reason why not
eschnett has quit [Quit: eschnett]
<hkaiser> might require a little bit of extra initialization work, however
<Yorlik> OK - I'll try to figure it out - because this other issue kinda spoiled it for me - but probably that is not catch2 related in any way.
<hkaiser> what other issue?
<Yorlik> It was olved with some typedef and the registering macro
<Yorlik> You don't need to solve it
<Yorlik> jbknr__: ^^
<Yorlik> dang: jbjnr__:^^
<jbjnr__> <applause>
<Yorlik> I could do more, but I'd need someone to suffer changing my newbie diapers from time to time (= answering horrible questions and dealing with the Jedi Mind Trick madness coming out of that) :P
<hkaiser> Yorlik: sure, I'm here
<Yorlik> hkaiser: Could you imagine to do voice talks about focused topics and I'd try to make a writeup / .rst file out of it?
<hkaiser> sure
<Yorlik> Allright - when?
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
<Yorlik> hkaiser: ?
<hkaiser> Yorlik: I'll be out until Aug 20
<hkaiser> after that - any time
<Yorlik> OK - We should talk then - I'll get back to you :)
<jbjnr__> hkaiser: vacation?
<hkaiser> nod
eschnett has joined #ste||ar
<rori> hkaiser: yep !
<hkaiser> rori: see #4015, pls
<rori> yep
<rori> hkaiser: Sorry I wanted to put the files in /hpx/artifacts/ directory but I mixed up the typo your change works for me though
<hkaiser> rori: ok, thanks
hkaiser has quit [Quit: bye]
aserio has joined #ste||ar
hkaiser has joined #ste||ar
<heller> Yorlik: last time I checked, catch2 was not thread safe
<Yorlik> Ouch! - Gotta keep that in mind.
<Yorlik> Thanks for the heads-up!
aserio1 has joined #ste||ar
aserio has quit [Ping timeout: 276 seconds]
aserio1 is now known as aserio
aserio has quit [Read error: Connection reset by peer]
aserio has joined #ste||ar
aserio has quit [Ping timeout: 246 seconds]
aserio has joined #ste||ar
rori has quit [Quit: WeeChat 1.9.1]
nikunj has quit [Remote host closed the connection]
aserio has quit [Ping timeout: 250 seconds]
aserio has joined #ste||ar
hkaiser has quit [Quit: bye]
nikunj has joined #ste||ar
eschnett has quit [Quit: eschnett]
bobakk3r has joined #ste||ar
<bobakk3r> Hi.
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
<bobakk3r> Say I have a lot of data I need to copy into GPU memory when I use CUDA, OpenCL, SYCL, etc. in combination with HPX.
nikunj has quit [Remote host closed the connection]
<bobakk3r> How do I avoid having to copy everything back in every time I spin off an action?
nikunj has joined #ste||ar
<bobakk3r> Does HPX have any features that let you keep your buffers around until the locality gets another call?
<aserio> bobakk3r: I believe so
<aserio> bobakk3r: I have not worked with GPUs but my understanding is that once you move the data to the GPU you can execute kernels on the data there
<bobakk3r> As far as I understand it, components can get shuffled around, right?
<aserio> You can move components yes
<aserio> we call it migration
<aserio> though our current GPU strategy does not involve using components.... I think
<aserio> daissgr: How are we doing this in OctoTiger?
eschnett has joined #ste||ar
eschnett has quit [Quit: eschnett]
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
eschnett has joined #ste||ar
hkaiser has joined #ste||ar
eschnett has quit [Quit: eschnett]
hkaiser has quit [Read error: Connection reset by peer]
hkaiser has joined #ste||ar
<bobakk3r> Any other suggestions? I'm still not sure how I'm supposed to deal with this.
<bobakk3r> I assume I can ensure a component does not get migrated, right?
<aserio> By default components are not migrated
<aserio> HPX doesn't do anything you aren't telling it to do
<hkaiser> bobakk3r: managing GPU memory is completely on you, HPX does not do anything with it
<aserio> bobakk3r: here is a thesis which explains some of this http://stellar.cct.lsu.edu/pubs/daiss_masters_thesis_05.0.18.pdf
<aserio> it looks like what you are interested in is on page 42
<aserio> good luck!
aserio has quit [Quit: aserio]
<bobakk3r> aserio: that looks like what I was looking for. Thanks!
<bobakk3r> hkaiser: right. I just wasn't sure how to manage it elegantly.
<diehlpk_work> hkaiser, see pm and I am confused
<bobakk3r> The problem I'm working on is very well-suited to the "tree-of-futures" style of program, if it weren't for the fact that the naive implementation would have to load ~5 GB into GPU memory every few milliseconds.
<bobakk3r> But I'm pretty sure I know how to make it work now.
<hkaiser> bobakk3r: even if you reference that memory from a component somehow (which lives in CPU memory) this shouldn't be a problem as components are only moved if you trigger it yourself
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
nikunj has quit [Remote host closed the connection]
nikunj has joined #ste||ar
eschnett has joined #ste||ar