hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoD: https://developers.google.com/season-of-docs/
Yorlik has quit [Disconnected by services]
Yorlik has joined #ste||ar
hkaiser has joined #ste||ar
hkaiser has quit [Quit: bye]
nikunj has quit [Remote host closed the connection]
hkaiser has joined #ste||ar
jgurhem has joined #ste||ar
hkaiser has quit [Quit: bye]
daissgr has joined #ste||ar
hkaiser has joined #ste||ar
daissgr has quit [Ping timeout: 252 seconds]
<hkaiser> simbergm: yt?
<simbergm> hkaiser: here
<hkaiser> hey
<hkaiser> wrt the util module
<hkaiser> what's your rationale not to split it now?
<simbergm> I guess laziness mostly
<simbergm> let me have a look at the files again
<hkaiser> heh
<hkaiser> we're in no rush here, are we?
<simbergm> no, definitely not
<Yorlik> hkaiser: How precise is sleep_for / sleep_until? I am constantly getting shorter reentry times than I passed to the sleep_xyz (around 8 ms faster when 100 was targeted)
<hkaiser> Yorlik: I wouldn't know why sleep should ever return before the time has run out
<Yorlik> This code:
<Yorlik> hpx::this_thread::sleep_until( last_frame_time_point + ( 1000000000ns / framerate ) ); // artificially slow down loop
<Yorlik> // capture time
<Yorlik> new_ft = std::chrono::nanoseconds( std::chrono::high_resolution_clock::now( ) - last_frame_time_point );
<Yorlik> last_frame_time_point = std::chrono::high_resolution_clock::now( );
<hkaiser> we use a system clock underneath
<Yorlik> With a framerate of 10 I am getting 92 ms framerate
<Yorlik> stddev over 10 frames is 3ms
<hkaiser> Yorlik: no idea
<Yorlik> Strange
<hkaiser> simbergm: if I may suggest, let's do it right now - we won't ever go back if we don't
<hkaiser> simbergm: otoh, this is what I was mentioning in the very beginning, what's the right granularity - I don't have an answer
<simbergm> you're right, later is never
<simbergm> but do you have a good suggestion for a grouping?
<simbergm> sitting here with auriane trying to figure out how to group them
<hkaiser> simbergm: would the categories I mentioned be a starting point?
<simbergm> yes, they would
<simbergm> what do you count as data structures for example?
<simbergm> tuple and optional?
<simbergm> or more?
<hkaiser> variant, and any might be added
<hkaiser> at some point
<hkaiser> we do have any, right?
<hkaiser> what about range?
<simbergm> right, yes, they would need to come in later once they've been un-serialized
<simbergm> hashing? fibhash and jenkins hash?
<hkaiser> yes
<hkaiser> tagged_tuple and related types
<hkaiser> i.e. tagged_pair and tagged
<simbergm> also later
<simbergm> what would you do with something like always_void? unused?
<hkaiser> you were asking what would be data structures
<hkaiser> type-trait support
<simbergm> yeah, sorry, I moved on
<hkaiser> same as lazy_enable_if and similar
<simbergm> ok, we'll get something together
<hkaiser> binders/callables would be another one
<simbergm> I just don't want to have to come up with names for these ;)
<hkaiser> lol
<simbergm> sorry for being lazy
<simbergm> utils is such a good name
<simbergm> we'll do it after lunch :)
<simbergm> and thanks for calling me out
<hkaiser> any time - I'm here to serve ;-)
<simbergm> :P
hkaiser has quit [Quit: bye]
diehlpk_mobile has joined #ste||ar
rori has joined #ste||ar
diehlpk_mobile has quit [Quit: Yaaic - Yet another Android IRC client - http://www.yaaic.org]
diehlpk_mobile has joined #ste||ar
<simbergm> heller: yt?
hkaiser has joined #ste||ar
eschnett has quit [Quit: eschnett]
hkaiser has quit [Quit: bye]
hkaiser has joined #ste||ar
<jgurhem> Hi ! I looked into the 1d_stencil example. I am trying to adapt the seventh example to my case. I am using the partition_data as a tiles for my matrix but I'm getting compile error I don't understand. Could you help me ?
<heller> simbergm: what's up?
<heller> jgurhem: code and/or the actual compiler error will help ;)
<jgurhem> So here is the part which bothers me :
<jgurhem> static partition inv_part(partition const& A_p, partition const& inv_p) { using hpx::dataflow; using hpx::util::unwrapping; hpx::shared_future<partition_data> A_data = A_p.get_data(); return dataflow(hpx::launch::async, unwrapping([inv_p](partition_data const& A) -> partition { partition_
<jgurhem> data& m_inv = stepper::inv_core(A); return partition(inv_p.get_id(), m_inv); }), A_data); }
<jgurhem> Oh that was not a good idea
<simbergm> jgurhem: a gist is a good idea ;)
<heller> simbergm: good question :(
<jgurhem> When, I am compiling, it tells me that A is not captured by the lambda function and it can't call inv_core without object
<simbergm> how much did you adapt it? your whole source file might help
<simbergm> heller: :|
<simbergm> mind if I change it? or do you think there was a good reason for it that I shouldn't mess with before we figure out why?
hkaiser has quit [Ping timeout: 245 seconds]
hkaiser has joined #ste||ar
<simbergm> ah, you commented
<simbergm> I'll leave it as it is for now
<simbergm> jgurhem: inv_core needs to be static for you to be allowed to call it like that without an instance
<simbergm> heller: just in case you missed it the tuple pr needs some work still :/
<heller> Yeah
diehlpk_mobile has quit [Ping timeout: 276 seconds]
diehlpk has joined #ste||ar
hkaiser has quit [Quit: bye]
K-ballo1 has joined #ste||ar
K-ballo has quit [Ping timeout: 248 seconds]
K-ballo1 is now known as K-ballo
<jgurhem> simbergm Thank you but the compiler is still complaining. I seem to acces my partition_data the wrong way. I don't know how to do that properly. Could you give me some pointers ?
<simbergm> jgurhem: can you post the error message?
<jgurhem> Yeah sure
diehlpk has quit [Ping timeout: 248 seconds]
<simbergm> jgurhem: too many/few consts?
<simbergm> e.g. inv_part I'm sure actually modifies its arguments, but they're const
<jgurhem> simbergmThat's what I am thinking too. Is it ok to remove the const operator in this case ?
aserio has joined #ste||ar
<simbergm> not sure to be honest :/
<simbergm> I mean, removing const is always a viable option, but they were there most likely for a reason
<jgurhem> I was thinking the same. I will not remove them for now.
<jgurhem> I tried to fix the access problems I had in my core functions but now, I have very long errors related to the inner working of HPX.
<jgurhem> Like this one : https://pastebin.com/FHhHa6iZ
hkaiser has joined #ste||ar
<jgurhem> Actually, it seems that I did something wrong with my dataflows : https://pastebin.com/Rh8e67zu
<simbergm> jgurhem: not sure if that's what you're getting but passing references to async/dataflow should be wrapped in std::ref
<hkaiser> simbergm: should they?
<simbergm> hkaiser: should they not?
<simbergm> :/
<hkaiser> I'd say it depends, you could move them as well, or just let dataflow copy them
<simbergm> you're the reference here, tell me if I'm spreading lies
<simbergm> right
<hkaiser> I also think we have deprecated unwrapping
<hkaiser> it's 'unwrapped' nowadays
<hkaiser> no, I take that back
<hkaiser> it's unwrapped that's deprecated after all (I'm confused myself)
<jgurhem> simbergm Unfortunately my knowledge in C++ is quite poor so I don't how how to use std::ref. I'm going to take a look.
<simbergm> jgurhem: it's quite simple, you wrap arguments with std::ref ;)
<hkaiser> jgurhem: std::ref is a facility that forces your arguments to be passed around as references, otherwise dataflow may decide to create a copy in order to be able defer the execution of the function
<simbergm> hkaiser: what I meant was that if the function passed to async/dataflow takes an argument by reference, then that argument should be wrapped with std::ref, but that doesn't mean taking arguments by reference is the right thing to do
<simbergm> does that sound accurate?
<hkaiser> if your function takes arguments by reference those can still be moved/copied when invocing dataflow
<hkaiser> dataflow has to create a bound invocable (function plus arguments), if you use std::ref that invocable will hold std::reference_wrapper
<hkaiser> if you don't use std::ref, the invocable will hold the data directly either moved from user land or copied
<hkaiser> same for async
<hkaiser> in all cases the actual function can receive the args either by value, by const& or by &&
<hkaiser> dataflow/async know that the function is invoked once, so they attempt to move the args from the bound invocable to the called function
<hkaiser> the actual behavior is then defined by the actual signature of the bound function
<hkaiser> also, if you use std::ref, the function will reference the original data in userland, which also means the user has to make sure it outlives the function invocation
<hkaiser> does that make sense?
jgurhem has quit [Remote host closed the connection]
<simbergm> he exploded
<hkaiser> lol
jgurhem has joined #ste||ar
<jgurhem> Sorry, I got disconnected ! It makes sense. So I tried to put std::ref around *_data in the call to dataflow in https://github.com/jgurhem/HPX_LA/blob/dist_tmp/lu_tiled_dist.cpp#L322-L369 but the compiler still shows errors
<hkaiser> jgurhem: your function you pass to dataflow expects two arguments, but you only pass one to dataflow itself
<hkaiser> jgurhem: this is not a question using std::ref or now
<jgurhem> hkaiser Yes, you're right but I put this argument since then and I still get the error.
<hkaiser> the second argument to your function is a non-const reference, that might cause the issue
<hkaiser> make it const& instead
<jgurhem> In the lambda ?
<hkaiser> yes
bita has joined #ste||ar
<jgurhem> So now I have those errors : https://pastebin.com/BCdru3WL. They are the one I got at the start but if I put const in all the parameters of my *_core functions, I will try to modify const variables
<hkaiser> so yes, not a dataflow problem anymore
<jgurhem> Ok so my question becomes : how do I make my computations properly in my dataflow lambda ?
<jgurhem> Should I just put the body of my functions in the lambda ?
<hkaiser> shrug, not sure what's you intent
<hkaiser> the function your dataflow invokes receives some data and produces some
<hkaiser> whatever it receives are usually the arguments and whatever it produces is normally returned
<jgurhem> Oh I was trying modify some data and return the modified version.
<hkaiser> you can still do that
<hkaiser> reuse memory of some of the arguments
<hkaiser> do you need the data the argument refers to after modification (besides the returned value)?
<jgurhem> Yeah that would be better but I'm stuck with those const
<hkaiser> you might just be able to move the args, modify them and return by moving the modified data out
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
<jgurhem> Seems promising be more difficult than creating a new partition_data and returning it at the end of the function (and eventually copying data that will be modified but which modifications do not have to be transmitted)
hkaiser has quit [Quit: bye]
Amy2 has quit [Ping timeout: 252 seconds]
Amy2 has joined #ste||ar
<jgurhem> simbergm It finally compiled ! But I got this error at runtime : lu_tiled_dist: /gpfshome/mds/staff/jgurhem/install/boost/1.69.0/release/include/boost/smart_ptr/intrusive_ptr.hpp:193: T& boost::intrusive_ptr<T>::operator*() const [with T = hpx::naming::detail::id_type_impl]: Assertion `px != 0' failed. Do you have any idea on what can cause that ?
eschnett has joined #ste||ar
<simbergm> jgurhem: don't know but probably some sort of reference counting has been subverted :/
<jgurhem> simbergm That's too abstract for me. That'll take me a while to find the cause and fix it
<simbergm> jgurhem: yeah, sorry, I can't give you a better answer
<simbergm> but a call stack might help
<jgurhem> simbergm How can I get one ? There was nothing more. It was an assert I guess
<simbergm> gdb is your best bet, it'll most likely stop at the assert
<jgurhem> ouch
<simbergm> jgurhem by gdb I mean any debugger that you're used to using
<jgurhem> simbergm Unfortunately, I am not used to any of them. I should start using one of them
<simbergm> I can only recommend it, you'll need one sooner or later
<jgurhem> Yeah, that's why I'm trying to use valgrind and gdb when I have to
<jgurhem> simbergm how do I use more localities ?
<simbergm> jgurhem depends on the system
<simbergm> With slurm you just allocate more nodes
<jgurhem> simbergm (not using slurm)
<simbergm> With your old funky system try - - hpx:localities and - -hpx:node
<simbergm> hkaiser what else does one need to connect multiple localities manually?
<simbergm> But you should probably try to fix your problem first with one locality...
<jgurhem> You gave me the hint !
<jgurhem> I think I know what's the problem
<jgurhem> simbergm It ran until the end !
<jgurhem> My first distributed HPX program !
<simbergm> jgurhem nice!
jgurhem has quit [Ping timeout: 260 seconds]
aserio has quit [Ping timeout: 264 seconds]
jgurhem has joined #ste||ar
hkaiser has joined #ste||ar
jgurhem has quit [Ping timeout: 260 seconds]
aserio has joined #ste||ar
eschnett has quit [Quit: eschnett]
eschnett has joined #ste||ar
rori has quit [Quit: bye]
hkaiser has quit [Quit: bye]
eschnett has quit [Quit: eschnett]
aserio has quit [Quit: aserio]
eschnett has joined #ste||ar
Yorlik has quit [Ping timeout: 272 seconds]
<nikunj97> can anyone explain me the exact functions of a sliding semaphore. And what the parameters to wait and signal signify?
Yorlik has joined #ste||ar
<parsa> look under ##Sliding Semaphore
<nikunj97> parsa: thanks!
Yorlik has quit [Ping timeout: 245 seconds]
mdiers_1 has joined #ste||ar
mdiers_ has quit [Ping timeout: 248 seconds]
mdiers_1 is now known as mdiers_
Yorlik has joined #ste||ar