K-ballo changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
bita has joined #ste||ar
bita has quit [Ping timeout: 258 seconds]
hkaiser has joined #ste||ar
<gnikunj[m]> ms: it builds now. Thanks!
hkaiser has quit [Quit: bye]
diehlpk_work__ has joined #ste||ar
diehlpk_work_ has quit [Ping timeout: 260 seconds]
diehlpk_work_ has joined #ste||ar
diehlpk_work__ has quit [Ping timeout: 264 seconds]
hkaiser has joined #ste||ar
<gnikunj[m]> hkaiser: yt?
<hkaiser> here
<gnikunj[m]> I don't see executor_parameters_type used anywhere in the code
<hkaiser> gnikunj[m]: that tell the executor infrastructure what default parameters object to use
<gnikunj[m]> But I don't see it used anywhere in the executor code
<gnikunj[m]> aah, so in our case the executor takes 2 or 3 parameters. So executor_parameters_type will return the type of those parameters?
<hkaiser> no
<hkaiser> executor parameters are (an obsolete) concept we introduced in order to enable customization of things like chunk sizes
<gnikunj[m]> what will it be then?
diehlpk_work_ has quit [Ping timeout: 240 seconds]
<hkaiser> executor properties
<gnikunj[m]> properties as in?
<gnikunj[m]> I'm sorry if it's silly. I don't remember any of the executor stuff at this point :/
<hkaiser> see wg21.link/p0443
<gnikunj[m]> I'll look into it. Thanks!
<hkaiser> gnikunj[m]: there is also wg21.link/p2183
<gnikunj[m]> ok, thanks!
<hkaiser> gnikunj[m]: the actual proposal is wg21.link/p1393
<gnikunj[m]> I realized that this is coming from boost asio's executors. Is that correct?
<gnikunj[m]> or HPX did it first?
<hkaiser> no
<hkaiser> asio is part of the executor motivation
<hkaiser> hpx didn't have a part in properties, we did implement early versions of the executor proposal, though, and that's what we have today
<gnikunj[m]> got it.
<hkaiser> also, ms[m] (and others) have started to implement senders/receivers in hpx recently, he also implemented one property to control task placement
<gnikunj[m]> hkaiser: why do we have inline constexpr in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1393r0.html? is constexpr not inlined by default?
<gnikunj[m]> Also, what does = unspecified mean exactly?
<gonidelis[m]> hkaiser: yt?
<hkaiser> here
<hkaiser> gnikunj[m]: it means you can do what you want
<gnikunj[m]> oh~
<gonidelis[m]> hkaiser: what's a reason for result of *i not eing projected?
<gnikunj[m]> hkaiser: what about inline constexpr?
<hkaiser> gonidelis[m]: you tell me, it's your code
<gonidelis[m]> hkaiser: is it because the projection can be "encapsulated" into the predicate?
<hkaiser> gonidelis[m]: not sure what you mean
<hkaiser> what I mean was that instead of p(*i) you should have p(proj(*i))
diehlpk_work has joined #ste||ar
<gonidelis[m]> i thought your "should the result of *i be projected?" question was rhetorical
<gonidelis[m]> hkaiser: ^^
<gonidelis[m]> srinivasyadav227: hello :)
<gonidelis[m]> wellcome
<srinivasyadav227> 😀 Hello
<hkaiser> srinivasyadav227: download and build hpx, run some examples, read the docs
<srinivasyadav227> ok thank you, let you know if im stuck somewhere or done with it! :-)
<hkaiser> srinivasyadav227: sure, please ask questions here
bita has joined #ste||ar
<srinivasyadav227> hkaiser: ok
<gonidelis[m]> hkaiser: can i inverse `util:invoke` ? like `!util::invoke()`
<gonidelis[m]> or sth
<hkaiser> gonidelis[m]: invoke just invokes, f(...) is equivalent to invoke(f, ...)
<gonidelis[m]> yeah... that's what i thought
<gonidelis[m]> but what if f returns a bool
<gonidelis[m]> i can surely write !f(...) but not !invoke(f,...)
<hkaiser> the return type is the same
<gonidelis[m]> hkaiser: k
<gonidelis[m]> hkaiser: now what i was saying before is. since we have a predicate, what's the purpose of the projection
<gonidelis[m]> and the algorithm is a modifying one
<gonidelis[m]> oh it just popped.... nevermind
<gonidelis[m]> i got it
teonnik has quit [Quit: Idle for 30+ days]
<gnikunj[m]> hkaiser: As I was saying, hpx::kokkos::kok expects a kokkos executor and make_async_executor when used with kokkos executor complains about executor_parameters_type saying that there is no type named executor_parameters_type in class hpx::kokkos::executor<Kokkos::Experimental::HPX>
<hkaiser> gnikunj[m]: sure, but do we need the execution policy?
<hkaiser> async_replay/dataflow_replay etc. expect an executor as their first argument, not a policy
<gnikunj[m]> we need to do it the async_replay_executor way because it also expects N and validate function (if any) as arguments
<hkaiser> sure, but there is still no policy involved
<gnikunj[m]> yeah, I'm not using a policy atm. That was only to check if hpx::kokkos::kok could work.
<gnikunj[m]> right now it looks like: async_replay(
<gnikunj[m]> hpx::resiliency::experimental::make_replay_executor(exec, 3)
<gnikunj[m]> , 3, get_result)
<hkaiser> yes
<gnikunj[m]> exec here is hpx::kokkos::default_executor
<hkaiser> why do you need hpx::kokkos::kok for that?
<gnikunj[m]> I don't. It was just to see if I need it.
<hkaiser> you don't
<hkaiser> right
<gnikunj[m]> right
<hkaiser> so what's the issue, then?
<gnikunj[m]> Currently I'm using the make_replay_executor interface and passing it hpx::kokkos::default_executor
<hkaiser> correct
<gnikunj[m]> and it is complaining about no type named ‘executor_parameters_type’ in ‘class hpx::kokkos::executor<Kokkos::Experimental::HPX>’
<hkaiser> does that somehow involve the policy under the hood?
<gnikunj[m]> that's why I asked you what executor_parameters_type do under the hood
<hkaiser> ahh, then this is a bug in the hpx kokkos stuff
<hkaiser> it shouldn't directly refer to executor_parameters_type, but should use it only if its defined
<ms[m]> hkaiser, gnikunj I can add that typedef for you right away if you want, but it'll likely just be void
<gnikunj[m]> yeah, that's what I was trying to understand by looking into how executors work under the hood
<ms[m]> does it need to be anything useful?
<gnikunj[m]> ms: it isn't used in the constructs I'm using but it will be used in parallel_for and other similar constructs from what hkaiser shared
<hkaiser> gnikunj[m]: no, ms has it's own algorithm implementations
<ms[m]> right, the parallel algorithms won't work anyway with the kokkos executors, that's why we customize them
<hkaiser> right
<gnikunj[m]> aah. So that typedef should fix the issue for now.
<hkaiser> ms[m]: executor may or may not expose their default parameters type
<ms[m]> which will make things interesting if you want resiliency for parallel algorithms...
<gnikunj[m]> Unless I get into troubles again, in which case I'll post them here again
<hkaiser> parallel::execution::extract_executor_parameters is handling this
<ms[m]> right, I saw that only now
<hkaiser> ms[m]: yah, resiliency for parallel algorithms is a different can of worms, I think it wouldn't even work for the cpu implementations of the algorithms
<hkaiser> at least not right now
<srinivasyadav227> <gnikunj[m] "(this is the first time I'm writ"> I think I could help here, :-) ..and very much interested!
<gnikunj[m]> ms: why do you say that? If we have resilience executors already implemented, all they do is replay/replicate accordingly on failures.
<hkaiser> gnikunj[m]: we will see ;-)
<gnikunj[m]> hkaiser: that comment :D
<srinivasyadav227> if I am not wrong hkaiser gnikunj[m] gnikunj[m] are mentors for this organisation right ? :-)
<hkaiser> srinivasyadav227: gsoc?
<srinivasyadav227> yes
<hkaiser> yes, we'll do some mentoring
<srinivasyadav227> cool!.. looking forward to learn as much as I can :-)
<gonidelis[m]> hkaiser: is this better now?
<hkaiser> why no just do invoke(p, invoke(proj, *i))?
<gonidelis[m]> hkaiser: ok what's the use of invoked_projected then?
<hkaiser> that's a helper you can use if you don't want to implement algorithms yourself but reuse the std:: versions
<gonidelis[m]> ohhh ok got it
<gonidelis[m]> yeah sure... now it makes sense. since projections are not supported by std
<hkaiser> right
<gonidelis[m]> actually i had already tried your suggestion but it seemed suspiciously trivial ;p
<gonidelis[m]> although it compiled
<gonidelis[m]> Is there any way to `ctest` just `remove` without running the other remove* tests?
<hkaiser> specify the full target name
<gonidelis[m]> isn't `ctest -R tests.unit.modules.algorithms.remove` the full name?
<hkaiser> well, then it should do what you want
<gonidelis[m]> hkaiser: moreover here
<gonidelis[m]> we have the similar first - last situation.
<hkaiser> not sure what you're trying to say
<gonidelis[m]> if we return last then the return type is a sentinel, but then again last is returned when the element is not found and then iter=last
<gonidelis[m]> first = last ^^
<hkaiser> you have to return the first that was incremented appropriately
<gonidelis[m]> hkaiser: in these sequential find loops `return last` happens when we do not find the element within the sequence
<hkaiser> that will not work
<hkaiser> you have to return an iterator, not the sentinel
<gonidelis[m]> exactly... so I changed it in order to `return first` when it can't find the element
<hkaiser> which is correct
<hkaiser> as first will be incremented to last at this point
<gonidelis[m]> but it's the same, since first = last when the element is not found
<gonidelis[m]> yes that's what i thought.
<gonidelis[m]> \
<gonidelis[m]> i am sure CI will start screaming again though, any time soon
<gnikunj[m]> ms: please let me know when you update the typedef
<hkaiser> gnikunj[m]: I think he'd be happy to accept a PR ;-)
<gnikunj[m]> lol. Let me know where to add and I'll do it myself ;)
<hkaiser> gnikunj[m]: make it work and you'll know what to change
<gnikunj[m]> alright :D
<hkaiser> gnikunj[m]: let's talk if you're not sure
<gnikunj[m]> yes, that would be better.
<gnikunj[m]> I mean I don't have much experience with it so it'll be very helpful
<hkaiser> gnikunj[m]: if that typedef is missing, add it
<gnikunj[m]> on it
<hkaiser> gnikunj[m]: the idea is that executors might want to customize the way they handle various parameters
<hkaiser> the default is to operate as if everything was sequential
<gnikunj[m]> ohh, so that's why execute had std::invoke there
<gnikunj[m]> and we have async_execute for async execution
<hkaiser> don't think so std::invoke is unrelated
<hkaiser> ahh for seq execution - yes
<hkaiser> nod
<hkaiser> our executors are not really related to 443 anymore
<hkaiser> don't mix those things
<gnikunj[m]> got it. how are our executors then?
<hkaiser> ours are from an older version, I think p0443r3 or something, don't remember
<gnikunj[m]> ok
<hkaiser> with some extensions we have implemented, like the variadic arguments, those are not in p0443
<hkaiser> gnikunj[m]: so show me the error and I may be able to help
<gnikunj[m]> wait
<ms[m]> gnikunj: I can for sure add the typedef, as can you, but shouldn't the resiliency executors complaining about the missing typedefs also use extract_executor_parameters?
<gnikunj[m]> right, I can make them use that to default to seq
<hkaiser> it's a problem in our code
<ms[m]> also note that they kokkos executors return future<void>s
<hkaiser> that must use the extract_executor_parameters<> instead
<gnikunj[m]> hkaiser: yes, that's what ms pointed out
<gnikunj[m]> PR on its way
<hkaiser> same is probably true for the other executors
<gnikunj[m]> yy, it's in both of those
<hkaiser> ms[m]: nod, let's see if that makes things blow up
<gonidelis[m]> what is "checkout_code" ?
<hkaiser> no idea
<hkaiser> they might have a problem
<gonidelis[m]> it failed within 2 minutes
<hkaiser> let's re-trigger things in a while
<gonidelis[m]> ok
<gonidelis[m]> i think we are set on `remove` anyways
<gonidelis[m]> on to the next. whatever comes up in `remove` will be small work
<gonidelis[m]> little work^^
<hkaiser> ok
<gonidelis[m]> (imagine me using the phrase "small work" in cppnow :E )
<gonidelis[m]> gnikunj[m]:
<gnikunj[m]> gonidelis[m]: not every speaker is a native english speaker at cppnow :P
<gonidelis[m]> lol why does remove_copy(_if) applies when the pred is FALSE. that sounds reverse to me
<hkaiser> gonidelis[m]: it removes things if true
<gonidelis[m]> What a word game
<gonidelis[m]> Silly me
<gnikunj[m]> hkaiser: another problem. kokkos executors return future<void>. Doing a .get() on it won't have any functionality: https://github.com/STEllAR-GROUP/hpx/blob/master/libs/parallelism/resiliency/include/hpx/resiliency/async_replay_executor.hpp#L100
<gnikunj[m]> from the error log, doesn't seem like it's picking that one :/
<gnikunj[m]> /home/nk/install/hpx/include/hpx/resiliency/async_replay_executor.hpp:100:32: error: forming reference to void
<gnikunj[m]> 100 | auto&& result = f.get();
<hkaiser> it uses the return type of the function, probably - another thing to fix
<gnikunj[m]> yeah, on it
<gnikunj[m]> thanks for the heads up
<hkaiser> hab schon, ist ok
<hkaiser> ugg, wrong window
giannis has joined #ste||ar
giannis is now known as Guest2615
Guest2615 has quit [Quit: Connection closed]
hkaiser has quit [Quit: bye]
bita has quit [Ping timeout: 260 seconds]
<gnikunj[m]> ms: when you say kokkos executors return future<void>, what did you mean exactly?
<gnikunj[m]> ms: checked the code of hpx-kokkos. Understood what you meant.
<ms[m]> gnikunj: they should, or if they don't it should be possible to make them work (iirc async doesn't care anymore about the return type), but you're certainly right that I haven't added tests for it
<ms[m]> Did you try it?
<gnikunj[m]> ms yes I did. They complain about type mismatch of futures (void vs resulting type of function)
<ms[m]> Ah, ok, so then I didn't remember correctly
<ms[m]> That's possible to relax in hpx though
<gnikunj[m]> I was debugging the code of async_replay_helper when I saw the resultant type from executor does not match the function return type
<gnikunj[m]> ms how would you relax it though?
<gnikunj[m]> I mean if a function return type is T then you want .get() to return T as well and not void
<ms[m]> Either the return type is ignored or we require that the return type is void for kokkos executors
hkaiser has joined #ste||ar
<gnikunj[m]> ms why's that? Does kokkos not have an equivalent of std/hpx::async?
<gnikunj[m]> I only see for each and reduction algorithms in the tutorial sections
<gnikunj[m]> hkaiser: yt?
<hkaiser> gnikunj[m]: here now
<gnikunj[m]> hkaiser: are you available for a quick call?
<hkaiser> yah, sec
<hkaiser> gnikunj[m]: https://lsu.zoom.us/j/3340410194