hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
K-ballo has quit [Quit: K-ballo]
hkaiser has quit [Quit: bye]
<lsl88> Hi! Sorry for the time of the day. Am I on the right track if I finish reading the manual, try to see things that could be improved - ie tables that are difficult to follow, more information that should be added to fully understand one chapter and so on- and afterwards read the examples?
<simbergm> lsl88: yep, that sounds great
<simbergm> remember that we're not expecting you to fix the whole manual, but of course the more you read the better your overview of the current state will be, so what you're doing is much appreciated
Amy1 has quit [Ping timeout: 258 seconds]
Amy1 has joined #ste||ar
nikunj has joined #ste||ar
lsl88 has quit [Ping timeout: 245 seconds]
Coldblackice has quit [Ping timeout: 268 seconds]
<Yorlik> I just started to eliminate C style casts from my allocators and replace them with C++ style casts. I am ending up with some really ugly code though, like:
<Yorlik> return reinterpret_cast<void*>( reinterpret_cast<uint64_t>(topPagePtr_) + reinterpret_cast<uint64_t>(pagealloc::COMMIT_ERROR) );
<Yorlik> Any hints for getting the best of both worlds, like pointer arithmetic and low level void* spammage but also as much type safety as possibnle?
<Yorlik> I tried to find some good sources for learning and ended up with the C++ guidelines - I wonder if there are any good tutorial resources for this area of work.
hkaiser has joined #ste||ar
<Yorlik> Heyo hkaiser! If you read up a couple of lines .. would you pls answer some newbie issue with pointer arithmetic coding style? (C vs C++ casts)?
<hkaiser> uhh
<hkaiser> I don't know anything about pointers ;-)
<Yorlik> You'r joking, right? Or do you imply you never ever use pointer again?
<hkaiser> you want to have type safety but at the same moment you use reinterpret_cast
<hkaiser> that does not go together
<hkaiser> but seriously
<Yorlik> I wonder if I should create new types
<hkaiser> you don't need to cast a pointer first to uint64 and then to void*
<hkaiser> a simple static_cast<void*>(ptr) will do that
<Yorlik> I want to add an error code to it
<hkaiser> casting back requires a reinterpret_cast, however
<Yorlik> The pointer has 12 free bits at the bottom
<hkaiser> urgs
<Yorlik> So i want to use it as a tagged pointer to a memory page
<hkaiser> stick with whatever you had with C-style casts, the C++ version will not give you anything more
<Yorlik> It might just raise an alarm flag if I see C style casts in my code and avoid them elsewhere
<hkaiser> you can also create a special type that acts as the tagged pointer
<Yorlik> I wonder if a special class with static functions for tagging could do the trick, so i would encapsulate away the unsafe parts of the code
<Yorlik> That shouldn't cause any serious runtime issues in terms of performance / memory usage
<Yorlik> I mean - the code works already - I just got a ton of squiggles and I try to avoid pragma warning away everything
<Yorlik> cpmressed_ptr_type - lol
<Yorlik> I like the name already and the underlying type
<zao> Yorlik: standards-wise you should only cast between regular pointer types and integer types that are of the same size, and not rely on the integer value except that it can roundtrip ptr to int to ptr.
<zao> uintptr_t is such a suitably sized type.
<Yorlik> Yeah - my code kinda assumes 64 bit machines
<Yorlik> But what once we get 128 bit machines ... :D
<Yorlik> Or we try to run HPX on a Z80 ...
<Yorlik> But seriously: i really wann write safe code and avoid making it ugly
<zao> There are 64-bit machines with trap representations for example. All I’m saying is that you’re skirting toward UB territory.
<Yorlik> What's a trap representation?
<hkaiser> Yorlik: use size_t instead of uitn64_t, that is guaranteed to be large enough to store a pointer
<hkaiser> also, the compressed_ptr is probably all you need
<hkaiser> hides all ugliness
<Yorlik> I'll study it
<zao> size_t is technically supposed to only be able to hold the size of an array, which on _most_ architectures is the right size :P
<hkaiser> zao: you sure?
<zao> They may have changed the wording around that since 03.
<zao> Yorlik: You know how floating point numbers can be NaN, and in some CPU modes they cause hardware exceptions?
<Yorlik> Not really
<Yorlik> I always kinda assumed NaN would be a special reserved value
<zao> Special bit patterns that are Not-a-Number, which propagate through numeric operations.
<Yorlik> But I'm blank on that
<zao> On some architectures, notably Itanium, there's Not-a-Thing, which holds for regular integers.
<Yorlik> They are hardware defined?
<Yorlik> I think I might just make a special MemoryPagePtr class form my purposes as an excercise and taking inspiration from your compressed Ptr
<Yorlik> And avoid any dynamic / virtual stuff in it
<Yorlik> Yesterday I ended up teaching catch2 to test against segfaults ... :D
<zao> I looked them up, seems like NaT:s are not directly part of the value representation, so they're not necessarily a problem here :D
<zao> I'm sure your code is "fine", it's just the coercion to a type that's not necessarily of pointer size that's iffy.
<zao> hkaiser: cppreference seems to imply that it only needs to be able to store the maximum size of an object, which could technically be smaller than a pointer.
<Yorlik> The problem is, that strategically other developers might use my code in the future, write new allocators or poke around in existing code and all this allocation stuff is so deep in the system I really want to make it as rock solid as i can.
<zao> It's the kind of reading that ##c++ would love to have :P
<zao> Personally I'm mostly scared of overly clever compilers optimizing my code according to spec.
<Yorlik> When doing arithmetic on memory cells - is there a c++ type that is guaranteed to always be able represent a single cell? Should I use byte or char?
<Yorlik> It's more a theoreical question, but after all void* has no type and I need to do arithmetic on memory cells, on pages too etc.
<Yorlik> So I believe I should represent them propely instead of doinf wild void* casts all over the place
<Yorlik> Sry - my typing is horrible again - need more coffee ... :)
nikunj has quit [Remote host closed the connection]
Coldblackice has joined #ste||ar
<zao> Yorlik: You might want to read https://en.cppreference.com/w/cpp/language/object
<zao> `unsigned char` has been a good bet in the past as it has unsurprising signedness, haven't done C++ since std::byte appeared.
<Yorlik> IC - thanks for the link! Seems unsigned char and byte are equivalent. I might just use byte for the lowlevel thing because its more precise imo.
<Yorlik> With "char" you always think of something that prints.
<Yorlik> But a byte I can print as char, int or hex
<zao> Doesn't matter much when you're operating on the pointer type, but char has implementation-defined signedness, which is always fun.
<Yorlik> BTW - there is another interesting detail in that article: "For example, multiple floating-point bit patterns represent the same special value NaN. "
<Yorlik> So - not all NaNs are equal internally - didn't know that
<zao> There's like a million of them.
<Yorlik> wow
<Yorlik> I think I'll se them as neglected orphans ...
* Yorlik feels some wird pity for the NaNs all at a sudden
<zao> f32 has a 8-bit exponent, where an exponent of 0xFF is all NaNs, 0x00 is all denormals.
<zao> (well, not all)
<Yorlik> I'm totally undereducated in this IEEE stuff - first stumbled over it when working with Lua - seems there is some learning to do.
<Yorlik> I start wondering why would we ever need void*? Any pointer to a memory cell could be byte* - couldn't/shouldn't it?
<hkaiser> byte* implies a type, void* does not
<zao> There's some value in a vocabulary type that has some semantic distinctions.
<Yorlik> You mean because byte* would imply the traget has 1 byte length and void* is totally open to any interpretation?
<hkaiser> yes
<zao> byte* also has some alignment assumptions, and sometimes it's great not being able to dereference or do arithmetic.
<Yorlik> It somehow makes sense to not have pointer arihmetic for void*, still it is a bit awkward to use casts to byte or char to actually do arithmetic with void*. otoh - maybe the problem is the ease of misusing void*.
<hkaiser> Yorlik: doing pointer arithmetic with (u)char* implies an object size of one (byte)
<Yorlik> Yes - a single memory cell.
<hkaiser> instead of foo* p = ... ; ((char*)p) + sizeof(foo) you could write ++p
<Yorlik> The problem appears at the interface to the OS, which always requires void* to reserve an address space and to commit a page. I think I'll just put the conversion closely there and above use pages or my object types.
<hkaiser> any pointer is default convertible to void*
<hkaiser> so you only need to cast when you go from a void* to foo*
<hkaiser> (well any non-member function pointer, but those are not really 'pointers' anyways)
<Yorlik> My low level API is like this(different implementations per OS under the hood):
<Yorlik> / get system memory pagesize
<Yorlik> static size_t get_mempage_size( );
<Yorlik> const size_t pagesize = get_mempage_size( ); // 0x1000; // 4096 bytes
<Yorlik> / try to reserve the given amount of virtual memory, page aligned and grainsized to pages
<Yorlik> static void* vreserve( size_t sz );
<Yorlik> / Give up reservation of the region pointed into (usually you never do that)
<Yorlik> static bool vunreserve( void* ptr );
<Yorlik> / commit pages with given address and size and back it up with real memory rounded up to full pages
<Yorlik> static void* vcommit( void* ptr, size_t sz );
<Yorlik> / give up physical backup of pages WHICH ARE FULLY COVERED by the given range
<Yorlik> static bool vuncommit( void* ptr, size_t sz );
<Yorlik> So - here I can't really avoid the void*
<hkaiser> I didn't say you should
<Yorlik> But I think I'll make my allocators all strictly typed.
<Yorlik> I'm just trying to find out where to do what and how to minimize risk of abuse.
<hkaiser> implement typed allocators, what those do under the hood is their business
<Yorlik> I have one allocator that is typed to pages and is ued by the object allocators
<Yorlik> like this: template <size_t PAGESIZE> struct LinearPageAllocator;
<Yorlik> So pagesize is 4096 for now
<Yorlik> And then on top my typed one:
<Yorlik> template <typename T, FreelistType FL>class ContiguousPoolAllocator ;
<Yorlik> FreeList can be a set or a vector
<Yorlik> so i can control the policy what to reuse foirst: lowest or last freed
lsl88 has joined #ste||ar
<hkaiser> lsl88: you're on the roll!
<lsl88> ok, thanks :)
K-ballo has joined #ste||ar
<lsl88> I haven't finished reading the whole manual, but I have some ideas, can I share them here?
<hkaiser> lsl88: absolutely!
<lsl88> Something that I find is that tables somethimes are broken, specially when converting the manual to the PDF version.
<hkaiser> yes, pdf generation is always finicky
<hkaiser> we just implemented it
<lsl88> it happens in almost all the chapters, specially when a field is big
<hkaiser> nod
<hkaiser> I think this shouldn't be the main priority, html is the main target anyways
<lsl88> oh, I see
<lsl88> maybe a solution could be rearranging the info there
<hkaiser> I have no idea what we could do about this... you?
<hkaiser> ok
<lsl88> I am thinking of maybe instead of having tables, have it in text with bullets.
<hkaiser> hmmm
<hkaiser> it's several pieces of information for each of the counters
<hkaiser> using a table sounds logical
<hkaiser> we might reduce some of the repitition in text
<hkaiser> or somehow fused table entries - not sure
<lsl88> yes, I was going to say that some info is repeated
<hkaiser> or somehow have a separate page for each of the counters we could link from a smaller table
<hkaiser> lsl88: does spinx allow to specify table field widths, that might help as well
<hkaiser> sphinx*
<lsl88> I am taking a look
<lsl88> yes, apparently we can specify the width of each field
<hkaiser> might work - at least its worth a try
<lsl88> yes, I am also thinking of changing the description field to the second place
<lsl88> WDYT?
<hkaiser> sounds good to me
<lsl88> and some of the info is duplicated or exchanged, that's why I was thinking of having a list instead of a table. For instance: in AGAS Performance Counters (https://stellar-group.github.io/hpx/docs/sphinx/branches/master/html/manual/optimizing_hpx_applications.html#id9) I believe the Description field is number 4 instead of Parameters
<lsl88> and in Parcel layer performance counters (https://stellar-group.github.io/hpx/docs/sphinx/branches/master/html/manual/optimizing_hpx_applications.html#id10) if you read the Description field, after the first paragraph you have the same info duplicated
<lsl88> (in the meantime I am fixing small typos)
<hkaiser> right
<hkaiser> lsl88: feel free to combine typo fixes into bigger pull requests
<lsl88> (hope you don't mind having pull requests)
<hkaiser> not at all
<lsl88> yes, I was going to say that I end up doing small pull requests but just because I am fixing them as I see them
nikunj has joined #ste||ar
<nikunj> hkaiser: yt?
<hkaiser> here
<nikunj> did you get time to check the PR?
<hkaiser> have not looked yet, sorry
<nikunj> I was about to make changes in the benchmarks to make sure there are no cache inconsistencies
<hkaiser> k
<nikunj> ohh, then I'll not make changes for replicate one's
<hkaiser> no, pls go ahead
<nikunj> alright
<nikunj> the PR is pretty simple and the implementation is pretty straightforward
<nikunj> so I don't think there's any problems in the implementation
<nikunj> I'll go ahead with all benchmarks then
<hkaiser> k
<Yorlik> I start liking test driven development. All the red tests act as a todo list for me, right in my IDE.
<Yorlik> If I need to remiond myself of sth I just make a quick failing test.
<Yorlik> TEST_CASE( "Household::GetCoffee" ) {
<Yorlik> CHECK(false);
<Yorlik> }
<Yorlik> :D
<nikunj> is rostam still down?
<hkaiser> nikunj: yes, will be up tomorrow only - cooling was down
<nikunj> ok
<nikunj> I have written the benchmarks and scripts
<nikunj> I'll run them tomorrow morning
<hkaiser> ok
<Yorlik> Oh man ... "Don't cast away const" and "cannot initialize T* from const T* " - this is nuts.
<Yorlik> If I have a const base pointer and i want to derive a non const pointer pointing to an item in the buffer ...
<Yorlik> I get a love hate relationship with the squiggles - lol
<hkaiser> Yorlik: casting away constness is a sign of bad/flawed/incomplete design
<Yorlik> How would you solve the usage of a buffer as array wuith a const base pointer?
<Yorlik> I mean - I'm totally open to learn
<Yorlik> But at this low level there can be no compromises conserning performance
<Yorlik> And really: I don't understand how copying a const value in a formuala could be bad design, when it's used to compute a derived value
<Yorlik> like nonconst a = const b + nonconst c
<Yorlik> I'm pondering to install GSL and just use spans for my buffers
<Yorlik> spans look cool to me in this area of wild void* and T* pointers
<hkaiser> Yorlik: don't make the base-ptr const in the first place ;-)
<Yorlik> But it never ever changes and should never ever be changed
<Yorlik> If anything in this part of the code should be const its that pointer
<Yorlik> I mean practically - yes It probably wouldn't hurt to make it non const - however - I'm trying to internalize the meaning of constness
<Yorlik> If not I am missing something conceptiually
<Yorlik> pracxtically it all works nicely
<Yorlik> I'm just trying to understand the underlying C++ consepts
<Yorlik> I think the base problem lies here:
<Yorlik> const testdata* td_array_base = cpa_td.allocate (itemcount);
<Yorlik> giving the base pointer to my allocated are a the type of the stored items is wrong in the forst place
<Yorlik> It should be void
<Yorlik> If I want to create an array like structure from this pointer I should either use a typed span or a typed non const pointer
<Yorlik> This nonconst typed pointer would then be used in all array arithmetic
<zao> (note the distinction between a pointer-to-const an da const-pointer-to-mutable)
<Yorlik> Its about the pointer itself - the data is all mutable
<hkaiser> well, if you use the base ptr to access a non-const member, then the ptr shouldn't be const
<Yorlik> I think the issue it, that const void* myBase and T* myArrayBase are different in meaning
<hkaiser> Yorlik: well, if the pointer itself is const then you shouldn't have problems accessing a non-const element
<Yorlik> The problem comes when initializing a typed non const pointer from the const pointer to the base
<hkaiser> a const pointer is a 'foo* const' while a ptr to a const is a 'foo const*'
<Yorlik> const testdata* td_array_base = cpa_td.allocate (itemcount); testdata* ptr = td_array_base; does not work
<hkaiser> sure it doesn't
<Yorlik> The data is not const right?
<hkaiser> it is
<Yorlik> not by this definition
<Yorlik> Really?
<hkaiser> if you need a ptr that is const write foo* const = ...
<zao> Note that `const T*` is `T const*`.
<zao> If you want a const pointer, `T* const`.
<hkaiser> one more reason not to use east-const
<Yorlik> Dang - I want the pointer itself to be const, not the data
<zao> Yorlik is using west-const, which is horrible ;)
<hkaiser> that's what I meant
<Yorlik> Thats a matter of taste
<hkaiser> it's a matter of consistency
<Yorlik> How would you write the above code correctly?
<hkaiser> foo* const = allocate(...);
<Yorlik> testdata * const td_array_base = cpa_td.allocate (itemcount); testdata* ptr = td_array_base; ?
<hkaiser> you essentially wrote: foo const* = allocate(...)
<hkaiser> yes
<Yorlik> FFS - lol
<zao> The only way to get the const on the left side there would be `using FooPtr = foo*; const FooPtr = ..`
<hkaiser> urgs
<Yorlik> rofl