<hkaiser>
also, you have almost no work at all - how many timestep do you run?
<parsa>
hkaiser: 10K timesteps
<parsa>
had a typo, it was running for the default 45 timesteps instead of 10K… i have to redo
<parsa>
update: 10k was a bad choice. it's too long and completely hides migration
<hkaiser>
parsa: isn't that what we want?
<parsa>
i mean, it's ideal, but it would make all cases, even the blocking ones look the same
hkaiser has quit [Quit: bye]
<peltonp1>
is there a way to launch multiple localities on a desktop or how do you test distributed programs without a cluster?
<zao>
peltonp1: Yup. A locality is just a process, batch systems just help orchestrate the launching of multiple processes and letting them know which other processes there are to communicate with.
<zao>
You can start them yourself and give arguments to let them know which one is the first rank and then tell the others to connect to the them. There might be some helpful information in the bundled Python scripts that are used by the test suite.
gonidelis[m] has quit [*.net *.split]
gnikunj[m] has quit [*.net *.split]
norbert[m] has quit [*.net *.split]
rori has quit [*.net *.split]
gonidelis[m] has joined #ste||ar
gnikunj[m] has joined #ste||ar
norbert[m] has joined #ste||ar
rori has joined #ste||ar
hkaiser has joined #ste||ar
<hkaiser>
parsa: yt?
<parsa>
parsa: yeah
<hkaiser>
parsa: hey
<hkaiser>
parsa: I'd be available to talk whenever you have time
<parsa>
i'm all set
<hkaiser>
sec
bita has joined #ste||ar
<hkaiser>
parsa: yt?
<parsa>
yes
<hkaiser>
parsa: for the shifted case, could you change the measurements such that it starts off with the shifted placement and migrate it into the optimal solution instead of doing it the other way around?
<hkaiser>
I think that would be more consistent with the impaired case
<hkaiser>
because if you start with the optimal case introducing non-optimal data placement usingn migration will not end up with a use case that corresponds what people would like to have
<parsa>
okay. so just to check: we want 4 new sets of data: impaired case with no migration, shifted case but shift the data when creating the components blocked, overlapped, and without migration. right?
<parsa>
hkaiser: ^
<hkaiser>
parsa: overall: start impaired, migrate in place blocked and overlapped; start shifted, migrate in place blocked and overlapped; baseline (no migration, optimal data placement); impaired (no migration); and shifted (no migration)
<hkaiser>
so three baseline measurements, and 4 migration scenarios
<hkaiser>
one baseline and the overlapped migrations are there - no need to redo those
<hkaiser>
what I think we would need is two baseline measurements of the impaired (start load-imbalanced) and shifted (start with bad data locality) cases
<hkaiser>
and the shifted migration scenarios, just not start with the good placement, but start with the bad placement
<hkaiser>
parsa: does this make sense?
<parsa>
not yet, still going through it
<hkaiser>
parsa: sorry, I misspoke
<parsa>
i get the baseline measurements part… no migration... 1: optimal; 2: impaired, 3: all data shifted to the neighbor at start)
<hkaiser>
I say 'one baseline and the overlapped migrations are there - no need to redo those' but I meant 'one baseline and the impaired migrations are there - no need to redo those'
<hkaiser>
yes
<hkaiser>
now the migration scenarios are simple
<hkaiser>
the impaired are ok, no need to redo
<hkaiser>
what would be nice is to have shifted migration measurements (overlapped and blocked) but start with the shifted placement and migrate things to the optimal layout
<parsa>
well we don't have the impaired without migration… i'll run on one node a couple of times and get you the average
<hkaiser>
parsa: yes, thanks
<parsa>
hkaiser: impaired case is 641.877 seconds… i'll email the rest once i get them
<hkaiser>
parsa: on one locality? or two?
<hkaiser>
parsa: btw, the overall scaling is almost a factor of 9 when going from two to ten nodes - that seems to be over the top and needs explanation
<hkaiser>
if this 641.877s is on one node, what's the optimal base line for one node?
parsa[m] has joined #ste||ar
<hkaiser>
parsa[m]: can you read back?
<parsa[m]>
I’ll be back in an hour
<hkaiser>
ok
<parsa>
hkaiser: that is on one node… i did run it on two nodes for good measure, and as expected, get the same exec time
<parsa>
right… going from 2 to 8 localities and getting a speedup of 14 :O
<parsa>
1* to 8 localities
<hkaiser>
parsa: can you run the others on one node as well
<hkaiser>
at least the optimal baseline?
<parsa>
yes
<hkaiser>
cool, thanks
<hkaiser>
I think the data we'll have is more than sufficient, then
<parsa>
hkaiser: ping
<hkaiser>
here
<parsa>
i've figure putting the raw data in a gsheet is expedient... see link in pm