<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Ivo (& Isaac),
<div class=""><br class="">
</div>
<div class=""> I have just checked in a new version of GSAS-II (#3855). I found a number of sections of code where we looped for every histogram through the entire GPX file or through complex data structures. This caused the time for these steps to increase
as N**2. For small GPX files this was so quick as to be negligible, but made a huge difference for your “long.gpx” file.</div>
<div class=""><br class="">
</div>
<div class=""> I changed a fair amount of code (and hopefully did not break anything), but the speed is now way faster for both startup and for fitting each histogram in a very large sequential refinement. On my laptop, the time for each histogram in long.gpx
has gone from 80 sec down to 6 sec. I don’t have timing for the refinement startup, but I know that I trimmed at least a few minutes off that. Histogram plotting has gone from ~60 sec. down to ~2 sec. Single-histogram plotting should actually be a bit faster
than that, but looking into that will be for another time. </div>
<div class=""><br class="">
</div>
<div class=""> I think that you will find there is a bit of a slowdown with GPX files having thousands of histograms — that is unavoidable as certain steps require reading the entire GPX file or searching through all the information, but my hope is that
those delays are only at the beginning and end of the sequential refinement, but the speed within will not be much affected. There are also some potential memory issues in that there are probably some arrays that should be deleted after each histogram (also
for another occasion). Nonetheless, GSAS-II should now be good for sequential fitting for least 10,000 histograms in a single GPX file. Please let me know your experience.</div>
<div class=""><br class="">
</div>
<div class="">Brian</div>
<div class=""><br class="">
</div>
On Mar 4, 2019, at 3:33 AM, Ivo Alxneit via GSAS-II<br class="">
<<a href="mailto:gsas-ii@aps.anl.gov" class="">gsas-ii@aps.anl.gov</a><br class="">
<<a href="mailto:gsas-ii@aps.anl.gov" class="">mailto:gsas-ii@aps.anl.gov</a>> <<a href="mailto:gsas-ii@aps.anl.gov" class="">mailto:gsas-ii@aps.anl.gov</a>>> wrote:<br class="">
<br class="">
Dear all<br class="">
<br class="">
I am having a problem with sequential refinements in GSASII. I have a<br class="">
large data set where the changes to the few fitted parameters (scale<br class="">
factor, one lattice parameter, three background parameters) are minor<br class="">
and slow. If I fit a single pattern, the fit takes about one second<br class="">
including saving the data. If I do a sequential refinement of less than<br class="">
100 patterns (starting values are very close to final values in all<br class="">
pattern) time to fit one pattern remains about the same. If I work with<br class="">
1200 pattern the time increases to about three seconds. Finally, if I<br class="">
use the whole series of 8500 patten the time to fit a single pattern<br class="">
increases to about 45 seconds!<br class="">
<br class="">
From my understanding the time to fit a single pattern in a sequential<br class="">
refinement should be approximately constant as the fits are independent<br class="">
of each other. I might expect a small increase of the time because the<br class="">
data structure (project) becomes larger and so does the file that is<br class="">
saved after each fit (8500 pattern: 1GB). The reality, however, shows a<br class="">
more than linear increase of the time-per-pattern. How can this happen?<br class="">
Where is more than 98% of the time spent?<br class="">
<br class="">
In the same context. Shouldn't parallel execution of a sequential<br class="">
refinement be "trivial" to be implemented: Each CPU grabs the next<br class="">
pattern waiting to be refined and works on it. "Copy previous results"<br class="">
would turn into "Get results from last refined pattern". These are<br class="">
starting values and it should not make much of a difference if the are<br class="">
from pattern n-1 or n-x (x being typically not too large).<br class="">
<br class="">
Any insight is appreciated.<br class="">
-- <br class="">
<div class="">
<div>Dr. Ivo Alxneit</div>
</div>
<div><br class="">
</div>
</body>
</html>