<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Hi Ivo, </div>
<div class=""><br class="">
</div>
<div class=""> You do not comment on what scale/type of computer you are using for this. It would be interesting to know if this is Windows, Mac or Linux and how much memory the computer has. This could simply be a memory-swap problem, as the way GSAS-II
works is to keep the entire GPX file in memory and that is going to create a much bigger demand with a 1Gb project. If you have access to big-memory machine it would be interesting to know if this makes a difference. </div>
<div class=""><br class="">
</div>
<div class=""> Bob and I spoke about this earlier and he suggested that a likely reason for the slowdown has to do with the way we handle file IO. After every pattern is fit, the entire GPX file is written and probably reread as well. If you would be willing,
we can put some debug statements into the code that would allow seeing where the 45 seconds are being consumed. Reorganizing the sequential refinement code to avoid some of this should then be possible. Just to confirm, you see the 45 seconds/pattern with
your 1 Gb project, regardless if you set the fitting range as say 10 patterns or the entire range. Correct? </div>
<div class=""><br class="">
</div>
<div class=""> Parallel processing is also certainly a possibility. It can already be done with scripting (see <span style="font-size: 12px; text-indent: -36px;" class="">O’Donnell, J. H., Dreele, R. B. V., Chan, M. K. Y. & Toby, B. H. (2018).</span><i style="font-size: 12px; text-indent: -36px;" class="">
Journal of Applied Crystallography</i><span style="font-size: 12px; text-indent: -36px;" class="">
</span><b style="font-size: 12px; text-indent: -36px;" class="">51</b><span style="font-size: 12px; text-indent: -36px;" class="">, 1244-1250.</span><span style="font-size: 12px; text-indent: -36px;" class="">) </span>— reprint on request, but the most straightforward
approach of simply assigning each processor every n-th pattern would require n copies of the GPX file, which would amplify the memory problems described in the 1st paragraph. </div>
<div class=""><br class="">
</div>
<div class=""> Please let us know if you have any experience with use of different types of machines, and if you are willing to do some timing runs for us. </div>
<div class=""><br class="">
</div>
<div class="">Brian</div>
<div class=""><br class="">
</div>
<div>
<blockquote type="cite" class="">
<div class="">On Mar 4, 2019, at 3:33 AM, Ivo Alxneit via GSAS-II <<a href="mailto:gsas-ii@aps.anl.gov" class="">gsas-ii@aps.anl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Dear all<br class="">
<br class="">
I am having a problem with sequential refinements in GSASII. I have a<br class="">
large data set where the changes to the few fitted parameters (scale<br class="">
factor, one lattice parameter, three background parameters) are minor<br class="">
and slow. If I fit a single pattern, the fit takes about one second<br class="">
including saving the data. If I do a sequential refinement of less than<br class="">
100 patterns (starting values are very close to final values in all<br class="">
pattern) time to fit one pattern remains about the same. If I work with<br class="">
1200 pattern the time increases to about three seconds. Finally, if I<br class="">
use the whole series of 8500 patten the time to fit a single pattern<br class="">
increases to about 45 seconds!<br class="">
<br class="">
From my understanding the time to fit a single pattern in a sequential<br class="">
refinement should be approximately constant as the fits are independent<br class="">
of each other. I might expect a small increase of the time because the<br class="">
data structure (project) becomes larger and so does the file that is<br class="">
saved after each fit (8500 pattern: 1GB). The reality, however, shows a<br class="">
more than linear increase of the time-per-pattern. How can this happen?<br class="">
Where is more than 98% of the time spent?<br class="">
<br class="">
In the same context. Shouldn't parallel execution of a sequential<br class="">
refinement be "trivial" to be implemented: Each CPU grabs the next<br class="">
pattern waiting to be refined and works on it. "Copy previous results"<br class="">
would turn into "Get results from last refined pattern". These are<br class="">
starting values and it should not make much of a difference if the are<br class="">
from pattern n-1 or n-x (x being typically not too large).<br class="">
<br class="">
Any insight is appreciated.<br class="">
-- <br class="">
Dr. Ivo Alxneit<br class="">
Catalysis for Energy Group<br class="">
Bioenergy and Catalysis Laboratory phone: +41 56 310 4092<br class="">
Paul Scherrer Institute fax: +41 56 310 2688<br class="">
CH-5232 Villigen gnupg key: 0x515E30C7<br class="">
Switzerland<br class="">
<a href="https://www.psi.ch/ceg/" class="">https://www.psi.ch/ceg/</a><br class="">
<br class="">
_______________________________________________<br class="">
GSAS-II mailing list<br class="">
GSAS-II@aps.anl.gov<br class="">
https://mailman.aps.anl.gov/mailman/listinfo/gsas-ii</div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>