initial commit

This commit is contained in:
2026-08-07 15:56:42 +09:00
commit 91ad25aca9
1012 changed files with 159314 additions and 0 deletions
@@ -0,0 +1,131 @@
%FILELIST Check for file queue
% checks for mat/dat files
% ** p p structure
%
% returns:
% ++ p p structure
% ++ status_ok status flag
%
% see also: scans.get_queue
% Academic License Agreement
%
% Source Code
%
% Introduction
% This license agreement sets forth the terms and conditions under which the PAUL SCHERRER INSTITUT (PSI), CH-5232 Villigen-PSI, Switzerland (hereafter "LICENSOR")
% will grant you (hereafter "LICENSEE") a royalty-free, non-exclusive license for academic, non-commercial purposes only (hereafter "LICENSE") to use the cSAXS
% ptychography MATLAB package computer software program and associated documentation furnished hereunder (hereafter "PROGRAM").
%
% Terms and Conditions of the LICENSE
% 1. LICENSOR grants to LICENSEE a royalty-free, non-exclusive license to use the PROGRAM for academic, non-commercial purposes, upon the terms and conditions
% hereinafter set out and until termination of this license as set forth below.
% 2. LICENSEE acknowledges that the PROGRAM is a research tool still in the development stage. The PROGRAM is provided without any related services, improvements
% or warranties from LICENSOR and that the LICENSE is entered into in order to enable others to utilize the PROGRAM in their academic activities. It is the
% LICENSEEs responsibility to ensure its proper use and the correctness of the results.
% 3. THE PROGRAM IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR
% A PARTICULAR PURPOSE AND NONINFRINGEMENT OF ANY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. IN NO EVENT SHALL THE LICENSOR, THE AUTHORS OR THE COPYRIGHT
% HOLDERS BE LIABLE FOR ANY CLAIM, DIRECT, INDIRECT OR CONSEQUENTIAL DAMAGES OR OTHER LIABILITY ARISING FROM, OUT OF OR IN CONNECTION WITH THE PROGRAM OR THE USE
% OF THE PROGRAM OR OTHER DEALINGS IN THE PROGRAM.
% 4. LICENSEE agrees that it will use the PROGRAM and any modifications, improvements, or derivatives of PROGRAM that LICENSEE may create (collectively,
% "IMPROVEMENTS") solely for academic, non-commercial purposes and that any copy of PROGRAM or derivatives thereof shall be distributed only under the same
% license as PROGRAM. The terms "academic, non-commercial", as used in this Agreement, mean academic or other scholarly research which (a) is not undertaken for
% profit, or (b) is not intended to produce works, services, or data for commercial use, or (c) is neither conducted, nor funded, by a person or an entity engaged
% in the commercial use, application or exploitation of works similar to the PROGRAM.
% 5. LICENSEE agrees that it shall make the following acknowledgement in any publication resulting from the use of the PROGRAM or any translation of the code into
% another computing language:
% "Data processing was carried out using the cSAXS ptychography MATLAB package developed by the Science IT and the coherent X-ray scattering (CXS) groups, Paul
% Scherrer Institut, Switzerland."
%
% Additionally, any publication using the package, or any translation of the code into another computing language should cite for difference map:
% P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, F. Pfeiffer, High-resolution scanning X-ray diffraction microscopy, Science 321, 379382 (2008).
% (doi: 10.1126/science.1158573),
% for maximum likelihood:
% P. Thibault and M. Guizar-Sicairos, Maximum-likelihood refinement for coherent diffractive imaging, New J. Phys. 14, 063004 (2012).
% (doi: 10.1088/1367-2630/14/6/063004),
% for mixed coherent modes:
% P. Thibault and A. Menzel, Reconstructing state mixtures from diffraction measurements, Nature 494, 6871 (2013). (doi: 10.1038/nature11806),
% and/or for multislice:
% E. H. R. Tsai, I. Usov, A. Diaz, A. Menzel, and M. Guizar-Sicairos, X-ray ptychography with extended depth of field, Opt. Express 24, 2908929108 (2016).
% (doi: 10.1364/OE.24.029089).
% 6. Except for the above-mentioned acknowledgment, LICENSEE shall not use the PROGRAM title or the names or logos of LICENSOR, nor any adaptation thereof, nor the
% names of any of its employees or laboratories, in any advertising, promotional or sales material without prior written consent obtained from LICENSOR in each case.
% 7. Ownership of all rights, including copyright in the PROGRAM and in any material associated therewith, shall at all times remain with LICENSOR, and LICENSEE
% agrees to preserve same. LICENSEE agrees not to use any portion of the PROGRAM or of any IMPROVEMENTS in any machine-readable form outside the PROGRAM, nor to
% make any copies except for its internal use, without prior written consent of LICENSOR. LICENSEE agrees to place the following copyright notice on any such copies:
% © All rights reserved. PAUL SCHERRER INSTITUT, Switzerland, Laboratory for Macromolecules and Bioimaging, 2017.
% 8. The LICENSE shall not be construed to confer any rights upon LICENSEE by implication or otherwise except as specifically set forth herein.
% 9. DISCLAIMER: LICENSEE shall be aware that Phase Focus Limited of Sheffield, UK has an international portfolio of patents and pending applications which relate
% to ptychography and that the PROGRAM may be capable of being used in circumstances which may fall within the claims of one or more of the Phase Focus patents,
% in particular of patent with international application number PCT/GB2005/001464. The LICENSOR explicitly declares not to indemnify the users of the software
% in case Phase Focus or any other third party will open a legal action against the LICENSEE due to the use of the program.
% 10. This Agreement shall be governed by the material laws of Switzerland and any dispute arising out of this Agreement or use of the PROGRAM shall be brought before
% the courts of Zürich, Switzerland.
function [ p, status_ok ] = filelist(p)
import utils.verbose
p.queue.path = utils.abspath(p.queue.path);
% Prepare folders for queue if not done yet
if ~exist(fullfile(p.queue.path,'in_progress'))
mkdir(fullfile(p.queue.path,'in_progress'));
end
if ~exist(fullfile(p.queue.path,'failed'))
mkdir(fullfile(p.queue.path,'failed'));
end
fext = 'dat';
status_ok = true;
verbose(1,['p.queue.path is active, touching folder and looking for files in the queue in ' p.queue.path]);
[status, result] = system(['touch ' p.queue.path '.'], '-echo');
files_recons = dir([p.queue.path 'scan*.dat']);
if isempty(files_recons)
files_recons = dir(fullfile(p.queue.path,'*.mat'));
fext = 'mat';
end
% Now move this file to another folder
% Found one file to reconstruct
if ~isempty(files_recons)
if p.queue.recon_latest_first
p.queue.file_this_recons = files_recons(end).name;
else
p.queue.file_this_recons = files_recons(1).name;
end
verbose(1,['Found file in queue ' fullfile(p.queue.path,p.queue.file_this_recons)]);
% now move it quickly before someone else will take it
try
io.movefile_fast(fullfile(p.queue.path,p.queue.file_this_recons),fullfile(p.queue.path,'in_progress'))
verbose(1,['Moving file to ' fullfile(p.queue.path,'in_progress')]);
catch
verbose(1,['Failed moving file to ' fullfile(p.queue.path,'in_progress')]);
pause(1);
status_ok = false;
end
% and finally parse the file
verbose(2,['Parsing file ' fullfile(p.queue.path,p.queue.file_this_recons)]);
p = parse_queue_file(fullfile(p.queue.path,'in_progress',p.queue.file_this_recons),p, fext);
verbose(2,['Succesfully parsed file ' fullfile(p.queue.path,p.queue.file_this_recons)]);
else
verbose(1,'Did not find enough files in queue, pausing %gs seconds and then exiting', p.queue.file_queue_timeout)
pause(p.queue.file_queue_timeout)
status_ok = false;
end
end
@@ -0,0 +1,140 @@
% p = ptycho_parse_omny_dat(filename,p)
% It parses dat files from queue to modify a ptychography
% p.structure. This allows for instance to change the scan number or
% another field. It also allows to define fields that do not exist.
% Inputs
% filename Filename with path of the dat file
% p Optional input, if it is given the output will be equal to p
% in all fields that are not modified
% Output
% p_out Structure with new, or modified fields.
% Academic License Agreement
%
% Source Code
%
% Introduction
% This license agreement sets forth the terms and conditions under which the PAUL SCHERRER INSTITUT (PSI), CH-5232 Villigen-PSI, Switzerland (hereafter "LICENSOR")
% will grant you (hereafter "LICENSEE") a royalty-free, non-exclusive license for academic, non-commercial purposes only (hereafter "LICENSE") to use the cSAXS
% ptychography MATLAB package computer software program and associated documentation furnished hereunder (hereafter "PROGRAM").
%
% Terms and Conditions of the LICENSE
% 1. LICENSOR grants to LICENSEE a royalty-free, non-exclusive license to use the PROGRAM for academic, non-commercial purposes, upon the terms and conditions
% hereinafter set out and until termination of this license as set forth below.
% 2. LICENSEE acknowledges that the PROGRAM is a research tool still in the development stage. The PROGRAM is provided without any related services, improvements
% or warranties from LICENSOR and that the LICENSE is entered into in order to enable others to utilize the PROGRAM in their academic activities. It is the
% LICENSEEs responsibility to ensure its proper use and the correctness of the results.
% 3. THE PROGRAM IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR
% A PARTICULAR PURPOSE AND NONINFRINGEMENT OF ANY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. IN NO EVENT SHALL THE LICENSOR, THE AUTHORS OR THE COPYRIGHT
% HOLDERS BE LIABLE FOR ANY CLAIM, DIRECT, INDIRECT OR CONSEQUENTIAL DAMAGES OR OTHER LIABILITY ARISING FROM, OUT OF OR IN CONNECTION WITH THE PROGRAM OR THE USE
% OF THE PROGRAM OR OTHER DEALINGS IN THE PROGRAM.
% 4. LICENSEE agrees that it will use the PROGRAM and any modifications, improvements, or derivatives of PROGRAM that LICENSEE may create (collectively,
% "IMPROVEMENTS") solely for academic, non-commercial purposes and that any copy of PROGRAM or derivatives thereof shall be distributed only under the same
% license as PROGRAM. The terms "academic, non-commercial", as used in this Agreement, mean academic or other scholarly research which (a) is not undertaken for
% profit, or (b) is not intended to produce works, services, or data for commercial use, or (c) is neither conducted, nor funded, by a person or an entity engaged
% in the commercial use, application or exploitation of works similar to the PROGRAM.
% 5. LICENSEE agrees that it shall make the following acknowledgement in any publication resulting from the use of the PROGRAM or any translation of the code into
% another computing language:
% "Data processing was carried out using the cSAXS ptychography MATLAB package developed by the Science IT and the coherent X-ray scattering (CXS) groups, Paul
% Scherrer Institut, Switzerland."
%
% Additionally, any publication using the package, or any translation of the code into another computing language should cite for difference map:
% P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, F. Pfeiffer, High-resolution scanning X-ray diffraction microscopy, Science 321, 379382 (2008).
% (doi: 10.1126/science.1158573),
% for maximum likelihood:
% P. Thibault and M. Guizar-Sicairos, Maximum-likelihood refinement for coherent diffractive imaging, New J. Phys. 14, 063004 (2012).
% (doi: 10.1088/1367-2630/14/6/063004),
% for mixed coherent modes:
% P. Thibault and A. Menzel, Reconstructing state mixtures from diffraction measurements, Nature 494, 6871 (2013). (doi: 10.1038/nature11806),
% and/or for multislice:
% E. H. R. Tsai, I. Usov, A. Diaz, A. Menzel, and M. Guizar-Sicairos, X-ray ptychography with extended depth of field, Opt. Express 24, 2908929108 (2016).
% (doi: 10.1364/OE.24.029089).
% 6. Except for the above-mentioned acknowledgment, LICENSEE shall not use the PROGRAM title or the names or logos of LICENSOR, nor any adaptation thereof, nor the
% names of any of its employees or laboratories, in any advertising, promotional or sales material without prior written consent obtained from LICENSOR in each case.
% 7. Ownership of all rights, including copyright in the PROGRAM and in any material associated therewith, shall at all times remain with LICENSOR, and LICENSEE
% agrees to preserve same. LICENSEE agrees not to use any portion of the PROGRAM or of any IMPROVEMENTS in any machine-readable form outside the PROGRAM, nor to
% make any copies except for its internal use, without prior written consent of LICENSOR. LICENSEE agrees to place the following copyright notice on any such copies:
% © All rights reserved. PAUL SCHERRER INSTITUT, Switzerland, Laboratory for Macromolecules and Bioimaging, 2017.
% 8. The LICENSE shall not be construed to confer any rights upon LICENSEE by implication or otherwise except as specifically set forth herein.
% 9. DISCLAIMER: LICENSEE shall be aware that Phase Focus Limited of Sheffield, UK has an international portfolio of patents and pending applications which relate
% to ptychography and that the PROGRAM may be capable of being used in circumstances which may fall within the claims of one or more of the Phase Focus patents,
% in particular of patent with international application number PCT/GB2005/001464. The LICENSOR explicitly declares not to indemnify the users of the software
% in case Phase Focus or any other third party will open a legal action against the LICENSEE due to the use of the program.
% 10. This Agreement shall be governed by the material laws of Switzerland and any dispute arising out of this Agreement or use of the PROGRAM shall be brought before
% the courts of Zürich, Switzerland.
function p_out = parse_queue_file(filename,p, fext)
import utils.update_param
if nargin < 1
error('At least one argument, filename, should be given')
return
end
if exist('p')
if ~isstruct(p)
error('Second argument, p, must be a structure')
return
else
p_out = p;
end
else
p_out = struct;
end
if ~exist(filename,'file')
warning(sprintf('I did not find the file: %s. Output structure will be then the same as input structure', filename))
return
end
switch fext
case 'dat'
fid = fopen(filename,'r');
tline = fgetl(fid);
while ischar(tline)
str_parts = strsplit(tline, ' ');
if numel(str_parts)>1
fname = strtrim(str_parts{1});
if strcmpi(fname(1:2), 'p.')
% found p entry
val = [];
for ii=2:numel(str_parts)
if ~isempty(strtrim(str_parts{ii}))
if ~isnan(str2double(str_parts{ii}))
% found number
val = [val, str2double(str_parts{ii})];
else
% found char
val = [val, strtrim(str_parts{ii})];
end
end
end
p_out.(fname(3:end)) = val;
elseif strcmpi(str_parts{1}, 'samplename')
p_out.samplename = strjoin(strtrim(str_parts(2:end)), '_');
% keep only printable chars (prevent users to fill put unicode characters)
p_out.samplename = p_out.samplename(isstrprop(p_out.samplename, 'print'));
% remove spaces from the file name
p_out.samplename = replace(p_out.samplename, ' ', '_');
end
end
tline = fgetl(fid);
end
fclose(fid);
case 'mat'
% load mat file and make sure that the paths are updated correctly
f = load(filename);
p = update_param(p, f.p);
p_out = core.ptycho_prepare_paths(p, true);
otherwise
error('Unknown file extension %s.', fext);
end
end
@@ -0,0 +1,91 @@
%REMOTE_QUEUE
% Academic License Agreement
%
% Source Code
%
% Introduction
% This license agreement sets forth the terms and conditions under which the PAUL SCHERRER INSTITUT (PSI), CH-5232 Villigen-PSI, Switzerland (hereafter "LICENSOR")
% will grant you (hereafter "LICENSEE") a royalty-free, non-exclusive license for academic, non-commercial purposes only (hereafter "LICENSE") to use the cSAXS
% ptychography MATLAB package computer software program and associated documentation furnished hereunder (hereafter "PROGRAM").
%
% Terms and Conditions of the LICENSE
% 1. LICENSOR grants to LICENSEE a royalty-free, non-exclusive license to use the PROGRAM for academic, non-commercial purposes, upon the terms and conditions
% hereinafter set out and until termination of this license as set forth below.
% 2. LICENSEE acknowledges that the PROGRAM is a research tool still in the development stage. The PROGRAM is provided without any related services, improvements
% or warranties from LICENSOR and that the LICENSE is entered into in order to enable others to utilize the PROGRAM in their academic activities. It is the
% LICENSEEs responsibility to ensure its proper use and the correctness of the results.
% 3. THE PROGRAM IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR
% A PARTICULAR PURPOSE AND NONINFRINGEMENT OF ANY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. IN NO EVENT SHALL THE LICENSOR, THE AUTHORS OR THE COPYRIGHT
% HOLDERS BE LIABLE FOR ANY CLAIM, DIRECT, INDIRECT OR CONSEQUENTIAL DAMAGES OR OTHER LIABILITY ARISING FROM, OUT OF OR IN CONNECTION WITH THE PROGRAM OR THE USE
% OF THE PROGRAM OR OTHER DEALINGS IN THE PROGRAM.
% 4. LICENSEE agrees that it will use the PROGRAM and any modifications, improvements, or derivatives of PROGRAM that LICENSEE may create (collectively,
% "IMPROVEMENTS") solely for academic, non-commercial purposes and that any copy of PROGRAM or derivatives thereof shall be distributed only under the same
% license as PROGRAM. The terms "academic, non-commercial", as used in this Agreement, mean academic or other scholarly research which (a) is not undertaken for
% profit, or (b) is not intended to produce works, services, or data for commercial use, or (c) is neither conducted, nor funded, by a person or an entity engaged
% in the commercial use, application or exploitation of works similar to the PROGRAM.
% 5. LICENSEE agrees that it shall make the following acknowledgement in any publication resulting from the use of the PROGRAM or any translation of the code into
% another computing language:
% "Data processing was carried out using the cSAXS ptychography MATLAB package developed by the Science IT and the coherent X-ray scattering (CXS) groups, Paul
% Scherrer Institut, Switzerland."
%
% Additionally, any publication using the package, or any translation of the code into another computing language should cite for difference map:
% P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, F. Pfeiffer, High-resolution scanning X-ray diffraction microscopy, Science 321, 379382 (2008).
% (doi: 10.1126/science.1158573),
% for maximum likelihood:
% P. Thibault and M. Guizar-Sicairos, Maximum-likelihood refinement for coherent diffractive imaging, New J. Phys. 14, 063004 (2012).
% (doi: 10.1088/1367-2630/14/6/063004),
% for mixed coherent modes:
% P. Thibault and A. Menzel, Reconstructing state mixtures from diffraction measurements, Nature 494, 6871 (2013). (doi: 10.1038/nature11806),
% and/or for multislice:
% E. H. R. Tsai, I. Usov, A. Diaz, A. Menzel, and M. Guizar-Sicairos, X-ray ptychography with extended depth of field, Opt. Express 24, 2908929108 (2016).
% (doi: 10.1364/OE.24.029089).
% 6. Except for the above-mentioned acknowledgment, LICENSEE shall not use the PROGRAM title or the names or logos of LICENSOR, nor any adaptation thereof, nor the
% names of any of its employees or laboratories, in any advertising, promotional or sales material without prior written consent obtained from LICENSOR in each case.
% 7. Ownership of all rights, including copyright in the PROGRAM and in any material associated therewith, shall at all times remain with LICENSOR, and LICENSEE
% agrees to preserve same. LICENSEE agrees not to use any portion of the PROGRAM or of any IMPROVEMENTS in any machine-readable form outside the PROGRAM, nor to
% make any copies except for its internal use, without prior written consent of LICENSOR. LICENSEE agrees to place the following copyright notice on any such copies:
% © All rights reserved. PAUL SCHERRER INSTITUT, Switzerland, Laboratory for Macromolecules and Bioimaging, 2017.
% 8. The LICENSE shall not be construed to confer any rights upon LICENSEE by implication or otherwise except as specifically set forth herein.
% 9. DISCLAIMER: LICENSEE shall be aware that Phase Focus Limited of Sheffield, UK has an international portfolio of patents and pending applications which relate
% to ptychography and that the PROGRAM may be capable of being used in circumstances which may fall within the claims of one or more of the Phase Focus patents,
% in particular of patent with international application number PCT/GB2005/001464. The LICENSOR explicitly declares not to indemnify the users of the software
% in case Phase Focus or any other third party will open a legal action against the LICENSEE due to the use of the program.
% 10. This Agreement shall be governed by the material laws of Switzerland and any dispute arising out of this Agreement or use of the PROGRAM shall be brought before
% the courts of Zürich, Switzerland.
function [p, status_ok] = remote_queue(p)
import utils.verbose
import utils.update_param
status_ok = true;
verbose(1,['Remote queue is active, touching folder and looking for files in the queue in ' p.queue.remote_path]);
[~, ~] = system(['touch ' p.queue.remote_path '.'], '-echo');
files_recons = dir([p.queue.remote_path '*.mat']);
% Found one file to reconstruct
if ~isempty(files_recons)
if p.queue.recon_latest_first
p.queue.file_this_recons = files_recons(end).name;
else
p.queue.file_this_recons = files_recons(1).name;
end
verbose(1,['Found file in queue ' fullfile(p.queue.remote_path,p.queue.file_this_recons)]);
f = load(fullfile(p.queue.remote_path,p.queue.file_this_recons));
p = update_param(f.p, p);
% Now move this file to another folder
try
movefile(fullfile(p.queue.remote_path,p.queue.file_this_recons),fullfile(p.queue.remote_path,'in_progress'))
verbose(1,['Moving file to ' fullfile(p.queue.remote_path,'in_progress')]);
catch
verbose(1,['Failed moving file to ' fullfile(p.queue.remote_path,'in_progress')]);
pause(1);
status_ok = false;
end
else
verbose(1,'Did not find enough files in queue, pausing 10 seconds and then exiting')
pause(10)
status_ok = false;
end
end