srwpy

Synchrotron Radiation Workshop (SRW)

SRW is a physical optics computer code for calculation of detailed characteristics of Synchrotron Radiation (SR) generated by relativistic electrons in magnetic fields of arbitrary configuration and for simulation of the radiation wavefront propagation through optical systems of beamlines.

Frequency-domain near-field methods are used for the SR calculation, and the Fourier-optics based approach is generally used for the wavefront propagation simulation. The code enables both fully- and partially-coherent radiation propagation simulations in steady-state and in frequency-/time-dependent regimes. With these features, the code has already proven its utility for a large number of applications in infrared, UV, soft and hard X-ray spectral range, in such important areas as analysis of spectral performances of new synchrotron radiation sources, optimization of user beamlines, development of new optical elements, source and beamline diagnostics, and even complete simulation of SR based experiments. Besides the SR applications, the code can be efficiently used for various simulations involving conventional lasers and other sources. SRW versions interfaced to Python and to IGOR Pro (WaveMetrics), as well as cross-platform library with C API, are available.

In the following writing, it is assumed that SRW_Dev is absolute path to the full SRW directory (obtained e.g. after downloading from repository).

I. Using pre-compiled SRW libraries and clients / bindings:

The last ~"clean" release of SRW for IGOR Pro and for Python can be found in SRW_Dev/env/release, in particular:

• installers (of compressed packages) can be found in SRW_Dev/env/release/install;
• unpacked folder of SRW for IGOR Pro (for Windows only) is: SRW_Dev/env/release/srw_igor;
  This folder contains ReadMe.txt file with general "start-up" notes; detailed documentation for IGOR Pro version can be found in: SRW_Dev/env/release/srw_igor/SRW Help/SRW Help.ifn file (in IGOR formatted notebook format);
• unpacked folder of SRW for Python (for Windows and Linux) is: SRW_Dev/env/release/srw_python; this folder contains ReadMe.txt file with general "start-up" notes.

The most recent pre-releases and current work versions of SRW for Python and for IGOR Pro can be found in: SRW_Dev/env/work.

Testing of the pre-compiled SRW libraries and clients / bindings can be done using examples included both to Python and IGOR Pro versions of SRW (see "Checking the examples" sections below for different platforms).

II. Compiling and testing SRW Library and its Python and IGOR Pro bindings on Windows

II.1. Compiling SRW library and Python binding using MS Visual C++

II.1.1

Microsoft Visual C++ 2015 (or later version) solution file (SRW.sln), which includes 4 projects:

• SRW Library (file SRWLIB.vcxproj),
• SRW Python client / binding (file SRWLClientPython.vcxproj),
• SRW IGOR Pro client / binding (file SRWLClientIgor.vcxproj),
• SRW C demo client (file SRWLClientC.vcxproj), can be found in SRW_Dev/cpp/vc.

The SRWLClientPython project file allows for compiling srwlpy.pyd shared library, i.e. SRW for Python 2.7 or/and 3.x (64-bit or 32-bit); SRWLClientIgor allows for compiling SRW.xop shared library, i.e. SRW for IGOR Pro (32-bit only). Free Microsoft Visual Studio Community 2015 (or later versions) can be used.

To compile SRW library supporting OpenMP based parallel calculations (e.g. for XFEL applications):

• In the Visual C++ Configuration Manager, select "Release_omp" version of the SRWLIB project, then re-compile SRWLIB and SRWLClientPython under the "x64" Solution Platform to produce a 64-bit version of SRW for Python supporting OpenMP based parallel calculations.
• Note that the "Release_omp" version of the SRWLIB project has only a few differences with respect to the standard version: the "_WITH_OMP" preprocessor definition is added to Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions, the "Open MP Support" option is set to "Yes (/openmp)" in Configuration Properties -> C/C++ -> Language -> Open MP Support, and the linking is made with older versions of the FFTW library (FFTW 2.5), whereas the "normal" compilation / linking is with the FFTW 3.8.

II.2. Checking the examples

II.2.1

The SRW for Python examples can be tested using e.g. "IDLE" (Python native GUI). To do so, start this application (e.g. from Windows Start menu), open an example file in it, e.g. SRW_Dev\env\work\srw_python\SRWLIB_Example01.py, and run it from the IDLE.

Alternatively, the example scripts can be executed from the Windows Command Prompt, e.g. from within the SRW_Dev\env\work\srw_python directory. For convenience, correct path to python.exe file may need to be specified in the Windows system PATH variable prior to these tests.

II.2.2

The SRW for IGOR Pro examples can be tested from "SRWE" and "SRWP" menus, "Help" sub-menus, of the IGOR Pro.

III. Compiling and testing SRW Library and its Python binding on Linux

III.1. Compiling SRW library and Python binding

This can be done either using Python "setuptools" module (see section III.1.1 below) or without it (see section III.1.2).

III.1.1. Compiling using Python "setuptools" module

Make sure the "setuptools" module of the Python version you would like to use is properly installed and configured. If this is done, the compilation and installation is simple:

cd SRW_Dev
make all

To compile SRW library supporting OpenMP based parallel calculations (e.g. for XFEL applications) add "MODE=omp" after "make all":

make all MODE=omp

This should compile libsrw.a and srwlpy.so, and copy srwlpy.so to SRW_Dev/env/work/srw_python/

III.1.2. Compiling without "setuptools"

III.1.2.1. Download and compile fftw-2.1.5 or/and fftw-3.3.8 library as required for SRW

Make sure files fftw-3.3.8.tar.gz and fftw-2.1.5.tar.gz are located in SRW_Dev/ext_lib directory (if necessary, download these files from FFTW site, probably http://www.fftw.org/download.html).
Do the following to compile fftw-3.3.8 for using single-precision numbers as required for most FFT-based operations in SRW:

cd SRW_Dev/ext_lib
tar -zxvf fftw-3.3.8.tar.gz
cd fftw-3.3.8
./configure --enable-float --with-pic

Manually (using editor) add -fPIC option to CFLAGS in Makefile

make -j8 && cp .libs/libfftw3f.a ../

Do the following to compile fftw-3.3.8 for using double-precision numbers required for some FFT-based operations in SRW:

cd SRW_Dev/ext_lib
tar -zxvf fftw-3.3.8.tar.gz
cd fftw-3.3.8
./configure --with-pic

Manually (using editor) add -fPIC option to CFLAGS in Makefile

make -j8 && cp .libs/libfftw3.a ../

Do the following to compile fftw-2.1.5 for using single-precision numbers required for supporting OpenMP based parallel calculations in SRW:

cd SRW_Dev/ext_lib
tar -zxvf fftw-2.1.5.tar.gz
cd fftw-2.1.5
./configure --enable-float --with-pic

Manually (using editor) add -fPIC option to CFLAGS in Makefile

make -j8 && cp fftw/.libs/libfftw.a ../

III.1.2.2. Compiling the SRW library and Python binding

cd SRW_Dev/cpp/gcc

Make sure Python 3.3 or higher (or Python 2.7) is installed.
In the SRW_Dev/cpp/gcc/Makefile, modify/correct PYPATH and PYFLAGS variables, i.e. specify path to Python header and library files. Depending on Linux environment, it may also be necessary to modify the name of compiler to be used, e.g.:

CC  = gcc
CXX = g++
#CC  = cc
#CXX = c++

After this, execute the following:

rm libsrw.a
make all

To compile SRW library in the mode supporting OpenMP based parallel calculations (e.g. for XFEL applications) add "MODE=omp" after "make all":

make all MODE=omp

Then copy srwlpy.so to SRW_Dev/env/work/srw_python/:

cp srwlpy.so ../../env/work/srw_python/

III.2. Checking the examples

Make sure the path to Python 3.x (or 2.7) is added to the PATH variable and "srw_python" to PYTHONPATH variable:

export PATH="$PATH:<absolute path to Python 3.x>" # this is not necessary if you install python using the distro's package manager
export PYTHONPAH="$PYTHONPATH:SRW_Dev/env/work/srw_python/" #temporary solution

or

echo "export PYTHONPATH=$PYTHONPATH:SRW_Dev/env/work/srw_python/" >> ~/.bashrc #permanent solution for a single user

Setting up PYTHONPATH allows to import srwlpy module from any directory. Testing of the examples would preferably done in the srw_python directory:

cd SRW_Dev/env/work/srw_python
python SRWLIB_ExampleXX.py

IV. Compiling and testing SRW Library and its Python binding on Mac OSX

Try to follow the steps described in section III (describing options for compiling and testing SRW on Linux).

We were informed that the actions described in III.1.1 lead to successful compilation on OSX 10.14.5 after the following modifications in SRW_Dev/cpp/gcc/Makefile:
Change CXX variable as follows:

#CXX = c++
CXX = g++ -stdlib=libc++ -mmacosx-version-min=10.9

Make sure to explicitly use the C++ compiler (CXX) for compiling all *.cpp files, e.g.:

%.o: $(SH_SRC_PARSE_DIR)/%.cpp
    $(CXX) $(CFLAGS) -c $<

It may be necessary also to set also the CC variable to came value as CXX (?):

#CC = cc
CC = g++ -stdlib=libc++ -mmacosx-version-min=10.9

Previously, we were informed that the actions described in III.1.2.2 lead to successful compilation with gcc/g++ provided by Xcode 10.1, after the following modifications in SRW_Dev/cpp/gcc/Makefile:

CC  = gcc
CXX = g++
#CC  = cc
#CXX = c++
...
PYPATH=/Library/Frameworks/Python.framework/Versions/3.6
PYFLAGS=-I$(PYPATH)/include/python3.6m -I$(PYPATH)/include/python3.6m -L$(PYPATH)/lib/python3.6/config-3.6m-darwin -lpython3.6m -ldl

The correct path and flags can be obtained e.g. by executing from command line:

python3-config --includes --ldflags

and removing the option "-framework"

With earlier versions of Xcode, the following manipulations, consisting in installation of "macports" and obtaining the whole gcc toolchain, were reported to be successful:

sudo port install gcc47

Modify the SRW_Dev/cpp/gcc/Makefile so that CC=<path to macports>/gcc and CXX=<path to macports>/g++, and proceed to the compilation as described in III.1.2.2.

V. Compiling and testing SRW Library and its Python binding on Windows and Linux (via CMake/Pip)

Run the following in a Visual Studio Developer Command Line/Linux Terminal:

cmake -B build
cmake --build build -j

The pip installable version of the package can be obtained by running the following in a Visual Studio Developer Command Line/Linux Terminal:

cd env/python
pip install -e .

VI. GPU Acceleration of SRW

SRW has basic support for GPU acceleration of some routines through CUDA. Compilation of SRW with GPU acceleration requires the CUDA HPC SDK or CUDA Toolkit to be installed. To compile on Linux, run the following from the SRW base directory:

MODE=cuda make

To compile on Windows, ensure that CUDA Toolkit v12.4 is installed, open the SRW solution in Visual Studio, set the target for the SRWLIB project to the _cuda variant and update the library and include paths. You may also have to copy the following DLLs from the HPC SDK or CUDA Toolkit install into the env/python/srwpy directory to resolve the DLL load failed while importing srwlpy error:

• cufft64_11.dll

Authors and Contributors to SRW project

O. Chubar (ESRF - SOLEIL - BNL)
P. Elleaume (ESRF)
J. Chavanne (ESRF)
R. Celestre (ESRF)
P. Dumas (SOLEIL)
O. Marcouille (SOLEIL)
L. Samoylova (E-XFEL)
A. Buzmakov (E-XFEL)
G. Geloni (E-XFEL)
I. Agapov (E-XFEL)
J. Sutter (DIAMOND)
D. Laundy (DIAMOND)
A. He (BNL)
R. Coles (BNL)
R. Li (BNL)
M. Rakitin (BNL)
H. Goel (SBU - BNL)
N. Canestrari (ESRF - BNL)
A. Suvorov (BNL)
R. Reininger (ANL)
X. Shi (ANL)
R. Lindberg (ANL)
L. Rebuffi (ELETTRA - ANL)
D. Bruhwiler (RadiaSoft LLC)
R. Nagler (RadiaSoft LLC)
P. Moeller (RadiaSoft LLC)
B. Nash (RadiaSoft LLC)