Command line tools


Examine the contents of an EuXFEL proposal directory, run directory, or HDF5 file:

# Proposal directory
lsxfel /gpfs/exfel/exp/XMPL/201750/p700000

# Run directory
lsxfel /gpfs/exfel/exp/XMPL/201750/p700000/raw/r0002

# Single file
lsxfel /gpfs/exfel/exp/XMPL/201750/p700000/proc/r0002/CORR-R0034-AGIPD00-S00000.h5
--detail <source-pattern>

Show more detail on the keys and data of the sources selected by a pattern like */XGM/*. Only applies when inspecting a single run or file. Can be used several times to select different patterns.

This option can make lsxfel considerably slower.


Check the structure of an EuXFEL run or HDF5 file:

extra-data-validate /gpfs/exfel/exp/XMPL/201750/p700000/raw/r0002

If it finds problems with the data, the program will produce a list of them and exit with status 1.


Stream data from files in the Karabo bridge format. See Streaming data over ZeroMQ for more information.

karabo-bridge-serve-files /gpfs/exfel/exp/XMPL/201750/p700000/proc/r0005 4321
--source <source>

Only sources matching the string <source> will be streamed. Default is ‘*’, serving as a global wildcard for all sources.

--key <key>

Only data sets keyed by the string <key> will be streamed. Default is ‘*’, serving as a global wildcard for all keys.


If the file data contains multiple detector modules as separate sources, i. e. for big area detectors (AGIPD, LPD and DSSC), append these into one single source.


Add mock timestamps if missing in the original meta-data.

These two options above - appended module sources and dummy timestamps - are required if streamed data shall be provided to OnDA.

-z <type>, --socket-type <type>

The ZMQ socket type to use, one of PUB, PUSH or REP. Default: REP.


Use the infiniband network interface (ib0) if it’s present.


Make a virtual CXI file to access AGIPD/LPD detector data from a specified run:

extra-data-make-virtual-cxi /gpfs/exfel/exp/XMPL/201750/p700000/proc/r0003 -o xmpl-3.cxi
-o <path>, --output <path>

The filename to write. Defaults to creating a file in the proposal’s scratch directory.

--min-modules <number>

Include trains where at least N modules have data (default 9).

--fill-value <dataset> <value>

Set the fill value for dataset (one of data, gain or mask). The defaults are different in different cases:

  • data (raw, uint16): 0

  • data (proc, float32): NaN

  • gain: 0

  • mask: 0xffffffff


Exclude ‘Suspect’ train IDs from the data to assemble. This can fix some problems with bad train IDs.


Check how the files are stored:

extra-data-locality /gpfs/exfel/exp/XMPL/201750/p700000/raw/r0002

The file reading may hang for a long time if files are unavailable or require staging in dCache from the tape. The program helps finding problem files.

If it finds problems with the data locality, the program will produce a list of files located on tape, lost or at unknown locality and exit with the non-zero status.