{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Inspecting available data\n", "\n", "The `.info()` method provides an overview of the data in an opened run or file:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# of trains: 579\n", "Duration: 0:00:57.9\n", "First train ID: 507096934\n", "Last train ID: 507097512\n", "\n", "16 detector modules (SPB_DET_AGIPD1M-1)\n", " e.g. module SPB_DET_AGIPD1M-1 0 : 512 x 128 pixels\n", " SPB_DET_AGIPD1M-1/DET/0CH0:xtdf\n", " 250 frames per train, up to 144750 frames total\n", "\n", "2 instrument sources (excluding detectors):\n", " - SA1_XTD2_XGM/XGM/DOOCS:output\n", " - SPB_XTD9_XGM/XGM/DOOCS:output\n", "\n", "18 control sources:\n", " - ACC_SYS_DOOCS/CTRL/BEAMCONDITIONS\n", " - SA1_XTD2_ATT/MDL/MAIN\n", " - SA1_XTD2_MIRR-1/MOTOR/HMRY\n", " - SA1_XTD2_XGM/XGM/DOOCS\n", " - SPB_IRU_AGIPD1M/MOTOR/Z_STEPPER\n", " - SPB_IRU_AGIPD1M/PSC/HV\n", " - SPB_IRU_AGIPD1M/TSENS/H1_T_EXTHOUS\n", " - SPB_IRU_AGIPD1M/TSENS/H2_T_EXTHOUS\n", " - SPB_IRU_AGIPD1M/TSENS/Q1_T_BLOCK\n", " - SPB_IRU_AGIPD1M/TSENS/Q2_T_BLOCK\n", " - SPB_IRU_AGIPD1M/TSENS/Q3_T_BLOCK\n", " - SPB_IRU_AGIPD1M/TSENS/Q4_T_BLOCK\n", " - SPB_IRU_AGIPD1M1/CTRL/MC1\n", " - SPB_IRU_AGIPD1M1/CTRL/MC2\n", " - SPB_IRU_VAC/GAUGE/GAUGE_FR_6\n", " - SPB_RR_SYS/MDL/BUNCH_PATTERN\n", " - SPB_RR_SYS/TSYS/X2TIMER2\n", " - SPB_XTD9_XGM/XGM/DOOCS\n", "\n" ] } ], "source": [ "from extra_data import RunDirectory\n", "\n", "run = RunDirectory(\"/gpfs/exfel/exp/XMPL/201750/p700000/raw/r0010\")\n", "run.info()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The [lsxfel command](cli.rst#lsxfel) can give similar information at the command line.\n", "\n", "The train IDs included in the run are available as a simple list:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[507096934, 507096935, 507096936, 507096937, 507096938, 507096939, 507096940, 507096941, 507096942, 507096943]\n" ] } ], "source": [ "print(run.train_ids[:10])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And the source names are available as a set:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "frozenset({'ACC_SYS_DOOCS/CTRL/BEAMCONDITIONS',\n", " 'SA1_XTD2_ATT/MDL/MAIN',\n", " 'SA1_XTD2_MIRR-1/MOTOR/HMRY',\n", " 'SA1_XTD2_XGM/XGM/DOOCS',\n", " 'SA1_XTD2_XGM/XGM/DOOCS:output',\n", " 'SPB_DET_AGIPD1M-1/DET/0CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/10CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/11CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/12CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/13CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/14CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/15CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/1CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/2CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/3CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/4CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/5CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/6CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/7CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/8CH0:xtdf',\n", " 'SPB_DET_AGIPD1M-1/DET/9CH0:xtdf',\n", " 'SPB_IRU_AGIPD1M/MOTOR/Z_STEPPER',\n", " 'SPB_IRU_AGIPD1M/PSC/HV',\n", " 'SPB_IRU_AGIPD1M/TSENS/H1_T_EXTHOUS',\n", " 'SPB_IRU_AGIPD1M/TSENS/H2_T_EXTHOUS',\n", " 'SPB_IRU_AGIPD1M/TSENS/Q1_T_BLOCK',\n", " 'SPB_IRU_AGIPD1M/TSENS/Q2_T_BLOCK',\n", " 'SPB_IRU_AGIPD1M/TSENS/Q3_T_BLOCK',\n", " 'SPB_IRU_AGIPD1M/TSENS/Q4_T_BLOCK',\n", " 'SPB_IRU_AGIPD1M1/CTRL/MC1',\n", " 'SPB_IRU_AGIPD1M1/CTRL/MC2',\n", " 'SPB_IRU_VAC/GAUGE/GAUGE_FR_6',\n", " 'SPB_RR_SYS/MDL/BUNCH_PATTERN',\n", " 'SPB_RR_SYS/TSYS/X2TIMER2',\n", " 'SPB_XTD9_XGM/XGM/DOOCS',\n", " 'SPB_XTD9_XGM/XGM/DOOCS:output'})" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "run.all_sources" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can see control and instrument sources separately,\n", "but for data analysis this distinction is often not important." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "assert run.all_sources == (run.control_sources | run.instrument_sources)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Within each source, the data is organised under keys.\n", "You can look at one source and use the `.keys()` method to see its keys:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'data.intensityAUXSa1TD',\n", " 'data.intensityAUXSa3TD',\n", " 'data.intensityAUXTD',\n", " 'data.intensitySa1SigmaTD',\n", " 'data.intensitySa1TD',\n", " 'data.intensitySa3SigmaTD',\n", " 'data.intensitySa3TD',\n", " 'data.intensitySigmaTD',\n", " 'data.intensityTD',\n", " 'data.trainId',\n", " 'data.xSa1SigmaTD',\n", " 'data.xSa1TD',\n", " 'data.xSa3SigmaTD',\n", " 'data.xSa3TD',\n", " 'data.xSigmaTD',\n", " 'data.xTD',\n", " 'data.ySa1SigmaTD',\n", " 'data.ySa1TD',\n", " 'data.ySa3SigmaTD',\n", " 'data.ySa3TD',\n", " 'data.ySigmaTD',\n", " 'data.yTD'}" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "run['SA1_XTD2_XGM/XGM/DOOCS:output'].keys()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instrument sources may have multiple entries recorded for each train,\n", "and may be missing data for some trains.\n", "You can see how many entries there are for each train with `.data_counts()`.\n", "E.g. for this AGIPD detector module, the counts are the number of frames in each train:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "507096934 0\n", "507096935 0\n", "507096936 0\n", "507096937 0\n", "507096938 0\n", " ... \n", "507097185 250\n", "507097186 250\n", "507097187 250\n", "507097188 250\n", "507097189 250\n", "Length: 256, dtype: uint64" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "run['SPB_DET_AGIPD1M-1/DET/11CH0:xtdf', 'image.data'].data_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This method returns a pandas series.\n", "The index (the numbers shown on the left) are train IDs." ] } ], "metadata": { "kernelspec": { "display_name": "xfel", "language": "python", "name": "xfel" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 2 }