{ "cells": [ { "cell_type": "markdown", "id": "3ef277a8", "metadata": {}, "source": [ "# Summarizing pupillometric data\n", "\n", "`pypillometry` summarizes a dataset at several levels:\n", "\n", "- **one dataset**: its contents (`print(d)`, `summary()`), the quality of\n", " the data (`quality_report()`) and how much of it is masked, and why\n", " (`layer_stats()`)\n", "- **several datasets**: the same summaries as one table with a row per\n", " dataset, e.g., to find participants with poor data\n", "- **trials**: one value per event or trial (`stat_per_event()`), together\n", " with the events and their metadata\n", "\n", "Many of these results are dicts that are displayed as a table: per\n", "channel (e.g., `layer_stats()`: channel -> layer -> statistics) or with\n", "one row per trial (a dict of columns). `.to_pandas()` converts them to a\n", "`pandas.DataFrame`." ] }, { "cell_type": "code", "execution_count": 1, "id": "b98be3f3", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:19.830326Z", "iopub.status.busy": "2026-11-10T08:27:19.829748Z", "iopub.status.idle": "2026-11-10T08:27:20.349248Z", "shell.execute_reply": "2026-11-10T08:27:20.347756Z" } }, "outputs": [], "source": [ "import pypillometry as pp\n", "from pypillometry import u\n", "import pandas as pd\n", "\n", "pp.logging_set_level(\"WARNING\")" ] }, { "cell_type": "markdown", "id": "c02c1890", "metadata": {}, "source": [ "## One dataset\n", "\n", "We detect and interpolate blinks and mask the gaze outside the screen.\n", "Printing the object gives an overview of its contents and the history of\n", "operations:" ] }, { "cell_type": "code", "execution_count": 2, "id": "7c00c6ae", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:20.353157Z", "iopub.status.busy": "2026-11-10T08:27:20.352593Z", "iopub.status.idle": "2026-11-10T08:27:21.536709Z", "shell.execute_reply": "2026-11-10T08:27:21.536009Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "EyeData(rlmw_002_short, 1.4MiB):\n", " n : 20465\n", " sampling_rate : 500.0\n", " eyes : ['left', 'right']\n", " data : ['left_x', 'left_y', 'left_pupil', 'right_x', 'right_y', 'right_pupil']\n", " nevents : 40\n", " experimental_setup: {'screen_resolution': (1280, 1024), 'physical_screen_size_mm': (300.0, 200.0), 'eye_to_screen_distance_mm': 600.0, 'camera_position': 'not set'}\n", " calibration : not available\n", " duration_minutes : 0.6822\n", " start_min : 0.0\n", " end_min : 0.6821\n", " params : {}\n", " nblinks : {'left_pupil': 14, 'right_pupil': 13}\n", " blinks : {'left_pupil': {'n': 14, 'mean': 217.1429, 'sd': 236.4701, 'min': 48.0, 'max': 984.0, 'total_duration': 3040.0, 'units': 'ms'}, 'right_pupil': {'n': 13, 'mean': 138.0, 'sd': 84.8564, 'min': 44.0, 'max': 404.0, 'total_duration': 1794.0, 'units': 'ms'}}\n", " glimpse : EyeDataDict(vars=6,n=20465,shape=(20465,)): \n", " left_x (float64): 655.6, 655.7, 655.0, 654.5, 655.0...\n", " left_y (float64): 599.9, 598.9, 597.6, 597.8, 597.8...\n", " left_pupil (float64): 1121.0, 1122.0, 1124.0, 1126.0, 1126.0...\n", " right_x (float64): 773.1, 773.8, 774.8, 776.3, 776.7...\n", " right_y (float64): 594.2, 593.9, 596.5, 597.4, 597.3...\n", " right_pupil (float64): 949.0, 951.0, 956.0, 959.0, 960.0...\n", "\n", " History:\n", " *\n", " └ reset_time()\n", " └ pupil.blinks_detect()\n", " └ pupil.blinks_interpolate()\n", " └ gaze.mask_offscreen()\n", "\n" ] } ], "source": [ "d = pp.get_example_data(\"rlmw_002_short\")\n", "d = d | pp.pupil.blinks_detect() | pp.pupil.blinks_interpolate() | pp.gaze.mask_offscreen()\n", "print(d)" ] }, { "cell_type": "markdown", "id": "72e75be7", "metadata": {}, "source": [ "`summary()` returns the same information as a `dict`:" ] }, { "cell_type": "code", "execution_count": 3, "id": "de9b8c61", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.540828Z", "iopub.status.busy": "2026-11-10T08:27:21.540461Z", "iopub.status.idle": "2026-11-10T08:27:21.550627Z", "shell.execute_reply": "2026-11-10T08:27:21.549511Z" } }, "outputs": [ { "data": { "text/plain": [ "{'name': 'rlmw_002_short',\n", " 'n': 20465,\n", " 'sampling_rate': 500.0,\n", " 'eyes': ['left', 'right'],\n", " 'data': ['left_x',\n", " 'left_y',\n", " 'left_pupil',\n", " 'right_x',\n", " 'right_y',\n", " 'right_pupil'],\n", " 'nevents': 40,\n", " 'experimental_setup': {'screen_resolution': (1280, 1024),\n", " 'physical_screen_size_mm': (300.0, 200.0),\n", " 'eye_to_screen_distance_mm': 600.0,\n", " 'camera_position': 'not set'},\n", " 'calibration': 'not available',\n", " 'duration_minutes': 0.6821666666666667,\n", " 'start_min': 0.0,\n", " 'end_min': 0.6821333333333333,\n", " 'params': {},\n", " 'nblinks': {'left_pupil': 14, 'right_pupil': 13},\n", " 'blinks': {'left_pupil': {'n': 14,\n", " 'mean': 217.14285714285714,\n", " 'sd': 236.4701301411652,\n", " 'min': 48.0,\n", " 'max': 984.0,\n", " 'total_duration': 3040.0,\n", " 'units': 'ms'},\n", " 'right_pupil': {'n': 13,\n", " 'mean': 138.0,\n", " 'sd': 84.85643985352782,\n", " 'min': 44.0,\n", " 'max': 404.0,\n", " 'total_duration': 1794.0,\n", " 'units': 'ms'}}}" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "d.summary()" ] }, { "cell_type": "markdown", "id": "ac9f8bcc", "metadata": {}, "source": [ "### Data quality\n", "\n", "The quality report rates missing data, noise, blinks, gaze precision,\n", "the agreement of the eyes and the calibration as \"good\", \"ok\", \"poor\" or\n", "\"bad\" (see [Data quality](../user_guide/quality.rst)):" ] }, { "cell_type": "code", "execution_count": 4, "id": "2545fbb5", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.553520Z", "iopub.status.busy": "2026-11-10T08:27:21.553282Z", "iopub.status.idle": "2026-11-10T08:27:21.622109Z", "shell.execute_reply": "2026-11-10T08:27:21.620518Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "

Quality report — rlmw_002_short\n", " (EyeData)

\n", " ok\n", "
\n", "\n", "
Recording
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
duration0.682 min
sampling rate500 Hz
samples20465
events40
time gaps0
\n", "
Pupil
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
leftright
missing data0.0% good0.0% good
dropouts0 /min good0 /min good
longest dropout0 s good0 s good
noise0.094% good0.11% good
noise (SD)1.05 1.05
jumps0 /min good0 /min good
\n", "
Gaze
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
leftright
missing data2.7% good1.9% good
off-screen0.21% good0.24% good
precision0.0202 deg good0.02 deg good
\n", "
Binocular
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left/right
pupil correlation0.914 good
gaze disparity1.61 deg ok
\n", "
Blinks
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
leftright
number14 13
blink rate20.5 /min ok19.1 /min good
mean duration217 ms 138 ms
time in blinks7.4% good4.4% good
long (>500 ms)1 0
\n", "
Artifacts
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
leftright
pupil masked0.0% good0.0% good
interpolated9.9% good6.0% good
gaze masked2.9% good2.1% good
masked off-screen0.21% 0.24%
other artifacts0.0% 0.0%
\n", "\n", "\n", "
\n", " Thresholds\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pupil.missing_pctgood <= 10, ok <= 20, poor <= 40, else bad
pupil.dropout_rategood <= 30, ok <= 60, poor <= 150, else bad
pupil.longest_dropoutgood <= 2, ok <= 5, poor <= 30, else bad
pupil.noisegood <= 0.5, ok <= 1, poor <= 2, else bad
pupil.jump_rategood <= 1, ok <= 10, poor <= 100, else bad
gaze.missing_pctgood <= 10, ok <= 20, poor <= 40, else bad
gaze.dropout_rategood <= 30, ok <= 60, poor <= 150, else bad
gaze.longest_dropoutgood <= 2, ok <= 5, poor <= 30, else bad
gaze.offscreen_pctgood <= 5, ok <= 15, poor <= 30, else bad
gaze.precision_deggood <= 0.05, ok <= 0.1, poor <= 0.2, else bad
binocular.pupil_correlationgood >= 0.9, ok >= 0.8, poor >= 0.6, else bad
binocular.gaze_disparity_deggood <= 1, ok <= 2, poor <= 3, else bad
blinks.rategood <= 20, ok <= 35, poor <= 70, else bad
blinks.time_pctgood <= 10, ok <= 20, poor <= 35, else bad
artifacts.pupil_masked_pctgood <= 10, ok <= 20, poor <= 35, else bad
artifacts.gaze_masked_pctgood <= 10, ok <= 20, poor <= 35, else bad
artifacts.interpolated_pctgood <= 10, ok <= 20, poor <= 30, else bad
calibration.mean_errorgood <= 0.5, ok <= 1, poor <= 1.5, else bad
calibration.max_errorgood <= 1, ok <= 1.5, poor <= 2, else bad
\n", "
The overall label is the worst label of all measures. Hover over a measure for a description.
\n", "
\n", "
" ], "text/plain": [ "Quality report: rlmw_002_short (EyeData)\n", "Overall: \u001b[33mOK\u001b[0m\n", "\n", "Recording\n", " duration 0.682 min\n", " sampling rate 500 Hz\n", " samples 20465\n", " events 40\n", " time gaps 0\n", "\n", "Pupil left right\n", " missing data 0.0% \u001b[32mgood\u001b[0m 0.0% \u001b[32mgood\u001b[0m\n", " dropouts 0 /min \u001b[32mgood\u001b[0m 0 /min \u001b[32mgood\u001b[0m\n", " longest dropout 0 s \u001b[32mgood\u001b[0m 0 s \u001b[32mgood\u001b[0m\n", " noise 0.094% \u001b[32mgood\u001b[0m 0.11% \u001b[32mgood\u001b[0m\n", " noise (SD) 1.05 1.05\n", " jumps 0 /min \u001b[32mgood\u001b[0m 0 /min \u001b[32mgood\u001b[0m\n", "\n", "Gaze left right\n", " missing data 2.7% \u001b[32mgood\u001b[0m 1.9% \u001b[32mgood\u001b[0m\n", " off-screen 0.21% \u001b[32mgood\u001b[0m 0.24% \u001b[32mgood\u001b[0m\n", " precision 0.0202 deg \u001b[32mgood\u001b[0m 0.02 deg \u001b[32mgood\u001b[0m\n", "\n", "Binocular left/right\n", " pupil correlation 0.914 \u001b[32mgood\u001b[0m\n", " gaze disparity 1.61 deg \u001b[33mok \u001b[0m\n", "\n", "Blinks left right\n", " number 14 13\n", " blink rate 20.5 /min \u001b[33mok \u001b[0m 19.1 /min \u001b[32mgood\u001b[0m\n", " mean duration 217 ms 138 ms\n", " time in blinks 7.4% \u001b[32mgood\u001b[0m 4.4% \u001b[32mgood\u001b[0m\n", " long (>500 ms) 1 0\n", "\n", "Artifacts left right\n", " pupil masked 0.0% \u001b[32mgood\u001b[0m 0.0% \u001b[32mgood\u001b[0m\n", " interpolated 9.9% \u001b[32mgood\u001b[0m 6.0% \u001b[32mgood\u001b[0m\n", " gaze masked 2.9% \u001b[32mgood\u001b[0m 2.1% \u001b[32mgood\u001b[0m\n", " masked off-screen 0.21% 0.24%\n", " other artifacts 0.0% 0.0%" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "report = d.quality_report()\n", "report" ] }, { "cell_type": "markdown", "id": "6b5c6187", "metadata": {}, "source": [ "`report.issues()` lists the measures that are \"poor\" or \"bad\" (none here),\n", "and `report.to_dict(flat=True)` gives all values as a flat `dict`, e.g.,\n", "for a table of many datasets (see below).\n", "\n", "### Masked data\n", "\n", "`layer_stats()` describes each mask layer of each channel: the number of\n", "stretches, their durations and the percentage of the samples they cover.\n", "The entry `masked` counts all unusable samples of a channel (see\n", "[Missing data](../user_guide/missing_data.rst)). It is a dict (channel ->\n", "layer -> statistics) that is displayed as a table:" ] }, { "cell_type": "code", "execution_count": 5, "id": "9a03a083", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.626244Z", "iopub.status.busy": "2026-11-10T08:27:21.625439Z", "iopub.status.idle": "2026-11-10T08:27:21.665828Z", "shell.execute_reply": "2026-11-10T08:27:21.664366Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nmeansdminmaxtotal_durationunitspercent
channellayer
left_xoffscreen140.0060000.0041400.0020.0160.084sec0.205228
masked110.1096360.0301490.0580.1461.206sec2.946494
left_yoffscreen140.0060000.0041400.0020.0160.084sec0.205228
masked110.1096360.0301490.0580.1461.206sec2.946494
left_pupilblink140.2171430.2364700.0480.9843.040sec7.427315
interpolated110.3694550.2982800.1161.0264.064sec9.929147
masked0NaNNaNNaNNaN0.000sec0.000000
right_xoffscreen150.0065330.0039640.0020.0160.098sec0.239433
masked100.0876000.0257340.0420.1200.876sec2.140239
right_yoffscreen150.0065330.0039640.0020.0160.098sec0.239433
masked100.0876000.0257340.0420.1200.876sec2.140239
right_pupilblink130.1380000.0848560.0440.4041.794sec4.383093
interpolated130.1890770.0815530.0960.4442.458sec6.005375
masked0NaNNaNNaNNaN0.000sec0.000000
\n", "
" ], "text/plain": [ " n mean sd min max \\\n", "channel layer \n", "left_x offscreen 14 0.006000 0.004140 0.002 0.016 \n", " masked 11 0.109636 0.030149 0.058 0.146 \n", "left_y offscreen 14 0.006000 0.004140 0.002 0.016 \n", " masked 11 0.109636 0.030149 0.058 0.146 \n", "left_pupil blink 14 0.217143 0.236470 0.048 0.984 \n", " interpolated 11 0.369455 0.298280 0.116 1.026 \n", " masked 0 NaN NaN NaN NaN \n", "right_x offscreen 15 0.006533 0.003964 0.002 0.016 \n", " masked 10 0.087600 0.025734 0.042 0.120 \n", "right_y offscreen 15 0.006533 0.003964 0.002 0.016 \n", " masked 10 0.087600 0.025734 0.042 0.120 \n", "right_pupil blink 13 0.138000 0.084856 0.044 0.404 \n", " interpolated 13 0.189077 0.081553 0.096 0.444 \n", " masked 0 NaN NaN NaN NaN \n", "\n", " total_duration units percent \n", "channel layer \n", "left_x offscreen 0.084 sec 0.205228 \n", " masked 1.206 sec 2.946494 \n", "left_y offscreen 0.084 sec 0.205228 \n", " masked 1.206 sec 2.946494 \n", "left_pupil blink 3.040 sec 7.427315 \n", " interpolated 4.064 sec 9.929147 \n", " masked 0.000 sec 0.000000 \n", "right_x offscreen 0.098 sec 0.239433 \n", " masked 0.876 sec 2.140239 \n", "right_y offscreen 0.098 sec 0.239433 \n", " masked 0.876 sec 2.140239 \n", "right_pupil blink 1.794 sec 4.383093 \n", " interpolated 2.458 sec 6.005375 \n", " masked 0.000 sec 0.000000 " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stats = d.layer_stats(units=\"sec\")\n", "stats" ] }, { "cell_type": "code", "execution_count": 6, "id": "97bd8f9c", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.669439Z", "iopub.status.busy": "2026-11-10T08:27:21.668929Z", "iopub.status.idle": "2026-11-10T08:27:21.675152Z", "shell.execute_reply": "2026-11-10T08:27:21.673870Z" } }, "outputs": [ { "data": { "text/plain": [ "7.4273149279257265" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stats[\"left_pupil\"][\"blink\"][\"percent\"] # % of the samples in blinks" ] }, { "cell_type": "markdown", "id": "44dd71ad", "metadata": {}, "source": [ "After the interpolation, the pupil channels have no unusable samples left\n", "(`masked`: 0 stretches), because interpolated samples are usable; the gaze is still\n", "masked during blinks and off the screen.\n", "\n", "`blink_stats()` is a shortcut for the blink rows. Durations of the\n", "recording are given by `get_duration()` and, if periods are excluded\n", "(e.g., breaks), `excluded_duration()`:" ] }, { "cell_type": "code", "execution_count": 7, "id": "0e34552b", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.678451Z", "iopub.status.busy": "2026-11-10T08:27:21.678011Z", "iopub.status.idle": "2026-11-10T08:27:21.687121Z", "shell.execute_reply": "2026-11-10T08:27:21.685853Z" } }, "outputs": [ { "data": { "text/plain": [ "({'n': 14,\n", " 'mean': 217.14285714285714,\n", " 'sd': 236.4701301411652,\n", " 'min': 48.0,\n", " 'max': 984.0,\n", " 'total_duration': 3040.0,\n", " 'units': 'ms'},\n", " 0.6821666666666667,\n", " 0.0)" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "d.blink_stats()[\"left_pupil\"], d.get_duration(\"min\"), d.excluded_duration(\"min\")" ] }, { "cell_type": "markdown", "id": "d1ca2e94", "metadata": {}, "source": [ "## Several datasets\n", "\n", "In a study, there is one dataset per participant. Here, we use three\n", "example recordings and detect blinks in each of them:" ] }, { "cell_type": "code", "execution_count": 8, "id": "277515a3", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.690380Z", "iopub.status.busy": "2026-11-10T08:27:21.689991Z", "iopub.status.idle": "2026-11-10T08:27:21.808953Z", "shell.execute_reply": "2026-11-10T08:27:21.807497Z" } }, "outputs": [], "source": [ "datasets = [pp.get_example_data(\"rlmw_002_short\"),\n", " pp.EyeData.from_file(\"../../data/test.pd\"),\n", " pp.EyeData.from_file(\"../../data/example_shortblinks.pd\")]\n", "datasets = [d | pp.pupil.blinks_detect() for d in datasets]" ] }, { "cell_type": "markdown", "id": "d4c140e8", "metadata": {}, "source": [ "A table of their summaries (some of the columns):" ] }, { "cell_type": "code", "execution_count": 9, "id": "3cf3503f", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.812561Z", "iopub.status.busy": "2026-11-10T08:27:21.812142Z", "iopub.status.idle": "2026-11-10T08:27:21.840686Z", "shell.execute_reply": "2026-11-10T08:27:21.838928Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namensampling_rateduration_minutesneventsnblinks
0rlmw_002_short20465500.00.68216740{'left_pupil': 14, 'right_pupil': 13}
1test_ro_ka_si_hu60001500.02.00003356{'left_pupil': 65}
255910_1_wu12501500.00.41670010{'left_pupil': 22}
\n", "
" ], "text/plain": [ " name n sampling_rate duration_minutes nevents \\\n", "0 rlmw_002_short 20465 500.0 0.682167 40 \n", "1 test_ro_ka_si_hu 60001 500.0 2.000033 56 \n", "2 55910_1_wu 12501 500.0 0.416700 10 \n", "\n", " nblinks \n", "0 {'left_pupil': 14, 'right_pupil': 13} \n", "1 {'left_pupil': 65} \n", "2 {'left_pupil': 22} " ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tab = pd.DataFrame([d.summary() for d in datasets])\n", "tab[[\"name\", \"n\", \"sampling_rate\", \"duration_minutes\", \"nevents\", \"nblinks\"]]" ] }, { "cell_type": "markdown", "id": "973e1eec", "metadata": {}, "source": [ "The quality reports as a table, one row per dataset, with the overall\n", "label and the missing pupil data of the left eye:" ] }, { "cell_type": "code", "execution_count": 10, "id": "350f5e08", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.844923Z", "iopub.status.busy": "2026-11-10T08:27:21.844436Z", "iopub.status.idle": "2026-11-10T08:27:21.894948Z", "shell.execute_reply": "2026-11-10T08:27:21.893580Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameoverallpupil.left.missing_pctpupil.left.missing_pct.labelblinks.left.rateblinks.left.rate.label
0rlmw_002_shortok2.741266good20.522844ok
1test_ro_ka_si_hupoor0.000000good32.499458ok
255910_1_wubad0.000000good52.795776poor
\n", "
" ], "text/plain": [ " name overall pupil.left.missing_pct \\\n", "0 rlmw_002_short ok 2.741266 \n", "1 test_ro_ka_si_hu poor 0.000000 \n", "2 55910_1_wu bad 0.000000 \n", "\n", " pupil.left.missing_pct.label blinks.left.rate blinks.left.rate.label \n", "0 good 20.522844 ok \n", "1 good 32.499458 ok \n", "2 good 52.795776 poor " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "quality = pd.DataFrame([d.quality_report().to_dict(flat=True) for d in datasets])\n", "quality[[\"name\", \"overall\", \"pupil.left.missing_pct\", \"pupil.left.missing_pct.label\",\n", " \"blinks.left.rate\", \"blinks.left.rate.label\"]]" ] }, { "cell_type": "markdown", "id": "bb30e15e", "metadata": {}, "source": [ "Datasets that need a closer look can be selected by their labels, and\n", "`issues()` tells what is wrong with them:" ] }, { "cell_type": "code", "execution_count": 11, "id": "98e7f6ac", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.898384Z", "iopub.status.busy": "2026-11-10T08:27:21.897951Z", "iopub.status.idle": "2026-11-10T08:27:21.907634Z", "shell.execute_reply": "2026-11-10T08:27:21.906685Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameoverall
1test_ro_ka_si_hupoor
255910_1_wubad
\n", "
" ], "text/plain": [ " name overall\n", "1 test_ro_ka_si_hu poor\n", "2 55910_1_wu bad" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "quality[quality[\"overall\"].isin([\"poor\", \"bad\"])][[\"name\", \"overall\"]]" ] }, { "cell_type": "code", "execution_count": 12, "id": "9fded822", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.910924Z", "iopub.status.busy": "2026-11-10T08:27:21.910683Z", "iopub.status.idle": "2026-11-10T08:27:21.919361Z", "shell.execute_reply": "2026-11-10T08:27:21.918344Z" } }, "outputs": [ { "data": { "text/plain": [ "[('pupil',\n", " 'left',\n", " QualityMetric(name='jump_rate', title='jumps', value=155.98752099832015, unit='/min', label='bad', description='number of changes larger than 5% of the median pupil size from one sample to the next, per minute (ignoring 50 ms around missing data); indicates artifacts, e.g., during saccades', thresholds=(1, 10, 100))),\n", " ('blinks',\n", " 'left',\n", " QualityMetric(name='rate', title='blink rate', value=52.795776337892974, unit='/min', label='poor', description='number of detected blinks per minute (median across people about 10 during screen tasks and 13 at rest; varies a lot between people)', thresholds=(20, 35, 70))),\n", " ('blinks',\n", " 'left',\n", " QualityMetric(name='time_pct', title='time in blinks', value=np.float64(21.926245900327974), unit='%', label='poor', description='percentage of the recording covered by blinks (and the samples around them that were removed)', thresholds=(10, 20, 35))),\n", " ('artifacts',\n", " 'left',\n", " QualityMetric(name='pupil_masked_pct', title='pupil masked', value=np.float64(21.926245900327974), unit='%', label='poor', description=\"percentage of masked pupil samples (missing data, blinks and other artifacts; interpolated samples only with the 'strict' mask policy)\", thresholds=(10, 20, 35)))]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "datasets[2].quality_report().issues()" ] }, { "cell_type": "markdown", "id": "3d0041ae", "metadata": {}, "source": [ "The layer statistics of all datasets in one table, with a column for the\n", "dataset:" ] }, { "cell_type": "code", "execution_count": 13, "id": "be304432", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.923145Z", "iopub.status.busy": "2026-11-10T08:27:21.922933Z", "iopub.status.idle": "2026-11-10T08:27:21.950796Z", "shell.execute_reply": "2026-11-10T08:27:21.949641Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
channellayernmeansdminmaxtotal_durationunitspercentname
0left_pupilblink14217.142857236.47013048.0984.03040.0ms7.427315rlmw_002_short
1left_pupilmasked14217.142857236.47013048.0984.03040.0ms7.427315rlmw_002_short
2right_pupilblink13138.00000084.85644044.0404.01794.0ms4.383093rlmw_002_short
3right_pupilmasked13138.00000084.85644044.0404.01794.0ms4.383093rlmw_002_short
4left_pupilblink65225.230769247.91206724.0990.014640.0ms12.199797test_ro_ka_si_hu
5left_pupilmasked65225.230769247.91206724.0990.014640.0ms12.199797test_ro_ka_si_hu
6left_pupilblink22249.18181868.717231184.0538.05482.0ms21.92624655910_1_wu
7left_pupilmasked22249.18181868.717231184.0538.05482.0ms21.92624655910_1_wu
\n", "
" ], "text/plain": [ " channel layer n mean sd min max \\\n", "0 left_pupil blink 14 217.142857 236.470130 48.0 984.0 \n", "1 left_pupil masked 14 217.142857 236.470130 48.0 984.0 \n", "2 right_pupil blink 13 138.000000 84.856440 44.0 404.0 \n", "3 right_pupil masked 13 138.000000 84.856440 44.0 404.0 \n", "4 left_pupil blink 65 225.230769 247.912067 24.0 990.0 \n", "5 left_pupil masked 65 225.230769 247.912067 24.0 990.0 \n", "6 left_pupil blink 22 249.181818 68.717231 184.0 538.0 \n", "7 left_pupil masked 22 249.181818 68.717231 184.0 538.0 \n", "\n", " total_duration units percent name \n", "0 3040.0 ms 7.427315 rlmw_002_short \n", "1 3040.0 ms 7.427315 rlmw_002_short \n", "2 1794.0 ms 4.383093 rlmw_002_short \n", "3 1794.0 ms 4.383093 rlmw_002_short \n", "4 14640.0 ms 12.199797 test_ro_ka_si_hu \n", "5 14640.0 ms 12.199797 test_ro_ka_si_hu \n", "6 5482.0 ms 21.926246 55910_1_wu \n", "7 5482.0 ms 21.926246 55910_1_wu " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.concat([d.layer_stats(variables=\"pupil\").to_pandas().reset_index().assign(name=d.name)\n", " for d in datasets], ignore_index=True)" ] }, { "cell_type": "markdown", "id": "ea0647e8", "metadata": {}, "source": [ "## Trials\n", "\n", "`stat_per_event()` computes one value per event, e.g., the mean pupil\n", "size in the second after each stimulus. Combined with the events and\n", "their metadata (see [Event metadata](event_metadata.ipynb)), this gives a\n", "table with one row per trial, e.g., for a statistical model:" ] }, { "cell_type": "code", "execution_count": 14, "id": "40831963", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.953776Z", "iopub.status.busy": "2026-11-10T08:27:21.953536Z", "iopub.status.idle": "2026-11-10T08:27:21.968760Z", "shell.execute_reply": "2026-11-10T08:27:21.967568Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
onsetlabelleft_pupilright_pupil
05677.0S_1_1_51002.974000831.318000
19575.0S_2_6_21179.907489966.048000
212574.0S_3_2_11269.6500001077.462000
316473.0S_4_4_11153.294000956.178000
420388.0S_5_3_51098.470000912.224000
523987.0S_6_4_51096.668000914.322000
627002.0S_7_3_1987.372000832.730000
730601.0S_8_6_51077.352000915.889262
834799.0S_9_4_6955.512000797.592000
938415.0S_10_2_51064.282000888.112000
\n", "
" ], "text/plain": [ " onset label left_pupil right_pupil\n", "0 5677.0 S_1_1_5 1002.974000 831.318000\n", "1 9575.0 S_2_6_2 1179.907489 966.048000\n", "2 12574.0 S_3_2_1 1269.650000 1077.462000\n", "3 16473.0 S_4_4_1 1153.294000 956.178000\n", "4 20388.0 S_5_3_5 1098.470000 912.224000\n", "5 23987.0 S_6_4_5 1096.668000 914.322000\n", "6 27002.0 S_7_3_1 987.372000 832.730000\n", "7 30601.0 S_8_6_5 1077.352000 915.889262\n", "8 34799.0 S_9_4_6 955.512000 797.592000\n", "9 38415.0 S_10_2_5 1064.282000 888.112000" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "d = datasets[0]\n", "epochs = d.get_epochs(\"S_\", (0, 1000) * u.ms)\n", "means = d.stat_per_event(epochs, variables=\"pupil\")\n", "trials = pp.Table({**epochs.events.to_table(), **means})\n", "trials" ] }, { "cell_type": "markdown", "id": "8461c29f", "metadata": {}, "source": [ "For segments around events (see the\n", "[ERPD tutorial](erpds.ipynb)), `summary()` gives their shape and the\n", "missing data per channel; the measures of gaze in regions of interest per\n", "trial (`seg.gaze.dwell_time()`, `seg.gaze.first_entry()`) are tables like\n", "the one above (see [Regions of interest](rois.ipynb)):" ] }, { "cell_type": "code", "execution_count": 15, "id": "15c3be5f", "metadata": { "execution": { "iopub.execute_input": "2026-11-10T08:27:21.971610Z", "iopub.status.busy": "2026-11-10T08:27:21.971332Z", "iopub.status.idle": "2026-11-10T08:27:21.978805Z", "shell.execute_reply": "2026-11-10T08:27:21.977798Z" } }, "outputs": [ { "data": { "text/plain": [ "{'name': 'S_',\n", " 'channels': ['left_pupil', 'right_pupil'],\n", " 'n_segments': 10,\n", " 'n_timepoints': 1500,\n", " 'window': (-500.0, 2498.0),\n", " 'percent_masked': {'left_pupil': 9.806666666666667,\n", " 'right_pupil': 5.733333333333333},\n", " 'sampling_rate': 500.0,\n", " 'n_averages': 0,\n", " 'n_trials': 10}" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg = d.get_segments(d.get_epochs(\"S_\", (-500, 2500) * u.ms), variables=\"pupil\")\n", "seg.summary()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.15.1" } }, "nbformat": 4, "nbformat_minor": 5 }