Intervals#
Many analyses look at stretches of data rather than single time points: the second after a stimulus, the time between trial start and response, blinks, or periods with bad data. In pypillometry, such stretches are stored as Intervals, a list of (start, end) pairs with their units and, if they belong to events, the event labels. Windows around events are Epochs: the event onsets with one window.
This tutorial shows where intervals come from, how to change and combine them, and how to use them in an analysis.
[1]:
import pypillometry as pp
from pypillometry import u
import numpy as np
import matplotlib.pyplot as plt
pp.logging_set_level("WARNING")
d = pp.get_example_data("rlmw_002_short")
Epochs: windows around events#
get_epochs() creates one window for each selected event. The first argument selects the events (a string contained in the label, a function, or an Events object; see the Events tutorial). The second gives the window relative to the event onset, here from 200 ms before until 1000 ms after each stimulus. The result is an Epochs object (onsets and window in ms); it can be used wherever intervals are expected, and stim.to_intervals() gives its spans as
Intervals:
[2]:
stim = d.get_epochs("S_", (-200, 1000) * u.ms)
stim
[2]:
Epochs 'S_' | 10 epochs, window (-200, 1000) ms
as_pandas() shows the epochs as a table, together with the events they belong to:
[3]:
stim.as_pandas().head()
[3]:
| onset | start | end | label | |
|---|---|---|---|---|
| 0 | 5677.0 | 5477.0 | 6677.0 | S_1_1_5 |
| 1 | 9575.0 | 9375.0 | 10575.0 | S_2_6_2 |
| 2 | 12574.0 | 12374.0 | 13574.0 | S_3_2_1 |
| 3 | 16473.0 | 16273.0 | 17473.0 | S_4_4_1 |
| 4 | 20388.0 | 20188.0 | 21388.0 | S_5_3_5 |
Note: Plain numbers without units are ms, like everywhere in pypillometry. Sample indices need units="samples": at 500 Hz, (-200, 1000) with units="samples" means 400 ms before until 2000 ms after the event.
[4]:
d.get_epochs("S_", (-200, 1000)) # plain numbers are ms
[4]:
Epochs 'S_' | 10 epochs, window (-200, 1000) ms
Intervals between two events#
get_intervals() takes two selectors, start and end; each interval spans from a start event to the next end event. Here, from the start of each trial (F events) to the feedback (R events). By default, an interval spans exactly from the first to the second event; a window extends or shortens it (e.g., window=(-200, 500) * u.ms starts 200 ms earlier and ends 500 ms later):
[5]:
trials = d.get_intervals(start="F_", end="R_", units="ms")
trials.as_pandas().head(3)
[5]:
| start | end | duration | units | label | onset | |
|---|---|---|---|---|---|---|
| 0 | 5177.0 | 7010.0 | 1833.0 | ms | F_1 to R_1_15 | 5177.0 |
| 1 | 8776.0 | 10808.0 | 2032.0 | ms | F_2 to R_2_20 | 8776.0 |
| 2 | 12374.0 | 13957.0 | 1583.0 | ms | F_3 to R_3_18 | 12374.0 |
stats() summarizes the durations:
[6]:
trials.stats()
[6]:
{'n': 10,
'mean': 1791.1,
'sd': 219.8792623236671,
'min': 1333.0,
'max': 2033.0,
'total_duration': 17911.0,
'units': 'ms'}
Other sources of intervals#
Detection methods store their results as mask layers; getters return them as intervals. For example, after pupil.blinks_detect(), get_blinks() returns one Intervals object per eye and variable (in ms). Detecting on a copy (d | pp.pupil.blinks_detect()) leaves the data unchanged, so you can check the blinks first:
[7]:
blinks = (d | pp.pupil.blinks_detect()).get_blinks()
blinks["left_pupil"]
[7]:
Intervals 'left_pupil_blinks' | 14 intervals (total duration: 3040.00 ms), 217.14 ms +/- 236.47 ms, [48.00 ms, 984.00 ms]
Blinks stored in the object can be retrieved with get_blinks(), and masked samples with mask_as_intervals(). You can also create intervals directly:
[8]:
pp.Intervals([(1, 2), (5, 7.5)] * u.sec, label="bad data")
[8]:
Intervals 'bad data' | 2 intervals (total duration: 3.50 sec), 1.75 sec +/- 0.75 sec, [1.00 sec, 2.50 sec]
Units#
to_units() converts intervals to other time units:
[9]:
trials.to_units("sec").as_pandas().head(3)
[9]:
| start | end | duration | units | label | onset | |
|---|---|---|---|---|---|---|
| 0 | 5.177 | 7.010 | 1.833 | sec | F_1 to R_1_15 | 5.177 |
| 1 | 8.776 | 10.808 | 2.032 | sec | F_2 to R_2_20 | 8.776 |
| 2 | 12.374 | 13.957 | 1.583 | sec | F_3 to R_3_18 | 12.374 |
Intervals in sample indices (units="samples") are only valid for the data object they came from: after slicing or downsampling, or in another recording, the same indices point to other samples. Converting between sample indices and time therefore needs the data object, whose time vector gives the time of each sample (a time is converted to the first sample at or after it). Here, the blinks as sample indices:
[10]:
blinks["left_pupil"].to_units("samples", d).as_pandas().head(3)
[10]:
| start | end | duration | units | |
|---|---|---|---|---|
| 0 | 2704 | 2748 | 44 | samples |
| 1 | 3781 | 3832 | 51 | samples |
| 2 | 3932 | 3956 | 24 | samples |
Changing intervals#
The following methods return new Intervals and leave the original unchanged.
filter() keeps intervals by duration, time range or a function of (start, end). Here, the trials that took longer than 1.8 seconds:
[11]:
trials.filter(min_duration=1.8 * u.sec)
[11]:
Intervals 'F_ to R_' | 7 intervals (total duration: 13445.00 ms), 1920.71 ms +/- 88.30 ms, [1800.00 ms, 2033.00 ms]
pad() extends each interval. For blinks, this is useful because the samples just before and after a blink are often distorted:
[12]:
left = blinks["left_pupil"]
left.pad(50 * u.ms, 100 * u.ms).as_pandas().head(3)
[12]:
| start | end | duration | units | |
|---|---|---|---|---|
| 0 | 5358.0 | 5596.0 | 238.0 | ms |
| 1 | 7512.0 | 7764.0 | 252.0 | ms |
| 2 | 7814.0 | 8012.0 | 198.0 | ms |
merge() combines intervals that overlap (or are closer than distance):
[13]:
print(len(left), "->", len(left.merge(distance=200 * u.ms)))
14 -> 12
Combining intervals#
+ (or pp.concat_intervals() for many at once, or a list or dict) puts the intervals of two objects together. For instance, the blinks of both eyes. Overlapping intervals are kept separate until you call merge():
[14]:
both_eyes = pp.concat_intervals(blinks["left_pupil"], blinks["right_pupil"])
both_eyes = both_eyes.merge()
both_eyes
[14]:
Intervals 'combined' | 15 intervals (total duration: 3480.00 ms), 232.00 ms +/- 232.02 ms, [48.00 ms, 984.00 ms]
- removes the parts of one set of intervals that overlap with another. Here, the parts of each trial without blinks in either eye:
[15]:
clean_trials = trials - both_eyes
clean_trials.as_pandas().head()
[15]:
| start | end | duration | units | |
|---|---|---|---|---|
| 0 | 5177.0 | 5408.0 | 231.0 | ms |
| 1 | 5496.0 | 7010.0 | 1514.0 | ms |
| 2 | 8776.0 | 10484.0 | 1708.0 | ms |
| 3 | 12374.0 | 13957.0 | 1583.0 | ms |
| 4 | 15973.0 | 17822.0 | 1849.0 | ms |
Trials with a blink are split into several pieces.
Plotting intervals#
plot() draws the intervals on a time line. The plots of the data shade intervals with highlight= (converted to the units of the plot); with a list of Intervals, highlight_color= gives one color for each. Here, trials are green and blinks red:
[16]:
plt.figure(figsize=(12, 4))
d.plot.plot_pupil(plot_range=(0, 20) * u.sec, show_blinks=False,
show_events=False, highlight=[trials, both_eyes],
highlight_color=["green", "red"]);
d.plot.plot_intervals() shows the data in each interval in a small plot of its own, e.g., to check every blink (for blinks, plot_blinks() does the same):
[17]:
d.plot.plot_intervals(stim, eyes="left",
variables="pupil", nrow=2, ncol=5, figsize=(14, 5));
Using intervals#
Intervals are the input for many methods of the data objects.
Masking and interpolating: mask_intervals() marks the samples in the intervals as invalid, interpolate_intervals() replaces them by interpolated values. Here we mask the padded blinks in the pupil data:
[18]:
d2 = d.mask_intervals(both_eyes.pad(50 * u.ms, 100 * u.ms),
variables="pupil", inplace=False)
d2.data.count_masked()
[18]:
2791
One value per interval: stat_per_event() applies a function to the data in each interval, e.g., the mean pupil size after each stimulus:
[19]:
d2.stat_per_event(intervals=stim, variables="pupil")
[19]:
{'left_pupil': array([1003.03142329, 1188.35538752, 1281.70833333, 1160.34333333,
1100.505 , 1095.62833333, 993.23 , 1113.09907121,
963.89 , 1065.74666667]),
'right_pupil': array([ 830.89833641, 977.61625709, 1095.90166667, 969.57833333,
916.80833333, 920.22833333, 843.105 , 932.53560372,
808.16333333, 889.60166667])}
Segments: get_segments() cuts the data in each epoch into a SegmentedEyeData object (time axis from the window; Intervals only if they all have the same length), the basis for event-related pupil dilations (see the ERPD tutorial):
[20]:
d2.get_segments(stim, "left_pupil")
[20]:
SegmentedEyeData — S_
Channels (mean over segments; red line = event onset)
Epochs
Epochs 'S_' | 10 epochs, window (-200, 1000) ms
Masks and intervals#
Intervals can be converted to a boolean mask with one entry per sample of a data object, and back:
[21]:
mask = both_eyes.to_mask(d)
print(mask.shape, mask.sum(), "samples in blinks")
pp.Intervals.from_mask(mask, d, units="ms")
(20465,) 1740 samples in blinks
[21]:
Intervals | 15 intervals (total duration: 3480.00 ms), 232.00 ms +/- 232.02 ms, [48.00 ms, 984.00 ms]