.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_tutorials/plot_20_pipelines.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_tutorials_plot_20_pipelines.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_tutorials_plot_20_pipelines.py:


.. _tut-pipelines:

Build and adjust a pipeline
===========================

A pipeline is a list of named steps. Each step is a small scikit-learn
style estimator: its parameters are set when it is created, ``fit`` learns
what it needs from a recording, ``transform`` applies it, and afterwards it
reports what it did (``qc_``) and draws it (``plot``).

This tutorial builds the recommended MEG pipeline for a Neuromag system
step by step on MNE's ``sample`` recording, looks at every step, and then
changes it.

.. GENERATED FROM PYTHON SOURCE LINES 16-18

.. code-block:: Python
   :dedent: 1










.. GENERATED FROM PYTHON SOURCE LINES 21-37

.. code-block:: Python

    import tempfile
    from pathlib import Path

    import mne

    import meeg_utils as meu
    from meeg_utils import steps as S

    mne.set_log_level("ERROR")

    data_path = mne.datasets.sample.data_path()
    raw = mne.io.read_raw_fif(data_path / "MEG" / "sample" / "sample_audvis_raw.fif")
    raw.pick(["meg", "eog", "stim"]).load_data()
    raw.info["line_freq"] = 60.0
    meu.io.detect_system(raw)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    'neuromag'



.. GENERATED FROM PYTHON SOURCE LINES 38-43

The steps
---------
Neuromag systems come with a cross-talk and a fine-calibration file, which
SSS needs. In a BIDS dataset they are found automatically; here we pass
them.

.. GENERATED FROM PYTHON SOURCE LINES 43-58

.. code-block:: Python

    cross_talk = data_path / "SSS" / "ct_sparse_mgh.fif"
    calibration = data_path / "SSS" / "sss_cal_mgh.dat"

    pipe = meu.Pipeline(
        [
            ("bads", S.BadChannels("maxwell", cross_talk=cross_talk, calibration=calibration)),
            ("sss", S.Maxwell(cross_talk=cross_talk, calibration=calibration)),
            ("highpass", S.Filter(0.1, None)),
            ("resample", S.Resample(250.0)),
            ("line_noise", S.LineNoise("zapline-plus")),
            ("ica", S.ICA(20, picks="meg", method="infomax", labeler="megnet")),
        ]
    )
    pipe






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <table class="meu-pipeline"><thead><tr><th style="text-align:left">#</th><th style="text-align:left">Step</th><th style="text-align:left">Configuration (changed parameters)</th><th></th></tr></thead><tbody><tr><td style="text-align:left">0</td><td style="text-align:left"><b>bads</b></td><td style="text-align:left"><code>BadChannels(calibration=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat&#x27;),
                cross_talk=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif&#x27;),
                method=&#x27;maxwell&#x27;)</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">1</td><td style="text-align:left"><b>sss</b></td><td style="text-align:left"><code>Maxwell(calibration=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat&#x27;),
            cross_talk=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif&#x27;))</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">2</td><td style="text-align:left"><b>highpass</b></td><td style="text-align:left"><code>Filter(h_freq=None, l_freq=0.1)</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">3</td><td style="text-align:left"><b>resample</b></td><td style="text-align:left"><code>Resample(sfreq=250.0)</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">4</td><td style="text-align:left"><b>line_noise</b></td><td style="text-align:left"><code>LineNoise()</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">5</td><td style="text-align:left"><b>ica</b></td><td style="text-align:left"><code>ICA(labeler=&#x27;megnet&#x27;, n_components=20, picks=&#x27;meg&#x27;)</code></td><td style="text-align:left"></td></tr></tbody></table>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 59-62

This is the ``meg-erp`` preset; ``meu.Pipeline.preset("meg-erp",
system="neuromag")`` builds the same pipeline and finds the two files in
the BIDS dataset of the recording.

.. GENERATED FROM PYTHON SOURCE LINES 62-64

.. code-block:: Python

    clean = pipe.fit_transform(raw)








.. GENERATED FROM PYTHON SOURCE LINES 65-70

Bad channels
------------
:func:`mne.preprocessing.find_bad_channels_maxwell` scores every channel
against its SSS reconstruction. ``MEG 2443`` was already marked bad in the
file:

.. GENERATED FROM PYTHON SOURCE LINES 70-73

.. code-block:: Python

    print(pipe.qc_["bads"]["meg"])
    figs = pipe["bads"].plot("scores")




.. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_001.png
   :alt: Bad-channel detection scores (detected channels in red), Maxwell grad, Maxwell mag
   :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_001.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_001_2_00x.png 2.00x
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    {'bads': ['MEG 2313'], 'n_bads': 1, 'fraction': 0.0032679738562091504, 'by_criterion': {'noisy': ['MEG 2313'], 'flat': []}}




.. GENERATED FROM PYTHON SOURCE LINES 74-78

SSS
---
Signal-space separation keeps the fields from inside the head and rebuilds
the bad channels. The spectra before and after:

.. GENERATED FROM PYTHON SOURCE LINES 78-81

.. code-block:: Python

    print({k: pipe.qc_["sss"][k] for k in ("reconstructed", "n_basis", "power_change_db")})
    figs = pipe["sss"].plot("psd")




.. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_002.png
   :alt: Spectra before and after SSS, MAG, GRAD
   :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_002.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_002_2_00x.png 2.00x
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    {'reconstructed': ['MEG 2443', 'MEG 2313'], 'n_basis': 72, 'power_change_db': {'mag': np.float64(-9.28), 'grad': np.float64(-1.97)}}




.. GENERATED FROM PYTHON SOURCE LINES 82-86

Line noise
----------
ZapLine-plus removes the 60 Hz components and their harmonics, chunk by
chunk, without notching the spectrum:

.. GENERATED FROM PYTHON SOURCE LINES 86-88

.. code-block:: Python

    figs = pipe["line_noise"].plot("psd")




.. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_003.png
   :alt: Line noise (zapline-plus, 60 Hz), MAG, GRAD
   :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_003.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_003_2_00x.png 2.00x
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 89-94

ICA with MEGnet
---------------
MEGnet labels the 20 components (it was trained on 20 Infomax components
of 250 Hz data, which is why the pipeline resamples first). Components
whose artifact probability is at least 0.8 are removed:

.. GENERATED FROM PYTHON SOURCE LINES 94-97

.. code-block:: Python

    print(pipe.qc_["ica"]["labels"])
    figs = pipe["ica"].plot("labels")




.. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_004.png
   :alt: ICA labels (megnet); x = excluded
   :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_004.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_004_2_00x.png 2.00x
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ['brain/other', 'eye blink', 'brain/other', 'brain/other', 'heart beat', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'eye movement', 'brain/other', 'brain/other', 'brain/other', 'brain/other', 'heart beat', 'brain/other']




.. GENERATED FROM PYTHON SOURCE LINES 98-100

.. code-block:: Python

    figs = pipe["ica"].plot("components")




.. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_005.png
   :alt: ICA000 (mag) brain/other 1.00, ICA001 (mag) eye blink 1.00, ICA002 (mag) brain/other 1.00, ICA003 (mag) brain/other 1.00, ICA004 (mag) heart beat 1.00, ICA005 (mag) brain/other 1.00, ICA006 (mag) brain/other 1.00, ICA007 (mag) brain/other 1.00, ICA008 (mag) brain/other 1.00, ICA009 (mag) brain/other 1.00, ICA010 (mag) brain/other 1.00, ICA011 (mag) brain/other 1.00, ICA012 (mag) brain/other 1.00, ICA013 (mag) eye movement 0.65, ICA014 (mag) brain/other 1.00, ICA015 (mag) brain/other 1.00, ICA016 (mag) brain/other 1.00, ICA017 (mag) brain/other 1.00, ICA018 (mag) heart beat 0.50, ICA019 (mag) brain/other 1.00
   :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_005.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_005_2_00x.png 2.00x
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 101-103

The excluded components, with their spectrum and time course (these
figures need the data):

.. GENERATED FROM PYTHON SOURCE LINES 103-105

.. code-block:: Python

    figs = pipe["ica"].plot("properties", inst=raw)




.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_006.png
         :alt: ICA001 (mag), Segment image and ERP/ERF, Spectrum, Dropped segments: 0.00 %
         :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_006.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_006_2_00x.png 2.00x
         :class: sphx-glr-single-img

    *

      .. image-sg:: /auto_tutorials/images/sphx_glr_plot_20_pipelines_007.png
         :alt: ICA004 (mag), Segment image and ERP/ERF, Spectrum, Dropped segments: 0.00 %
         :srcset: /auto_tutorials/images/sphx_glr_plot_20_pipelines_007.png, /auto_tutorials/images/sphx_glr_plot_20_pipelines_007_2_00x.png 2.00x
         :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 106-117

If you disagree with a label, correct it and transform again; the
decomposition is kept:

.. code-block:: python

   pipe["ica"].relabel({4: "brain"})
   clean = pipe.transform(raw)

Change the pipeline
-------------------
Parameters are addressed as ``<step>__<parameter>``, as in scikit-learn:

.. GENERATED FROM PYTHON SOURCE LINES 117-120

.. code-block:: Python

    pipe.set_params(highpass__l_freq=1.0, ica__threshold=0.9)
    pipe["highpass"]






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-1 {
      /* Definition of color scheme common for light and dark mode */
      --sklearn-color-text: #000;
      --sklearn-color-text-muted: #666;
      --sklearn-color-line: gray;
      /* Definition of color scheme for unfitted estimators */
      --sklearn-color-unfitted-level-0: #fff5e6;
      --sklearn-color-unfitted-level-1: #f6e4d2;
      --sklearn-color-unfitted-level-2: #ffe0b3;
      --sklearn-color-unfitted-level-3: chocolate;
      /* Definition of color scheme for fitted estimators */
      --sklearn-color-fitted-level-0: #f0f8ff;
      --sklearn-color-fitted-level-1: #d4ebff;
      --sklearn-color-fitted-level-2: #b3dbfd;
      --sklearn-color-fitted-level-3: cornflowerblue;
    }

    #sk-container-id-1.light {
      /* Specific color for light theme */
      --sklearn-color-text-on-default-background: black;
      --sklearn-color-background: white;
      --sklearn-color-border-box: black;
      --sklearn-color-icon: #696969;
    }

    #sk-container-id-1.dark {
      --sklearn-color-text-on-default-background: white;
      --sklearn-color-background: #111;
      --sklearn-color-border-box: white;
      --sklearn-color-icon: #878787;
    }

    #sk-container-id-1 {
      color: var(--sklearn-color-text);
    }

    #sk-container-id-1 pre {
      padding: 0;
    }

    #sk-container-id-1 input.sk-hidden--visually {
      border: 0;
      clip: rect(1px 1px 1px 1px);
      clip: rect(1px, 1px, 1px, 1px);
      height: 1px;
      margin: -1px;
      overflow: hidden;
      padding: 0;
      position: absolute;
      width: 1px;
    }

    #sk-container-id-1 div.sk-dashed-wrapped {
      border: 1px dashed var(--sklearn-color-line);
      margin: 0 0.4em 0.5em 0.4em;
      box-sizing: border-box;
      padding-bottom: 0.4em;
      background-color: var(--sklearn-color-background);
    }

    #sk-container-id-1 div.sk-container {
      /* jupyter's `normalize.less` sets `[hidden] { display: none; }`
         but bootstrap.min.css set `[hidden] { display: none !important; }`
         so we also need the `!important` here to be able to override the
         default hidden behavior on the sphinx rendered scikit-learn.org.
         See: https://github.com/scikit-learn/scikit-learn/issues/21755 */
      display: inline-block !important;
      position: relative;
    }

    #sk-container-id-1 div.sk-text-repr-fallback {
      display: none;
    }

    div.sk-parallel-item,
    div.sk-serial,
    div.sk-item {
      /* draw centered vertical line to link estimators */
      background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));
      background-size: 2px 100%;
      background-repeat: no-repeat;
      background-position: center center;
    }

    /* Parallel-specific style estimator block */

    #sk-container-id-1 div.sk-parallel-item::after {
      content: "";
      width: 100%;
      border-bottom: 2px solid var(--sklearn-color-text-on-default-background);
      flex-grow: 1;
    }

    #sk-container-id-1 div.sk-parallel {
      display: flex;
      align-items: stretch;
      justify-content: center;
      background-color: var(--sklearn-color-background);
      position: relative;
    }

    #sk-container-id-1 div.sk-parallel-item {
      display: flex;
      flex-direction: column;
    }

    #sk-container-id-1 div.sk-parallel-item:first-child::after {
      align-self: flex-end;
      width: 50%;
    }

    #sk-container-id-1 div.sk-parallel-item:last-child::after {
      align-self: flex-start;
      width: 50%;
    }

    #sk-container-id-1 div.sk-parallel-item:only-child::after {
      width: 0;
    }

    /* Serial-specific style estimator block */

    #sk-container-id-1 div.sk-serial {
      display: flex;
      flex-direction: column;
      align-items: center;
      background-color: var(--sklearn-color-background);
      padding-right: 1em;
      padding-left: 1em;
    }


    /* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is
    clickable and can be expanded/collapsed.
    - Pipeline and ColumnTransformer use this feature and define the default style
    - Estimators will overwrite some part of the style using the `sk-estimator` class
    */

    /* Pipeline and ColumnTransformer style (default) */

    #sk-container-id-1 div.sk-toggleable {
      /* Default theme specific background. It is overwritten whether we have a
      specific estimator or a Pipeline/ColumnTransformer */
      background-color: var(--sklearn-color-background);
    }

    /* Toggleable label */
    #sk-container-id-1 label.sk-toggleable__label {
      cursor: pointer;
      display: flex;
      width: 100%;
      margin-bottom: 0;
      padding: 0.5em;
      box-sizing: border-box;
      text-align: center;
      align-items: center;
      justify-content: center;
      gap: 0.5em;
    }

    #sk-container-id-1 label.sk-toggleable__label .caption {
      font-size: 0.6rem;
      font-weight: lighter;
      color: var(--sklearn-color-text-muted);
    }

    #sk-container-id-1 label.sk-toggleable__label-arrow:before {
      /* Arrow on the left of the label */
      content: "▸";
      float: left;
      margin-right: 0.25em;
      color: var(--sklearn-color-icon);
    }

    #sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {
      color: var(--sklearn-color-text);
    }

    /* Toggleable content - dropdown */

    #sk-container-id-1 div.sk-toggleable__content {
      display: none;
      text-align: left;
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content pre {
      margin: 0.2em;
      border-radius: 0.25em;
      color: var(--sklearn-color-text);
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content.fitted pre {
      /* unfitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    #sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {
      /* Expand drop-down */
      display: block;
      width: 100%;
      overflow: visible;
    }

    #sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {
      content: "▾";
    }

    /* Pipeline/ColumnTransformer-specific style */

    #sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Estimator-specific style */

    /* Colorize estimator box */
    #sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-2);
    }

    #sk-container-id-1 div.sk-label label.sk-toggleable__label,
    #sk-container-id-1 div.sk-label label {
      /* The background is the default theme color */
      color: var(--sklearn-color-text-on-default-background);
    }

    /* On hover, darken the color of the background */
    #sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    /* Label box, darken color on hover, fitted */
    #sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Estimator label */

    #sk-container-id-1 div.sk-label label {
      font-family: monospace;
      font-weight: bold;
      line-height: 1.2em;
    }

    #sk-container-id-1 div.sk-label-container {
      text-align: center;
    }

    /* Estimator-specific */
    #sk-container-id-1 div.sk-estimator {
      font-family: monospace;
      border: 1px dotted var(--sklearn-color-border-box);
      border-radius: 0.25em;
      box-sizing: border-box;
      margin-bottom: 0.5em;
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-estimator.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    /* on hover */
    #sk-container-id-1 div.sk-estimator:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-estimator.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Specification for estimator info (e.g. "i" and "?") */

    /* Common style for "i" and "?" */

    .sk-estimator-doc-link,
    a:link.sk-estimator-doc-link,
    a:visited.sk-estimator-doc-link {
      float: right;
      font-size: smaller;
      line-height: 1em;
      font-family: monospace;
      background-color: var(--sklearn-color-unfitted-level-0);
      border-radius: 1em;
      height: 1em;
      width: 1em;
      text-decoration: none !important;
      margin-left: 0.5em;
      text-align: center;
      /* unfitted */
      border: var(--sklearn-color-unfitted-level-3) 1pt solid;
      color: var(--sklearn-color-unfitted-level-3);
    }

    .sk-estimator-doc-link.fitted,
    a:link.sk-estimator-doc-link.fitted,
    a:visited.sk-estimator-doc-link.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
      border: var(--sklearn-color-fitted-level-3) 1pt solid;
      color: var(--sklearn-color-fitted-level-3);
    }

    /* On hover */
    div.sk-estimator:hover .sk-estimator-doc-link:hover,
    .sk-estimator-doc-link:hover,
    div.sk-label-container:hover .sk-estimator-doc-link:hover,
    .sk-estimator-doc-link:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-3);
      border: var(--sklearn-color-fitted-level-0) 1pt solid;
      color: var(--sklearn-color-unfitted-level-0);
      text-decoration: none;
    }

    div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,
    .sk-estimator-doc-link.fitted:hover,
    div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,
    .sk-estimator-doc-link.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-3);
      border: var(--sklearn-color-fitted-level-0) 1pt solid;
      color: var(--sklearn-color-fitted-level-0);
      text-decoration: none;
    }

    /* Span, style for the box shown on hovering the info icon */
    .sk-estimator-doc-link span {
      display: none;
      z-index: 9999;
      position: relative;
      font-weight: normal;
      right: .2ex;
      padding: .5ex;
      margin: .5ex;
      width: min-content;
      min-width: 20ex;
      max-width: 50ex;
      color: var(--sklearn-color-text);
      box-shadow: 2pt 2pt 4pt #999;
      /* unfitted */
      background: var(--sklearn-color-unfitted-level-0);
      border: .5pt solid var(--sklearn-color-unfitted-level-3);
    }

    .sk-estimator-doc-link.fitted span {
      /* fitted */
      background: var(--sklearn-color-fitted-level-0);
      border: var(--sklearn-color-fitted-level-3);
    }

    .sk-estimator-doc-link:hover span {
      display: block;
    }

    /* "?"-specific style due to the `<a>` HTML tag */

    #sk-container-id-1 a.estimator_doc_link {
      float: right;
      font-size: 1rem;
      line-height: 1em;
      font-family: monospace;
      background-color: var(--sklearn-color-unfitted-level-0);
      border-radius: 1rem;
      height: 1rem;
      width: 1rem;
      text-decoration: none;
      /* unfitted */
      color: var(--sklearn-color-unfitted-level-1);
      border: var(--sklearn-color-unfitted-level-1) 1pt solid;
    }

    #sk-container-id-1 a.estimator_doc_link.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
      border: var(--sklearn-color-fitted-level-1) 1pt solid;
      color: var(--sklearn-color-fitted-level-1);
    }

    /* On hover */
    #sk-container-id-1 a.estimator_doc_link:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-3);
      color: var(--sklearn-color-background);
      text-decoration: none;
    }

    #sk-container-id-1 a.estimator_doc_link.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-3);
    }

    .estimator-table {
        font-family: monospace;
    }

    .estimator-table summary {
        padding: .5rem;
        cursor: pointer;
    }

    .estimator-table summary::marker {
        font-size: 0.7rem;
    }

    .estimator-table details[open] {
        padding-left: 0.1rem;
        padding-right: 0.1rem;
        padding-bottom: 0.3rem;
    }

    .estimator-table .parameters-table {
        margin-left: auto !important;
        margin-right: auto !important;
        margin-top: 0;
    }

    .estimator-table .parameters-table tr:nth-child(odd) {
        background-color: #fff;
    }

    .estimator-table .parameters-table tr:nth-child(even) {
        background-color: #f6f6f6;
    }

    .estimator-table .parameters-table tr:hover {
        background-color: #e0e0e0;
    }

    .estimator-table table td {
        border: 1px solid rgba(106, 105, 104, 0.232);
    }

    /*
        `table td`is set in notebook with right text-align.
        We need to overwrite it.
    */
    .estimator-table table td.param {
        text-align: left;
        position: relative;
        padding: 0;
    }

    .user-set td {
        color:rgb(255, 94, 0);
        text-align: left !important;
    }

    .user-set td.value {
        color:rgb(255, 94, 0);
        background-color: transparent;
    }

    .default td {
        color: black;
        text-align: left !important;
    }

    .user-set td i,
    .default td i {
        color: black;
    }

    /*
        Styles for parameter documentation links
        We need styling for visited so jupyter doesn't overwrite it
    */
    a.param-doc-link,
    a.param-doc-link:link,
    a.param-doc-link:visited {
        text-decoration: underline dashed;
        text-underline-offset: .3em;
        color: inherit;
        display: block;
        padding: .5em;
    }

    /* "hack" to make the entire area of the cell containing the link clickable */
    a.param-doc-link::before {
        position: absolute;
        content: "";
        inset: 0;
    }

    .param-doc-description {
        display: none;
        position: absolute;
        z-index: 9999;
        left: 0;
        padding: .5ex;
        margin-left: 1.5em;
        color: var(--sklearn-color-text);
        box-shadow: .3em .3em .4em #999;
        width: max-content;
        text-align: left;
        max-height: 10em;
        overflow-y: auto;

        /* unfitted */
        background: var(--sklearn-color-unfitted-level-0);
        border: thin solid var(--sklearn-color-unfitted-level-3);
    }

    /* Fitted state for parameter tooltips */
    .fitted .param-doc-description {
        /* fitted */
        background: var(--sklearn-color-fitted-level-0);
        border: thin solid var(--sklearn-color-fitted-level-3);
    }

    .param-doc-link:hover .param-doc-description {
        display: block;
    }

    .copy-paste-icon {
        background-image: url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCA0NDggNTEyIj48IS0tIUZvbnQgQXdlc29tZSBGcmVlIDYuNy4yIGJ5IEBmb250YXdlc29tZSAtIGh0dHBzOi8vZm9udGF3ZXNvbWUuY29tIExpY2Vuc2UgLSBodHRwczovL2ZvbnRhd2Vzb21lLmNvbS9saWNlbnNlL2ZyZWUgQ29weXJpZ2h0IDIwMjUgRm9udGljb25zLCBJbmMuLS0+PHBhdGggZD0iTTIwOCAwTDMzMi4xIDBjMTIuNyAwIDI0LjkgNS4xIDMzLjkgMTQuMWw2Ny45IDY3LjljOSA5IDE0LjEgMjEuMiAxNC4xIDMzLjlMNDQ4IDMzNmMwIDI2LjUtMjEuNSA0OC00OCA0OGwtMTkyIDBjLTI2LjUgMC00OC0yMS41LTQ4LTQ4bDAtMjg4YzAtMjYuNSAyMS41LTQ4IDQ4LTQ4ek00OCAxMjhsODAgMCAwIDY0LTY0IDAgMCAyNTYgMTkyIDAgMC0zMiA2NCAwIDAgNDhjMCAyNi41LTIxLjUgNDgtNDggNDhMNDggNTEyYy0yNi41IDAtNDgtMjEuNS00OC00OEwwIDE3NmMwLTI2LjUgMjEuNS00OCA0OC00OHoiLz48L3N2Zz4=);
        background-repeat: no-repeat;
        background-size: 14px 14px;
        background-position: 0;
        display: inline-block;
        width: 14px;
        height: 14px;
        cursor: pointer;
    }
    </style><body><div id="sk-container-id-1" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>Filter(h_freq=None, l_freq=1.0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator fitted sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-1" type="checkbox" checked><label for="sk-estimator-id-1" class="sk-toggleable__label fitted sk-toggleable__label-arrow"><div><div>Filter</div></div><div><span class="sk-estimator-doc-link fitted">i<span>Fitted</span></span></div></label><div class="sk-toggleable__content fitted" data-param-prefix="">
            <div class="estimator-table">
                <details>
                    <summary>Parameters</summary>
                    <table class="parameters-table">
                      <tbody>
                    
            <tr class="user-set">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('l_freq',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">l_freq</td>
                <td class="value">1.0</td>
            </tr>
    

            <tr class="user-set">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('h_freq',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">h_freq</td>
                <td class="value">None</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('picks',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">picks</td>
                <td class="value">None</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('method',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">method</td>
                <td class="value">&#x27;fir&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('iir_params',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">iir_params</td>
                <td class="value">None</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('phase',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">phase</td>
                <td class="value">&#x27;zero&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('l_trans_bandwidth',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">l_trans_bandwidth</td>
                <td class="value">&#x27;auto&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('h_trans_bandwidth',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">h_trans_bandwidth</td>
                <td class="value">&#x27;auto&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('filter_length',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">filter_length</td>
                <td class="value">&#x27;auto&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('fir_window',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">fir_window</td>
                <td class="value">&#x27;hamming&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('fir_design',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">fir_design</td>
                <td class="value">&#x27;firwin&#x27;</td>
            </tr>
    

            <tr class="default">
                <td><i class="copy-paste-icon"
                     onclick="copyToClipboard('n_jobs',
                              this.parentElement.nextElementSibling)"
                ></i></td>
                <td class="param">n_jobs</td>
                <td class="value">None</td>
            </tr>
    
                      </tbody>
                    </table>
                </details>
            </div>
        </div></div></div></div></div><script>function copyToClipboard(text, element) {
        // Get the parameter prefix from the closest toggleable content
        const toggleableContent = element.closest('.sk-toggleable__content');
        const paramPrefix = toggleableContent ? toggleableContent.dataset.paramPrefix : '';
        const fullParamName = paramPrefix ? `${paramPrefix}${text}` : text;

        const originalStyle = element.style;
        const computedStyle = window.getComputedStyle(element);
        const originalWidth = computedStyle.width;
        const originalHTML = element.innerHTML.replace('Copied!', '');

        navigator.clipboard.writeText(fullParamName)
            .then(() => {
                element.style.width = originalWidth;
                element.style.color = 'green';
                element.innerHTML = "Copied!";

                setTimeout(() => {
                    element.innerHTML = originalHTML;
                    element.style = originalStyle;
                }, 2000);
            })
            .catch(err => {
                console.error('Failed to copy:', err);
                element.style.color = 'red';
                element.innerHTML = "Failed!";
                setTimeout(() => {
                    element.innerHTML = originalHTML;
                    element.style = originalStyle;
                }, 2000);
            });
        return false;
    }

    document.querySelectorAll('.copy-paste-icon').forEach(function(element) {
        const toggleableContent = element.closest('.sk-toggleable__content');
        const paramPrefix = toggleableContent ? toggleableContent.dataset.paramPrefix : '';
        const paramName = element.parentElement.nextElementSibling
            .textContent.trim().split(' ')[0];
        const fullParamName = paramPrefix ? `${paramPrefix}${paramName}` : paramName;

        element.setAttribute('title', fullParamName);
    });


    /**
     * Adapted from Skrub
     * https://github.com/skrub-data/skrub/blob/403466d1d5d4dc76a7ef569b3f8228db59a31dc3/skrub/_reporting/_data/templates/report.js#L789
     * @returns "light" or "dark"
     */
    function detectTheme(element) {
        const body = document.querySelector('body');

        // Check VSCode theme
        const themeKindAttr = body.getAttribute('data-vscode-theme-kind');
        const themeNameAttr = body.getAttribute('data-vscode-theme-name');

        if (themeKindAttr && themeNameAttr) {
            const themeKind = themeKindAttr.toLowerCase();
            const themeName = themeNameAttr.toLowerCase();

            if (themeKind.includes("dark") || themeName.includes("dark")) {
                return "dark";
            }
            if (themeKind.includes("light") || themeName.includes("light")) {
                return "light";
            }
        }

        // Check Jupyter theme
        if (body.getAttribute('data-jp-theme-light') === 'false') {
            return 'dark';
        } else if (body.getAttribute('data-jp-theme-light') === 'true') {
            return 'light';
        }

        // Guess based on a parent element's color
        const color = window.getComputedStyle(element.parentNode, null).getPropertyValue('color');
        const match = color.match(/^rgb\s*\(\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*\)\s*$/i);
        if (match) {
            const [r, g, b] = [
                parseFloat(match[1]),
                parseFloat(match[2]),
                parseFloat(match[3])
            ];

            // https://en.wikipedia.org/wiki/HSL_and_HSV#Lightness
            const luma = 0.299 * r + 0.587 * g + 0.114 * b;

            if (luma > 180) {
                // If the text is very bright we have a dark theme
                return 'dark';
            }
            if (luma < 75) {
                // If the text is very dark we have a light theme
                return 'light';
            }
            // Otherwise fall back to the next heuristic.
        }

        // Fallback to system preference
        return window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
    }


    function forceTheme(elementId) {
        const estimatorElement = document.querySelector(`#${elementId}`);
        if (estimatorElement === null) {
            console.error(`Element with id ${elementId} not found.`);
        } else {
            const theme = detectTheme(estimatorElement);
            estimatorElement.classList.add(theme);
        }
    }

    forceTheme('sk-container-id-1');</script></body>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 121-123

Steps can be added, replaced and removed by name, and a pipeline can be
sliced:

.. GENERATED FROM PYTHON SOURCE LINES 123-127

.. code-block:: Python

    pipe.insert_after("line_noise", "lowpass", S.Filter(None, 40.0))
    pipe.remove("resample")
    list(pipe.named_steps), pipe[:2]





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (['bads', 'sss', 'highpass', 'line_noise', 'lowpass', 'ica'], Pipeline(steps=[('bads',
                     BadChannels(calibration=PosixPath('/home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat'),
                                 cross_talk=PosixPath('/home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif'),
                                 method='maxwell')),
                    ('sss',
                     Maxwell(calibration=PosixPath('/home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat'),
                             cross_talk=PosixPath('/home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif')))]))



.. GENERATED FROM PYTHON SOURCE LINES 128-129

A pipeline is saved as YAML, to rerun it or to publish it with a paper:

.. GENERATED FROM PYTHON SOURCE LINES 129-133

.. code-block:: Python

    fname = Path(tempfile.mkdtemp()) / "pipeline.yaml"
    pipe.to_yaml(fname)
    print(fname.read_text())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    meeg_utils_version: 0.2.0
    copy: true
    steps:
    - name: bads
      class: meeg_utils.steps.channels:BadChannels
      params:
        calibration: /home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat
        cross_talk: /home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif
        h_freq: 40.0
        limit: 7.0
        method: maxwell
        origin: auto
        psd: false
        random_state: 42
        ransac: true
        reject_by_annotation: null
        robust_reference: true
    - name: sss
      class: meeg_utils.steps.maxwell:Maxwell
      params:
        bad_condition: error
        calibration: /home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat
        coord_frame: head
        cross_talk: /home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif
        destination: null
        ext_order: 3
        head_pos: null
        int_order: 8
        origin: auto
        regularize: in
        remove_chpi: true
        skip_by_annotation:
        - edge
        - bad_acq_skip
        st_correlation: 0.98
        st_duration: null
    - name: highpass
      class: meeg_utils.steps.filtering:Filter
      params:
        filter_length: auto
        fir_design: firwin
        fir_window: hamming
        h_freq: null
        h_trans_bandwidth: auto
        iir_params: null
        l_freq: 1.0
        l_trans_bandwidth: auto
        method: fir
        n_jobs: null
        phase: zero
        picks: null
    - name: line_noise
      class: meeg_utils.steps.line_noise:LineNoise
      params:
        adaptive_params: null
        fline: null
        method: zapline-plus
        n_harmonics: null
        n_remove: auto
        nfft: 1024
    - name: lowpass
      class: meeg_utils.steps.filtering:Filter
      params:
        filter_length: auto
        fir_design: firwin
        fir_window: hamming
        h_freq: 40.0
        h_trans_bandwidth: auto
        iir_params: null
        l_freq: null
        l_trans_bandwidth: auto
        method: fir
        n_jobs: null
        phase: zero
        picks: null
    - name: ica
      class: meeg_utils.steps.ica:ICA
      params:
        decim: null
        exclude_labels: null
        fit_h_freq: 100.0
        fit_l_freq: 1.0
        fit_params: null
        labeler: megnet
        max_iter: auto
        method: infomax
        n_components: 20
        picks: meg
        random_state: 42
        reject_by_annotation: true
        threshold: 0.9





.. GENERATED FROM PYTHON SOURCE LINES 134-143

``meu.Pipeline.from_yaml(fname)`` reads it back, and ``meu run
pipeline.yaml --sources bids/ --out derivatives/`` runs it from the command
line.

MEG and EEG together
--------------------
:class:`~meeg_utils.steps.ByChannelType` runs separate steps on each
channel type and writes the results back into one recording, e.g. SSS for
the MEG and PREP plus an average reference for the EEG:

.. GENERATED FROM PYTHON SOURCE LINES 143-158

.. code-block:: Python

    combined = meu.Pipeline(
        [
            (
                "by_type",
                S.ByChannelType(
                    {
                        "meg": [("sss", S.Maxwell(cross_talk=cross_talk, calibration=calibration))],
                        "eeg": [("bads", S.BadChannels("prep")), ("reference", S.Reference())],
                    }
                ),
            ),
            ("highpass", S.Filter(0.1, None)),
        ]
    )
    combined





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <table class="meu-pipeline"><thead><tr><th style="text-align:left">#</th><th style="text-align:left">Step</th><th style="text-align:left">Configuration (changed parameters)</th><th></th></tr></thead><tbody><tr><td style="text-align:left">0</td><td style="text-align:left"><b>by_type</b></td><td style="text-align:left"><code>ByChannelType(branches={&#x27;eeg&#x27;: [(&#x27;bads&#x27;, BadChannels(method=&#x27;prep&#x27;)),
                                    (&#x27;reference&#x27;, Reference())],
                            &#x27;meg&#x27;: [(&#x27;sss&#x27;,
                                     Maxwell(calibration=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/sss_cal_mgh.dat&#x27;),
                                             cross_talk=PosixPath(&#x27;/home/runner/mne_data/MNE-sample-data/SSS/ct_sparse_mgh.fif&#x27;)))]})</code></td><td style="text-align:left"></td></tr><tr><td style="text-align:left">1</td><td style="text-align:left"><b>highpass</b></td><td style="text-align:left"><code>Filter(h_freq=None, l_freq=0.1)</code></td><td style="text-align:left"></td></tr></tbody></table>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (2 minutes 13.774 seconds)


.. _sphx_glr_download_auto_tutorials_plot_20_pipelines.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_20_pipelines.ipynb <plot_20_pipelines.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_20_pipelines.py <plot_20_pipelines.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_20_pipelines.zip <plot_20_pipelines.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_