Data Science Tooling on Linux: Getting R and Python Right

R
python
software
linux

Notes on managing R and Python environments for data science on Linux, and the surprisingly painful interactions between them.

Author

Matthias Mittner

Published

2026

Note

This is a living document and I will add to it as I change things or encounter issues.

If you do data science on Linux, you’ve probably been bitten by environment issues at some point. A system upgrade breaks your R packages. A Python dependency pulls in a conflicting version. cmdstan refuses to compile because the compiler you’re using doesn’t match the one your libraries were built with. You spend an afternoon fixing tooling instead of doing actual work.

I’ve been through all of these, repeatedly, over the years. I even wrote a post about using conda for R back in 2015 that I have since abandoned entirely. After a lot of trial and error, I’ve arrived at a setup that seems to work for me: rig + pak for R, conda for Python, and some careful discipline around keeping them from interfering with each other. That last part turns out to be the hard part. I’m writing this mostly for my own future reference, but maybe it saves someone else an afternoon of debugging.

R: rig + pak

For years, I managed R the way most Linux users do: install it from the system repositories and hope for the best. This works until you need a different R version for a project, or until your distribution ships an R version that’s too old (or too new) for a package you depend on. There are various ways to deal with this (I’ve tried most of them), and what currently seems to work best for me is the combination of rig for managing R versions and pak for installing packages.

rig: R version management

rig is a command-line tool for installing and managing multiple R versions on Linux (and macOS). It downloads pre-built R binaries from Posit (formerly RStudio) and installs them under /opt/R/. You can switch between versions, set a default, and they don’t interfere with each other.

# install rig (see https://github.com/r-lib/rig for details)
# then install R versions
$ rig add release     # latest stable
$ rig add 4.4.3       # specific version
$ rig default release # set default
$ rig list
* name   version  aliases
------------------------------------------
* 4.5.3           release

Each version gets its own library path (e.g., /home/user/R/x86_64-pc-linux-gnu-library/4.5), so packages installed for one version don’t conflict with another. This is a big improvement over the system R, where a single apt upgrade can pull the rug out from under your package library.

The plucky problem

I’m running Ubuntu 25.04 (Plucky Puffin) which, at the time of writing, is not yet supported by rig. The pre-built R binaries are available for Ubuntu 24.04 (Noble Numbat) but not for plucky. Fortunately, the noble binaries work perfectly on plucky — the two releases are close enough that there are no compatibility issues that I’ve noticed so far. The trick is to tell rig to pretend you’re on noble. In .bashrc:

# Rig for R version management (does not support my ubuntu yet)
RIG_PLATFORM=ubuntu-24.04

This makes rig download and install the noble binaries, and they run just fine.

But there’s a second part to this workaround. pak can install pre-compiled binary packages from Posit Public Package Manager (PPM) instead of compiling everything from source (which on Linux can take a very long time for packages with heavy C/C++ dependencies). PPM also doesn’t know about plucky yet, so you need to tell R to use the noble repository. In .Rprofile:

options(repos = c(CRAN = "https://packagemanager.posit.co/cran/__linux__/noble/latest"))

Without this line, pak will either fall back to compiling from source or just fail. With it in place, pak::pak("tidyverse") installs the entire tidyverse in seconds. This is one of those things that you set up once and then forget about until the next Ubuntu upgrade breaks it again.

pak: fast package management

The R versions installed by rig come with pak pre-installed, so there’s nothing extra to set up beyond the .Rprofile line above. pak is a package manager that resolves dependencies properly, installs packages in parallel, and—with the PPM repository—gives you pre-compiled binaries on Linux. It’s noticeably faster and more reliable than install.packages(), and I’ve had far fewer dependency resolution headaches with it.

Why not conda for R?

I used to use conda for R (see my old post). It worked, sort of, but it always felt like a second-class citizen in the conda ecosystem. The R packages on conda-forge lagged behind CRAN, building your own conda recipes for R packages was tedious, and the interaction between conda’s R and system libraries caused headaches I could have done without. The R ecosystem has since developed its own tooling with rig and pak, and for now, I don’t see a reason to go back to conda for R.

Python: conda

For Python, I’ve stuck with conda (specifically miniconda). The Python ecosystem has no shortage of environment managers — venv, pipenv, poetry, uv, and probably three new ones since I started writing this post. I keep coming back to conda because it manages non-Python dependencies as well. In scientific computing, you regularly depend on compiled libraries like MKL, OpenBLAS, HDF5, or Qt, and pure-Python tools can’t help you there.

I create separate conda environments for different projects or toolchains:

$ conda env list
base                 *   /home/user/miniconda3
fmri                     /home/user/miniconda3/envs/fmri
hddm                     /home/user/miniconda3/envs/hddm
mne                      /home/user/miniconda3/envs/mne
psychopy                 /home/user/miniconda3/envs/psychopy

Each environment is self-contained with its own Python version and libraries. When I need PsychoPy with a specific Qt version, or MNE-Python with specific dependencies, there’s no conflict.

The typical setup is straightforward:

$ conda create -n myproject python=3.11
$ conda activate myproject
$ pip install whatever-you-need  # or conda install
$ conda env export > environment.yml  # freeze for reproducibility

The hard part: non-Python/non-R tools

This is where things get genuinely tricky. Many data science workflows depend on tools that are neither R packages nor Python packages. The prime example in my work is CmdStan, the command-line interface to the Stan probabilistic programming language. CmdStan needs to compile C++ code, and it’s where the conda/system boundary causes real pain.

CmdStan and the compiler problem

CmdStan compiles Stan models into standalone executables using a C++ compiler. When you install CmdStan (via cmdstanr in R or cmdstanpy in Python), it downloads the source and compiles it. The question is: which compiler does it use?

If you install CmdStan from your rig-managed R (outside conda), it uses the system compiler (/usr/bin/g++). The compiled CmdStan binary and all Stan models you compile will be linked against the system’s standard library (libstdc++).

If you then try to use this same CmdStan installation from within a conda environment, you can run into trouble. Conda environments often ship their own libstdc++ (and other libraries) that may be a different version from the system one. When the CmdStan binary (compiled with the system compiler) tries to load conda’s libstdc++, you get symbol errors like:

/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found

or the reverse — the conda-compiled binary can’t find symbols in the system library.

The same issue bites in the other direction: if you install CmdStan inside a conda environment (where conda’s compiler toolchain is active), the resulting binaries may not work outside that environment.

My solution

I keep CmdStan installations under ~/.cmdstan/ and install them from the rig-managed R, outside of any conda environment:

# make sure conda is deactivated!
$ R -e 'cmdstanr::install_cmdstan()'
$ ls ~/.cmdstan/
cmdstan-2.35.0
cmdstan-2.36.0
cmdstan-2.38.0

For Python-based Stan work (cmdstanpy), I install it in the relevant conda environment and point it to a CmdStan that was compiled within that same environment. This avoids any compiler mismatch.

The lesson I’ve taken from this: don’t cross the conda/system boundary for compiled tools. If something was compiled with the system compiler, use it from outside conda. If it was compiled inside a conda environment, use it from within that environment. It sounds obvious in hindsight, but I’ve lost enough afternoons to this that I think it’s worth stating explicitly.

The conda/rig R shadowing issue

There is one more gotcha. When conda is active, it puts its own bin/ directory at the front of PATH. If there’s an R installation inside the active conda environment (or in the base environment), it will shadow the rig-managed R:

$ conda activate base
$ which R
/home/user/miniconda3/bin/R    # conda's R, NOT rig's
$ R --version
R version 4.4.3 ...

$ conda deactivate
$ which R
/usr/local/bin/R               # rig's R
$ R --version
R version 4.5.3 ...

This is why I keep R work and Python work strictly separate. When I’m doing R work, I make sure conda is deactivated. When I need Python, I activate the appropriate conda environment. I also set QUARTO_R in my .bashrc to make sure Quarto always finds the rig-managed R, even when conda is active:

export QUARTO_R=/usr/local/bin/R

What I’ve learned so far

None of this is necessarily the “right” way to do things — it’s just what currently works for me. If I had to distill it into a few rules of thumb:

  • R: rig + pak, and do your R work outside of conda.
  • Python: conda, with per-project environments.
  • Don’t mix compilers. Keep track of what was compiled with which toolchain and don’t expect binaries to work across the conda/system boundary.
  • On unsupported Ubuntu versions, the previous LTS binaries tend to work fine — set RIG_PLATFORM and the PPM repo to match.
  • Deactivate conda before doing R work. The R shadowing issue is subtle and will confuse you if you forget.

This setup has survived a couple of Ubuntu upgrades and Stan releases without breaking, which is more than I can say for my previous approaches. I’ll update this post when something inevitably goes wrong again.