Ctrl+K

Managing conda based distributions and installing modules

Managing conda based distributions and installing modules#

As mentioned in the previous section, adding functionality to Python is done through modules and packages. If you are using the Miniforge distribution, only a minimal python installation is installed, as modules can be added as needed over time. Alternatively, the anaconda distribution provides the most useful numerical packages in the installer already, however occasionally you likely will need to add additional modules or packages. In this section we will explain how to install packages and how to keep your installation up to date.

Learning outcomes#

In this section you will learn

  • ways to install modules in Python

  • how to find and install modules using the conda command

  • how to find and install modules with pip the python package manager

  • how to keep your installed packages up to date

References and further reading#

Modules vs packages#

Before we dive into the details of let us briefly discuss the difference between a module and a package.

Module#

A module is a file containing Python functions and statements, which is added with the import statement. If you create a file fibonacci.py in this directory with the contents

c = 2.99792458e8

def fib(n):    # write Fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

We can import and run the functions or access the constants in this file.

import fibonacci

fibonacci.fib(5)
print("The speed of light in vacuum is= {} m/s".format(fibonacci.c))

0 1 1 2 3 
The speed of light in vacuum is= 299792458.0 m/s

Package#

A package on the other hand is a collection of modules under a single namespace. Generally most external installations are packages, examples are numpy, matplotlib, scipy. Modules within a package are separated with dot (and we can go many layers deep). For example:

import numpy as np

np.polynomial.chebyshev

<module 'numpy.polynomial.chebyshev' from '/home/jschrod/.virtualenvs/jupyterbook2/lib/python3.10/site-packages/numpy/polynomial/chebyshev.py'>

np.polynomial.chebyshev.chebadd

<function numpy.polynomial.chebyshev.chebadd(c1, c2)>

Package installation#

Generally speaking, Python can find modules to import in a number of locations (see https://www.devdungeon.com/content/python-import-syspath-and-pythonpath-tutorial) and it is therefore possible to install packages using many different ways, however it is generally good to use a package manager. The most common ways to install external packages are:

  • Through the distribution (Miniforge or Anaconda)

    • using the conda command

  • Using the pip the Python package manager

  • From source (not covered here)

  • Using the operating system package manager (mainly Linux, also not covered here)

Installation using conda#

We recommend beginners to use a Python distribution based on the conda package manager, because it provides most of the commonly used Python packages, and simplifies the installation of packages that require additional libraries to work correctly, which does occasionally cause difficulties when packages are installed with pip. Also, conda can manage version dependencies between multiple packages automatically and can be used to create individual python environments.

Installation using conda#

conda is the preferred command-line tool to install and manage packages and environments in the Miniforge or Anaconda Python distribution. Below a few useful conda commands:

Install a package#

conda install PACKAGE_NAME

Update a package#

conda update PACKAGE_NAME

Uninstall a package#

conda remove PACKAGE_NAME

Create an environment#

conda create --name ENVIRONMENT_NAME python

Activate an environment#

conda activate ENVIRONMENT_NAME

Deactivate an environment#

conda deactivate

Search available packages#

conda search SEARCH_TERM

Install package from specific source like for example from conda-forge#

conda install -c conda-forge PACKAGE_NAME

List installed packages#

conda list --name ENVIRONMENT_NAME

Keeping a conda distribution up to date#

One of the advantages of using a conda based distribution is that it is straight forward to keep packages up to date. This is achieved using the conda command. You can find a detailed discussion of updating a conda distribution on thewebsite. In short if you want to update a single package simply use conda update [pkg name]. If you want to update the full distribution use conda update --all.

Package installation with pip#

Sometimes packages can not be found in the extensive public conda repository conda-forge, however they can often be found on PyPI the python package index. The tool for searching and installing packages from PyPI is pip the python package manager.

pip is a command-line tool for searching, installing removing and upgrading packages.

Searching for packages#

The way to find packages on PyPI used to be the search command pip search [pkgname], however unfortunately the PyPI package index had been hit by too many request from a large network of servers that made maintaining the tool unsustainable.

There are currently two solutions:

  1. Search on the website https://pypi.org

  2. Install pip-search and use the pip_search command-line tool (note the - in the package name, but _ in the command)

!pip_search numpy

Installing packages from PyPI#

To install the package use the pip install [pkgname] command.

Installing from github#

Sometimes a package might not be on PyPI but can be found on e.g. github. You can use the pip install -e <url> switch that installs from a given URL. So for example to install bokeh from git use pip install -e git+https://github.com/bokeh/bokeh. Note this also works for different version control systems (the git+ part in the URL) or other websites like e.g. gitlab.com

Removing and upgrading#

Removing#

Removing packages is straight forward using pip uninstall [pkgname].

Upgrading#

To upgrade a package use pip install --upgrade [pkgname]. There is no command to upgrade all packages in one go, however it is possible to get a list of upgradable packages which is pip list --outdated

pip help#

pip offers an extensive help system which can be accessed with pip help and pip help [command].

A note on virtual environments#

There is often advised to use virtual environments. This can be advantageous if one needs different package versions for different projects or to generally manage your environments separately and avoid to install all packages globaly. A discussion of the different ways to create and use virtual environments can be found here

Exercises#

  • Use pip or conda to install the VISA bindings for Python (note that unfortunately there are also bindings to the VISA card API, so check you are installing the correct module)

  • Install pyvisa-sim and pyvisa-py a simulated backend and a python backend for the Python VISA bindings

  • Find all the installed packages via pip

  • What does the -e option for pip install do. How can it be used to working with packages?

Contents