Use Spack to install dependencies and configure the project build

../_images/Shared-Build-Infrastructure.png

The Shared Build Infrastructure turns a specified target into installed dependencies and a CMake cached configuration file. We chose spack as the core tool to accomplish this task.

Spack is used to install the dependencies and generate a configuration file for the build. In RADIUSS projects, Uberenv drives Spack which itself is configured with RADIUSS Spack Configs. We call this the Shared Build Infrastructure, as a build counter part of the Shared CI Infrastructure: both are used by several RADIUSS projects to unify their workflow. Neither the CI nor the Build infrastructure are dependent on each other. They may supplement each other, with RADIUSS Spack Configs providing predefined CI jobs with spack specs of interest.

Note

In Spack, packages that inherit from the CachedCMakePackage class generate a CMake Cached configuration file during the project build. MFEM stands out as the only non-CMake project that also generates a configuration file that is used in the CI workflow with Uberenv.

Why Spack

Spack provides a single context to express toolchains, machine setup and build sequence. Using it allows us to share configuration files that describe toolchain and machine setup. RADIUSS Spack Configs is the repository where RADIUSS projects Spack configuration is shared.

Spack is increasingly used to install dependency trees of large simulation codes. As such, it makes sense to use Spack early in the development process.

Note

We are not promoting a “Spack everywhere” strategy. But we advocate that Spack should be one of the ways to configure and build your project, since it will likely be built that way when used in production.

End product

The typical end product is a script that takes a Spack spec as an input, and returns the configuration file generated by Spack after installing the dependencies for the given spec.

We rely on Uberenv to facilitate the setup of a local and isolated spack instance that will be used to build the project dependencies. We strongly suggest that you start with Uberenv to benefit from a reliable Spack usage in your CI (tried and tested) and keep your script simple.

Uberenv Guide

The role of Uberenv is to set up your Spack instance and then drive Spack to install your project dependencies and generate the configuration file.

Note

Uberenv will create a directory uberenv_libs containing a Spack instance with the required project dependencies installed. Spack then generates a CMake configuration file (<config_dependent_name>.cmake) at the root of the project repository.

One common source of error when using Uberenv is when the uberenv_libs folder is out of date after a Spack update. To resolve, make sure this uberenv_libs is deleted before running uberenv for the first time after an update because it needs to be regenerated.

Main steps

  1. Get uberenv.py script.

    Use git submodule add to get Uberenv into a uberenv directory.

  2. Edit the .uberenv_config.json file.

    Create the .uberenv_config.json file in a directory that is a parent of the uberenv directory. Projects typically place the file in the top-level directory of its repository. Set your project package name, and other parameters like Spack reference commit/tag.

    Note

    We recommend using the Spack reference mentionned in RADIUSS Spack Configs closest parent tag. In the RADIUSS Spack Configs history, look for the tag that is the closest parent of the checkout commit. It is made of the latest supported release version of RAJA and Umpire together with the Spack reference used to vet RADIUSS Spack Configs. Newer or older Spack references may not work out of the box.

  3. Add RADIUSS Spack Configs submodule.

    • Use git submodule add to get RADIUSS Spack Configs in a second submodule or custom location.

    • In .uberenv_config.json set spack_configs_path to point to <some_relative_path>/radiuss-spack-configs.

  4. Add custom packages.

    Radiuss Spack Configs now gathers the RADIUSS custom spack packages.

    If you need to make local modifications to your project package, we suggest creating a branch in RADIUSS Spack Configs as it is likely your changes will be need by projects depending on yours.

    Then, in .uberenv_config.json, set spack_packages_path to point to <some_relative_path>/radiuss-spack-configs/packages

    However, you can still use packages defined locally instead of the RADIUSS Spack Configs ones. Let’s say you place them in <some_relative_path>/packages/<package_name>/package.py.

    Then, in .uberenv_config.json, set spack_packages_path to point to <some_relative_path>/packages

  5. Make sure that the package.py file for your project generates a CMake configuration file.

    This is usually done adding a specific stage to the package. In particular, Spack now supports this for CMake build system with the CachedCMakePackages class. (see Setup your Spack package to generate a configuration file for details, and Umpire, CHAI, RAJA for implementation examples).

Get the shared Spack configuration

We share Spack configuration files in RADIUSS Spack Configs. In this repo you will find:

  • config.yaml for Spack general configuration.

  • modules.yaml for modules creation by Spack.

  • One compilers.yaml and packages.yaml per system type, describing the installed toolchain on each machine.

  • a packages directory containing some Spack packages tuned for our needs.

Depending on the machine/system, we may or may not provide a spack configuration allowing you to use it right away. Please refer to RADIUSS Spack Configs documentation about adding a new machine. This will be welcome by the RADIUSS teams using it!

Note

MacOS (darwin): it is not trivial to provide a universal configuration for MacOS. Instead, developers will likely have to complete the packages.yaml file in order to adapt the location and version of externally installed dependencies. MacOS is not available on LC systems, the Spack configuration is provided as-is, for development use.

Setup your Spack package to generate a configuration file

We want to build the dependencies with Spack and then build the project with those dependencies outside of Spack. We need to generate a CMake configuration file that reproduces the configuration Spack would have generated in the same context. It should contain all the information necessary to build your project with the described toolchain and dependencies.

In particular, the configuration file should setup:

  • flags corresponding with the target requested (Release, Debug).

  • paths to compilers and other toolkits (e.g. cuda), etc.

  • paths to installed dependencies.

  • any options that may impact the build.

This provides an easy way to build your project based on Spack configuration while only using CMake and a traditional developer workflow.

CMake projects: Spack CachedCMakePackage

CMake is strongly recommended to use the RADIUSS CI workflow, mostly because of this step. With CMake, we generate a cache file describing the configuration necessary to build the code for a project. This is supported in Spack as soon as your package inherits from CachedCMakePackage.

When your package is ported, stopping an installation after the initconfig phase will prevent Spack from building your project after the CMake configuration file is generated.

Non-CMake projects: Custom implementation

The only example of a non-CMake project that has adopted this workflow is MFEM. Although it is using a Makefile build system in its Spack packages, MFEM is generating a configuration file that can be used just like a CMake configuration file. We adapted the implementation of the package to mimic the mechanism available in CMake-based packages. You may use that as an example.