Reference Guide¶

This page provides a reference guide of the ReFrame API for writing regression tests covering all the relevant details. Internal data structures and APIs are covered only to the extent that might be helpful to the final user of the framework.

Regression test classes and related utilities¶

class reframe.RegressionTest(name=None, prefix=None)¶: This is an alias of reframe.core.pipeline.RegressionTest.

New in version 2.13.

class reframe.RunOnlyRegressionTest(*args, **kwargs)¶: This is an alias of reframe.core.pipeline.RunOnlyRegressionTest.

New in version 2.13.

class reframe.CompileOnlyRegressionTest(*args, **kwargs)¶: This is an alias of reframe.core.pipeline.CompileOnlyRegressionTest.

New in version 2.13.

@reframe.simple_test¶: This is an alias of reframe.core.decorators.simple_test().

New in version 2.13.

@reframe.parameterized_test(inst=[])¶: This is an alias of reframe.core.decorators.parameterized_test().

New in version 2.13.

reframe.core.decorators.parameterized_test(*inst)[source]¶

Class decorator for registering multiple instantiations of a test class.

The decorated class must derive from reframe.core.pipeline.RegressionTest. This decorator is also available directly under the reframe module.

Parameters:	inst – The different instantiations of the test. Each instantiation argument may be either a sequence or a mapping.

New in version 2.13.

Note

This decorator does not instantiate any test. It only registers them. The actual instantiation happens during the loading phase of the test.

reframe.core.decorators.simple_test(cls)[source]¶

Class decorator for registering parameterless tests with ReFrame.

The decorated class must derive from reframe.core.pipeline.RegressionTest. This decorator is also available directly under the reframe module.

New in version 2.13.

reframe.core.decorators.required_version(*versions)[source]¶

Class decorator for specifying the required ReFrame versions for the following test.

If the test is not compatible with the current ReFrame version it will be skipped.

Parameters:

versions –

A list of ReFrame version specifications that this test is allowed to run. A version specification string can have one of the following formats:

VERSION: Specifies a single version.

2. {OP}VERSION, where {OP} can be any of >, >=, <, <=, == and !=. For example, the version specification string '>=2.15' will only allow the following test to be loaded only by ReFrame 2.15 and higher. The ==VERSION specification is the equivalent of VERSION.

V1..V2: Specifies a range of versions.

You can specify multiple versions with this decorator, such as @required_version('2.13', '>=2.16'), in which case the test will be selected if any of the versions is satisfied, even if the versions specifications are conflicting.

New in version 2.13.

class reframe.core.pipeline.RegressionTest(name=None, prefix=None)[source]¶

Bases: object

Base class for regression tests.

All regression tests must eventually inherit from this class. This class provides the implementation of the pipeline phases that the regression test goes through during its lifetime.

Parameters:	name – The name of the test. If `None`, the framework will try to assign a unique and human-readable name to the test. prefix – The directory prefix of the test. If `None`, the framework will set it to the directory containing the test file.

Note

The name and prefix arguments are just maintained for backward compatibility to the old (prior to 2.13) syntax of regression tests. Users are advised to use the new simplified syntax for writing regression tests. Refer to the ReFrame Tutorial for more information.

This class is also directly available under the top-level reframe module.

Changed in version 2.13.

build_system¶

The build system to be used for this test. If not specified, the framework will try to figure it out automatically based on the value of sourcepath.

This field may be set using either a string referring to a concrete build system class name (see build systems) or an instance of reframe.core.buildsystems.BuildSystem. The former is the recommended way.

Type:	`str` or `reframe.core.buildsystems.BuildSystem`.
Default:	`None`.

New in version 2.14.

check_performance()[source]¶

The performance checking phase of the regression test pipeline.

Raises:	reframe.core.exceptions.SanityError – If the performance check fails.

check_sanity()[source]¶

The sanity checking phase of the regression test pipeline.

Raises:	reframe.core.exceptions.SanityError – If the sanity check fails.

cleanup(remove_files=False, unload_env=True)[source]¶

The cleanup phase of the regression test pipeline.

Parameters:	remove_files – If `True`, the stage directory associated with this test will be removed. unload_env – If `True`, the environment that was used to run this test will be unloaded.

compile(**compile_opts)[source]¶

The compilation phase of the regression test pipeline.

Parameters:	compile_opts – Extra options to be passed to the programming environment for compiling the source code of the test.
Raises:	reframe.core.exceptions.ReframeError – In case of errors.

compile_wait()[source]¶: Wait for compilation phase to finish.

New in version 2.13.

current_environ¶

The programming environment that the regression test is currently executing with.

This is set by the framework during the setup() phase.

Type:	`reframe.core.environments.Environment`.

current_partition¶

The system partition the regression test is currently executing on.

This is set by the framework during the setup() phase.

Type:	`reframe.core.systems.SystemPartition`.

current_system¶

The system the regression test is currently executing on.

This is set by the framework during the initialization phase.

Type:	`reframe.core.runtime.HostSystem`.

descr¶

A detailed description of the test.

Type:	`str`
Default:	`self.name`

exclusive_access¶

Specify whether this test needs exclusive access to nodes.

Type:	boolean
Default:	`False`

executable¶

The name of the executable to be launched during the run phase.

Type:	`str`
Default:	`os.path.join('.', self.name)`

executable_opts¶

List of options to be passed to the executable.

Type:	`list[str]`
Default:	`[]`

extra_resources¶

Extra resources for this test.

This field is for specifying custom resources needed by this test. These resources are defined in the configuration of a system partition. For example, assume that two additional resources, named gpu and datawarp, are defined in the configuration file as follows:

'resources': {
    'gpu': [
        '--gres=gpu:{num_gpus_per_node}'
    ],
    'datawarp': [
        '#DW jobdw capacity={capacity}',
        '#DW stage_in source={stagein_src}'
    ]
}

A regression test then may instantiate the above resources by setting the extra_resources attribute as follows:

self.extra_resources = {
    'gpu': {'num_gpus_per_node': 2}
    'datawarp': {
        'capacity': '100GB',
        'stagein_src': '/foo'
    }
}

The generated batch script (for Slurm) will then contain the following lines:

#SBATCH --gres=gpu:2
#DW jobdw capacity=100GB
#DW stage_in source=/foo

Notice that if the resource specified in the configuration uses an alternative directive prefix (in this case #DW), this will replace the standard prefix of the backend scheduler (in this case #SBATCH)

If the resource name specified in this variable does not match a resource name in the partition configuration, it will be simply ignored. The num_gpus_per_node attribute translates internally to the _rfm_gpu resource, so that setting self.num_gpus_per_node = 2 is equivalent to the following:

self.extra_resources = {'_rfm_gpu': {'num_gpus_per_node': 2}}

Type:	`dict[str, dict[str, object]]`
Default:	`{}`

Note

New in version 2.8.

Changed in version 2.9.

A new more powerful syntax was introduced that allows also custom job script directive prefixes.

info()[source]¶

Provide live information of a running test.

This method is used by the front-end to print the status message during the test’s execution. This function is also called to provide the message for the check_info logging attribute. By default, it returns a message reporting the test name, the current partition and the current programming environment that the test is currently executing on.

Returns:	a string with an informational message about this test

Note

When overriding this method, you should pay extra attention on how you use the RegressionTest’s attributes, because this method may be called at any point of the test’s lifetime.

New in version 2.10.

is_local()[source]¶

Check if the test will execute locally.

A test executes locally if the local attribute is set or if the current partition’s scheduler does not support job submission.

job¶

The job descriptor associated with this test.

This is set by the framework during the setup() phase.

Type:	`reframe.core.schedulers.Job`.

keep_files¶

List of files to be kept after the test finishes.

By default, the framework saves the standard output, the standard error and the generated shell script that was used to run this test.

These files will be copied over to the framework’s output directory during the cleanup() phase.

Directories are also accepted in this field.

Relative path names are resolved against the stage directory.

Type:	`list[str]`
Default:	`[]`

local¶

Always execute this test locally.

Type:	boolean
Default:	`False`

logger¶

A logger associated with the this test.

You can use this logger to log information for your test.

maintainers¶

List of people responsible for this test.

When the test fails, this contact list will be printed out.

Type:	`list[str]`
Default:	`[]`

modules¶

List of modules to be loaded before running this test.

These modules will be loaded during the setup() phase.

Type:	`list[str]`
Default:	`[]`

name¶

The name of the test.

Type:	string that can contain any character except `/`

num_cpus_per_task¶

Number of CPUs per task required by this test.

Ignored if None.

Type:	integral or `None`
Default:	`None`

num_gpus_per_node¶

Number of GPUs per node required by this test.

Type:	integral
Default:	`0`

num_tasks¶

Number of tasks required by this test.

If the number of tasks is set to 0, ReFrame will try to use all the available nodes of a reservation. A reservation must be specified through the –reservation command-line option, otherwise the regression test will fail during submission. ReFrame will try to run the test on all the nodes of the reservation that satisfy the selection criteria of the current virtual partition (i.e., constraints and/or partitions).

Type:	integral
Default:	`1`

Note

Changed in version 2.9: Added support for running the test using all the nodes of the specified reservation if the number of tasks is set to 0.

num_tasks_per_core¶

Number of tasks per core required by this test.

Ignored if None.

Type:	integral or `None`
Default:	`None`

num_tasks_per_node¶

Number of tasks per node required by this test.

Ignored if None.

Type:	integral or `None`
Default:	`None`

num_tasks_per_socket¶

Number of tasks per socket required by this test.

Ignored if None.

Type:	integral or `None`
Default:	`None`

outputdir¶

The output directory of the test.

This is set during the setup() phase.

New in version 2.13.

Type:	`str`.

perf_patterns¶

Patterns for verifying the performance of this test.

Refer to the ReFrame Tutorial for concrete usage examples.

If set to None, no performance checking will be performed.

Type:	A dictionary with keys of type `str` and deferrable expressions (i.e., the result of a sanity function) as values. `None` is also allowed.
Default:	`None`

poll()[source]¶

Poll the test’s state.

Returns:	`True` if the associated job has finished, `False` otherwise. If no job descriptor is yet associated with this test, `True` is returned.
Raises:	reframe.core.exceptions.ReframeError – In case of errors.

post_run¶

List of shell commands to execute after launching this job.

See pre_run for a more detailed description of the semantics.

Type:	`list` of `str`
Default:	`[]`

Note

New in version 2.10.

postbuild_cmd¶

List of shell commands to be executed after a successful compilation.

These commands are executed during the compilation phase and from inside the stage directory. Each entry in the list spawns a new shell.

Type:	`list[str]`
Default:	`[]`

pre_run¶

List of shell commands to execute before launching this job.

These commands do not execute in the context of ReFrame. Instead, they are emitted in the generated job script just before the actual job launch command.

Type:	`list` of `str`
Default:	`[]`

Note

New in version 2.10.

prebuild_cmd¶

List of shell commands to be executed before compiling.

These commands are executed during the compilation phase and from inside the stage directory. Each entry in the list spawns a new shell.

Type:	`list[str]`
Default:	`[]`

prefix¶

The prefix directory of the test.

Type:	`str`.

readonly_files¶

List of files or directories (relative to the sourcesdir) that will be symlinked in the stage directory and not copied.

You can use this variable to avoid copying very large files to the stage directory.

Type:	`list[str]`
Default:	`[]`

reference¶

The set of reference values for this test.

Refer to the ReFrame Tutorial for concrete usage examples.

Type:	A scoped dictionary with system names as scopes or `None`
Default:	`{}`

run()[source]¶

The run phase of the regression test pipeline.

This call is non-blocking. It simply submits the job associated with this test and returns.

sanity_patterns¶

Refer to the ReFrame Tutorial for concrete usage examples.

If set to None, a sanity error will be raised during sanity checking.

Type:	A deferrable expression (i.e., the result of a sanity function) or `None`
Default:	`None`

Note

Changed in version 2.9: The default behaviour has changed and it is now considered a sanity failure if this attribute is set to None.

If a test doesn’t care about its output, this must be stated explicitly as follows:

self.sanity_patterns = sn.assert_found(r'.*', self.stdout)

setup(partition, environ, **job_opts)[source]¶

The setup phase of the regression test pipeline.

Parameters:	partition – The system partition to set up this test for. environ – The environment to set up this test for. job_opts – Options to be passed through to the backend scheduler. When overriding this method users should always pass through `job_opts` to the base class method.
Raises:	reframe.core.exceptions.ReframeError – In case of errors.

sourcepath¶

The path to the source file or source directory of the test.

It must be a path relative to the sourcesdir, pointing to a subfolder or a file contained in sourcesdir. This applies also in the case where sourcesdir is a Git repository.

If it refers to a regular file, this file will be compiled using the SingleSource build system. If it refers to a directory, ReFrame will try to infer the build system to use for the project and will fall back in using the Make build system, if it cannot find a more specific one.

Type:	`str`
Default:	`''`

sourcesdir¶

The directory containing the test’s resources.

This directory may be specified with an absolute path or with a path relative to the location of the test. Its contents will always be copied to the stage directory of the test.

This attribute may also accept a URL, in which case ReFrame will treat it as a Git repository and will try to clone its contents in the stage directory of the test.

If set to None, the test has no resources an no action is taken.

Type:	`str` or `None`
Default:	`'src'`

Note

Changed in version 2.9: Allow None values to be set also in regression tests with a compilation phase

Changed in version 2.10: Support for Git repositories was added.

stagedir¶

The stage directory of the test.

This is set during the setup() phase.

Type:	`str`.

stderr¶

The name of the file containing the standard error of the test.

This is set during the setup() phase.

This attribute is evaluated lazily, so it can by used inside sanity expressions.

Type:	`str`.

stdout¶

The name of the file containing the standard output of the test.

This is set during the setup() phase.

This attribute is evaluated lazily, so it can by used inside sanity expressions.

Type:	`str`.

strict_check¶

Mark this test as a strict performance test.

If a test is marked as non-strict, the performance checking phase will always succeed, unless the --strict command-line option is passed when invoking ReFrame.

Type:	boolean
Default:	`True`

tags¶

Set of tags associated with this test.

This test can be selected from the frontend using any of these tags.

Type:	`set[str]`
Default:	an empty set

time_limit¶

Time limit for this test.

Time limit is specified as a three-tuple in the form (hh, mm, ss), with hh >= 0, 0 <= mm <= 59 and 0 <= ss <= 59.

Type:	`tuple[int]`
Default:	`(0, 10, 0)`

use_multithreading¶

Specify whether this tests needs simultaneous multithreading enabled.

Ignored if None.

Type:	boolean or `None`
Default:	`None`

valid_prog_environs¶

List of programming environments supported by this test.

Type:	`list[str]`
Default:	`[]`

Note

Changed in version 2.12: Programming environments can now be specified using wildcards.

valid_systems¶

List of systems supported by this test. The general syntax for systems is <sysname>[:<partname].

Type:	`list[str]`
Default:	`[]`

variables¶

Environment variables to be set before running this test.

These variables will be set during the setup() phase.

Type:	`dict[str, str]`
Default:	`{}`

wait()[source]¶

Wait for this test to finish.

Raises:	reframe.core.exceptions.ReframeError – In case of errors.

class reframe.core.pipeline.RunOnlyRegressionTest(name=None, prefix=None)[source]¶

Bases: reframe.core.pipeline.RegressionTest

Base class for run-only regression tests.

This class is also directly available under the top-level reframe module.

compile(**compile_opts)[source]¶

The compilation phase of the regression test pipeline.

This is a no-op for this type of test.

compile_wait()[source]¶

Wait for compilation phase to finish.

This is a no-op for this type of test.

run()[source]¶

The run phase of the regression test pipeline.

The resources of the test are copied to the stage directory and the rest of execution is delegated to the RegressionTest.run().

class reframe.core.pipeline.CompileOnlyRegressionTest(*args, **kwargs)[source]¶

Bases: reframe.core.pipeline.RegressionTest

Base class for compile-only regression tests.

These tests are by default local and will skip the run phase of the regression test pipeline.

The standard output and standard error of the test will be set to those of the compilation stage.

This class is also directly available under the top-level reframe module.

run()[source]¶

The run stage of the regression test pipeline.

Implemented as no-op.

setup(partition, environ, **job_opts)[source]¶

The setup stage of the regression test pipeline.

Similar to the RegressionTest.setup(), except that no job descriptor is set up for this test.

wait()[source]¶

Wait for this test to finish.

Implemented as no-op

Environments and Systems¶

class reframe.core.environments.Environment(name, modules=[], variables={}, **kwargs)[source]¶

Bases: object

This class abstracts away an environment to run regression tests.

It is simply a collection of modules to be loaded and environment variables to be set when this environment is loaded by the framework. Users may not create or modify directly environments.

is_loaded¶: True if this environment is loaded, False otherwise.

modules¶

The modules associated with this environment.

Type:	`list` of `str`

name¶

The name of this environment.

Type:	`str`

variables¶

The environment variables associated with this environment.

Type:	dictionary of `str` keys/values.

class reframe.core.environments.ProgEnvironment(name, modules=[], variables={}, cc='cc', cxx='CC', ftn='ftn', nvcc='nvcc', cppflags=None, cflags=None, cxxflags=None, fflags=None, ldflags=None, **kwargs)[source]¶

Bases: reframe.core.environments.Environment

A class representing a programming environment.

This type of environment adds also attributes for setting the compiler and compilation flags.

If compilation flags are set to None (the default, if not set otherwise in ReFrame’s configuration), they are not passed to the make invocation.

If you want to disable completely the propagation of the compilation flags to the make invocation, even if they are set, you should set the propagate attribute to False.

cc¶

The C compiler of this programming environment.

Type:	`str`

cflags¶

The C compiler flags of this programming environment.

Type:	`str` or `None`

cppflags¶

The preprocessor flags of this programming environment.

Type:	`str` or `None`

cxx¶

The C++ compiler of this programming environment.

Type:	`str` or `None`

cxxflags¶

The C++ compiler flags of this programming environment.

Type:	`str` or `None`

fflags¶

The Fortran compiler flags of this programming environment.

Type:	`str` or `None`

ftn¶

The Fortran compiler of this programming environment.

Type:	`str` or `None`

include_search_path¶

The include search path of this programming environment.

Type:	`list` of `str`
Default:	`[]`

ldflags¶

The linker flags of this programming environment.

Type:	`str` or `None`

propagate¶

Propagate the compilation flags to the make invocation.

Type:	`bool`
Default:	`True`

class reframe.core.environments.save_environment[source]¶

Bases: object

A context manager for saving and restoring the current environment.

class reframe.core.systems.System(name, descr=None, hostnames=[], partitions=[], prefix='.', stagedir=None, outputdir=None, perflogdir=None, resourcesdir='.', modules_system=None)[source]¶

Bases: object

A representation of a system inside ReFrame.

descr¶: The description of this system.

hostnames¶: The hostname patterns associated with this system.

modules_system¶: The modules system name associated with this system.

name¶: The name of this system.

outputdir¶: The ReFrame output directory prefix associated with this system.

partitions¶: All the system partitions associated with this system.

perflogdir¶: The ReFrame log directory prefix associated with this system.

prefix¶: The ReFrame prefix associated with this system.

resourcesdir¶

Global resources directory for this system.

You may use this directory for storing large resource files of your regression tests. See here on how to configure this.

Type:	`str`

stagedir¶: The ReFrame stage directory prefix associated with this system.

class reframe.core.systems.SystemPartition(name, descr=None, scheduler=None, launcher=None, access=[], environs=[], resources={}, local_env=None, max_jobs=1)[source]¶

Bases: object

A representation of a system partition inside ReFrame.

descr¶: A detailed description of this partition.

fullname¶

Return the fully-qualified name of this partition.

The fully-qualified name is of the form <parent-system-name>:<partition-name>.

Type:	str

launcher¶

The type of the backend launcher of this partition.

Returns:	a subclass of `reframe.core.launchers.JobLauncher`.

Note

New in version 2.8.

name¶

The name of this partition.

Type:	str

scheduler¶

The type of the backend scheduler of this partition.

Returns:	a subclass of `reframe.core.schedulers.Job`.

Note

Changed in version 2.8.

Prior versions returned a string representing the scheduler and job launcher combination.

Job schedulers and parallel launchers¶

class reframe.core.schedulers.Job(name, launcher, workdir='.', num_tasks=1, num_tasks_per_node=None, num_tasks_per_core=None, num_tasks_per_socket=None, num_cpus_per_task=None, use_smt=None, time_limit=(0, 10, 0), script_filename=None, stdout=None, stderr=None, pre_run=[], post_run=[], sched_account=None, sched_partition=None, sched_reservation=None, sched_nodelist=None, sched_exclude_nodelist=None, sched_exclusive_access=None, sched_options=[])[source]¶

Bases: abc.ABC

A job descriptor.

Caution

This is an abstract class. Users may not create jobs directly.

launcher¶

The parallel program launcher that will be used to launch the parallel executable of this job.

Type:	`reframe.core.launchers.JobLauncher`

options¶

Options to be passed to the backend job scheduler.

Type:	`list` of `str`
Default:	`[]`

class reframe.core.launchers.JobLauncher(options=[])[source]¶

Bases: abc.ABC

A job launcher.

A job launcher is the executable that actually launches a distributed program to multiple nodes, e.g., mpirun, srun etc.

Note

This is an abstract class. Regression tests may not instantiate this class directly.

Note

Changed in version 2.8: Job launchers do not get a reference to a job during their initialization.

command(job)[source]¶

The launcher command.

Parameters:	job – A `reframe.core.schedulers.Job` that will be used by this launcher to properly emit its options. Subclasses may override this method and emit options according the number of tasks associated to the job etc.
Returns:	a list of command line arguments (including the launcher executable).

options¶

List of options to be passed to the job launcher invocation.

Type:	`list` of `str`
Default:	`[]`

class reframe.core.launchers.LauncherWrapper(target_launcher, wrapper_command, wrapper_options=[])[source]¶

Bases: reframe.core.launchers.JobLauncher

Wrap a launcher object so as to modify its invocation.

This is useful for parallel debuggers. For example, to launch a regression test using the DDT debugger, you can do the following:

def setup(self, partition, environ, **job_opts):
    super().setup(partition, environ, **job_opts)
    self.job.launcher = LauncherWrapper(self.job.launcher, 'ddt',
                                        ['--offline'])

If the current system partition uses native Slurm for job submission, this setup will generate the following command in the submission script:

ddt --offline srun <test_executable>

If the current partition uses mpirun instead, it will generate

ddt --offline mpirun -np <num_tasks> ... <test_executable>

Parameters:	target_launcher – The launcher to wrap. wrapper_command – The wrapper command. wrapper_options – List of options to pass to the wrapper command.

reframe.core.launchers.registry.getlauncher(name)[source]¶

Get launcher by its registered name.

The available names are those specified in the configuration file.

This method may become handy in very special situations, e.g., testing an application that needs to replace the system partition launcher or if a different launcher must be used for a different programming environment.

For example, if you want to replace the current partition’s launcher with the local one, here is how you can achieve it:

def setup(self, partition, environ, **job_opts):
    super().setup(partition, environ, **job_opts)
    self.job.launcher = getlauncher('local')()

Note that this method returns a launcher class type and not an instance of that class. You have to instantiate it explicitly before assigning it to the launcher attribute of the job.

Note

New in version 2.8.

Parameters:	name – The name of the launcher to retrieve.
Returns:	The class of the launcher requested, which is a subclass of `reframe.core.launchers.JobLauncher`.
Raises:	reframe.core.exceptions.ConfigError – if no launcher is registered with that name.

reframe.core.launchers.registry.register_launcher(name, local=False)[source]¶

Class decorator for registering new job launchers.

Caution

This decorator is only relevant to developers of new job launchers.

Note

New in version 2.8.

Parameters:	name – The registration name of this launcher local – `True` if launcher may only submit local jobs, `False` otherwise.
Raises:	ValueError – if a job launcher is already registered with the same name.

Runtime services¶

class reframe.core.runtime.HostResources(prefix=None, stagedir=None, outputdir=None, perflogdir=None, timefmt=None)[source]¶

Bases: object

Resources associated with ReFrame execution on the current host.

Note

New in version 2.13.

output_prefix¶: The output prefix directory of ReFrame.

prefix¶

The prefix directory of ReFrame execution. This is always an absolute path.

Type:	`str`

Caution

Users may not set this field.

stage_prefix¶: The stage prefix directory of ReFrame.

class reframe.core.runtime.HostSystem(system, partname=None)[source]¶

Bases: object

The host system of the framework.

The host system is a representation of the system that the framework currently runs on.If the framework is properly configured, the host system is automatically detected. If not, it may be explicitly set by the user.

This class is mainly a proxy of reframe.core.systems.System that stores optionally a partition name and provides some additional functionality for manipulating system partitions.

All attributes of the reframe.core.systems.System may be accessed directly from this proxy.

Note

New in version 2.13.

partition(name)[source]¶

Return the system partition name.

Type:	`reframe.core.systems.SystemPartition`.

partitions¶

The partitions of this system.

Type:	`list[reframe.core.systems.SystemPartition]`.

class reframe.core.runtime.RuntimeContext(dict_config, sysdescr=None)[source]¶

Bases: object

The runtime context of the framework.

This class essentially groups the current host system and the associated resources of the framework on the current system.

There is a single instance of this class globally in the framework.

Note

New in version 2.13.

modules_system¶

The modules system used by the current host system.

Type:	`reframe.core.modules.ModulesSystem`.

resources¶

The framework resources.

Type:	`reframe.core.runtime.HostResources`

system¶

The current host system.

Type:	`reframe.core.runtime.HostSystem`

reframe.core.runtime.runtime()[source]¶

Retrieve the framework’s runtime context.

Type:	`reframe.core.runtime.RuntimeContext`

Note

New in version 2.13.

Modules System API¶

class reframe.core.modules.ModulesSystem(backend)[source]¶

A modules system abstraction inside ReFrame.

This class interfaces between the framework internals and the actual modules systems implementation.

conflicted_modules(name)[source]¶

Return the list of the modules conflicting with module name.

If module name resolves to multiple real modules, then the returned list will be the concatenation of the conflict lists of all the real modules.

This method returns a list of strings.

emit_load_commands(name)[source]¶: Return the appropriate shell command for loading module name.

emit_unload_commands(name)[source]¶: Return the appropriate shell command for unloading module name.

is_module_loaded(name)[source]¶

Check if module name is loaded.

If module name refers to multiple real modules, this method will return True only if all the referees are loaded.

load_mapping(mapping)[source]¶

Update the internal module mappings using a single mapping.

Parameters:	mapping – a string specifying the module mapping. Example syntax: `'m0: m1 m2'`.

load_mapping_from_file(filename)[source]¶: Update the internal module mappings from mappings read from file.

load_module(name, force=False)[source]¶

Load the module name.

If force is set, forces the loading, unloading first any conflicting modules currently loaded. If module name refers to multiple real modules, all of the target modules will be loaded.

Returns the list of unloaded modules as strings.

loaded_modules()[source]¶

Return a list of loaded modules.

This method returns a list of strings.

name¶: Return the name of this module system.

resolve_module(name)[source]¶

Resolve module name in the registered module map.

Returns:	the list of real modules names pointed to by `name`.
Raises:	`reframe.core.exceptions.ConfigError` if the mapping contains a cycle.

searchpath¶: The module system search path as a list of directories.

searchpath_add(*dirs)[source]¶: Add dirs to the module system search path.

searchpath_remove(*dirs)[source]¶: Remove dirs from the module system search path.

unload_all()[source]¶: Unload all loaded modules.

unload_module(name)[source]¶

Unload module name.

If module name refers to multiple real modules, all the referred to modules will be unloaded in reverse order.

version¶: Return the version of this module system.

Build systems¶

New in version 2.14.

ReFrame delegates the compilation of the regression test to a build system. Build systems in ReFrame are entities that are responsible for generating the necessary shell commands for compiling a code. Each build system defines a set of attributes that users may set in order to customize their compilation. An example usage is the following:

self.build_system = 'SingleSource'
self.build_system.cflags = ['-fopenmp']

Users simply set the build system to use in their regression tests and then they configure it. If no special configuration is needed for the compilation, users may completely ignore the build systems. ReFrame will automatically pick one based on the regression test attributes and will try to compile the code.

All build systems in ReFrame derive from the abstract base class reframe.core.buildsystems.BuildSystem. This class defines a set of common attributes, such us compilers, compilation flags etc. that all subclasses inherit. It is up to the concrete build system implementations on how to use or not these attributes.

class reframe.core.buildsystems.Autotools[source]¶

Bases: reframe.core.buildsystems.ConfigureBasedBuildSystem

A build system for compiling Autotools-based projects.

This build system will emit the following commands:

Create a build directory if builddir is not None and change to it.
Invoke configure to configure the project by setting the corresponding flags for compilers and compiler flags.
Issue make to compile the code.

class reframe.core.buildsystems.BuildSystem[source]¶

Bases: object

The abstract base class of any build system.

Concrete build systems inherit from this class and must override the emit_build_commands() abstract function.

cc¶

The C compiler to be used. If set to None and flags_from_environ is True, the compiler defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

cflags¶

The C compiler flags to be used. If set to None and flags_from_environ is True, the corresponding flags defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

cppflags¶

The preprocessor flags to be used. If set to None and flags_from_environ is True, the corresponding flags defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

cxx¶

The C++ compiler to be used. If set to None and flags_from_environ is True, the compiler defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

cxxflags¶

The C++ compiler flags to be used. If set to None and flags_from_environ is True, the corresponding flags defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

emit_build_commands(environ)[source]¶

Return the list of commands for building using this build system.

The build commands may always assume to be issued from the top-level directory of the code that is to be built.

Parameters:	environ (`reframe.core.environments.ProgEnvironment`) – The programming environment for which to emit the build instructions. The framework passes here the current programming environment.
Raises:	`BuildSystemError` in case of errors when generating the build instructions.

Note

This method is relevant only to developers of new build systems.

fflags¶

The Fortran compiler flags to be used. If set to None and flags_from_environ is True, the corresponding flags defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

flags_from_environ¶

Set compiler and compiler flags from the current programming environment if not specified otherwise.

Type:	`bool`
Default:	`True`

ftn¶

The Fortran compiler to be used. If set to None and flags_from_environ is True, the compiler defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

ldflags¶

The linker flags to be used. If set to None and flags_from_environ is True, the corresponding flags defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

nvcc¶

The CUDA compiler to be used. If set to None and flags_from_environ is True, the compiler defined in the current programming environment will be used.

Type:	`str`
Default:	`None`

class reframe.core.buildsystems.CMake[source]¶

Bases: reframe.core.buildsystems.ConfigureBasedBuildSystem

A build system for compiling CMake-based projects.

This build system will emit the following commands:

Create a build directory if builddir is not None and change to it.
Invoke cmake to configure the project by setting the corresponding CMake flags for compilers and compiler flags.
Issue make to compile the code.

class reframe.core.buildsystems.ConfigureBasedBuildSystem[source]¶

Bases: reframe.core.buildsystems.BuildSystem

Abstract base class for configured-based build systems.

builddir¶

The CMake build directory, where all the generated files will be placed.

Type:	`str`
Default:	`None`

config_opts¶

Additional configuration options to be passed to the CMake invocation.

Type:	`list[str]`
Default:	`[]`

make_opts¶

Options to be passed to the subsequent make invocation.

Type:	`list[str]`
Default:	`[]`

max_concurrency¶

Same as for the Make build system.

Type:	integer
Default:	`None`

srcdir¶

The top-level directory of the code.

This is set automatically by the framework based on the reframe.core.pipeline.RegressionTest.sourcepath attribute.

Type:	`str`
Default:	`None`

class reframe.core.buildsystems.Make[source]¶

Bases: reframe.core.buildsystems.BuildSystem

A build system for compiling codes using make.

The generated build command has the following form:

make -j [N] [-f MAKEFILE] [-C SRCDIR] CC='X' CXX='X' FC='X' NVCC='X' CPPFLAGS='X' CFLAGS='X' CXXFLAGS='X' FCFLAGS='X' LDFLAGS='X' OPTIONS

The compiler and compiler flags variables will only be passed if they are not None. Their value is determined by the corresponding attributes of BuildSystem. If you want to completely disable passing these variables to the make invocation, you should make sure not to set any of the correspoding attributes and set also the BuildSystem.flags_from_environ flag to False.

makefile¶

Instruct build system to use this Makefile. This option is useful when having non-standard Makefile names.

Type:	`str`
Default:	`None`

max_concurrency¶

Limit concurrency for make jobs. This attribute controls the -j option passed to make. If not None, make will be invoked as make -j max_concurrency. Otherwise, it will invoked as make -j.

Type:	integer
Default:	`None`

options¶

Append these options to the make invocation. This variable is also useful for passing variables or targets to make.

Type:	`list[str]`
Default:	`[]`

srcdir¶

The top-level directory of the code.

This is set automatically by the framework based on the reframe.core.pipeline.RegressionTest.sourcepath attribute.

Type:	`str`
Default:	`None`

class reframe.core.buildsystems.SingleSource[source]¶

Bases: reframe.core.buildsystems.BuildSystem

A build system for compiling a single source file.

The generated build command will have the following form:

COMP CPPFLAGS XFLAGS SRCFILE -o EXEC LDFLAGS

COMP is the required compiler for compiling SRCFILE. This build system will automatically detect the programming language of the source file and pick the correct compiler. See also the SingleSource.lang attribute.
CPPFLAGS are the preprocessor flags and are passed to any compiler.
XFLAGS is any of CFLAGS, CXXFLAGS or FCFLAGS depending on the programming language of the source file.
SRCFILE is the source file to be compiled. This is set up automatically by the framework. See also the SingleSource.srcfile attribute.
EXEC is the executable to be generated. This is also set automatically by the framework. See also the SingleSource.executable attribute.
LDFLAGS are the linker flags.

For CUDA codes, the language assumed is C++ (for the compilation flags) and the compiler used is BuildSystem.nvcc.

executable¶

The executable file to be generated.

This is set automatically by the framework based on the reframe.core.pipeline.RegressionTest.executable attribute.

Type:	`str` or `None`

include_path¶

The include path to be used for this compilation.

All the elements of this list will be appended to the BuildSystem.cppflags, by prepending to each of them the -I option.

Type:	`list[str]`
Default:	`[]`

lang¶

The programming language of the file that needs to be compiled. If not specified, the build system will try to figure it out automatically based on the extension of the source file. The automatically detected extensions are the following:

C: .c.

C++: .cc, .cp, .cxx, .cpp, .CPP, .c++ and .C.

Fortran: .f, .for, .ftn, .F, .FOR, .fpp, .FPP, .FTN, .f90, .f95, .f03, .f08, .F90, .F95, .F03 and .F08.

CUDA: .cu.

Type:	`str` or `None`

srcfile¶

The source file to compile. This is automatically set by the framework based on the reframe.core.pipeline.RegressionTest.sourcepath attribute.

Type:	`str` or `None`