Reference Guide¶
-
class
reframe.core.pipeline.
CompileOnlyRegressionTest
(*args, **kwargs)[source]¶ Bases:
reframe.core.pipeline.RegressionTest
Base class for compile-only regression tests.
These tests are by default local and will skip the run phase of the regression test pipeline.
The standard output and standard error of the test will be set to those of the compilation stage.
-
compile
(**compile_opts)[source]¶ The compilation stage of the regression test pipeline.
The standard output and standard error of this stage will be used as the standard output and error of the test.
-
setup
(partition, environ, **job_opts)[source]¶ The setup stage of the regression test pipeline.
Similar to the
RegressionTest.setup()
, except that no job descriptor is set up for this test.
-
-
class
reframe.core.pipeline.
RegressionTest
(name, prefix, system, resources)[source]¶ Bases:
object
Base class for regression tests.
All regression tests must eventually inherit from this class. This class provides the implementation of the pipeline phases that the regression test goes through during its lifetime.
Parameters: - name – The name of the test. This is the only argument that the users may specify freely.
- prefix – The directory prefix of the test.
You should initialize this to the directory containing the file that
defines the regression test.
You can achieve this by always passing
os.path.dirname(__file__)
. - system – The system that this regression test will run on. The framework takes care of initializing and passing correctly this argument.
- resources – An object managing the framework’s resources. The framework takes care of initializing and passing correctly this argument.
Concrete regression test subclasses should call the base constructor as follows:
class MyTest(RegressionTest): def __init__(self, my_test_args, **kwargs): super().__init__('mytest', os.path.dirname(__file__), **kwargs)
-
check_performance
()[source]¶ The performance checking phase of the regression test pipeline.
Returns: True
on success,False
otherwise, if the oldperf_patterns
syntax is used.Raises: - reframe.core.exceptions.SanityError – If the new syntax is used and the performance check fails.
- reframe.core.exceptions.ReframeError – In case of other errors.
-
check_sanity
()[source]¶ The sanity checking phase of the regression test pipeline.
Returns: True
on success,False
otherwise, if the oldsanity_patterns
syntax is used.Raises: - reframe.core.exceptions.SanityError – If the new syntax is used and the sanity check fails.
- reframe.core.exceptions.ReframeError – In case of other errors.
-
cleanup
(remove_files=False, unload_env=True)[source]¶ The cleanup phase of the regression test pipeline.
Parameters: - remove_files – If
True
, the stage directory associated with this test will be removed. - unload_env – If
True
, the environment that was used to run this test will be unloaded.
- remove_files – If
-
compile
(**compile_opts)[source]¶ The compilation phase of the regression test pipeline.
Parameters: compile_opts – Extra options to be passed to the programming environment for compiling the source code of the test. Raises: reframe.core.exceptions.ReframeError – In case of errors.
-
current_environ
¶ The programming environment that the regression test is currently executing with.
This is set by the framework during the
setup()
phase.Type: reframe.core.environments.Environment
.
-
current_partition
¶ The system partition the regression test is currently executing on.
This is set by the framework during the
setup()
phase.Type: reframe.core.systems.SystemPartition
.
-
current_system
¶ The system the regression test is currently executing on.
This is set by the framework during the initialization phase.
Type: reframe.core.systems.System
.
-
exclusive_access
¶ Specify whether this test needs exclusive access to nodes.
Type: boolean Default: False
-
executable
¶ The name of the executable to be launched during the run phase.
Type: str
Default: os.path.join('.', self.name)
-
executable_opts
¶ List of options to be passed to the
executable
.Type: list
ofstr
Default: []
-
extra_resources
¶ Extra resources for this test.
This field is for specifying custom resources needed by this test. These resources are defined in the configuration of a system partition. For example, assume that
num_accels
is defined as follows in the configuration file:'resources': { 'num_accels': [ '--gres=gpu:{num_accels}' ] }
A regression test then may define
extra_resources
as follows in order to get two accelerator devices:self.extra_resources = {'num_accels': 2}
The framework will then pass the option
--gres=gpu:2
to the backend scheduler.If the resource name specified in this variable does not match a resource name in the partition configuration, it will be simply ignored. The
num_gpus_per_node
attribute translates internally to thenum_gpus_per_node
resource, so that settingself.num_gpus_per_node = 2
is equivalent to the following:self.extra_resources = {'num_gpus_per_node': 2}
Type: dictionary with str
keys/valuesDefault: {}
Note
New in version 2.8.
-
is_local
()[source]¶ Check if the test will execute locally.
A test executes locally if the
local
attribute is set or if the current partition’s scheduler does not support job submission.
-
job
¶ The job descriptor associated with this test.
This is set by the framework during the
setup()
phase.Type: reframe.core.schedulers.Job
.
-
keep_files
¶ List of files to be kept after the test finishes.
By default, the framework saves the standard output, the standard error and the generated shell script that was used to run this test.
These files will be copied over to the framework’s output directory during the
cleanup()
phase.Directories are also accepted in this field.
Relative path names are resolved against the stage directory.
Type: list
ofstr
Default: []
-
local
¶ Always execute this test locally.
Type: boolean Default: False
-
logger
¶ A logger associated with the this test.
You can use this logger to log information for your test.
-
maintainers
¶ List of people responsible for this test.
When the test fails, this contact list will be printed out.
Type: list
ofstr
Default: []
-
modules
¶ List of modules to be loaded before running this test.
These modules will be loaded during the
setup()
phase.Type: list
ofstr
Default: []
-
name
¶ The name of the test.
Type: Alphanumeric string.
-
num_cpus_per_task
¶ Number of CPUs per task required by this test.
Ignored if
None
.Type: integer or None
Default: None
-
num_gpus_per_node
¶ Number of GPUs per node required by this test.
Type: integer Default: 0
-
num_tasks
¶ Number of tasks required by this test.
Type: integer Default: 1
-
num_tasks_per_core
¶ Number of tasks per core required by this test.
Ignored if
None
.Type: integer or None
Default: None
-
num_tasks_per_node
¶ Number of tasks per node required by this test.
Ignored if
None
.Type: integer or None
Default: None
-
num_tasks_per_socket
¶ Number of tasks per socket required by this test.
Ignored if
None
.Type: integer or None
Default: None
-
perf_patterns
¶ Patterns for verifying the performance of this test.
Refer to the ReFrame Tutorial for concrete usage examples.
If set to
None
, no performance checking will be performed.Type: A dictionary with keys of type str
and deferrable expressions (i.e., the result of a sanity function) as values.None
is also allowed.Default: None
-
poll
()[source]¶ Poll the test’s state.
Returns: True
if the associated job has finished,False
otherwise.If no job descriptor is yet associated with this test,
True
is returned.Raises: reframe.core.exceptions.ReframeError – In case of errors.
-
postbuild_cmd
¶ List of shell commands to be executed after a successful compilation.
These commands are executed during the compilation phase and from inside the stage directory.
Type: list
ofstr
Default: []
-
prebuild_cmd
¶ List of shell commands to be executed before compiling.
These commands are executed during the compilation phase and from inside the stage directory.
Type: list
ofstr
Default: []
-
readonly_files
¶ List of files or directories (relative to the
sourcesdir
) that will be symlinked in the stage directory and not copied.You can use this variable to avoid copying very large files to the stage directory.
Type: list
ofstr
Default: []
-
reference
¶ The set of reference values for this test.
Refer to the ReFrame Tutorial for concrete usage examples.
Type: A scoped dictionary with system names as scopes or None
Default: {}
-
run
()[source]¶ The run phase of the regression test pipeline.
This call is non-blocking. It simply submits the job associated with this test and returns.
-
sanity_patterns
¶ Patterns for verifying the sanity of this test.
Refer to the ReFrame Tutorial for concrete usage examples.
If set to
None
, no sanity checking will be performed.Type: A deferrable expression (i.e., the result of a sanity function) or None
Default: None
-
setup
(partition, environ, **job_opts)[source]¶ The setup phase of the regression test pipeline.
Parameters: - partition – The system partition to set up this test for.
- environ – The environment to set up this test for.
- job_opts – Options to be passed through to the backend scheduler.
When overriding this method users should always pass through
job_opts
to the base class method.
Raises: reframe.core.exceptions.ReframeError – In case of errors.
-
sourcepath
¶ The path to the source file or source directory of the test.
If not absolute, it is resolved against the
sourcesdir
directory.If it refers to a regular file, this file will be compiled (its language will be automatically recognized) and the produced executable will be placed in the test’s stage directory. If it refers to a directory, this will be copied to the test’s stage directory and
make
will be invoked in that.Type: str
Default: ''
-
sourcesdir
¶ The directory containing the test’s resources.
If set to
None
, the test has no resources.Type: str
orNone
Default: os.path.join(self.prefix, 'src')
-
stderr
¶ The name of the file containing the standard error of the test.
This is set during the
setup()
phase.This attribute is evaluated lazily, so it can by used inside sanity expressions.
Type: str
.
-
stdout
¶ The name of the file containing the standard output of the test.
This is set during the
setup()
phase.This attribute is evaluated lazily, so it can by used inside sanity expressions.
Type: str
.
-
strict_check
¶ Mark this test as a strict performance test.
If a test is marked as non-strict, the performance checking phase will always succeed, unless the
--strict
command-line option is passed when invoking ReFrame.Type: boolean Default: True
Set of tags associated with this test.
This test can be selected from the frontend using any of these tags.
Type: set
ofstr
Default: an empty set
-
time_limit
¶ Time limit for this test.
Time limit is specified as a three-tuple in the form
(hh, mm, ss)
, withhh >= 0
,0 <= mm <= 59
and0 <= ss <= 59
.Type: a three-tuple with the above properties. Default: (0, 10, 0)
-
use_multithreading
¶ Specify whether this tests needs simultaneous multithreading enabled.
Ignored if
None
.Type: boolean or None
Default: None
-
valid_prog_environs
¶ List of programming environmets supported by this test.
Type: list
ofstr
Default: []
-
valid_systems
¶ List of systems supported by this test. The general syntax for systems is
<sysname>[:<partname]
.Type: list
ofstr
.Default: []
-
class
reframe.core.pipeline.
RunOnlyRegressionTest
(name, prefix, system, resources)[source]¶ Bases:
reframe.core.pipeline.RegressionTest
Base class for run-only regression tests.
-
compile
(**compile_opts)[source]¶ The compilation phase of the regression test pipeline.
This is a no-op for this type of test.
-
run
()[source]¶ The run phase of the regression test pipeline.
The resources of the test are copied to the stage directory and the rest of execution is delegated to the
RegressionTest.run()
.
-
-
class
reframe.core.environments.
Environment
(name, modules=[], variables={}, **kwargs)[source]¶ Bases:
object
This class abstracts away an environment to run regression tests.
It is simply a collection of modules to be loaded and environment variables to be set when this environment is loaded by the framework. Users may not create or modify directly environments.
-
is_loaded
¶ True
if this environment is loaded,False
otherwise.
-
-
class
reframe.core.environments.
ProgEnvironment
(name, modules=[], variables={}, cc='cc', cxx='CC', ftn='ftn', cppflags=None, cflags=None, cxxflags=None, fflags=None, ldflags=None, **kwargs)[source]¶ Bases:
reframe.core.environments.Environment
A class representing a programming environment.
This type of environment adds also attributes for setting the compiler and compilation flags.
If compilation flags are set to
None
(the default, if not set otherwise in ReFrame’s configuration), they are not passed to themake
invocation.If you want to disable completely the propagation of the compilation flags to the
make
invocation, even if they are set, you should set thepropagate
attribute toFalse
.
-
class
reframe.core.environments.
save_environment
[source]¶ Bases:
object
A context manager for saving and restoring the current environment.
-
class
reframe.core.systems.
System
(name, descr=None, hostnames=[], partitions=[], prefix='.', stagedir=None, outputdir=None, logdir=None, resourcesdir='.', modules_system=None)[source]¶ Bases:
object
A representation of a system inside ReFrame.
-
descr
¶ The description of this system.
-
name
¶ The name of this system.
-
partition
(name)[source]¶ Get system partition with
name
.Returns: the requested SystemPartition
, orNone
if not found.
-
partitions
¶ Get all the active partitions of this system.
Returns: a list of SystemPartition
.
-
-
class
reframe.core.systems.
SystemPartition
(name, descr=None, scheduler=None, launcher=None, access=[], environs=[], resources={}, local_env=None, max_jobs=1)[source]¶ Bases:
object
A representation of a system partition inside ReFrame.
-
descr
¶ A detailed description of this partition.
-
fullname
¶ Return the fully-qualified name of this partition.
The fully-qualified name is of the form
<parent-system-name>:<partition-name>
.Type: str
-
launcher
¶ The type of the backend launcher of this partition.
Returns: a subclass of reframe.core.launchers.JobLauncher
.Note
New in version 2.8.
-
name
¶ The name of this partition.
Type: str
-
scheduler
¶ The type of the backend scheduler of this partition.
Returns: a subclass of reframe.core.schedulers.Job
.Note
Changed in version 2.8.
Prior versions returned a string representing the scheduler and job launcher combination.
-
-
class
reframe.core.schedulers.
Job
(name, command, launcher, environs=[], workdir='.', num_tasks=1, num_tasks_per_node=None, num_tasks_per_core=None, num_tasks_per_socket=None, num_cpus_per_task=None, use_smt=None, time_limit=(0, 10, 0), script_filename=None, stdout=None, stderr=None, sched_account=None, sched_partition=None, sched_reservation=None, sched_nodelist=None, sched_exclude_nodelist=None, sched_exclusive_access=None, sched_options=[])[source]¶ Bases:
abc.ABC
A job descriptor.
Note
This is an abstract class. Users may not create jobs directly.
-
launcher
¶ The parallel program launcher that will be used to launch the parallel executable of this job.
Type: reframe.core.launchers.JobLauncher
-
-
class
reframe.core.launchers.
JobLauncher
(options=[])[source]¶ Bases:
abc.ABC
A job launcher.
A job launcher is the executable that actually launches a distributed program to multiple nodes, e.g.,
mpirun
,srun
etc.Note
This is an abstract class. Regression tests may not instantiate this class directly.
Note
Changed in version 2.8: Job launchers do not get a reference to a job during their initialization.
-
class
reframe.core.launchers.
LauncherWrapper
(target_launcher, wrapper_command, wrapper_options=[])[source]¶ Bases:
reframe.core.launchers.JobLauncher
Wrap a launcher object so as to modify its invocation.
This is useful for parallel debuggers. For example, to launch a regression test using the DDT debugger, you can do the following:
def setup(self, partition, environ, **job_opts): super().setup(partition, environ, **job_opts) self.job.launcher = LauncherWrapper(self.job.launcher, 'ddt', ['--offline'])
If the current system partition uses native Slurm for job submission, this setup will generate the following command in the submission script:
ddt --offline srun <test_executable>
If the current partition uses
mpirun
instead, it will generateddt --offline mpirun -np <num_tasks> ... <test_executable>
Parameters: - target_launcher – The launcher to wrap.
- wrapper_command – The wrapper command.
- wrapper_options – List of options to pass to the wrapper command.
-
reframe.core.launchers.registry.
getlauncher
(name)[source]¶ Get launcher by its registered name.
The available names are those specified in the configuration file.
This method may become handy in very special situations, e.g., testing an application that needs to replace the system partition launcher or if a different launcher must be used for a different programming environment.
For example, if you want to replace the current partition’s launcher with the local one, here is how you can achieve it:
def setup(self, partition, environ, **job_opts): super().setup(partition, environ, **job_opts) self.job.launcher = getlauncher('local')()
Note that this method returns a launcher class type and not an instance of that class. You have to instantiate it explicitly before assigning it to the
launcher
attribute of the job.Parameters: name – The name of the launcher to retrieve. Returns: The class of the launcher requested, which is a subclass of reframe.core.launchers.JobLauncher
.Note
New in version 2.8.
-
reframe.core.launchers.registry.
register_launcher
(name, local=False)[source]¶ Class decorator for registering new job launchers.
Parameters: - name – The registration name of this launcher
- local –
True
if launcher may only submit local jobs,False
otherwise.
Note
New in version 2.8.
This method is only relevant to developers of new job launchers.