Utility Functions

New in version 3.3.

This is a collection of utility functions and classes that are used by the framework but can also be useful when writing regression tests. Functions or classes marked as draft should be used with caution, since they might change or be replaced without a deprecation warning.

General Utilities

class reframe.utility.MappingView(mapping)[source]

Bases: Mapping

A read-only view of a mapping.

See collections.abc.Mapping for a list of supported of operations.

get(key, default=None)[source]

Return the value mapped to key or default, if key does not exist.

  • key – The key to look up.

  • default – The default value to return if the key is not present.


The value associated to the requested key.


Return a set-like object providing a view on the underlying mapping’s items.


Return a set-like object providing a view on the underlying mapping’s keys.


Return a set-like object providing a view on the underlying mapping’s values.

class reframe.utility.OrderedSet(*args)[source]

Bases: MutableSet

An ordered set.

This container behaves like a normal set but remembers the insertion order of its elements. It can also inter-operate with standard Python sets.

Operations between ordered sets respect the order of the elements of the operands. For example, if x and y are both ordered sets, then x | y will be a new ordered set with the (unique) elements of x and y in the order they appear in x and y. The same holds for all the other set operations.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.


See same method in set.

class reframe.utility.ScopedDict(mapping={}, scope_sep=':', global_scope='*')[source]

Bases: UserDict

This is a special dictionary that imposes scopes on its keys.

When a key is not found, it will be searched up in the scope hierarchy. If not found even at the global scope, a KeyError will be raised.

A scoped dictionary is initialized using a two-level normal dictionary that defines the different scopes and the keys inside them. Scopes can be nested by concatenating them using the : separator by default: scope:subscope. Below is an example of a scoped dictionary that also demonstrates key lookup:

d = ScopedDict({
    'a': {'k1': 1, 'k2': 2},
    'a:b': {'k1': 3, 'k3': 4},
    '*': {'k1': 7, 'k3': 9, 'k4': 10}

assert d['a:k1'] == 1    # resolved in the scope 'a'
assert d['a:k3'] == 9    # resolved in the global scope
assert d['a:b:k1'] == 3  # resolved in the scope 'a:b'
assert d['a:b:k2'] == 2  # resolved in the scope 'a'
assert d['a:b:k4'] == 10 # resolved in the global scope
d['a:k5']    # KeyError
d['*:k2']    # KeyError

If no scope is specified in the key lookup, the global scope is assumed. For example, d['k1'] will return 7. The syntaxes d[':k1'] and d['*:k1'] are all equivalent. If you try to retrieve a whole scope, e.g., d['a:b'], KeyError will be raised. For retrieving scopes, you should use the scope() function.

Key deletion follows the same resolution mechanism as key retrieval, except that you are allowed to delete whole scopes. For example, del d['*'] will delete the global scope, such that subsequent access of d['a:k3'] will raise a KeyError. If a key specification matches both a key and scope, the key will be deleted and not the scope.

  • mapping

    A two-level mapping of the form

         scope1: {k1: v1, k2: v2},
         scope2: {k1: v1, k3: v3}

    Both the scope keys and the actual dictionary keys must be strings, otherwise a TypeError will be raised.

  • scope_sep – A character that separates the scopes.

  • global_scope – A key that represents the global scope.

property global_scope_mark

The key representing the global scope of this dictionary.


Retrieve a whole scope.


scope – The name of the scope to retrieve.


A dictionary with the keys that are within the requested scope.

property scope_separator

The scope separator of this dictionary.


Update this dictionary from the values of a two-level mapping as described above.


other – A two-level mapping defining scopes and keys.

class reframe.utility.SequenceView(container)[source]

Bases: Sequence

A read-only view of a sequence.

See collections.abc.Sequence for a list of supported of operations.


container – The container to create a view on.


TypeError – If the container does not fulfill the collections.abc.Sequence interface.


You can concatenate a SequenceView with a container of the same type as the underlying container of the view, in which case a new container with the concatenated elements will be returned.


Count occurrences of value in the container.


value – The value to search for.


The number of occurrences.

index(value, start=0, stop=None)[source]

Return the first index of value.

  • value – The value to search for.

  • start – The position where the search starts.

  • stop – The position where the search stops. The element at this position is not looked at. If None, this equals to the sequence’s length.


The index of the first element found that equals value.


ValueError – if the value is not present.


Same as the built-in all(), except that it returns False if iterable is empty.


Validate object attributes recursively.

This returns a function which you can call with the object to check. It will return True if the validate_fn() returns True for all object attributes recursively. If the object to be validated is an iterable, its elements will be validated individually.


validate_fn – A callable that validates an object. It takes a single argument, which is the object to validate.


A validation function that will perform the actual validation. It accepts a single argument, which is the object to validate. It returns a two-element tuple, containing the result of the validation as a boolean and a formatted string indicating the faulty attribute.


Objects defining __slots__ are passed directly to the validate_fn function.

New in version 3.3.


Decorator that caches the return value of the decorated function.

The function will only be called once and then the cached value will be returned each time.

reframe.utility.decamelize(s, delim='_')[source]

Decamelize a string.

For example, MyBaseClass will be converted to my_base_class. The delimiter may be changed by setting the delim argument.

  • s – A string in camel notation.

  • delim – The delimiter that will be used to separate words.


The converted string.

reframe.utility.find_modules(substr, environ_mapping=None)[source]

Return all modules in the current system that contain substr in their name.

This function is a generator and will yield tuples of partition, environment and module combinations for each partition of the current system and for each environment of a partition.

The environ_mapping argument allows you to map module name patterns to ReFrame environments. This is useful for flat module name schemes, in order to avoid incompatible combinations of modules and environments.

You can use this function to parametrize regression tests over the available environment modules. The following example will generate tests for all the available netcdf packages in the system:

class MyTest(rfm.RegressionTest):
    module_info = parameter(find_modules('netcdf'))

    def apply_module_info(self):
        s, e, m = self.module_info
        self.valid_systems = [s]
        self.valid_prog_environs = [e]
        self.modules = [m]

The following example shows the use of environ_mapping with flat module name schemes. In this example, the toolchain for which the package was built is encoded in the module’s name. Using the environ_mapping argument we can map module name patterns to ReFrame environments, so that invalid combinations are pruned:

my_find_modules = functools.partial(find_modules, environ_mapping={
    r'.*CrayGNU.*': 'PrgEnv-gnu',
    r'.*CrayIntel.*': 'PrgEnv-intel',
    r'.*CrayCCE.*': 'PrgEnv-cray'

class MyTest(rfm.RegressionTest):
    module_info = parameter(my_find_modules('GROMACS'))

    def apply_module_info(self):
        s, e, m = self.module_info
        self.valid_systems = [s]
        self.valid_prog_environs = [e]
        self.modules = [m]
  • substr – A substring that the returned module names must contain.

  • environ_mapping – A dictionary mapping regular expressions to environment names.


An iterator that iterates over tuples of the module, partition and environment name combinations that were found.

reframe.utility.import_module_from_file(filename, force=False)[source]

Import module from file.

  • filename – The path to the filename of a Python module.

  • force – Force reload of module in case it is already loaded.


The loaded Python module.


Check if an object can be copied with copy.deepcopy(), without performing the copy.

This is a superset of is_picklable(). It returns True also in the following cases:

  • The object defines a __copy__() method.

  • The object defines a __deepcopy__() method.

  • The object is a function.

  • The object is a builtin type.

New in version 3.3.


Check if an object can be pickled.

New in version 3.3.

reframe.utility.is_trivially_callable(fn, *, non_def_args=0)[source]

Check that a callable object is trivially callable.

An object is trivially callable when it can be invoked by providing just an expected number of non-default arguments to its call method. For example, (non-static) member functions expect a single argument without a default value, which will passed as cls or self during invocation depending on whether the function is a classmethod or not, respectively. On the other hand, member functions that are static methods are not passed any values by default when invoked. Therefore, these functions can only be trivially callable when their call method expects no arguments by default.

  • fn – A callable to be tested if its trivially callable.

  • non_def_args – The number of non-default arguments the callable fn expects when invoked.


This function returns True if the expected number of arguments matches the value of non_def_args. Otherwise, it returns False.


Return the longest sequence.

This function raises a TypeError if any of the iterables is not Sized.


iterables – The iterables to check.


The longest iterable.


Create an abbreviated string representation of the node list.

For example, the node list

['nid001', 'nid002', 'nid010', 'nid011', 'nid012', 'nid510', 'nid511']

will be abbreviated as follows:


New in version 3.5.3.


nodes – The node list to abbreviate.


The abbreviated list representation.


Expand the nodes in nodespec to a list of nodes.


nodespec – A node specification as the one returned by nodelist_abbrev()


The list of nodes corresponding to the given node specification.

New in version 4.0.0.

reframe.utility.ppretty(value, htchar=' ', lfchar='\n', indent=4, basic_offset=0, repr=<built-in function repr>)[source]

Format value in a pretty way.

If value is a container, this function will recursively format the container’s elements.

  • value – The value to be formatted.

  • htchar – Horizontal-tab character.

  • lfchar – Linefeed character.

  • indent – Number of htchar characters for every indentation level.

  • basic_offset – Basic offset for the representation, any additional indentation space is added to the basic_offset.

  • repr – A repr()-like function that will be used for printing values. This function is allowed to accept all the arguments of ppretty() except the repr argument.


A formatted string of value.

reframe.utility.repr(obj, htchar=' ', lfchar='\n', indent=4, basic_offset=0)[source]

A repr() replacement function for debugging purposes printing all object attributes recursively.

This function does not follow the standard repr() convention, but it prints each object as a set of key/value pairs along with its memory location. It also keeps track of the already visited objects, and abbreviates their representation.


obj – The object to be dumped. For the rest of the arguments, see ppretty().


The formatted object dump.


Return the shortest sequence.

This function raises a TypeError if any of the iterables is not Sized.


iterables – The iterables to check.


The shortest iterable.

class reframe.utility.temp_setattr(obj, attr, val)[source]

Bases: object

Context manager to temporarily change the attribute value of an object.

System Utilities

class reframe.utility.osext.change_dir(dir_name)[source]

Bases: object

Context manager to temporarily change the current working directory.


dir_name – The directory to temporarily change to.

reframe.utility.osext.concat_files(dst, *files, sep='\n', overwrite=False)[source]

Concatenate files into dst.

  • dst – The name of the output file.

  • files – The files to concatenate.

  • sep – The separator to use during concatenation.

  • overwrite – Overwrite the output file if it already exists.

  • TypeError – In case files it not an iterable object.

  • ValueError – In case output already exists and ovewrite is False.

reframe.utility.osext.copytree(src, dst, symlinks=False, ignore=None, copy_function=<function copy2>, ignore_dangling_symlinks=False, dirs_exist_ok=False)[source]

Compatibility version of shutil.copytree() for Python < 3.8.

This function will automatically delegate to shutil.copytree() for Python versions >= 3.8.

reframe.utility.osext.copytree_virtual(src, dst, file_links=None, symlinks=False, copy_function=<function copy2>, ignore_dangling_symlinks=False, dirs_exist_ok=False)[source]

Copy src to dst, but create symlinks for the files listed in file_links.

If file_links is empty or None, this is equivalent to copytree(). The rest of the arguments are passed as-is to copytree(). Paths in file_links must be relative to src. If you try to pass '.' in file_links, an OSError will be raised.


Return either the Cray Development Toolkit (CDT) version, the Cray Programming Environment (CPE) version or None if the version cannot be retrieved.


Return the Cray Linux Environment (CLE) release information.


filename – The file that contains the CLE release information


A named tuple with the following attributes that correspond to the release information: release, build, date, arch, network, patchset.


Expand environment variables in s and perform any command substitution.

This function is the same as os.path.expandvars(), except that it also recognizes the syntax of shell command substitution: $(cmd) or `cmd`.

Return the final target of a symlink chain.

If path is not a symlink, it will be returned as is.


Remove filename ignoring FileNotFoundError.

reframe.utility.osext.git_clone(url, targetdir=None, opts=None, timeout=5)[source]

Clone a git repository from a URL.

  • url – The URL to clone from.

  • opts – List of options to be passed to the git clone command

  • timeout – Timeout in seconds when checking if the url is a valid repository.

  • targetdir – The directory where the repository will be cloned to. If None, a new directory will be created with the repository name as if git clone {url} was issued.

reframe.utility.osext.git_repo_exists(url, timeout=5)[source]

Check if URL refers to a valid Git repository.

  • url – The URL to check.

  • timeout – Timeout in seconds.


True if URL is a Git repository, False otherwise or if timeout is reached.

reframe.utility.osext.git_repo_hash(commit='HEAD', short=True, wd=None)[source]

Return the SHA1 hash of a Git commit.

  • commit – The commit to look at.

  • short – Return a short hash. This always corresponds to the first 8 characters of the long hash. We don’t rely on Git for the short hash, since depending on the version it might return either 7 or 8 characters.

  • wd – Change to this directory before retrieving the hash. If None, ReFrame’s install prefix will be used.


The Git commit hash or None if the hash could not be retrieved.

reframe.utility.osext.inpath(entry, pathvar)[source]

Check if entry is in path.

  • entry – The entry to look for.

  • pathvar – A path variable in the form ‘entry1:entry2:entry3’.


True if the entry exists in the path variable, False otherwise.


Check if the current Python session is interactive.


Check if string is a URL.

reframe.utility.osext.mkstemp_path(*args, **kwargs)[source]

Create a temporary file and return its path.

This is a wrapper to tempfile.mkstemp() except that it closes the temporary file as soon as it creates it and returns the path.

args and kwargs passed through to tempfile.mkstemp().


Return the group name of the current OS user.

If the group name cannot be retrieved, None will be returned.


Return the name of the current OS user.

If the user name cannot be retrieved, None will be returned.

reframe.utility.osext.rmtree(*args, max_retries=3, **kwargs)[source]

Persistent version of shutil.rmtree().

If shutil.rmtree() fails with ENOTEMPTY or EBUSY, ignore the error and retry up to max_retries times to delete the directory.

This version of rmtree() is mostly provided to work around a race condition between when sacct reports a job as completed and when the Slurm epilog runs. See gh #291 for more information. Furthermore, it offers a work around for NFS file systems where stale file handles may be present during the rmtree() call, causing it to throw a busy device/resource error. See gh #712 for more information.

args and kwargs are passed through to shutil.rmtree().

If onerror is specified in kwargs and it is not None, this function is completely equivalent to shutil.rmtree().

  • args – Arguments to be passed through to shutil.rmtree().

  • max_reties – Maximum number of retries if the target directory cannot be deleted.

  • kwargs – Keyword arguments to be passed through to shutil.rmtree().

reframe.utility.osext.run_command(cmd, check=False, timeout=None, **kwargs)[source]

Run command synchronously.

This function will block until the command executes or the timeout is reached. It essentially calls run_command_async() and waits for the command’s completion.

  • cmd – The command to execute as a string or a sequence. See run_command_async() for more details.

  • check – Raise an error if the command exits with a non-zero exit code.

  • timeout – Timeout in seconds.

  • kwargs – Keyword arguments to be passed run_command_async().


A subprocess.CompletedProcess object with information about the command’s outcome.

reframe.utility.osext.run_command_async(cmd, stdout=-1, stderr=-1, shell=False, log=True, **popen_args)[source]

Run command asynchronously.

A wrapper to subprocess.Popen with the following tweaks:

  • It always passes universal_newlines=True to Popen.

  • If shell=False and cmd is a string, it will lexically split cmd using shlex.split(cmd).

  • cmd – The command to run either as a string or a sequence of arguments.

  • stdout – Same as the corresponding argument of Popen. Default is subprocess.PIPE.

  • stderr – Same as the corresponding argument of Popen. Default is subprocess.PIPE.

  • shell – Same as the corresponding argument of Popen.

  • log – Log the execution of the command through ReFrame’s logging facility.

  • popen_args – Any additional arguments to be passed to Popen.


A new Popen object.

reframe.utility.osext.samefile(path1, path2)[source]

Check if paths refer to the same file.

If paths exist, this is equivalent to os.path.samefile(). If only one of the paths exists and is a symbolic link, it will be followed and its final target will be compared to the other path. If both paths do not exist, a simple string comparison will be performed (after the paths have been normalized).

reframe.utility.osext.subdirs(dirname, recurse=False)[source]

Get the list of subdirectories of dirname including dirname.

If recurse is True, this function will retrieve all subdirectories in pre-order.

  • dirname – The directory to start searching.

  • recurse – If True, then recursively search for subdirectories.


The list of subdirectories found.

reframe.utility.osext.unique_abs_paths(paths, prune_children=True)[source]

Get the unique absolute paths from a given list of paths.

  • paths – An iterable of paths.

  • prune_children – Discard paths that are children of other paths in the list.


TypeError – In case paths it not an iterable object.

Type Checking Utilities

Dynamic recursive type checking of collections.

This module defines types for collections, such as lists, dictionaries etc., that you can use with the isinstance() builtin function to recursively type check all the elements of the collection. Suppose you have a list of integers, suchs as [1, 2, 3], the following checks should be true:

l = [1, 2, 3]
assert isinstance(l, List[int]) == True
assert isinstance(l, List[float]) == False

Aggregate types can be combined in an arbitrary depth, so that you can type check any complex data strcture:

d = {'a': [1, 2], 'b': [3, 4]}
assert isisntance(d, Dict) == True
assert isisntance(d, Dict[str, List[int]]) == True

This module offers the following aggregate types:


A list with elements of type T.


A set with elements of type T.


A dictionary with keys of type K and values of type V.


A tuple with elements of type T.


A tuple with n elements, whose types are exactly T1, T2, …, Tn in that order.


A string type whose members are all the strings matching the regular expression patt.

Implementation details

Internally, this module leverages metaclasses and the __isinstancecheck__() method to customize the behaviour of the isinstance() builtin.

By implementing also the __getitem__() accessor method, this module follows the look-and-feel of the type hints proposed in PEP484. This method returns a new type that is a subtype of the base container type. Using the facilities of abc.ABCMeta, builtin types, such as list, str etc. are registered as subtypes of the base container types offered by this module. The type hierarchy of the types defined in this module is the following (example shown for List, but it is analogous for the rest of the types):

    /   |
   /    |
  /     |
list  List[T]

In the above example T may refer to any type, so that List[List[int]] is an instance of List, but not an instance of List[int].

class reframe.utility.typecheck.Bool(*args, **kwargs)[source]

Bases: object

A boolean type accepting implicit conversions from strings.

This type represents a boolean value but allows implicit conversions from str. More specifically, the following conversions are supported:

  • The strings 'yes', 'true' and '1' are converted to True.

  • The strings 'no', 'false' and '0' are converted to False.

The built-in bool type is registered as a subclass of this type.

Boolean test variables that are meant to be set properly from the command line must be declared of this type and not bool.

class reframe.utility.typecheck.ConvertibleType(name, bases, namespace, **kwargs)[source]

Bases: ABCMeta

A type that support conversions from other types.

This is a metaclass that allows classes that use it to support arbitrary conversions from other types using a cast-like syntax without having to change their constructor:

new_obj = convertible_type(another_type)

For example, a class whose constructor accepts and int may need to support a cast-from-string conversion. This is particular useful if you want a custom-typed test variable to be able to be set from the command line using the -S option.

In order to support such conversions, a class must use this metaclass and define a class method, named as __rfm_cast_<type>__, for each of the type conversion that needs to support .

The following is an example of a class X that its normal constructor accepts two arguments but it also allows conversions from string:

class X(metaclass=ConvertibleType):
    def __init__(self, x, y):
        self.data = (x, y)

    def __rfm_cast_str__(cls, s):
        return X(*(int(x) for x in s.split(',', maxsplit=1)))

 assert X(2, 3).data == X('2,3').data

New in version 3.8.0.

Test Case Dependencies Management

Managing the test case “micro-dependencies” between two tests.

This module defines a set of basic functions that can be used with the how argument of the reframe.core.pipeline.RegressionTest.depends_on() function to control how the individual dependencies between the test cases of two tests are formed.

All functions take two arguments, the source and destination vertices of an edge in the test case dependency subgraph that connects two tests. In the relation “T0 depends on T1”, the source are the test cases of “T0” and the destination are the test cases of “T1.” The source and destination arguments are two-element tuples containing the names of the partition and the environment of the corresponding test cases. These functions return True if there is an edge connecting the two test cases or False otherwise.

A how function will be called by the framework multiple times when the test DAG is built. More specifically, for each test dependency relation, it will be called once for each test case combination of the two tests.

The how functions essentially split the test case subgraph of two dependent tests into fully connected components based on the values of their supported partitions and environments.

The How Test Dependencies Work In ReFrame page contains more information about test dependencies and shows visually the test case subgraph connectivity that the different how functions described here achieve.

New in version 3.3.

reframe.utility.udeps.by_case(src, dst)[source]

The test cases of two dependent tests will be split by partition and by environment.

Test cases from different partitions and different environments are independent.

reframe.utility.udeps.by_env(src, dst)[source]

The test cases of two dependent tests will be split by environment.

Test cases from different environments are independent.

reframe.utility.udeps.by_part(src, dst)[source]

The test cases of two dependent tests will be split by partition.

Test cases from different partitions are independent.

reframe.utility.udeps.by_xcase(src, dst)[source]

The test cases of two dependent tests will be split by the exclusive disjunction (XOR) of their partitions and environments.

Test cases from the same environment and the same partition are independent.

reframe.utility.udeps.by_xenv(src, dst)[source]

The test cases of two dependent tests will be split by the exclusive disjunction (XOR) of their environments.

Test cases from the same environment are independent.

reframe.utility.udeps.by_xpart(src, dst)[source]

The test cases of two dependent tests will be split by the exclusive disjunction (XOR) of their partitions.

Test cases from the same partition are independent.

reframe.utility.udeps.fully(src, dst)[source]

The test cases of two dependent tests will be fully connected.