Deferrable Functions Reference

Deferrable functions are the functions whose execution may be postponed to a later time after they are called. The key characteristic of these functions is that they store their arguments when they are called, and the execution itself does not occur until the function is evaluated either explicitly or implicitly.

ReFrame provides an ample set of deferrable utilities and it also allows users to write their own deferrable functions when needed. Please refer to “Understanding the Mechanism of Deferrable Functions” for a hands-on explanation on how deferrable functions work and how to create custom deferrable functions.

Explicit evaluation of deferrable functions

Deferrable functions may be evaluated at any time by calling evaluate() on their return value or by passing the deferred function itself to the evaluate() free function. These evaluate() functions take an optional bool argument cache, which can be used to cache the evaluation of the deferrable function. Hence, if caching is enabled on a given deferrable function, any subsequent calls to evaluate() will simply return the previously cached results.

Changed in version 3.8.0: Support of cached evaluation is added.

Implicit evaluation of deferrable functions

Deferrable functions may also be evaluated implicitly in the following situations:

• When you try to get their truthy value by either explicitly or implicitly calling bool on their return value. This implies that when you include the result of a deferrable function in an if statement or when you apply the and, or or not operators, this will trigger their immediate evaluation.

• When you try to iterate over their result. This implies that including the result of a deferrable function in a for statement will trigger its evaluation immediately.

• When you try to explicitly or implicitly get its string representation by calling str on its result. This implies that printing the return value of a deferrable function will automatically trigger its evaluation.

Categories of deferrable functions

Currently ReFrame provides three broad categories of deferrable functions:

1. Deferrable replacements of certain Python built-in functions. These functions simply delegate their execution to the actual built-ins.

2. Assertion functions. These functions are used to assert certain conditions and they either return True or raise SanityError with a message describing the error. Users may provide their own formatted messages through the msg argument. For example, in the following call to assert_eq() the {0} and {1} placeholders will obtain the actual arguments passed to the assertion function.

   assert_eq(a, 1, msg="{0} is not equal to {1}")
   

   If in the user provided message more placeholders are used than the arguments of the assert function (except the msg argument), no argument substitution will be performed in the user message.

3. Utility functions. They include, but are not limited to, functions to iterate over regex matches in a file, extracting and converting values from regex matches, computing statistical information on series of data etc.

Deferrable performance functions

Added in version 3.8.0.

Deferrable performance functions are a special type of deferrable functions which are intended for measuring a given quantity. Therefore, this kind of deferrable functions have an associated unit that can be used to interpret the return values from these functions. The unit of a deferrable performance function can be accessed through the public member unit. Regular deferrable functions can be promoted to deferrable performance functions using the make_performance_function() utility. Also, this utility allows to create performance functions directly from any callable.

List of deferrable functions and utilities

@reframe.utility.sanity.deferrable(func)

Deferrable decorator.

Converts the decorated free function into a deferrable function.

import reframe.utility.sanity as sn

@sn.deferrable
def myfunc(*args):
    do_sth()

reframe.utility.sanity.abs(x)

Replacement for the built-in abs() function.

reframe.utility.sanity.all(iterable)

Replacement for the built-in all() function.

reframe.utility.sanity.allx(iterable)

Same as the built-in all() function, except that it returns False if iterable is empty.

Added in version 2.13.

reframe.utility.sanity.and_(a, b)

Deferrable version of the and operator.

Returns:

a and b.

reframe.utility.sanity.any(iterable)

Replacement for the built-in any() function.

reframe.utility.sanity.assert_bounded(val, lower=None, upper=None, msg=None)

Assert that lower <= val <= upper.

Parameters:

• val – The value to check.

• lower – The lower bound. If None, it defaults to -inf.

• upper – The upper bound. If None, it defaults to inf.

• msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_eq(a, b, msg=None)

Assert that a == b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_false(x, msg=None)

Assert that x is evaluated to False.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_found(patt, filename, msg=None, encoding='utf-8')

Assert that regex pattern patt is found in the file filename.

Parameters:

• patt – The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.

• filename – The name of the file to examine or a file descriptor as in open(). Any OSError raised while processing the file will be propagated as a reframe.core.exceptions.SanityError.

• msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

• encoding – The name of the encoding used to decode the file.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_found_s(patt, string, msg=None)

Assert that regex pattern patt is found in the string string.

Parameters:

• patt – as in assert_found().

• string – The string to examine.

• msg – as in assert_found(). You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

Added in version 3.4.1.

reframe.utility.sanity.assert_ge(a, b, msg=None)

Assert that a >= b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_gt(a, b, msg=None)

Assert that a > b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_in(item, container, msg=None)

Assert that item is in container.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_le(a, b, msg=None)

Assert that a <= b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_lt(a, b, msg=None)

Assert that a < b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_ne(a, b, msg=None)

Assert that a != b.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_not_found(patt, filename, msg=None, encoding='utf-8')

Assert that regex pattern patt is not found in the file filename.

This is the inverse of assert_found().

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_not_found_s(patt, string, msg=None)

Assert that regex pattern patt is not found in string.

This is the inverse of assert_found_s().

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

Added in version 3.4.1.

reframe.utility.sanity.assert_not_in(item, container, msg=None)

Assert that item is not in container.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_reference(val, ref, lower_thres=None, upper_thres=None, msg=None)

Assert that value val respects the reference value ref.

Parameters:

• val – The value to check.

• ref – The reference value.

• lower_thres – The lower threshold value expressed as a negative decimal fraction of the reference value. Must be in [-1, 0] for ref >= 0.0 and in [-inf, 0] for ref < 0.0. If None, no lower thresholds is applied.

• upper_thres – The upper threshold value expressed as a decimal fraction of the reference value. Must be in [0, inf] for ref >= 0.0 and in [0, 1] for ref < 0.0. If None, no upper thresholds is applied.

• msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails or if the lower and upper thresholds do not have appropriate values.

reframe.utility.sanity.assert_true(x, msg=None)

Assert that x is evaluated to True.

Parameters:

msg – The error message to use if the assertion fails. You may use {0} … {N} as placeholders for the function arguments.

Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.avg(iterable)

Return the average of all the elements of iterable.

reframe.utility.sanity.chain(*iterables)

Replacement for the itertools.chain() function.

reframe.utility.sanity.contains(seq, key)

Deferrable version of the in operator.

Returns:

key in seq.

reframe.utility.sanity.count(iterable)

Return the element count of iterable.

This is similar to the built-in len(), except that it can also handle any argument that supports iteration, including generators.

reframe.utility.sanity.count_uniq(iterable)

Return the unique element count of iterable.

reframe.utility.sanity.defer(x)

Defer the evaluation of variable x.

Added in version 2.21.

reframe.utility.sanity.enumerate(iterable, start=0)

Replacement for the built-in enumerate() function.

reframe.utility.sanity.evaluate(expr, cache=False)

Evaluate a deferred expression.

If expr is not a deferred expression, it will be returned as is. If expr is a deferred expression and cache is True, the results of the deferred expression will be cached and subsequent calls to evaluate() on this deferred expression (when cache=False) will simply return the previously cached result.

Parameters:

• expr – The expression to be evaluated.

• cache – Cache the result of this evaluation.

Note

When the cache argument is passed as True, a deferred expression will always be evaluated and its results will be re-cached. This may replace any other results that may have been cached in previous evaluations.

Added in version 2.21.

Changed in version 3.8.0: The cache argument is added.

reframe.utility.sanity.extractall(patt, filename, tag=0, conv=None, encoding='utf-8')

Extract all values from the capturing group tag of a matching regex patt in the file filename.

Parameters:

• patt –

  The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.

• filename – The name of the file to examine or a file descriptor as in open().

• encoding – The name of the encoding used to decode the file.

• tag – The regex capturing group to be extracted. Group 0 refers always to the whole match. Since the file is processed line by line, this means that group 0 returns the whole line that was matched.

• conv – A callable or iterable of callables taking a single argument and returning a new value. If not an iterable, it will be used to convert the extracted values for all the capturing groups specified in tag. Otherwise, each conversion function will be used to convert the value extracted from the corresponding capturing group in tag. If more conversion functions are supplied than the corresponding capturing groups in tag, the last conversion function will be used for the additional capturing groups.

Returns:

A list of tuples of converted values extracted from the capturing groups specified in tag, if tag is an iterable. Otherwise, a list of the converted values extracted from the single capturing group specified in tag.

Raises:

reframe.core.exceptions.SanityError – In case of errors.

Changed in version 3.1: Multiple regex capturing groups are now supporetd via tag and multiple conversion functions can be used in conv.

reframe.utility.sanity.extractall_s(patt, string, tag=0, conv=None)

Extract all values from the capturing group tag of a matching regex patt in string.

Parameters:

• patt – as in extractall().

• string – The string to examine.

• tag – as in extractall().

• conv – as in extractall().

Returns:

same as extractall().

Added in version 3.4.1.

reframe.utility.sanity.extractiter(patt, filename, tag=0, conv=None, encoding='utf-8')

Get an iterator over the values extracted from the capturing group tag of a matching regex patt in the file filename.

This function is equivalent to extractall() except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.

reframe.utility.sanity.extractiter_s(patt, string, tag=0, conv=None)

Get an iterator over the values extracted from the capturing group tag of a matching regex patt in string.

This function is equivalent to extractall_s() except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.

Added in version 3.4.1.

reframe.utility.sanity.extractsingle(patt, filename, tag=0, conv=None, item=0, encoding='utf-8')

Extract a single value from the capturing group tag of a matching regex patt in the file filename.

This function is equivalent to extractall(patt, filename, tag, conv)[item], except that it raises a SanityError if item is out of bounds.

Parameters:

• patt – as in extractall().

• filename – as in extractall().

• encoding – as in extractall().

• tag – as in extractall().

• conv – as in extractall().

• item – the specific element to extract.

Returns:

The extracted value.

Raises:

reframe.core.exceptions.SanityError – In case of errors.

reframe.utility.sanity.extractsingle_s(patt, string, tag=0, conv=None, item=0)

Extract a single value from the capturing group tag of a matching regex patt in string.

This function is equivalent to extractall_s(patt, string, tag, conv)[item], except that it raises a SanityError if item is out of bounds.

Parameters:

• patt – as in extractall_s().

• string – as in extractall_s().

• tag – as in extractall_s().

• conv – as in extractall_s().

• item – the specific element to extract.

Returns:

The extracted value.

Raises:

reframe.core.exceptions.SanityError – In case of errors.

Added in version 3.4.1.

reframe.utility.sanity.filter(function, iterable)

Replacement for the built-in filter() function.

reframe.utility.sanity.findall(patt, filename, encoding='utf-8')

Get all matches of regex patt in filename.

Parameters:

• patt –

  The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.

• filename – The name of the file to examine.

• encoding – The name of the encoding used to decode the file.

Returns:

A list of raw regex match objects.

Raises:

reframe.core.exceptions.SanityError – In case an OSError is raised while processing filename.

reframe.utility.sanity.findall_s(patt, string)

Get all matches of regex patt in string.

Parameters:

• patt – as in findall()

• string – The string to examine.

Returns:

same as finall().

Added in version 3.4.1.

reframe.utility.sanity.finditer(patt, filename, encoding='utf-8')

Get an iterator over the matches of the regex patt in filename.

This function is equivalent to findall() except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.

reframe.utility.sanity.finditer_s(patt, string)

Get an iterator over the matches of the regex patt in string.

This function is equivalent to findall_s() except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.

Added in version 3.4.1.

reframe.utility.sanity.getattr(obj, attr, *args)

Replacement for the built-in getattr() function.

reframe.utility.sanity.getitem(container, item)

Get item from container.

container may refer to any container that can be indexed.

Raises:

reframe.core.exceptions.SanityError – In case item cannot be retrieved from container.

reframe.utility.sanity.glob(pathname, *, recursive=False)

Replacement for the glob.glob() function.

reframe.utility.sanity.hasattr(obj, name)

Replacement for the built-in hasattr() function.

reframe.utility.sanity.iglob(pathname, recursive=False)

Replacement for the glob.iglob() function.

reframe.utility.sanity.len(s)

Replacement for the built-in len() function.

reframe.utility.sanity.make_performance_function(func, unit, *args, **kwargs)

Convert a callable or deferred expression into a performance function.

If func is a deferred expression, the performance function will be built by extending this deferred expression into a deferred performance expression. Otherwise, a new deferred performance expression will be created from the function func(). The argument unit is the unit associated with the deferrable performance expression, and *args and **kwargs are the arguments to be captured by this deferred expression. See deferrable functions reference for further information on deferrable functions.

Added in version 3.8.0.

reframe.utility.sanity.map(function, *iterables)

Replacement for the built-in map() function.

reframe.utility.sanity.max(*args)

Replacement for the built-in max() function.

reframe.utility.sanity.min(*args)

Replacement for the built-in min() function.

reframe.utility.sanity.not_(a)

Deferrable version of the not operator.

Returns:

not a.

reframe.utility.sanity.or_(a, b)

Deferrable version of the or operator.

Returns:

a or b.

reframe.utility.sanity.path_exists(path)

Replacement for the os.path.exists() function.

Added in version 3.4.

reframe.utility.sanity.path_isdir(path)

Replacement for the os.path.isdir() function.

Added in version 3.4.

reframe.utility.sanity.path_isfile(path)

Replacement for the os.path.isfile() function.

Added in version 3.4.

reframe.utility.sanity.path_islink(path)

Replacement for the os.path.islink() function.

Added in version 3.4.

reframe.utility.sanity.print(obj, *, sep=' ', end='\n', file=None, flush=False)

Replacement for the built-in print() function.

The only difference is that this function takes a single object argument and it returns that, so that you can use it transparently inside a complex sanity expression. For example, you could write the following to print the matches returned from the extractall() function:

@sanity_function
def my_sanity_fn(self):
    return sn.assert_eq(
        sn.count(sn.print(sn.extractall(...))), 10
    )

If file is None, print() will print its arguments to the standard output. Unlike the builtin print() function, we don’t bind the file argument to sys.stdout by default. This would capture sys.stdout at the time this function is defined and would prevent it from seeing changes to sys.stdout, such as redirects, in the future.

Changed in version 3.4: This function accepts now a single object argument in contrast to the built-in print() function, which accepts multiple.

reframe.utility.sanity.reference_bounds(ref, lower_thres=None, upper_thres=None)

Calculate the absolute bounds from their fractional values.

See assert_reference() for more details.

Added in version 4.9.

reframe.utility.sanity.reversed(seq)

Replacement for the built-in reversed() function.

reframe.utility.sanity.round(number, *args)

Replacement for the built-in round() function.

reframe.utility.sanity.setattr(obj, name, value)

Replacement for the built-in setattr() function.

reframe.utility.sanity.sorted(iterable, *args)

Replacement for the built-in sorted() function.

reframe.utility.sanity.sum(iterable, *args)

Replacement for the built-in sum() function.

reframe.utility.sanity.zip(*iterables)

Replacement for the built-in zip() function.