Sanity Functions Reference

Sanity deferrable functions.

This module provides functions to be used with the sanity_patterns and :attr`perf_patterns <reframe.core.pipeline.RegressionTest.perf_patterns>`. The key characteristic of these functions is that they are not executed the time they are called. Instead they are evaluated at a later point by the framework (inside the check_sanity and check_performance methods). Any sanity function may be evaluated either explicitly or implicitly.

Explicit evaluation of sanity functions

Sanity functions may be evaluated at any time by calling the evaluate on their return value.

Implicit evaluation of sanity functions

Sanity functions may also be evaluated implicitly in the following situations:

  • When you try to get their truthy value by either explicitly or implicitly calling bool on their return value. This implies that when you include the result of a sanity function in an if statement or when you apply the and, or or not operators, this will trigger their immediate evaluation.
  • When you try to iterate over their result. This implies that including the result of a sanity function in a for statement will trigger its evaluation immediately.
  • When you try to explicitly or implicitly get its string representation by calling str on its result. This implies that printing the return value of a sanity function will automatically trigger its evaluation.

This module provides three categories of sanity functions:

  1. Deferrable replacements of certain Python built-in functions. These functions simply delegate their execution to the actual built-ins.

  2. Assertion functions. These functions are used to assert certain conditions and they either return True or raise SanityError with a message describing the error. Users may provide their own formatted messages through the msg argument. For example, in the following call to assert_eq() the {0} and {1} placeholders will obtain the actual arguments passed to the assertion function.

    assert_eq(a, 1, msg="{0} is not equal to {1}")
    

    If in the user provided message more placeholders are used than the arguments of the assert function (except the msg argument), no argument substitution will be performed in the user message.

  3. Utility functions. The are functions that you will normally use when defining sanity_patterns and perf_patterns. They include, but are not limited to, functions to iterate over regex matches in a file, extracting and converting values from regex matches, computing statistical information on series of data etc.

reframe.utility.sanity.abs(x)[source]

Replacement for the built-in abs() function.

reframe.utility.sanity.all(iterable)[source]

Replacement for the built-in all() function.

reframe.utility.sanity.and_(a, b)[source]

Deferrable version of the and operator.

Returns:a and b.
reframe.utility.sanity.any(iterable)[source]

Replacement for the built-in any() function.

reframe.utility.sanity.assert_bounded(val, lower=None, upper=None, msg=None)[source]

Assert that lower <= val <= upper.

Parameters:
  • val – The value to check.
  • lower – The lower bound. If None, it defaults to -inf.
  • upper – The upper bound. If None, it defaults to inf.
Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_eq(a, b, msg=None)[source]

Assert that a == b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_false(x, msg=None)[source]

Assert that x is evaluated to False.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_found(patt, filename, msg=None, encoding='utf-8')[source]

Assert that regex pattern patt is found in the file filename.

Parameters:
  • patt – The regex pattern to search. Any standard Python regular expression is accepted.
  • filename – The name of the file to examine. Any OSError raised while processing the file will be propagated as a reframe.core.exceptions.SanityError.
  • encoding – The name of the encoding used to decode the file.
Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails.

reframe.utility.sanity.assert_ge(a, b, msg=None)[source]

Assert that a >= b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_gt(a, b, msg=None)[source]

Assert that a > b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_in(item, container, msg=None)[source]

Assert that item is in container.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_le(a, b, msg=None)[source]

Assert that a <= b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_lt(a, b, msg=None)[source]

Assert that a < b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_ne(a, b, msg=None)[source]

Assert that a != b.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_not_found(patt, filename, msg=None, encoding='utf-8')[source]

Assert that regex pattern patt is not found in the file filename.

This is the inverse of assert_found().

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_not_in(item, container, msg=None)[source]

Assert that item is not in container.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.assert_reference(val, ref, lower_thres=None, upper_thres=None, msg=None)[source]

Assert that value val respects the reference value ref.

Parameters:
  • val – The value to check.
  • ref – The reference value.
  • lower_thres – The lower threshold value expressed as a negative decimal fraction of the reference value. Must be in [-1, 0]. If None, no lower thresholds is applied.
  • upper_thres – The upper threshold value expressed as a decimal fraction of the reference value. Must be in [0, 1]. If None, no upper thresholds is applied.
Returns:

True on success.

Raises:

reframe.core.exceptions.SanityError – if assertion fails or if the lower and upper thresholds do not have appropriate values.

reframe.utility.sanity.assert_true(x, msg=None)[source]

Assert that x is evaluated to True.

Returns:True on success.
Raises:reframe.core.exceptions.SanityError – if assertion fails.
reframe.utility.sanity.avg(iterable)[source]

Return the average of all the elements of iterable.

reframe.utility.sanity.chain(*iterables)[source]

Replacement for the itertools.chain() function.

reframe.utility.sanity.contains(seq, key)[source]

Deferrable version of the in operator.

Returns:key in seq.
reframe.utility.sanity.count(iterable)[source]

Return the element count of iterable.

This is similar to the built-in len(), except that it can also handle any argument that supports iteration, including generators.

reframe.utility.sanity.enumerate(iterable, start=0)[source]

Replacement for the built-in enumerate() function.

reframe.utility.sanity.extractall(patt, filename, tag=0, conv=None, encoding='utf-8')[source]

Extract all values from the capturing group tag of a matching regex patt in the file filename.

Parameters:
  • patt

    The regex pattern to search. Any standard Python regular expression is accepted.

  • filename – The name of the file to examine.
  • encoding – The name of the encoding used to decode the file.
  • tag – The regex capturing group to be extracted. Group 0 refers always to the whole match. Since the file is processed line by line, this means that group 0 returns the whole line that was matched.
  • conv – A callable that takes a single argument and returns a new value. If provided, it will be used to convert the extracted values before returning them.
Returns:

A list of the extracted values from the matched regex.

Raises:

reframe.core.exceptions.SanityError – In case of errors.

reframe.utility.sanity.extractiter(patt, filename, tag=0, conv=None, encoding='utf-8')[source]

Get an iterator over the values extracted from the capturing group tag of a matching regex patt in the file filename.

This function is equivalent to extractall() except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.

reframe.utility.sanity.extractsingle(patt, filename, tag=0, conv=None, item=0, encoding='utf-8')[source]

Extract a single value from the capturing group tag of a matching regex patt in the file filename.

This function is equivalent to extractall(patt, filename, tag, conv)[item], except that it raises a SanityError if item is out of bounds.

Parameters:
Returns:

The extracted value.

Raises:

reframe.core.exceptions.SanityError – In case of errors.

reframe.utility.sanity.filter(function, iterable)[source]

Replacement for the built-in filter() function.

reframe.utility.sanity.findall(patt, filename, encoding='utf-8')[source]

Get all matches of regex patt in filename.

Parameters:
  • patt

    The regex pattern to search. Any standard Python regular expression is accepted.

  • filename – The name of the file to examine.
  • encoding – The name of the encoding used to decode the file.
Returns:

A list of raw regex match objects.

Raises:

reframe.core.exceptions.SanityError – In case an OSError is raised while processing filename.

reframe.utility.sanity.finditer(patt, filename, encoding='utf-8')[source]

Get an iterator over the matches of the regex patt in filename.

This function is equivalent to findall() except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.

reframe.utility.sanity.getattr(obj, attr, *args)[source]

Replacement for the built-in getattr() function.

reframe.utility.sanity.getitem(container, item)[source]

Get item from container.

container may refer to any container that can be indexed.

Raises:reframe.core.exceptions.SanityError – In case item cannot be retrieved from container.
reframe.utility.sanity.glob(pathname, *, recursive=False)[source]

Replacement for the glob.glob() function.

reframe.utility.sanity.hasattr(obj, name)[source]

Replacement for the built-in hasattr() function.

reframe.utility.sanity.iglob(pathname, recursive=False)[source]

Replacement for the glob.iglob() function.

reframe.utility.sanity.len(s)[source]

Replacement for the built-in len() function.

reframe.utility.sanity.map(function, *iterables)[source]

Replacement for the built-in map() function.

reframe.utility.sanity.max(*args)[source]

Replacement for the built-in max() function.

reframe.utility.sanity.min(*args)[source]

Replacement for the built-in min() function.

reframe.utility.sanity.not_(a)[source]

Deferrable version of the not operator.

Returns:not a.
reframe.utility.sanity.or_(a, b)[source]

Deferrable version of the or operator.

Returns:a or b.
reframe.utility.sanity.reversed(seq)[source]

Replacement for the built-in reversed() function.

reframe.utility.sanity.round(number, *args)[source]

Replacement for the built-in round() function.

reframe.utility.sanity.sanity_function(func)
Decorator:Sanity function decorator.

Decorate any function to be used in sanity and/or performance patterns with this decorator:

@sanity_function
def myfunc(*args):
    do_sth()

This decorator is an alias to the reframe.core.deferrable.deferrable() decorator. The following function definition is equivalent to the above:

@deferrable
def myfunc(*args):
    do_sth()
reframe.utility.sanity.setattr(obj, name, value)[source]

Replacement for the built-in setattr() function.

reframe.utility.sanity.sorted(iterable, *args)[source]

Replacement for the built-in sorted() function.

reframe.utility.sanity.sum(iterable, *args)[source]

Replacement for the built-in sum() function.

reframe.utility.sanity.zip(*iterables)[source]

Replacement for the built-in zip() function.