Sanity Functions Reference¶
Sanity deferrable functions.
This module provides functions to be used with the sanity_patterns
and
:attr`perf_patterns <reframe.core.pipeline.RegressionTest.perf_patterns>`.
The key characteristic of these functions is that they are not executed the
time they are called. Instead they are evaluated at a later point by the
framework (inside the check_sanity
and check_performance
methods).
Any sanity function may be evaluated either explicitly or implicitly.
Explicit evaluation of sanity functions¶
Sanity functions may be evaluated at any time by calling the evaluate
on their return value.
Implicit evaluation of sanity functions¶
Sanity functions may also be evaluated implicitly in the following situations:
- When you try to get their truthy value by either explicitly or implicitly
calling
bool
on their return value. This implies that when you include the result of a sanity function in anif
statement or when you apply theand
,or
ornot
operators, this will trigger their immediate evaluation. - When you try to iterate over their result.
This implies that including the result of a sanity function in a
for
statement will trigger its evaluation immediately. - When you try to explicitly or implicitly get its string representation by
calling
str
on its result. This implies that printing the return value of a sanity function will automatically trigger its evaluation.
This module provides three categories of sanity functions:
Deferrable replacements of certain Python built-in functions. These functions simply delegate their execution to the actual built-ins.
Assertion functions. These functions are used to assert certain conditions and they either return
True
or raiseSanityError
with a message describing the error. Users may provide their own formatted messages through themsg
argument. For example, in the following call toassert_eq()
the{0}
and{1}
placeholders will obtain the actual arguments passed to the assertion function.assert_eq(a, 1, msg="{0} is not equal to {1}")
If in the user provided message more placeholders are used than the arguments of the assert function (except the
msg
argument), no argument substitution will be performed in the user message.Utility functions. The are functions that you will normally use when defining
sanity_patterns
andperf_patterns
. They include, but are not limited to, functions to iterate over regex matches in a file, extracting and converting values from regex matches, computing statistical information on series of data etc.
-
reframe.utility.sanity.
assert_bounded
(val, lower=None, upper=None, msg=None)[source]¶ Assert that
lower <= val <= upper
.Parameters: - val – The value to check.
- lower – The lower bound. If
None
, it defaults to-inf
. - upper – The upper bound. If
None
, it defaults toinf
.
Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_eq
(a, b, msg=None)[source]¶ Assert that
a == b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_false
(x, msg=None)[source]¶ Assert that
x
is evaluated toFalse
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_found
(patt, filename, msg=None, encoding='utf-8')[source]¶ Assert that regex pattern
patt
is found in the filefilename
.Parameters: - patt – The regex pattern to search. Any standard Python regular expression is accepted.
- filename – The name of the file to examine.
Any
OSError
raised while processing the file will be propagated as areframe.core.exceptions.SanityError
. - encoding – The name of the encoding used to decode the file.
Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_ge
(a, b, msg=None)[source]¶ Assert that
a >= b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_gt
(a, b, msg=None)[source]¶ Assert that
a > b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_in
(item, container, msg=None)[source]¶ Assert that
item
is incontainer
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_le
(a, b, msg=None)[source]¶ Assert that
a <= b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_lt
(a, b, msg=None)[source]¶ Assert that
a < b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_ne
(a, b, msg=None)[source]¶ Assert that
a != b
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_not_found
(patt, filename, msg=None, encoding='utf-8')[source]¶ Assert that regex pattern
patt
is not found in the filefilename
.This is the inverse of
assert_found()
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_not_in
(item, container, msg=None)[source]¶ Assert that
item
is not incontainer
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_reference
(val, ref, lower_thres=None, upper_thres=None, msg=None)[source]¶ Assert that value
val
respects the reference valueref
.Parameters: - val – The value to check.
- ref – The reference value.
- lower_thres – The lower threshold value expressed as a negative decimal
fraction of the reference value. Must be in [-1, 0]. If
None
, no lower thresholds is applied. - upper_thres – The upper threshold value expressed as a decimal fraction
of the reference value. Must be in [0, 1]. If
None
, no upper thresholds is applied.
Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails or if the lower and upper thresholds do not have appropriate values.
-
reframe.utility.sanity.
assert_true
(x, msg=None)[source]¶ Assert that
x
is evaluated toTrue
.Returns: True
on success.Raises: reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
chain
(*iterables)[source]¶ Replacement for the
itertools.chain()
function.
-
reframe.utility.sanity.
contains
(seq, key)[source]¶ Deferrable version of the
in
operator.Returns: key in seq
.
-
reframe.utility.sanity.
count
(iterable)[source]¶ Return the element count of
iterable
.This is similar to the built-in
len()
, except that it can also handle any argument that supports iteration, including generators.
-
reframe.utility.sanity.
enumerate
(iterable, start=0)[source]¶ Replacement for the built-in
enumerate()
function.
-
reframe.utility.sanity.
extractall
(patt, filename, tag=0, conv=None, encoding='utf-8')[source]¶ Extract all values from the capturing group
tag
of a matching regexpatt
in the filefilename
.Parameters: - patt –
The regex pattern to search. Any standard Python regular expression is accepted.
- filename – The name of the file to examine.
- encoding – The name of the encoding used to decode the file.
- tag – The regex capturing group to be extracted.
Group
0
refers always to the whole match. Since the file is processed line by line, this means that group0
returns the whole line that was matched. - conv – A callable that takes a single argument and returns a new value. If provided, it will be used to convert the extracted values before returning them.
Returns: A list of the extracted values from the matched regex.
Raises: reframe.core.exceptions.SanityError – In case of errors.
- patt –
-
reframe.utility.sanity.
extractiter
(patt, filename, tag=0, conv=None, encoding='utf-8')[source]¶ Get an iterator over the values extracted from the capturing group
tag
of a matching regexpatt
in the filefilename
.This function is equivalent to
extractall()
except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.
-
reframe.utility.sanity.
extractsingle
(patt, filename, tag=0, conv=None, item=0, encoding='utf-8')[source]¶ Extract a single value from the capturing group
tag
of a matching regexpatt
in the filefilename
.This function is equivalent to
extractall(patt, filename, tag, conv)[item]
, except that it raises aSanityError
ifitem
is out of bounds.Parameters: - patt – as in
extractall()
. - filename – as in
extractall()
. - encoding – as in
extractall()
. - tag – as in
extractall()
. - conv – as in
extractall()
. - item – the specific element to extract.
Returns: The extracted value.
Raises: reframe.core.exceptions.SanityError – In case of errors.
- patt – as in
-
reframe.utility.sanity.
filter
(function, iterable)[source]¶ Replacement for the built-in
filter()
function.
-
reframe.utility.sanity.
findall
(patt, filename, encoding='utf-8')[source]¶ Get all matches of regex
patt
infilename
.Parameters: - patt –
The regex pattern to search. Any standard Python regular expression is accepted.
- filename – The name of the file to examine.
- encoding – The name of the encoding used to decode the file.
Returns: A list of raw regex match objects.
Raises: reframe.core.exceptions.SanityError – In case an
OSError
is raised while processingfilename
.- patt –
-
reframe.utility.sanity.
finditer
(patt, filename, encoding='utf-8')[source]¶ Get an iterator over the matches of the regex
patt
infilename
.This function is equivalent to
findall()
except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.
-
reframe.utility.sanity.
getattr
(obj, attr, *args)[source]¶ Replacement for the built-in
getattr()
function.
-
reframe.utility.sanity.
getitem
(container, item)[source]¶ Get
item
fromcontainer
.container
may refer to any container that can be indexed.Raises: reframe.core.exceptions.SanityError – In case item
cannot be retrieved fromcontainer
.
-
reframe.utility.sanity.
glob
(pathname, *, recursive=False)[source]¶ Replacement for the
glob.glob()
function.
-
reframe.utility.sanity.
iglob
(pathname, recursive=False)[source]¶ Replacement for the
glob.iglob()
function.
-
reframe.utility.sanity.
map
(function, *iterables)[source]¶ Replacement for the built-in
map()
function.
-
reframe.utility.sanity.
reversed
(seq)[source]¶ Replacement for the built-in
reversed()
function.
-
reframe.utility.sanity.
sanity_function
(func)¶ Decorator: Sanity function decorator. Decorate any function to be used in sanity and/or performance patterns with this decorator:
@sanity_function def myfunc(*args): do_sth()
This decorator is an alias to the
reframe.core.deferrable.deferrable()
decorator. The following function definition is equivalent to the above:@deferrable def myfunc(*args): do_sth()
-
reframe.utility.sanity.
setattr
(obj, name, value)[source]¶ Replacement for the built-in
setattr()
function.