Deferrable Functions Reference¶
Deferrable functions are the functions whose execution may be postponed to a later time after they are called. The key characteristic of these functions is that they store their arguments when they are called, and the execution itself does not occur until the function is evaluated either explicitly or implicitly.
ReFrame provides an ample set of deferrable utilities and it also allows users to write their own deferrable functions when needed. Please refer to “Understanding the Mechanism of Deferrable Functions” for a hands-on explanation on how deferrable functions work and how to create custom deferrable functions.
Explicit evaluation of deferrable functions¶
Deferrable functions may be evaluated at any time by calling evaluate()
on their return value or by passing the deferred function itself to the evaluate()
free function.
These evaluate()
functions take an optional bool
argument cache
, which can be used to cache the evaluation of the deferrable function.
Hence, if caching is enabled on a given deferrable function, any subsequent calls to evaluate()
will simply return the previously cached results.
Changed in version 3.8.0: Support of cached evaluation is added.
Implicit evaluation of deferrable functions¶
Deferrable functions may also be evaluated implicitly in the following situations:
When you try to get their truthy value by either explicitly or implicitly calling
bool
on their return value. This implies that when you include the result of a deferrable function in anif
statement or when you apply theand
,or
ornot
operators, this will trigger their immediate evaluation.When you try to iterate over their result. This implies that including the result of a deferrable function in a
for
statement will trigger its evaluation immediately.When you try to explicitly or implicitly get its string representation by calling
str
on its result. This implies that printing the return value of a deferrable function will automatically trigger its evaluation.
Categories of deferrable functions¶
Currently ReFrame provides three broad categories of deferrable functions:
Deferrable replacements of certain Python built-in functions. These functions simply delegate their execution to the actual built-ins.
Assertion functions. These functions are used to assert certain conditions and they either return
True
or raiseSanityError
with a message describing the error. Users may provide their own formatted messages through themsg
argument. For example, in the following call toassert_eq()
the{0}
and{1}
placeholders will obtain the actual arguments passed to the assertion function.assert_eq(a, 1, msg="{0} is not equal to {1}")
If in the user provided message more placeholders are used than the arguments of the assert function (except the
msg
argument), no argument substitution will be performed in the user message.Utility functions. They include, but are not limited to, functions to iterate over regex matches in a file, extracting and converting values from regex matches, computing statistical information on series of data etc.
Deferrable performance functions¶
New in version 3.8.0.
Deferrable performance functions are a special type of deferrable functions which are intended for measuring a given quantity.
Therefore, this kind of deferrable functions have an associated unit that can be used to interpret the return values from these functions.
The unit of a deferrable performance function can be accessed through the public member unit
.
Regular deferrable functions can be promoted to deferrable performance functions using the make_performance_function()
utility.
Also, this utility allows to create performance functions directly from any callable.
List of deferrable functions and utilities¶
-
@
reframe.utility.sanity.
deferrable
(func)¶ Deferrable decorator.
Converts the decorated free function into a deferrable function.
import reframe.utility.sanity as sn @sn.deferrable def myfunc(*args): do_sth()
-
@
reframe.utility.sanity.
sanity_function
(func)¶ Please use the
reframe.core.pipeline.RegressionMixin.deferrable()
decorator when possible. Alternatively, please use thereframe.utility.sanity.deferrable()
decorator instead.Warning
Not to be mistaken with
sanity_function()
built-in.Deprecated since version 3.8.0.
-
reframe.utility.sanity.
allx
(iterable)[source]¶ Same as the built-in
all()
function, except that it returnsFalse
ifiterable
is empty.New in version 2.13.
-
reframe.utility.sanity.
assert_bounded
(val, lower=None, upper=None, msg=None)[source]¶ Assert that
lower <= val <= upper
.- Parameters
val – The value to check.
lower – The lower bound. If
None
, it defaults to-inf
.upper – The upper bound. If
None
, it defaults toinf
.msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.
- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_eq
(a, b, msg=None)[source]¶ Assert that
a == b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_false
(x, msg=None)[source]¶ Assert that
x
is evaluated toFalse
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_found
(patt, filename, msg=None, encoding='utf-8')[source]¶ Assert that regex pattern
patt
is found in the filefilename
.- Parameters
patt – The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.
filename – The name of the file to examine or a file descriptor as in
open()
. AnyOSError
raised while processing the file will be propagated as areframe.core.exceptions.SanityError
.msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.encoding – The name of the encoding used to decode the file.
- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_found_s
(patt, string, msg=None)[source]¶ Assert that regex pattern
patt
is found in the stringstring
.- Parameters
patt – as in
assert_found()
.string – The string to examine.
msg – as in
assert_found()
. You may use{0}
…{N}
as placeholders for the function arguments.
- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
New in version 3.4.1.
-
reframe.utility.sanity.
assert_ge
(a, b, msg=None)[source]¶ Assert that
a >= b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_gt
(a, b, msg=None)[source]¶ Assert that
a > b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_in
(item, container, msg=None)[source]¶ Assert that
item
is incontainer
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_le
(a, b, msg=None)[source]¶ Assert that
a <= b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_lt
(a, b, msg=None)[source]¶ Assert that
a < b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_ne
(a, b, msg=None)[source]¶ Assert that
a != b
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_not_found
(patt, filename, msg=None, encoding='utf-8')[source]¶ Assert that regex pattern
patt
is not found in the filefilename
.This is the inverse of
assert_found()
.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_not_found_s
(patt, string, msg=None)[source]¶ Assert that regex pattern
patt
is not found instring
.This is the inverse of
assert_found_s()
.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
New in version 3.4.1.
-
reframe.utility.sanity.
assert_not_in
(item, container, msg=None)[source]¶ Assert that
item
is not incontainer
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
assert_reference
(val, ref, lower_thres=None, upper_thres=None, msg=None)[source]¶ Assert that value
val
respects the reference valueref
.- Parameters
val – The value to check.
ref – The reference value.
lower_thres – The lower threshold value expressed as a negative decimal fraction of the reference value. Must be in [-1, 0] for ref >= 0.0 and in [-inf, 0] for ref < 0.0. If
None
, no lower thresholds is applied.upper_thres – The upper threshold value expressed as a decimal fraction of the reference value. Must be in [0, inf] for ref >= 0.0 and in [0, 1] for ref < 0.0. If
None
, no upper thresholds is applied.msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.
- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails or if the lower and upper thresholds do not have appropriate values.
-
reframe.utility.sanity.
assert_true
(x, msg=None)[source]¶ Assert that
x
is evaluated toTrue
.- Parameters
msg – The error message to use if the assertion fails. You may use
{0}
…{N}
as placeholders for the function arguments.- Returns
True
on success.- Raises
reframe.core.exceptions.SanityError – if assertion fails.
-
reframe.utility.sanity.
chain
(*iterables)[source]¶ Replacement for the
itertools.chain()
function.
-
reframe.utility.sanity.
contains
(seq, key)[source]¶ Deferrable version of the
in
operator.- Returns
key in seq
.
-
reframe.utility.sanity.
count
(iterable)[source]¶ Return the element count of
iterable
.This is similar to the built-in
len()
, except that it can also handle any argument that supports iteration, including generators.
-
reframe.utility.sanity.
enumerate
(iterable, start=0)[source]¶ Replacement for the built-in
enumerate()
function.
-
reframe.utility.sanity.
evaluate
(expr, cache=False)[source]¶ Evaluate a deferred expression.
If
expr
is not a deferred expression, it will be returned as is. Ifexpr
is a deferred expression andcache
isTrue
, the results of the deferred expression will be cached and subsequent calls toevaluate()
on this deferred expression (whencache=False
) will simply return the previously cached result.- Parameters
expr – The expression to be evaluated.
cache – Cache the result of this evaluation.
Note
When the
cache
argument is passed asTrue
, a deferred expression will always be evaluated and its results will be re-cached. This may replace any other results that may have been cached in previous evaluations.New in version 2.21.
Changed in version 3.8.0: The
cache
argument is added.
-
reframe.utility.sanity.
extractall
(patt, filename, tag=0, conv=None, encoding='utf-8')[source]¶ Extract all values from the capturing group
tag
of a matching regexpatt
in the filefilename
.- Parameters
patt –
The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.
filename – The name of the file to examine or a file descriptor as in
open()
.encoding – The name of the encoding used to decode the file.
tag – The regex capturing group to be extracted. Group
0
refers always to the whole match. Since the file is processed line by line, this means that group0
returns the whole line that was matched.conv – A callable or iterable of callables taking a single argument and returning a new value. If not an iterable, it will be used to convert the extracted values for all the capturing groups specified in
tag
. Otherwise, each conversion function will be used to convert the value extracted from the corresponding capturing group intag
. If more conversion functions are supplied than the corresponding capturing groups intag
, the last conversion function will be used for the additional capturing groups.
- Returns
A list of tuples of converted values extracted from the capturing groups specified in
tag
, iftag
is an iterable. Otherwise, a list of the converted values extracted from the single capturing group specified intag
.- Raises
reframe.core.exceptions.SanityError – In case of errors.
Changed in version 3.1: Multiple regex capturing groups are now supporetd via
tag
and multiple conversion functions can be used inconv
.
-
reframe.utility.sanity.
extractall_s
(patt, string, tag=0, conv=None)[source]¶ - Extract all values from the capturing group
tag
of a matching regex patt
instring
.- arg patt
as in
extractall()
.
- ‘ :arg string: The string to examine.
- arg tag
as in
extractall()
.- arg conv
as in
extractall()
.- returns
same as
extractall()
.
New in version 3.4.1.
- Extract all values from the capturing group
-
reframe.utility.sanity.
extractiter
(patt, filename, tag=0, conv=None, encoding='utf-8')[source]¶ Get an iterator over the values extracted from the capturing group
tag
of a matching regexpatt
in the filefilename
.This function is equivalent to
extractall()
except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.
-
reframe.utility.sanity.
extractiter_s
(patt, string, tag=0, conv=None)[source]¶ Get an iterator over the values extracted from the capturing group
tag
of a matching regexpatt
instring
.This function is equivalent to
extractall_s()
except that it returns a generator object, instead of a list, which you can use to iterate over the extracted values.New in version 3.4.1.
-
reframe.utility.sanity.
extractsingle
(patt, filename, tag=0, conv=None, item=0, encoding='utf-8')[source]¶ Extract a single value from the capturing group
tag
of a matching regexpatt
in the filefilename
.This function is equivalent to
extractall(patt, filename, tag, conv)[item]
, except that it raises aSanityError
ifitem
is out of bounds.- Parameters
patt – as in
extractall()
.filename – as in
extractall()
.encoding – as in
extractall()
.tag – as in
extractall()
.conv – as in
extractall()
.item – the specific element to extract.
- Returns
The extracted value.
- Raises
reframe.core.exceptions.SanityError – In case of errors.
-
reframe.utility.sanity.
extractsingle_s
(patt, string, tag=0, conv=None, item=0)[source]¶ Extract a single value from the capturing group
tag
of a matching regexpatt
instring
.This function is equivalent to
extractall_s(patt, string, tag, conv)[item]
, except that it raises aSanityError
ifitem
is out of bounds.- Parameters
patt – as in
extractall_s()
.string – as in
extractall_s()
.tag – as in
extractall_s()
.conv – as in
extractall_s()
.item – the specific element to extract.
- Returns
The extracted value.
- Raises
reframe.core.exceptions.SanityError – In case of errors.
New in version 3.4.1.
-
reframe.utility.sanity.
filter
(function, iterable)[source]¶ Replacement for the built-in
filter()
function.
-
reframe.utility.sanity.
findall
(patt, filename, encoding='utf-8')[source]¶ Get all matches of regex
patt
infilename
.- Parameters
patt –
The regex pattern to search. Any standard Python regular expression is accepted. The re.MULTILINE flag is set for the pattern search.
filename – The name of the file to examine.
encoding – The name of the encoding used to decode the file.
- Returns
A list of raw regex match objects.
- Raises
reframe.core.exceptions.SanityError – In case an
OSError
is raised while processingfilename
.
-
reframe.utility.sanity.
findall_s
(patt, string)[source]¶ Get all matches of regex
patt
instring
.- Parameters
patt – as in
findall()
string – The string to examine.
- Returns
same as
finall()
.
New in version 3.4.1.
-
reframe.utility.sanity.
finditer
(patt, filename, encoding='utf-8')[source]¶ Get an iterator over the matches of the regex
patt
infilename
.This function is equivalent to
findall()
except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.
-
reframe.utility.sanity.
finditer_s
(patt, string)[source]¶ Get an iterator over the matches of the regex
patt
instring
.This function is equivalent to
findall_s()
except that it returns a generator object instead of a list, which you can use to iterate over the raw matches.New in version 3.4.1.
-
reframe.utility.sanity.
getattr
(obj, attr, *args)[source]¶ Replacement for the built-in
getattr()
function.
-
reframe.utility.sanity.
getitem
(container, item)[source]¶ Get
item
fromcontainer
.container
may refer to any container that can be indexed.- Raises
reframe.core.exceptions.SanityError – In case
item
cannot be retrieved fromcontainer
.
-
reframe.utility.sanity.
glob
(pathname, *, recursive=False)[source]¶ Replacement for the
glob.glob()
function.
-
reframe.utility.sanity.
iglob
(pathname, recursive=False)[source]¶ Replacement for the
glob.iglob()
function.
-
reframe.utility.sanity.
make_performance_function
(func, unit, *args, **kwargs)[source]¶ Convert a callable or deferred expression into a performance function.
If
func
is a deferred expression, the performance function will be built by extending this deferred expression into a deferred performance expression. Otherwise, a new deferred performance expression will be created from the functionfunc()
. The argumentunit
is the unit associated with the deferrable performance expression, and*args
and**kwargs
are the arguments to be captured by this deferred expression. See deferrable functions reference for further information on deferrable functions.New in version 3.8.0.
-
reframe.utility.sanity.
map
(function, *iterables)[source]¶ Replacement for the built-in
map()
function.
-
reframe.utility.sanity.
path_exists
(path)[source]¶ Replacement for the
os.path.exists()
function.New in version 3.4.
-
reframe.utility.sanity.
path_isdir
(path)[source]¶ Replacement for the
os.path.isdir()
function.New in version 3.4.
-
reframe.utility.sanity.
path_isfile
(path)[source]¶ Replacement for the
os.path.isfile()
function.New in version 3.4.
-
reframe.utility.sanity.
path_islink
(path)[source]¶ Replacement for the
os.path.islink()
function.New in version 3.4.
-
reframe.utility.sanity.
print
(obj, *, sep=' ', end='\n', file=None, flush=False)[source]¶ Replacement for the built-in
print()
function.The only difference is that this function takes a single object argument and it returns that, so that you can use it transparently inside a complex sanity expression. For example, you could write the following to print the matches returned from the
extractall()
function:@sanity_function def my_sanity_fn(self): return sn.assert_eq( sn.count(sn.print(sn.extract_all(...))), 10 )
If
file
is None,print()
will print its arguments to the standard output. Unlike the builtinprint()
function, we don’t bind thefile
argument tosys.stdout
by default. This would capturesys.stdout
at the time this function is defined and would prevent it from seeing changes tosys.stdout
, such as redirects, in the future.Changed in version 3.4: This function accepts now a single object argument in contrast to the built-in
print()
function, which accepts multiple.
-
reframe.utility.sanity.
reversed
(seq)[source]¶ Replacement for the built-in
reversed()
function.
-
reframe.utility.sanity.
setattr
(obj, name, value)[source]¶ Replacement for the built-in
setattr()
function.