aria.utils
¶
General-purpose utilities package.
aria.utils.archive
¶
Archive utilities.
-
aria.utils.archive.
extract_archive
(source)¶
-
aria.utils.archive.
is_archive
(source)¶
-
aria.utils.archive.
tar
(source, destination)¶
-
aria.utils.archive.
untar
(archive, destination=None)¶
-
aria.utils.archive.
unzip
(archive, destination=None)¶
-
aria.utils.archive.
zip
(source, destination)¶
aria.utils.argparse
¶
Enhancements to Python’s argparse
module.
-
class
aria.utils.argparse.
ArgumentParser
(prog=None, usage=None, description=None, epilog=None, version=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True)¶ Bases:
argparse.ArgumentParser
Enhanced argument parser.
Applied patch to fix this issue.
-
add_flag_argument
(name, help_true=None, help_false=None, default=False)¶ Adds a flag argument as two arguments:
--my-flag
and--no-my-flag
.
-
aria.utils.caching
¶
Caching utilities.
-
class
aria.utils.caching.
HasCachedMethods
(method_cache=None)¶ Bases:
object
Provides convenience methods for working with
cachedmethod
.
-
class
aria.utils.caching.
cachedmethod
(func)¶ Bases:
object
Decorator for caching method return values.
The implementation is thread-safe.
Supports
cache_info
to be compatible with Python 3’sfunctools.lru_cache
. Note that the statistics are combined for all instances of the class.Won’t use the cache if not called when bound to an object, allowing you to override the cache.
Adapted from this solution.
-
ENABLED
= True¶
-
cache_info
()¶
-
reset_cache_info
()¶
-
aria.utils.collections
¶
Additional collection classes and collection utilities.
-
class
aria.utils.collections.
FrozenDict
(*args, **kwargs)¶ Bases:
collections.OrderedDict
An immutable ordered dict.
After initialization it will raise
TypeError
exceptions if modification is attempted.Note that objects stored in the dict may not be immutable.
-
class
aria.utils.collections.
FrozenList
(*args, **kwargs)¶ Bases:
list
An immutable list.
After initialization it will raise
TypeError
exceptions if modification is attempted.Note that objects stored in the list may not be immutable.
-
append
(value)¶
-
extend
(values)¶
-
insert
(index, value)¶
-
-
class
aria.utils.collections.
StrictDict
(items=None, key_class=None, value_class=None, wrapper_function=None, unwrapper_function=None)¶ Bases:
collections.OrderedDict
An ordered dict that raises
TypeError
exceptions when keys or values of the wrong type are used.
-
class
aria.utils.collections.
StrictList
(items=None, value_class=None, wrapper_function=None, unwrapper_function=None)¶ Bases:
list
A list that raises
TypeError
exceptions when objects of the wrong type are inserted.-
append
(value)¶
-
extend
(values)¶
-
insert
(index, value)¶
-
-
aria.utils.collections.
cls_name
(cls)¶
-
aria.utils.collections.
copy_locators
(target, source)¶ Copies over
_locator
for all elements, recursively.Assumes that target and source have exactly the same list/dict structure.
-
aria.utils.collections.
deepcopy_fast
(obj)¶ The builtin
deepcopy
is very slow due to detection of loops and other errors.This version is surprisingly much faster.
-
aria.utils.collections.
deepcopy_with_locators
(value)¶ Like
deepcopy()
, but also copies over locators.
-
aria.utils.collections.
is_removable
(_container, _key, v)¶
-
aria.utils.collections.
merge
(dict_a, dict_b, copy=True, strict=False, path=None)¶ Merges dicts, recursively.
Parameters: - dict_a – target dict (will be modified)
- dict_b – source dict (will not be modified)
- copy – if True, will use
deepcopy_fast()
on each merged element - strict – if True, will raise a ValueError if there are key conflicts, otherwise will override exiting values
- path – for internal use in strict mode
Returns: dict_a, after the merge
-
aria.utils.collections.
prune
(value, is_removable_function=<function is_removable>)¶ Deletes
None
and empty lists and dicts, recursively.
aria.utils.console
¶
Abstraction API above terminal color libraries.
-
class
aria.utils.console.
Colored
¶ Bases:
object
-
static
black
(string, always=False, bold=False)¶
-
static
blue
(string, always=False, bold=False)¶
-
static
cyan
(string, always=False, bold=False)¶
-
static
green
(string, always=False, bold=False)¶
-
static
magenta
(string, always=False, bold=False)¶
-
static
red
(string, always=False, bold=False)¶
-
static
white
(string, always=False, bold=False)¶
-
static
yellow
(string, always=False, bold=False)¶
-
static
-
class
aria.utils.console.
TopologyStylizer
(indentation=0)¶ Bases:
object
-
indent
(*args, **kwds)¶
-
static
literal_style
(value)¶
-
static
meta_style
(value)¶
-
static
node_style
(value)¶
-
static
property_style
(value)¶
-
static
required_style
(value)¶
-
static
type_style
(value)¶
-
write
(string)¶
-
-
aria.utils.console.
indent
(*args, **kwds)¶
-
aria.utils.console.
puts
(string='', newline=True, stream=<open file '<stdout>', mode 'w'>)¶
aria.utils.exceptions
¶
Utilities for extracting and formatting Python exceptions.
-
aria.utils.exceptions.
get_exception_as_string
(exc_type, exc_val, traceback)¶
-
aria.utils.exceptions.
print_exception
(e, full=True, cause=False, traceback=None)¶ Prints the exception with nice colors and such.
-
aria.utils.exceptions.
print_traceback
(traceback=None, print_last_stack=False)¶ Prints the traceback with nice colors and such.
-
aria.utils.exceptions.
wrap_if_needed
(exception)¶
aria.utils.file
¶
File utilities.
-
aria.utils.file.
makedirs
(path)¶ Enhancement of
os.makedirs()
that doesn’t fail if the directory already exists.
-
aria.utils.file.
remove_if_exists
(path)¶
aria.utils.formatting
¶
String formatting and string-based format utilities.
-
class
aria.utils.formatting.
JsonAsRawEncoder
(*args, **kwargs)¶ Bases:
json.encoder.JSONEncoder
A
JSONEncoder
that will use theas_raw
property of objects if available.-
raw_encoder_default
(obj)¶
-
-
class
aria.utils.formatting.
YamlAsRawDumper
(stream, default_style=None, default_flow_style=None, canonical=None, indent=None, width=None, allow_unicode=None, line_break=None, encoding=None, explicit_start=None, explicit_end=None, version=None, tags=None, block_seq_indent=None, top_level_colon_align=None, prefix_colon=None)¶ Bases:
ruamel.yaml.dumper.RoundTripDumper
A
RoundTripDumper
that will use theas_raw
property of objects if available.-
represent_data
(data)¶
-
-
aria.utils.formatting.
as_agnostic
(value)¶ Converts subclasses of list and dict to standard lists and dicts, and Unicode strings to non-Unicode if possible, recursively.
Useful for creating human-readable output of structures.
-
aria.utils.formatting.
as_raw
(value)¶ Converts values using their
as_raw
property, if it exists, recursively.
-
aria.utils.formatting.
as_raw_dict
(value)¶ Assuming value is a dict, converts its values using
as_raw()
. The keys are left as is.
-
aria.utils.formatting.
as_raw_list
(value)¶ Assuming value is a list, converts its values using
as_raw()
.
-
aria.utils.formatting.
decode_dict
(data)¶
-
aria.utils.formatting.
decode_list
(data)¶
-
aria.utils.formatting.
json_dumps
(value, indent=2)¶ JSON dumps that supports Unicode and the
as_raw
property of objects if available.
-
aria.utils.formatting.
pluralize
(noun)¶
-
aria.utils.formatting.
safe_repr
(value)¶ Like
repr()
, but callsas_raw()
andas_agnostic()
first.
-
aria.utils.formatting.
safe_str
(value)¶ Like
str
coercion, but makes sure that Unicode strings are properly encoded, and will never returnNone
.
-
aria.utils.formatting.
string_list_as_string
(strings)¶ Nice representation of a list of strings.
-
aria.utils.formatting.
yaml_dumps
(value, indent=2)¶ YAML dumps that supports Unicode and the
as_raw
property of objects if available.
-
aria.utils.formatting.
yaml_loads
(value)¶
aria.utils.http
¶
HTTP utilities.
-
aria.utils.http.
download_file
(url, destination=None, logger=None, progress_handler=None)¶ Download file.
Parameters: - url (basestring) – URL from which to download
- destination (basestring) – path where the file should be saved or
None
to auto-generate
Returns: path where the file was saved
Return type: Raises: - exceptions.IOError –
- requests.exceptions.RequestException –
aria.utils.imports
¶
Utilities for dynamically loading Python code.
-
aria.utils.imports.
import_fullname
(name, paths=None)¶ Imports a variable or class based on a full name, optionally searching for it in the paths.
-
aria.utils.imports.
import_modules
(name)¶ Imports a module and all its sub-modules, recursively. Relies on modules defining a
MODULES
attribute listing their sub-module names.
-
aria.utils.imports.
load_attribute
(attribute_path)¶ Dynamically load an attribute based on the path to it. E.g.
some_package.some_module.some_attribute
, will loadsome_attribute
from thesome_package.some_module
module.
aria.utils.openclose
¶
Utilities for working with open/close patterns.
aria.utils.process
¶
Process utilities.
-
aria.utils.process.
append_to_path
(*args, **kwargs)¶ Appends one or more paths to the system path of an environment. The environment will be that of the current process unless another is passed using the ‘env’ keyword argument. :param args: paths to append :param kwargs: ‘env’ may be used to pass a custom environment to use
-
aria.utils.process.
append_to_pythonpath
(*args, **kwargs)¶ Appends one or more paths to the python path of an environment. The environment will be that of the current process unless another is passed using the ‘env’ keyword argument. :param args: paths to append :param kwargs: ‘env’ may be used to pass a custom environment to use
aria.utils.specification
¶
Utilities for cross-referencing code with specification documents.
-
aria.utils.specification.
implements_specification
(section, spec)¶ Decorator for specification implementations.
Used for documentation and standards compliance.
aria.utils.threading
¶
Threading utilities.
-
class
aria.utils.threading.
BlockingExecutor
(print_exceptions=False)¶ Bases:
aria.utils.threading.Executor
Executes tasks in the current thread.
-
exceptions
¶
-
returns
¶
-
submit
(func, *args, **kwargs)¶
-
-
class
aria.utils.threading.
DaemonThread
(*args, **kwargs)¶ Bases:
threading.Thread
-
run
()¶ We’re overriding
Thread.run
in order to avoid annoying (but harmless) error messages during shutdown. The problem is that CPython nullifies the global state _before_ shutting down daemon threads, so that exceptions might happen, and thenThread.__bootstrap_inner
prints them out.Our solution is to swallow these exceptions here.
The side effect is that uncaught exceptions in our own thread code will _not_ be printed out as usual, so it’s our responsibility to catch them in our code.
-
-
class
aria.utils.threading.
ExceptionThread
(*args, **kwargs)¶ Bases:
threading.Thread
A thread from which top level exceptions can be retrieved or re-raised.
-
is_error
()¶
-
raise_error_if_exists
()¶
-
run
()¶
-
-
class
aria.utils.threading.
Executor
(print_exceptions=False)¶ Bases:
object
Executes tasks.
Makes sure to gather all returned results and thrown exceptions in one place, in order of task submission.
Example:
def sum(arg1, arg2): return arg1 + arg2 executor = FixedThreadPoolExecutor(10) try: for value in range(100): executor.submit(sum, value, value) executor.drain() except: executor.close() executor.raise_first() print executor.returns
You can also use it with the Python
with
keyword, in which case you don’t need to callclose
explicitly:with FixedThreadPoolExecutor(10) as executor: for value in range(100): executor.submit(sum, value, value) executor.drain() executor.raise_first() print executor.returns
-
close
()¶ Blocks until all current tasks finish execution and all worker threads are dead.
You cannot submit tasks anymore after calling this.
This is called automatically upon exit if you are using the
with
keyword.
-
drain
()¶ Blocks until all current tasks finish execution, but leaves the worker threads alive.
-
exceptions
¶ The raised exceptions from all tasks, in order of submission.
-
raise_first
()¶ If exceptions were thrown by any task, then the first one will be raised.
This is rather arbitrary: proper handling would involve iterating all the exceptions. However, if you want to use the “raise” mechanism, you are limited to raising only one of them.
-
returns
¶ The returned values from all tasks, in order of submission.
-
submit
(func, *args, **kwargs)¶ Submit a task for execution.
The task will be called ASAP on the next available worker thread in the pool.
Raises: ExecutorException – if cannot be submitted
-
-
exception
aria.utils.threading.
ExecutorException
¶ Bases:
exceptions.Exception
-
class
aria.utils.threading.
FixedThreadPoolExecutor
(size=None, timeout=None, print_exceptions=False)¶ Bases:
aria.utils.threading.Executor
Executes tasks in a fixed thread pool.
Parameters: - size – number of threads in the pool; if
None
will use an optimal number for the platform - timeout – timeout in seconds for all blocking operations (
None
means no timeout) - print_exceptions – set to
True
in order to print exceptions from tasks
-
close
()¶
-
drain
()¶
-
exceptions
¶
-
is_alive
¶ True if any of the worker threads are alive.
-
returns
¶
-
submit
(func, *args, **kwargs)¶
- size – number of threads in the pool; if
-
class
aria.utils.threading.
LockedList
(*args, **kwargs)¶ Bases:
list
A list that supports the
with
keyword with a built-in lock.Though Python lists are thread-safe in that they will not raise exceptions during concurrent access, they do not guarantee atomicity. This class will let you gain atomicity when needed.
aria.utils.type
¶
Type utilities.
-
aria.utils.type.
canonical_type
(type_name)¶ Return the canonical type for any Python, YAML, or TOSCA type name or alias, or
None
if unsupported.Parameters: type_name – Type name (case insensitive)
-
aria.utils.type.
canonical_type_name
(value)¶ Returns the canonical TOSCA type name of a primitive value, or
None
if unknown.For a list of TOSCA type names, see the TOSCA Simple Profile v1.0 cos01 specification
-
aria.utils.type.
convert_value_to_type
(str_value, python_type_name)¶ Converts a value to a specific Python primitive type.
Parameters: python_type_name – Python primitive type name (case insensitive) Raises: ValueError – for unsupported types or conversion failure
-
aria.utils.type.
full_type_name
(value)¶ The full class name of a type or instance.
-
aria.utils.type.
validate_value_type
(value, type_name)¶ Validate that a value is of a specific type. Supports Python, YAML, and TOSCA type names and aliases.
Parameters: type_name – type name (case insensitive) Raises: ValueError – on type mismatch
aria.utils.uris
¶
URI utilities.
-
aria.utils.uris.
as_file
(uri)¶ If the URI is a file (either the
file
scheme or no scheme), then returns the normalized path. Otherwise, returnsNone
.
aria.utils.uuid
¶
UUID generation utilities.
-
aria.utils.uuid.
generate_uuid
(length=None, variant='base57')¶ A random string with varying degrees of guarantee of universal uniqueness.
Parameters: variant – base57
(the default) uses a mix of upper and lowercase alphanumerics ensuring no visually ambiguous characters; default length 22alphanumeric
uses lowercase alphanumeric; default length 25uuid
uses lowercase hexadecimal in the classic UUID format, including dashes; length is always 36hex
uses lowercase hexadecimal characters but has no guarantee of uniqueness; default length of 5
aria.utils.validation
¶
Validation utilities.
-
class
aria.utils.validation.
ValidatorMixin
¶ Bases:
object
A mix0in that should be added to classes that require validating user input.
-
classmethod
validate_callable
(argument_name, argument)¶ Validate
argument
is callable
-
classmethod
validate_in_choice
(name, argument, choices)¶ Validate
argument
is inchoices
-
classmethod
validate_instance
(argument_name, argument, expected_type)¶ Validate
argument
is a instance ofexpected_type
-
classmethod
validate_type
(argument_name, argument, expected_type)¶ Validate
argument
is a subclass ofexpected_type
-
classmethod
-
aria.utils.validation.
validate_function_arguments
(func, func_kwargs)¶ Validates all required arguments are supplied to
func
and that no additional arguments are supplied.
aria.utils.versions
¶
Verion string utilities.
-
class
aria.utils.versions.
VersionString
(value=None)¶ Bases:
unicode
Version string that can be compared, sorted, made unique in a set, and used as a unique dict key.
The primary part of the string is one or more dot-separated natural numbers. Trailing zeroes are treated as redundant, e.g. “1.0.0” == “1.0” == “1”.
An optional qualifier can be added after a “-“. The qualifier can be a natural number or a specially treated prefixed natural number, e.g. “1.1-beta1” > “1.1-alpha2”. The case of the prefix is ignored.
Numeric qualifiers will always be greater than prefixed integer qualifiers, e.g. “1.1-1” > “1.1-beta1”.
Versions without a qualifier will always be greater than their equivalents with a qualifier, e.g. e.g. “1.1” > “1.1-1”.
Any value that does not conform to this format will be treated as a zero version, which would be lesser than any non-zero version.
For efficient list sorts use the
key
property, e.g.:sorted(versions, key=lambda x: x.key)
-
NULL
= u''¶
-