xdoctest.utils package¶
Submodules¶
Module contents¶
Most of these utilities exist in ubelt, but we copy them here to keep xdoctest as a package with minimal dependencies, whereas ubelt includes a larger set of utilities.
This __init__ file is generated using mkinit:
mkinit xdoctest.utils
- class xdoctest.utils.CaptureStdout(suppress=True, enabled=True, **kwargs)[source]¶
Bases:
CaptureStream
Context manager that captures stdout and stores it in an internal stream
- Parameters:
suppress (bool, default=True) – if True, stdout is not printed while captured
enabled (bool, default=True) – does nothing if this is False
Example
>>> self = CaptureStdout(suppress=True) >>> print('dont capture the table flip (╯°□°)╯︵ ┻━┻') >>> with self: ... text = 'capture the heart ♥' ... print(text) >>> print('dont capture look of disapproval ಠ_ಠ') >>> assert isinstance(self.text, str) >>> assert self.text == text + '\n', 'failed capture text'
Example
>>> self = CaptureStdout(suppress=False) >>> with self: ... print('I am captured and printed in stdout') >>> assert self.text.strip() == 'I am captured and printed in stdout'
Example
>>> self = CaptureStdout(suppress=True, enabled=False) >>> with self: ... print('dont capture') >>> assert self.text is None
- class xdoctest.utils.CaptureStream[source]¶
Bases:
object
Generic class for capturing streaming output from stdout or stderr
- class xdoctest.utils.NiceRepr[source]¶
Bases:
object
Defines __str__ and __repr__ in terms of __nice__ function Classes that inherit NiceRepr must define __nice__
Example
>>> class Foo(NiceRepr): ... pass >>> class Bar(NiceRepr): ... def __nice__(self): ... return 'info' >>> foo = Foo() >>> bar = Bar() >>> assert str(bar) == '<Bar(info)>' >>> assert repr(bar).startswith('<Bar(info) at ') >>> assert 'object at' in str(foo) >>> assert 'object at' in repr(foo)
- class xdoctest.utils.PythonPathContext(dpath, index=0)[source]¶
Bases:
object
Context for temporarily adding a dir to the PYTHONPATH.
Used in testing, and used as a helper in certain ubelt functions.
Warning
Even though this context manager takes precautions, this modifies
sys.path
, and things can go wrong when that happens. This is generally safe as long as nothing else you do inside of this context modifies the path. If the path is modified in this context, we will try to detect it and warn.- Variables:
Example
>>> import sys >>> with PythonPathContext('foo', -1): >>> assert sys.path[-1] == 'foo' >>> assert sys.path[-1] != 'foo' >>> with PythonPathContext('bar', 0): >>> assert sys.path[0] == 'bar' >>> assert sys.path[0] != 'bar'
Example
>>> # xdoctest: +REQUIRES(module:pytest) >>> # Mangle the path inside the context >>> import sys >>> self = PythonPathContext('foo', 0) >>> self.__enter__() >>> sys.path.insert(0, 'mangled') >>> import pytest >>> with pytest.warns(UserWarning): >>> self.__exit__(None, None, None)
Example
>>> # xdoctest: +REQUIRES(module:pytest) >>> import sys >>> self = PythonPathContext('foo', 0) >>> self.__enter__() >>> sys.path.remove('foo') >>> import pytest >>> with pytest.raises(RuntimeError): >>> self.__exit__(None, None, None)
- Parameters:
dpath (str | PathLike) – directory to insert into the PYTHONPATH
index (int) – position to add to. Typically either -1 or 0.
- class xdoctest.utils.TeeStringIO(redirect=None)[source]¶
Bases:
StringIO
An IO object that writes to itself and another IO stream.
- Variables:
redirect (io.IOBase) – The other stream to write to.
Example
>>> redirect = io.StringIO() >>> self = TeeStringIO(redirect)
- isatty()[source]¶
Returns true of the redirect is a terminal.
Note
Needed for IPython.embed to work properly when this class is used to override stdout / stderr.
- property encoding¶
Gets the encoding of the redirect IO object
Example
>>> redirect = io.StringIO() >>> assert TeeStringIO(redirect).encoding is None >>> assert TeeStringIO(None).encoding is None >>> assert TeeStringIO(sys.stdout).encoding is sys.stdout.encoding >>> redirect = io.TextIOWrapper(io.StringIO()) >>> assert TeeStringIO(redirect).encoding is redirect.encoding
- class xdoctest.utils.TempDir(persist=False)[source]¶
Bases:
object
Context for creating and cleaning up temporary files. Used in testing.
Example
>>> with TempDir() as self: >>> dpath = self.dpath >>> assert exists(dpath) >>> assert not exists(dpath)
Example
>>> self = TempDir() >>> dpath = self.ensure() >>> assert exists(dpath) >>> self.cleanup() >>> assert not exists(dpath)
- class xdoctest.utils.TempDoctest(docstr, modname=None)[source]¶
Bases:
object
Creates a temporary file containing a module-level doctest for testing
Example
>>> from xdoctest import core >>> self = TempDoctest('>>> a = 1') >>> doctests = list(core.parse_doctestables(self.modpath)) >>> assert len(doctests) == 1
- xdoctest.utils.add_line_numbers(source, start=1, n_digits=None)[source]¶
Prefixes code with line numbers
- Parameters:
source (str | List[str])
start (int)
n_digits (int | None)
- Returns:
List[str] | str
Example
>>> print(chr(10).join(add_line_numbers(['a', 'b', 'c']))) 1 a 2 b 3 c >>> print(add_line_numbers(chr(10).join(['a', 'b', 'c']))) 1 a 2 b 3 c
- xdoctest.utils.codeblock(block_str)[source]¶
Wraps multiline string blocks and returns unindented code. Useful for templated code defined in indented parts of code.
- Parameters:
block_str (str) – typically in the form of a multiline string
- Returns:
the unindented string
- Return type:
Example
>>> # Simulate an indented part of code >>> if True: ... # notice the indentation on this will be normal ... codeblock_version = codeblock( ... ''' ... def foo(): ... return 'bar' ... ''' ... ) ... # notice the indentation and newlines on this will be odd ... normal_version = (''' ... def foo(): ... return 'bar' ... ''') >>> assert normal_version != codeblock_version >>> print('Without codeblock') >>> print(normal_version) >>> print('With codeblock') >>> print(codeblock_version)
- xdoctest.utils.color_text(text, color)[source]¶
Colorizes text a single color using ansii tags.
- Parameters:
text (str) – text to colorize
color (str) – may be one of the following: yellow, blink, lightgray, underline, darkyellow, blue, darkblue, faint, fuchsia, black, white, red, brown, turquoise, bold, darkred, darkgreen, reset, standout, darkteal, darkgray, overline, purple, green, teal, fuscia
- Returns:
- colorized text.
If pygments is not installed plain text is returned.
- Return type:
Example
>>> import sys >>> if sys.platform.startswith('win32'): >>> import pytest >>> pytest.skip() >>> text = 'raw text' >>> from xdoctest import utils >>> from xdoctest.utils import util_str >>> if utils.modname_to_modpath('pygments') and not util_str.NO_COLOR: >>> # Colors text only if pygments is installed >>> import pygments >>> print('pygments = {!r}'.format(pygments)) >>> ansi_text1 = color_text(text, 'red') >>> print('ansi_text1 = {!r}'.format(ansi_text1)) >>> ansi_text = utils.ensure_unicode(ansi_text1) >>> prefix = utils.ensure_unicode('\x1b[31') >>> print('prefix = {!r}'.format(prefix)) >>> print('ansi_text = {!r}'.format(ansi_text)) >>> assert ansi_text.startswith(prefix) >>> assert color_text(text, None) == 'raw text' >>> else: >>> # Otherwise text passes through unchanged >>> assert color_text(text, 'red') == 'raw text' >>> assert color_text(text, None) == 'raw text'
- xdoctest.utils.ensure_unicode(text)[source]¶
Casts bytes into utf8 (mostly for python2 compatibility)
- Parameters:
text (str)
- Returns:
str
References
http://stackoverflow.com/questions/12561063/python-extract-data-from-file
CommandLine
python -m xdoctest.utils ensure_unicode
Example
>>> assert ensure_unicode('my ünicôdé strįng') == 'my ünicôdé strįng' >>> assert ensure_unicode('text1') == 'text1' >>> assert ensure_unicode('text1'.encode('utf8')) == 'text1' >>> assert ensure_unicode('text1'.encode('utf8')) == 'text1' >>> import codecs >>> assert (codecs.BOM_UTF8 + 'text»¿'.encode('utf8')).decode('utf8')
- xdoctest.utils.ensuredir(dpath, mode=1023)[source]¶
Ensures that directory will exist. creates new dir with sticky bits by default
- Parameters:
dpath (str) – dir to ensure. Can also be a tuple to send to join
mode (int) – octal mode of directory (default 0o1777)
- Returns:
path - the ensured directory
- Return type:
- xdoctest.utils.highlight_code(text, lexer_name='python', **kwargs)[source]¶
Highlights a block of text using ansi tags based on language syntax.
- Parameters:
text (str) – plain text to highlight
lexer_name (str) – name of language
**kwargs – passed to pygments.lexers.get_lexer_by_name
- Returns:
- texthighlighted text
If pygments is not installed, the plain text is returned.
- Return type:
CommandLine
python -c "import pygments.formatters; print(list(pygments.formatters.get_all_formatters()))"
Example
>>> text = 'import xdoctest as xdoc; print(xdoc)' >>> new_text = highlight_code(text) >>> print(new_text)
- xdoctest.utils.import_module_from_name(modname)[source]¶
Imports a module from its string name (i.e.
__name__
)This is a simple wrapper around
importlib.import_module()
, but is provided as a companion function toimport_module_from_path()
, which contains functionality not provided in the Python standard library.- Parameters:
modname (str) – module name
- Returns:
module
- Return type:
ModuleType
- SeeAlso:
Example
>>> # test with modules that won't be imported in normal circumstances >>> # todo write a test where we guarantee this >>> # xdoctest: +SKIP("ubelt dependency") >>> import sys >>> modname_list = [ >>> 'pickletools', >>> 'lib2to3.fixes.fix_apply', >>> ] >>> #assert not any(m in sys.modules for m in modname_list) >>> modules = [ub.import_module_from_name(modname) for modname in modname_list] >>> assert [m.__name__ for m in modules] == modname_list >>> assert all(m in sys.modules for m in modname_list)
- xdoctest.utils.import_module_from_path(modpath, index=-1)[source]¶
Imports a module via a filesystem path.
This works by modifying
sys.path
, importing the module name, and then attempting to undo the change to sys.path. This function may produce unexpected results in the case where the imported module itself itself modifiessys.path
or if there is another conflicting module with the same name.- Parameters:
modpath (str | PathLike) – Path to the module on disk or within a zipfile. Paths within a zipfile can be given by
<path-to>.zip/<path-inside-zip>.py
.index (int) – Location at which we modify PYTHONPATH if necessary. If your module name does not conflict, the safest value is -1, However, if there is a conflict, then use an index of 0. The default may change to 0 in the future.
- Returns:
the imported module
- Return type:
ModuleType
References
- Raises:
IOError - when the path to the module does not exist –
ImportError - when the module is unable to be imported –
Note
If the module is part of a package, the package will be imported first. These modules may cause problems when reloading via IPython magic
This can import a module from within a zipfile. To do this modpath should specify the path to the zipfile and the path to the module within that zipfile separated by a colon or pathsep. E.g. “/path/to/archive.zip:mymodule.pl”
Warning
It is best to use this with paths that will not conflict with previously existing modules.
If the modpath conflicts with a previously existing module name. And the target module does imports of its own relative to this conflicting path. In this case, the module that was loaded first will win.
For example if you try to import ‘/foo/bar/pkg/mod.py’ from the folder structure:
- foo/ +- bar/ +- pkg/ + __init__.py |- mod.py |- helper.py
If there exists another module named
pkg
already insys.modules
and mod.py contains the codefrom . import helper
, Python will assume helper belongs to thepkg
module already insys.modules
. This can cause a NameError or worse — an incorrect helper module.- SeeAlso:
Example
>>> # xdoctest: +SKIP("ubelt dependency") >>> import xdoctest >>> modpath = xdoctest.__file__ >>> module = ub.import_module_from_path(modpath) >>> assert module is xdoctest
Example
>>> # Test importing a module from within a zipfile >>> # xdoctest: +SKIP("ubelt dependency") >>> import zipfile >>> from xdoctest import utils >>> import os >>> from os.path import join, expanduser, normpath >>> dpath = expanduser('~/.cache/xdoctest') >>> dpath = utils.ensuredir(dpath) >>> #dpath = utils.TempDir().ensure() >>> # Write to an external module named bar >>> external_modpath = join(dpath, 'bar.py') >>> # For pypy support we have to write this using with >>> with open(external_modpath, 'w') as file: >>> file.write('testvar = 1') >>> internal = 'folder/bar.py' >>> # Move the external bar module into a zipfile >>> zippath = join(dpath, 'myzip.zip') >>> with zipfile.ZipFile(zippath, 'w') as myzip: >>> myzip.write(external_modpath, internal) >>> # Import the bar module from within the zipfile >>> modpath = zippath + ':' + internal >>> modpath = zippath + os.path.sep + internal >>> module = ub.import_module_from_path(modpath) >>> assert normpath(module.__name__) == normpath('folder/bar') >>> assert module.testvar == 1
Example
>>> import pytest >>> # xdoctest: +SKIP("ubelt dependency") >>> with pytest.raises(IOError): >>> ub.import_module_from_path('does-not-exist') >>> with pytest.raises(IOError): >>> ub.import_module_from_path('does-not-exist.zip/')
- xdoctest.utils.indent(text, prefix=' ')[source]¶
Indents a block of text
- Parameters:
text (str) – text to indent
prefix (str) – prefix to add to each line (default = ‘ ‘)
- Returns:
indented text
- Return type:
CommandLine
python -m xdoctest.utils ensure_unicode
Example
>>> text = 'Lorem ipsum\ndolor sit amet' >>> prefix = ' ' >>> result = indent(text, prefix) >>> assert all(t.startswith(prefix) for t in result.split('\n'))
- xdoctest.utils.is_modname_importable(modname, sys_path=None, exclude=None)[source]¶
Determines if a modname is importable based on your current sys.path
- Parameters:
modname (str) – name of module to check
sys_path (list | None, default=None) – if specified overrides
sys.path
exclude (list | None) – list of directory paths. if specified prevents these directories from being searched.
- Returns:
True if the module can be imported
- Return type:
Example
>>> is_modname_importable('xdoctest') True >>> is_modname_importable('not_a_real_module') False >>> is_modname_importable('xdoctest', sys_path=[]) False
- xdoctest.utils.modname_to_modpath(modname, hide_init=True, hide_main=False, sys_path=None)[source]¶
Finds the path to a python module from its name.
Determines the path to a python module without directly import it
Converts the name of a module (__name__) to the path (__file__) where it is located without importing the module. Returns None if the module does not exist.
- Parameters:
modname (str) – The name of a module in
sys_path
.hide_init (bool) – if False, __init__.py will be returned for packages. Defaults to True.
hide_main (bool) – if False, and
hide_init
is True, __main__.py will be returned for packages, if it exists. Defaults to False.sys_path (None | List[str | PathLike]) – The paths to search for the module. If unspecified, defaults to
sys.path
.
- Returns:
modpath - path to the module, or None if it doesn’t exist
- Return type:
str | None
Example
>>> modname = 'xdoctest.__main__' >>> modpath = modname_to_modpath(modname, hide_main=False) >>> assert modpath.endswith('__main__.py') >>> modname = 'xdoctest' >>> modpath = modname_to_modpath(modname, hide_init=False) >>> assert modpath.endswith('__init__.py') >>> # xdoctest: +REQUIRES(CPython) >>> modpath = basename(modname_to_modpath('_ctypes')) >>> assert 'ctypes' in modpath
- xdoctest.utils.modpath_to_modname(modpath, hide_init=True, hide_main=False, check=True, relativeto=None)[source]¶
Determines importable name from file path
Converts the path to a module (__file__) to the importable python name (__name__) without importing the module.
The filename is converted to a module name, and parent directories are recursively included until a directory without an __init__.py file is encountered.
- Parameters:
modpath (str) – Module filepath
hide_init (bool) – Removes the __init__ suffix. Defaults to True.
hide_main (bool) – Removes the __main__ suffix. Defaults to False.
check (bool) – If False, does not raise an error if modpath is a dir and does not contain an __init__ file. Defaults to True.
relativeto (str | None) – If specified, all checks are ignored and this is considered the path to the root module. Defaults to None.
Todo
- [ ] Does this need modification to support PEP 420?
- Returns:
modname
- Return type:
- Raises:
ValueError – if check is True and the path does not exist
Example
>>> from xdoctest import static_analysis >>> modpath = static_analysis.__file__.replace('.pyc', '.py') >>> modpath = modpath.replace('.pyc', '.py') >>> modname = modpath_to_modname(modpath) >>> assert modname == 'xdoctest.static_analysis'
Example
>>> import xdoctest >>> assert modpath_to_modname(xdoctest.__file__.replace('.pyc', '.py')) == 'xdoctest' >>> assert modpath_to_modname(dirname(xdoctest.__file__.replace('.pyc', '.py'))) == 'xdoctest'
Example
>>> # xdoctest: +REQUIRES(CPython) >>> modpath = modname_to_modpath('_ctypes') >>> modname = modpath_to_modname(modpath) >>> assert modname == '_ctypes'
Example
>>> modpath = '/foo/libfoobar.linux-x86_64-3.6.so' >>> modname = modpath_to_modname(modpath, check=False) >>> assert modname == 'libfoobar'
- xdoctest.utils.normalize_modpath(modpath, hide_init=True, hide_main=False)[source]¶
Normalizes __init__ and __main__ paths.
- Parameters:
modpath (str | PathLike) – path to a module
hide_init (bool) – if True, always return package modules as __init__.py files otherwise always return the dpath. Defaults to True.
hide_main (bool) – if True, always strip away main files otherwise ignore __main__.py. Defaults to False.
- Returns:
a normalized path to the module
- Return type:
str | PathLike
Note
Adds __init__ if reasonable, but only removes __main__ by default
Example
>>> from xdoctest import static_analysis as module >>> modpath = module.__file__ >>> assert normalize_modpath(modpath) == modpath.replace('.pyc', '.py') >>> dpath = dirname(modpath) >>> res0 = normalize_modpath(dpath, hide_init=0, hide_main=0) >>> res1 = normalize_modpath(dpath, hide_init=0, hide_main=1) >>> res2 = normalize_modpath(dpath, hide_init=1, hide_main=0) >>> res3 = normalize_modpath(dpath, hide_init=1, hide_main=1) >>> assert res0.endswith('__init__.py') >>> assert res1.endswith('__init__.py') >>> assert not res2.endswith('.py') >>> assert not res3.endswith('.py')
- xdoctest.utils.split_modpath(modpath, check=True)[source]¶
Splits the modpath into the dir that must be in PYTHONPATH for the module to be imported and the modulepath relative to this directory.
- Parameters:
modpath (str) – module filepath
check (bool) – if False, does not raise an error if modpath is a directory and does not contain an
__init__.py
file.
- Returns:
(directory, rel_modpath)
- Return type:
- Raises:
ValueError – if modpath does not exist or is not a package
Example
>>> from xdoctest import static_analysis >>> modpath = static_analysis.__file__.replace('.pyc', '.py') >>> modpath = abspath(modpath) >>> dpath, rel_modpath = split_modpath(modpath) >>> recon = join(dpath, rel_modpath) >>> assert recon == modpath >>> assert rel_modpath == join('xdoctest', 'static_analysis.py')
- xdoctest.utils.strip_ansi(text)[source]¶
Removes all ansi directives from the string.
- Parameters:
text (str)
- Returns:
str
References
http://stackoverflow.com/questions/14693701/remove-ansi https://stackoverflow.com/questions/13506033/filtering-out-ansi-escape-sequences
Examples
>>> line = '\t\u001b[0;35mBlabla\u001b[0m \u001b[0;36m172.18.0.2\u001b[0m' >>> escaped_line = strip_ansi(line) >>> assert escaped_line == '\tBlabla 172.18.0.2'