xdoctest.static_analysis module¶
The core logic that allows for xdoctest to parse source statically
- class xdoctest.static_analysis.CallDefNode(callname, lineno, docstr, doclineno, doclineno_end, args=None)[source]¶
Bases:
object
- Variables:
lineno_end (None | int) – the line number the docstring ends on (if known)
- Parameters:
callname (str) – the name of the item containing the docstring.
lineno (int) – the line number the item containing the docstring.
docstr (str) – the docstring itself
doclineno (int) – the line number (1 based) the docstring begins on
doclineno_end (int) – the line number (1 based) the docstring ends on
args (None | ast.arguments) – arguments from static analysis
TopLevelVisitor
.
- class xdoctest.static_analysis.TopLevelVisitor(source=None)[source]¶
Bases:
NodeVisitor
Parses top-level function names and docstrings
For other
visit_<classname>
values see [MeetTheNodes].References
CommandLine
python -m xdoctest.static_analysis TopLevelVisitor
- Variables:
Example
>>> from xdoctest.static_analysis import * # NOQA >>> from xdoctest import utils >>> source = utils.codeblock( ''' def foo(): """ my docstring """ def subfunc(): pass def bar(): pass class Spam(object): def eggs(self): pass @staticmethod def hams(): pass @property def jams(self): return 3 @jams.setter def jams2(self, x): print('ignoring') @jams.deleter def jams(self, x): print('ignoring') ''') >>> self = TopLevelVisitor.parse(source) >>> callnames = set(self.calldefs.keys()) >>> assert callnames == { >>> 'foo', 'bar', 'Spam', 'Spam.eggs', 'Spam.hams', >>> 'Spam.jams'} >>> assert self.calldefs['foo'].docstr.strip() == 'my docstring' >>> assert 'subfunc' not in self.calldefs
- Parameters:
source (None | str)
- classmethod parse(source)[source]¶
main entry point
executes parsing algorithm and populates self.calldefs
- Parameters:
source (str)
- xdoctest.static_analysis.parse_static_calldefs(source=None, fpath=None)[source]¶
Statically finds top-level callable functions and methods in python source
- Parameters:
source (str) – python text
fpath (str) – filepath to read if source is not specified
- Returns:
- mapping from callnames to CallDefNodes, which contain
info about the item with the doctest.
- Return type:
Dict[str, CallDefNode]
Example
>>> from xdoctest import static_analysis >>> fpath = static_analysis.__file__.replace('.pyc', '.py') >>> calldefs = parse_static_calldefs(fpath=fpath) >>> assert 'parse_static_calldefs' in calldefs
- xdoctest.static_analysis.parse_static_value(key, source=None, fpath=None)[source]¶
Statically parse a constant variable’s value from python code.
TODO: This does not belong here. Move this to an external static analysis library.
- Parameters:
key (str) – name of the variable
source (str) – python text
fpath (str) – filepath to read if source is not specified
- Returns:
object
Example
>>> from xdoctest.static_analysis import parse_static_value >>> key = 'foo' >>> source = 'foo = 123' >>> assert parse_static_value(key, source=source) == 123 >>> source = 'foo = "123"' >>> assert parse_static_value(key, source=source) == '123' >>> source = 'foo = [1, 2, 3]' >>> assert parse_static_value(key, source=source) == [1, 2, 3] >>> source = 'foo = (1, 2, "3")' >>> assert parse_static_value(key, source=source) == (1, 2, "3") >>> source = 'foo = {1: 2, 3: 4}' >>> assert parse_static_value(key, source=source) == {1: 2, 3: 4} >>> source = 'foo = None' >>> assert parse_static_value(key, source=source) == None >>> #parse_static_value('bar', source=source) >>> #parse_static_value('bar', source='foo=1; bar = [1, foo]')
- xdoctest.static_analysis.package_modpaths(pkgpath, with_pkg=False, with_mod=True, followlinks=True, recursive=True, with_libs=False, check=True)[source]¶
Finds sub-packages and sub-modules belonging to a package.
- Parameters:
pkgpath (str) – path to a module or package
with_pkg (bool) – if True includes package __init__ files (default = False)
with_mod (bool) – if True includes module files (default = True)
exclude (list) – ignores any module that matches any of these patterns
recursive (bool) – if False, then only child modules are included
with_libs (bool) – if True then compiled shared libs will be returned as well
check (bool) – if False, then then pkgpath is considered a module even if it does not contain an __init__ file.
- Yields:
str – module names belonging to the package
References
http://stackoverflow.com/questions/1707709/list-modules-in-py-package
Example
>>> from xdoctest.static_analysis import * >>> pkgpath = modname_to_modpath('xdoctest') >>> paths = list(package_modpaths(pkgpath)) >>> print('\n'.join(paths)) >>> names = list(map(modpath_to_modname, paths)) >>> assert 'xdoctest.core' in names >>> assert 'xdoctest.__main__' in names >>> assert 'xdoctest' not in names >>> print('\n'.join(names))
- xdoctest.static_analysis.is_balanced_statement(lines, only_tokens=False, reraise=0)[source]¶
Checks if the lines have balanced braces and quotes.
- Parameters:
lines (List[str]) – list of strings, one for each line
- Returns:
True if the statement is balanced, otherwise False
- Return type:
CommandLine
xdoctest -m xdoctest.static_analysis is_balanced_statement:0
References
https://stackoverflow.com/questions/46061949/parse-until-complete
Example
>>> from xdoctest.static_analysis import * # NOQA >>> assert is_balanced_statement(['print(foobar)']) >>> assert is_balanced_statement(['foo = bar']) is True >>> assert is_balanced_statement(['foo = (']) is False >>> assert is_balanced_statement(['foo = (', "')(')"]) is True >>> assert is_balanced_statement( ... ['foo = (', "'''", ")]'''", ')']) is True >>> assert is_balanced_statement( ... ['foo = ', "'''", ")]'''", ')']) is False >>> #assert is_balanced_statement(['foo = ']) is False >>> #assert is_balanced_statement(['== ']) is False >>> lines = ['def foo():', '', ' x = 1', 'assert True', ''] >>> assert is_balanced_statement(lines)
Example
>>> from xdoctest.static_analysis import * >>> source_parts = [ >>> 'setup(', >>> " name='extension',", >>> ' ext_modules=[', >>> ' CppExtension(', >>> " name='extension',", >>> " sources=['extension.cpp'],", >>> " extra_compile_args=['-g'])),", >>> ' ],', >>> ] >>> print('\n'.join(source_parts)) >>> assert not is_balanced_statement(source_parts) >>> source_parts = [ >>> 'setup(', >>> " name='extension',", >>> ' ext_modules=[', >>> ' CppExtension(', >>> " name='extension',", >>> " sources=['extension.cpp'],", >>> " extra_compile_args=['-g']),", >>> ' ],', >>> ' cmdclass={', >>> " 'build_ext': BuildExtension", >>> ' })', >>> ] >>> print('\n'.join(source_parts)) >>> assert is_balanced_statement(source_parts)
Example
>>> lines = ['try: raise Exception'] >>> is_balanced_statement(lines, only_tokens=1) True >>> is_balanced_statement(lines, only_tokens=0) False
Example
>>> # Cause a failure case on 3.12 >>> from xdoctest.static_analysis import * >>> lines = ['3, 4]', 'print(len(x))'] >>> is_balanced_statement(lines, only_tokens=1) False
- xdoctest.static_analysis.extract_comments(source)[source]¶
Returns the text in each comment in a block of python code. Uses tokenize to account for quotations.
- Parameters:
source (str | List[str])
CommandLine
python -m xdoctest.static_analysis extract_comments
Example
>>> from xdoctest import utils >>> source = utils.codeblock( >>> ''' # comment 1 a = '# not a comment' # comment 2 c = 3 ''') >>> comments = list(extract_comments(source)) >>> assert comments == ['# comment 1', '# comment 2'] >>> comments = list(extract_comments(source.splitlines())) >>> assert comments == ['# comment 1', '# comment 2']