diff --git a/README.md b/README.md index 6eb71e9..ce261c2 100644 --- a/README.md +++ b/README.md @@ -1,54 +1,40 @@ # docstub -> [!NOTE] -> In early development! +> [!NOTE] In early development! +> Expect bugs, missing features, and incomplete documentation. +> Docstub is still evaluating which features it needs to support as the community gives feedback. +> Several features are experimental and included to make adoption of docstub easier. +> Long-term, some of these might be discouraged or removed as docstub matures. -A command line tool to generate Python stub files (PYI) from type descriptions -in NumPyDoc style docstrings. +docstub is a command-line tool to generate [Python stub files](https://typing.python.org/en/latest/guides/writing_stubs.html) (i.e., PYI files) from type descriptions found in [numpydoc](https://numpydoc.readthedocs.io)-style docstrings. +Many packages in the scientific Python ecosystem already describe expected parameter and return types in their docstrings. +Docstub aims to take advantage of these and help with the adoption of type annotations. +It does so by supporting widely used readable conventions such as `array of dtype` or `iterable of int(s)` which it translates into valid type annotations. -## Installation -To try out docstub, for now, we recommend installing docstub directly from this -repo: +## Installation & getting started -```shell -pip install 'docstub [optional] @ git+https://github.com/scientific-python/docstub' -``` +Please refer to the [user guide](doc/user_guide.md) to get started with docstub. -## Usage & configuration - -```shell -cd examples/ -docstub example_pkg/ -``` -will create stub files for `example_pkg/` in `examples/example_pkg-stubs/`. -For now, refer to `docstub --help` for more. - - -### Declare imports and synonyms - -Types in docstrings can and are used without having to import them. However, -when docstub creates stub files from these docstrings it actually needs to -know how to import those unknown types. - -> [!TIP] -> docstub already knows about types in Python's `typing` or `collections.abc` -> modules. That means you can just use types like `Literal` or `Sequence`. - -While docstub is smart enough to find some types via static analysis of -definitions in the given source directory, it must be told about other types -for now. To do so, refer to the syntax and comments in the -`default_config.toml`. +## Contributing +The best way you can help and contribute right now is by trying docstub out! +Feedback to what features might still be missing or where it breaks for you would be greatly appreciated. +Pointers to where the documentation is confusing and unclear. -## Contributing +Since docstub is still in early development there isn't an official contribution guide yet. +Features and API are still being heavily extended and the internal structure is still somewhat in flux. +That said, if that only entices you, feel free to open a PR. +But please do check in with an issue before you do so. -TBD +Our project follows the [Scientific Python's Code of Conduct](https://scientific-python.org/code_of_conduct/). ## Acknowledgements Thanks to [docs2stubs](https://github.com/gramster/docs2stubs) by which this project was heavily inspired and influenced. + +And thanks to CZI for supporting this work with an [EOSS grant](https://chanzuckerberg.com/eoss/proposals/from-library-to-protocol-scikit-image-as-an-api-reference/). diff --git a/doc/command_line.md b/doc/command_line.md new file mode 100644 index 0000000..168893e --- /dev/null +++ b/doc/command_line.md @@ -0,0 +1,41 @@ +# Command line reference + +Running +``` +docstub --help +``` +will print + + + + +```plain +Usage: docstub [OPTIONS] PACKAGE_PATH + + Generate Python stub files with type annotations from docstrings. + + Given a path `PACKAGE_PATH` to a Python package, generate stub files for it. + Type descriptions in docstrings will be used to fill in missing inline type + annotations or to override them. + +Options: + --version Show the version and exit. + -o, --out-dir PATH Set output directory explicitly. Stubs will be directly + written into that directory while preserving the + directory structure under `PACKAGE_PATH`. Otherwise, + stubs are generated inplace. + --config PATH Set one or more configuration file(s) explicitly. + Otherwise, it will look for a `pyproject.toml` or + `docstub.toml` in the current directory. + --group-errors Group identical errors together and list where they + occurred. Will delay showing errors until all files have + been processed. Otherwise, simply report errors as the + occur. + --allow-errors INT Allow this many or fewer errors. If docstub reports + more, exit with error code '1'. This is useful to adopt + docstub gradually. [default: 0; x>=0] + -v, --verbose Print more details (repeatable). + -h, --help Show this message and exit. +``` + + diff --git a/doc/typing_syntax.md b/doc/typing_syntax.md new file mode 100644 index 0000000..682aef6 --- /dev/null +++ b/doc/typing_syntax.md @@ -0,0 +1,124 @@ +# Typing syntax in docstrings + +> [!NOTE] In early development! +> Expect bugs, missing features, and incomplete documentation. +> Docstub is still evaluating which features it needs to support as the community gives feedback. +> Several features are experimental and included to make adoption of docstub easier. +> Long-term, some of these might be discouraged or removed as docstub matures. + +Docstub defines its own [grammar](../src/docstub/doctype.lark) to parse and transform type information in docstrings into valid type annotations. +This grammar fully supports [Python's conventional typing syntax](https://typing.python.org/en/latest/index.html). +So any type annotation that is valid in Python, can be used in a docstrings as is. +In addition, docstub extends this syntax with several "natural language" expressions that are commonly used in the scientific Python ecosystem. + +Docstrings are expected to follow the NumPyDoc style: +``` +Section name +------------ +name : annotation, optional, extra_info + Description. +``` + +- `name` might be the name of a parameter or attribute. + Other sections like "Returns" or "Yields" are supported. +- `annotation` the actual type information that will be transformed into the type annotation. +- `optional` and `extra_info` can be appended to provide additional information. + Their presence and content doesn't currently affect the resulting type annotation. + + +## Unions + +In addition to Python's conventional shorthand `|` syntax for [union types](https://typing.python.org/en/latest/spec/concepts.html#union-types), you can use `or` to join types. + +| Docstring type | Python type annotation | +|----------------|------------------------| +| `X or Y` | `X \| Y` | +| `int or float` | `int \| float` | + + +## Containers + +The content of containers can be typed using a `CONTAINER of X` like form. +This extends the basic subscription syntax for [generics](https://typing.python.org/en/latest/spec/generics.html#generics). + +| Docstring type | Python type annotation | +|-------------------------|------------------------| +| `CONTAINER of X` | `CONTAINER[X]` | +| `CONTAINER of (X or Y)` | `CONTAINER[X \| Y]` | + +For the simple case `CONTAINER of X`, where `X` is a name, you can append `(s)` to indicate the plural form. +E.g., `list of float(s)`. + +Variants of for [**tuples**](https://typing.python.org/en/latest/spec/tuples.html) + +| Docstring type | Python type annotation | +|---------------------|------------------------| +| `tuple of (X, Y)` | `tuple[X, Y]` | +| `tuple of (X, ...)` | `tuple[X, ...]` | + +and **mappings** exist. + +| Docstring type | Python type annotation | +|----------------------|------------------------| +| `MAPPING of {X: Y}` | `MAPPING[X, Y]` | +| `dict of {str: int}` | `dict[str, int]` | + + +> [!TIP] +> While it is possible to nest these variants repeatedly, it is discouraged to do so to keep type descriptions readable. +> For complex annotations with nested containers, consider using Python's conventional syntax. +> In the future, docstub may warn against or disallow nesting these natural language variants. + + +## Shape and dtype syntax for arrays + +This expression allows adding shape and datatype information for data structures like [NumPy arrays](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html). + +`array` and `ndarray`, and `array-like` and `array_like` can be used interchange-ably. + +| Docstring type | Python type annotation | +|-----------------------------|------------------------| +| `array of DTYPE` | `ndarray[DTYPE]` | +| `ndarray of dtype DTYPE` | `ndarray[DTYPE]` | +| `array-like of DTYPE` | `ArrayLike[DTYPE]` | +| `array_like of dtype DTYPE` | `ArrayLike[DTYPE]` | + +> [!NOTE] +> Noting the **shape** of an array in the docstring is supported. +> However, Python's typing system is not yet able to express this information. +> It is therefore not included in the resulting type annotation. + +| Docstring type | Python type annotation | +|--------------------------|------------------------| +| `(3,) array of DTYPE` | `ndarray[DTYPE]` | +| `(X, Y) array of DTYPE` | `ndarray[DTYPE]` | +| `([P,] M, N) array-like` | `ArrayLike` | +| `(M, ...) ndarray` | `ArrayLike` | + + +## Literals + +[Literals](https://typing.python.org/en/latest/spec/literal.html#literals) indicate a concrete value instead of type. +Instead of using [`typing.Literal`](https://docs.python.org/3/library/typing.html#typing.Literal), you can enclose literal values in `{...}` in docstrings. + +| Docstring type | Python type annotation | +|----------------|------------------------| +| `{1, 2, 3}` | `Literal[1, 2, 3]` | +| `{1, 2, 3}` | `Literal[1, 2, 3]` | + +> [!TIP] +> Enclosing a single value `{X}` is currently allowed but discouraged. +> Instead consider the more explicit `Literal[X]`. + + +## reStructuredText role + +Since docstrings are also used to generate documentation with Sphinx, you may want to use [restructuredText roles](https://docutils.sourceforge.io/docs/ref/rst/roles.html) in your type annotations. +Docstub allows for this anywhere where a qualified name can be used. + +| Docstring type | Python type annotation | +|----------------------|------------------------| +| `` `X` `` | `X` | +| ``:ref:`X` `` | `X` | +| ``:class:`Y.X` `` | `Y.X` | +| ``:py:class:`Y.X` `` | `Y.X` | diff --git a/doc/user_guide.md b/doc/user_guide.md new file mode 100644 index 0000000..5fc2da3 --- /dev/null +++ b/doc/user_guide.md @@ -0,0 +1,161 @@ +# User guide + +> [!NOTE] In early development! +> Expect bugs, missing features, and incomplete documentation. +> Docstub is still evaluating which features it needs to support as the community gives feedback. +> Several features are experimental and included to make adoption of docstub easier. +> Long-term, some of these might be discouraged or removed as docstub matures. + + +## Installation + +While a docstub package is already available on PyPI, we recommend trying out docstub by installing directly from GitHub with + +```shell +pip install 'docstub [optional] @ git+https://github.com/scientific-python/docstub' +``` + +If you want to pin to a certain commit you can append `@COMMIT_SHA` to the repo URL above. + + +## Getting started + +Consider a simple example with the following documented function + + + + +```python +# example.py + +def example_metric(image, *, mask=None, sigma=1.0, method='standard'): + """Pretend to calculate a local metric between two images. + + Parameters + ---------- + image : array-like + First image. + mask : array of dtype uint8, optional + Second image. + sigma : float or Iterable of float, default: 1.0 + Sigma value for each dimension in `image`. A single value is broadcast + to all dimensions. + method : {'standard', 'modified'}, default: 'standard' + The method to use for calculating the metric. + + Returns + ------- + metric : ndarray of dtype float + """ + pass +``` + + + +Feeding this input to docstub with + +```shell +docstub example.py +``` + +will create `example.pyi` in the same directory + + + + +```python +# File generated with docstub + +from collections.abc import Iterable +from typing import Literal + +import numpy as np +from numpy.typing import ArrayLike, NDArray + +def example_metric( + image: ArrayLike, + *, + mask: NDArray[np.uint8] | None = ..., + sigma: float | Iterable[float] = ..., + method: Literal["standard", "modified"] = ... +) -> NDArray[float]: ... +``` + + + +There are several interesting things to note here: + +- Many existing conventions that the scientific Python ecosystem uses, will work out of the box. + In this case, docstub knew how to translate `array-like`, `array of dtype uint8` into a valid type annotation in the stub file. + In a similar manner, `or` can be used as a "natural language" alternative to `|`. + You can find more details in [Typing syntax in docstrings](typing_syntax.md). + +- Optional arguments that default to `None` are recognized and a `| None` is appended automatically if the type doesn't include it already. + The `optional` or `default = ...` part don't influence the annotation. + +- Common container types from Python's standard library such as `Iterable` can be used and a necessary import will be added automatically. + + +## Using types & nicknames + +To translate a type from a docstring into a valid type annotation, docstub needs to know where that type originates from and how to import it. +Out of the box, docstub will know about builtin types such as `int` or `bool` that don't need an import, and types in `typing`, `collections.abc` from Python's standard library. +It will source these from the Python environment it is installed in. +In addition to that, docstub will collect all types in the package directory you are running it on. + +However, if you want to use types from third-party libraries you can tell docstub about them in a configuration file. +Docstub will look for a `pyproject.toml` or `docstub.toml` in the current working directory. +Or, you can point docstub at TOML file(s) explicitly using the `--config` option. +In these configuration file(s) you can declare external types directly with + +```toml +[tool.docstub.types] +Path = "pathlib" +Figure = "matplotlib.pyplot" +``` + +This will enable using `Path` and `Figure` anywhere in docstrings. +Alternatively, you can declare an entire prefix with + +```toml +[tool.docstub.type_prefixes] +ski = "skimage" +"sklearn.tree" = "sklearn.tree" +``` + +which will enable any type that is prefixed with `ski.` or `sklearn.tree.`, e.g. `ski.transform.AffineTransform` or `sklearn.tree.DecisionTreeClassifier`. + +In both of these cases, docstub doesn't check that these types actually exist. +Testing the generated stubs with a type checker is recommended. + +> [!TIP] Limitations & roadmap +> Docstub currently collects types statically. +> So it won't see compiled modules and won't be able to generate stubs for them. +> For now, you can add stubs for compiled modules yourself and docstub will include these in the generated output. +> Support for dynamic type collection is on the roadmap. + + +The codebase docstub is running on may already use existing conventions to refer to common types (or you may want to do so). +Docstub refers to these alternatives as "type nicknames". +You can declare type nicknames in a configuration file with +```toml +[tool.docstub.type_nicknames] +func = "Callable" +``` + + +## Adopting docstub gradually + +Adopting docstub on a large codebase may initially generate many errors. +Two command line options can help addressing these errors gradually: + +* `--group-errors` will group identical errors together. + This helps identifying common groups of errors that may be addressed in one go. + +* `--allow-errors` puts an upper limit (["ratchet"](https://qntm.org/ratchet)) on the number of allowed errors. + This way you can adjust the upper bound of allowed errors as they are addressed. + Useful, if you are running in docstub in continuous integration. + +> [!TIP] Get in touch! +> If you are trying out docstub and have feedback or problems, we'd love to hear from you! +> Feel welcome to [open an issue](https://github.com/scientific-python/docstub/issues/new/choose) 🚀. diff --git a/pyproject.toml b/pyproject.toml index 2efef8b..08011a0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -105,6 +105,9 @@ testpaths = [ [tool.coverage] run.source = ["docstub"] +[tool.docstub.types] +Path = "pathlib" + [tool.docstub.type_prefixes] cst = "libcst" lark = "lark" diff --git a/src/docstub/_analysis.py b/src/docstub/_analysis.py index f196624..eacac37 100644 --- a/src/docstub/_analysis.py +++ b/src/docstub/_analysis.py @@ -193,6 +193,8 @@ def __post_init__(self): raise ValueError("builtin cannot contain import information") elif self.import_name is None: raise ValueError("non builtin must at least define an `import_name`") + if self.import_alias is not None and "." in self.import_alias: + raise ValueError("`import_alias` can't contain a '.'") def __repr__(self) -> str: if self.builtin_name: @@ -416,8 +418,8 @@ class TypeMatcher: Attributes ---------- types : dict[str, KnownImport] - prefixes : dict[str, KnownImport] - aliases : dict[str, str] + type_prefixes : dict[str, KnownImport] + type_nicknames : dict[str, str] successful_queries : int unknown_qualnames : list current_module : Path | None @@ -434,19 +436,19 @@ def __init__( self, *, types=None, - prefixes=None, - aliases=None, + type_prefixes=None, + type_nicknames=None, ): """ Parameters ---------- types : dict[str, KnownImport] - prefixes : dict[str, KnownImport] - aliases : dict[str, str] + type_prefixes : dict[str, KnownImport] + type_nicknames : dict[str, str] """ self.types = types or common_known_imports() - self.prefixes = prefixes or {} - self.aliases = aliases or {} + self.type_prefixes = type_prefixes or {} + self.type_nicknames = type_nicknames or {} self.successful_queries = 0 self.unknown_qualnames = [] @@ -499,7 +501,7 @@ def match(self, search_name): ) # Replace alias - search_name = self.aliases.get(search_name, search_name) + search_name = self.type_nicknames.get(search_name, search_name) if type_origin is None and self.current_module: # Try scope of current module @@ -516,7 +518,7 @@ def match(self, search_name): if type_origin is None: # Try a subset of the qualname (first 'a.b.c', then 'a.b' and 'a') for partial_qualname in reversed(accumulate_qualname(search_name)): - type_origin = self.prefixes.get(partial_qualname) + type_origin = self.type_prefixes.get(partial_qualname) if type_origin: type_name = search_name break diff --git a/src/docstub/_cache.py b/src/docstub/_cache.py index 607a69f..0a10575 100644 --- a/src/docstub/_cache.py +++ b/src/docstub/_cache.py @@ -96,7 +96,7 @@ def __init__(self, *, func, serializer, cache_dir, name): """ Parameters ---------- - func : callable + func : Callable The function whose output shall be cached. serializer : FuncSerializer An interface that matches the given `func`. It must implement the diff --git a/src/docstub/_cli.py b/src/docstub/_cli.py index 7af74a7..6128096 100644 --- a/src/docstub/_cli.py +++ b/src/docstub/_cli.py @@ -28,18 +28,20 @@ logger = logging.getLogger(__name__) -def _load_configuration(config_path=None): +def _load_configuration(config_paths=None): """Load and merge configuration from CWD and optional files. Parameters ---------- - config_path : Path + config_paths : list[Path] Returns ------- config : ~.Config """ - config = Config.from_toml(Config.DEFAULT_CONFIG_PATH) + config = Config.from_toml(Config.TEMPLATE_PATH) + numpy_config = Config.from_toml(Config.NUMPY_PATH) + config = config.merge(numpy_config) pyproject_toml = Path.cwd() / "pyproject.toml" if pyproject_toml.is_file(): @@ -53,9 +55,9 @@ def _load_configuration(config_path=None): add_config = Config.from_toml(docstub_toml) config = config.merge(add_config) - if config_path: - logger.info("using %s", config_path) - add_config = Config.from_toml(config_path) + for path in config_paths: + logger.info("using %s", path) + add_config = Config.from_toml(path) config = config.merge(add_config) return config @@ -126,7 +128,10 @@ def report_execution_time(): click.echo(f"Finished in {formated_duration}") +# Preserve click.command below to keep type checker happy +# docstub: off @click.command() +# docstub: on @click.version_option(__version__) @click.argument("root_path", type=click.Path(exists=True), metavar="PACKAGE_PATH") @click.option( @@ -134,19 +139,25 @@ def report_execution_time(): "--out-dir", type=click.Path(file_okay=False), metavar="PATH", - help="Set output directory explicitly. Otherwise, stubs are generated inplace.", + help="Set output directory explicitly. " + "Stubs will be directly written into that directory while preserving the directory " + "structure under `PACKAGE_PATH`. " + "Otherwise, stubs are generated inplace.", ) @click.option( "--config", - "config_path", + "config_paths", type=click.Path(exists=True, dir_okay=False), metavar="PATH", - help="Set configuration file explicitly.", + multiple=True, + help="Set one or more configuration file(s) explicitly. " + "Otherwise, it will look for a `pyproject.toml` or `docstub.toml` in the " + "current directory.", ) @click.option( "--group-errors", is_flag=True, - help="Group identical errors together and list where they occured. " + help="Group identical errors together and list where they occurred. " "Will delay showing errors until all files have been processed. " "Otherwise, simply report errors as the occur.", ) @@ -157,12 +168,13 @@ def report_execution_time(): show_default=True, metavar="INT", help="Allow this many or fewer errors. " - "If docstub reports more, exit with error code '1'.", + "If docstub reports more, exit with error code '1'. " + "This is useful to adopt docstub gradually.", ) @click.option("-v", "--verbose", count=True, help="Print more details (repeatable).") @click.help_option("-h", "--help") @report_execution_time() -def main(root_path, out_dir, config_path, group_errors, allow_errors, verbose): +def main(root_path, out_dir, config_paths, group_errors, allow_errors, verbose): """Generate Python stub files with type annotations from docstrings. Given a path `PACKAGE_PATH` to a Python package, generate stub files for it. @@ -174,7 +186,7 @@ def main(root_path, out_dir, config_path, group_errors, allow_errors, verbose): ---------- root_path : Path out_dir : Path - config_path : Path + config_paths : list[Path] group_errors : bool allow_errors : int verbose : str @@ -191,7 +203,7 @@ def main(root_path, out_dir, config_path, group_errors, allow_errors, verbose): "or type references won't work." ) - config = _load_configuration(config_path) + config = _load_configuration(config_paths) types = common_known_imports() types |= _collect_types(root_path) @@ -200,7 +212,7 @@ def main(root_path, out_dir, config_path, group_errors, allow_errors, verbose): for type_name, module in config.types.items() } - prefixes = { + type_prefixes = { prefix: ( KnownImport(import_name=module, import_alias=prefix) if module != prefix @@ -210,7 +222,9 @@ def main(root_path, out_dir, config_path, group_errors, allow_errors, verbose): } reporter = GroupedErrorReporter() if group_errors else ErrorReporter() - matcher = TypeMatcher(types=types, prefixes=prefixes, aliases=config.type_aliases) + matcher = TypeMatcher( + types=types, type_prefixes=type_prefixes, type_nicknames=config.type_nicknames + ) stub_transformer = Py2StubTransformer(matcher=matcher, reporter=reporter) if not out_dir: diff --git a/src/docstub/_config.py b/src/docstub/_config.py index af100eb..d5df49a 100644 --- a/src/docstub/_config.py +++ b/src/docstub/_config.py @@ -9,11 +9,12 @@ @dataclasses.dataclass(frozen=True, slots=True, kw_only=True) class Config: - DEFAULT_CONFIG_PATH: ClassVar[Path] = Path(__file__).parent / "default_config.toml" + TEMPLATE_PATH: ClassVar[Path] = Path(__file__).parent / "config_template.toml" + NUMPY_PATH: ClassVar[Path] = Path(__file__).parent / "numpy_config.toml" types: dict[str, str] = dataclasses.field(default_factory=dict) type_prefixes: dict[str, str] = dataclasses.field(default_factory=dict) - type_aliases: dict[str, str] = dataclasses.field(default_factory=dict) + type_nicknames: dict[str, str] = dataclasses.field(default_factory=dict) _source: tuple[Path, ...] = () @@ -36,17 +37,6 @@ def from_toml(cls, path): logger.debug("created Config from %s", path) return config - @classmethod - def from_default(cls): - """Create a configuration with default values. - - Returns - ------- - config : Self - """ - config = cls.from_toml(cls.DEFAULT_CONFIG_PATH) - return config - def merge(self, other): """Merge contents with other and return a copy_with Config instance. @@ -63,7 +53,7 @@ def merge(self, other): new = Config( types=self.types | other.types, type_prefixes=self.type_prefixes | other.type_prefixes, - type_aliases=self.type_aliases | other.type_aliases, + type_nicknames=self.type_nicknames | other.type_nicknames, _source=self._source + other._source, ) logger.debug("merged Config from %s", new._source) @@ -82,7 +72,7 @@ def __repr__(self) -> str: @staticmethod def validate(mapping): - for name in ["types", "type_prefixes", "type_aliases"]: + for name in ["types", "type_prefixes", "type_nicknames"]: table = mapping[name] if not isinstance(table, dict): raise TypeError(f"{name} must be a dict") diff --git a/src/docstub/_utils.py b/src/docstub/_utils.py index ecf748f..bbd55bd 100644 --- a/src/docstub/_utils.py +++ b/src/docstub/_utils.py @@ -327,7 +327,10 @@ def key(message): groups[group_name] = [] groups[group_name].append(message) - for (short, details), group in groups.items(): + # Show largest groups last + groups_by_size = sorted(groups.items(), key=lambda x: len(x[1])) + + for (short, details), group in groups_by_size: formatted = click.style(short, bold=True) if len(group) > 1: formatted = f"{formatted} (x{len(group)})" diff --git a/src/docstub/config_template.toml b/src/docstub/config_template.toml new file mode 100644 index 0000000..a277ba8 --- /dev/null +++ b/src/docstub/config_template.toml @@ -0,0 +1,33 @@ +[tool.docstub] + +# Types and their external modules to use in docstrings. +# Docstub can't yet automatically discover where to import types from other +# packages from. Instead, you can provide this information explicitly. +# Any type on the left side will be associated with the given "module" on the +# right side. +# +# Examples: +# Path = "pathlib" +# Will allow using "Path" and use "from pathlib import Path". +# +# NDArray = "numpy.typing" +# Will allow "NDarray" and use "from numpy.typing import NDArray". +[tool.docstub.types] + +# Prefixes for external modules to match types in docstrings. +# Docstub can't yet automatically discover where to import types from other +# packages from. Instead, you can provide this information explicitly. +# Any type in a docstring whose prefix matches the name given on the left side, +# will be associated with the given "module" on the right side. +# +# Examples: +# np = "numpy" +# Will match `np.uint8` and `np.typing.NDarray` and use "import numpy as np". +# +# plt = "matplotlib.pyplot +# Will match `plt.Figure` use `import matplotlib.pyplot as plt`. +[tool.docstub.type_prefixes] + +# Nicknames for types that can be used in docstrings to describe valid Python +# types or annotations. +[tool.docstub.type_nicknames] diff --git a/src/docstub/default_config.toml b/src/docstub/numpy_config.toml similarity index 55% rename from src/docstub/default_config.toml rename to src/docstub/numpy_config.toml index 5f94ff1..087e623 100644 --- a/src/docstub/default_config.toml +++ b/src/docstub/numpy_config.toml @@ -1,51 +1,20 @@ [tool.docstub] # Types and their external modules to use in docstrings. -# Docstub can't yet automatically discover where to import types from other -# packages from. Instead, you can provide this information explicitly. -# Any type on the left side will be associated with the given "module" on the -# right side. -# -# Examples: -# Path = "pathlib" -# Will allow using "Path" and use "from pathlib import Path". -# -# NDArray = "numpy.typing" -# Will allow "NDarray" and use "from numpy.typing import NDArray". [tool.docstub.types] -Path = "pathlib" NDArray = "numpy.typing" ArrayLike = "numpy.typing" # Prefixes for external modules to match types in docstrings. -# Docstub can't yet automatically discover where to import types from other -# packages from. Instead, you can provide this information explicitly. -# Any type in a docstring whose prefix matches the name given on the left side, -# will be associated with the given "module" on the right side. -# -# Examples: -# np = "numpy" -# Will match `np.uint8` and `np.typing.NDarray` and use "import numpy as np". -# -# plt = "matplotlib.pyplot -# Will match `plt.Figure` use `import matplotlib.pyplot as plt`. [tool.docstub.type_prefixes] np = "numpy" numpy = "numpy" -# Specify human-friendly aliases that can be used in docstrings to describe -# valid Python types or annotations. -[tool.docstub.type_aliases] -iterable = "Iterable" -callable = "Callable" -function = "Callable" -func = "Callable" -sequence = "Sequence" -mapping = "Mapping" - -# NumPy +# Nicknames for types that can be used in docstrings to describe valid Python +# types or annotations. +[tool.docstub.type_nicknames] scalar = "np.ScalarType" integer = "np.integer" signedinteger = "np.signedinteger" diff --git a/stubtest_allow.txt b/stubtest_allow.txt index 016a47f..7a9ed55 100644 --- a/stubtest_allow.txt +++ b/stubtest_allow.txt @@ -1,4 +1,3 @@ docstub\._version\..* docstub\..*\.__match_args__$ docstub._cache.FuncSerializer.__type_params__ -docstub._cli.main diff --git a/tests/test_analysis.py b/tests/test_analysis.py index 91a6f09..c23bada 100644 --- a/tests/test_analysis.py +++ b/tests/test_analysis.py @@ -5,6 +5,12 @@ from docstub._analysis import KnownImport, TypeCollector, TypeMatcher +class Test_KnownImport: + def test_dot_in_alias(self): + with pytest.raises(ValueError, match=".*can't contain a '\.'"): + KnownImport(import_name="foo.bar.baz", import_alias="bar.baz") + + @pytest.fixture def module_factory(tmp_path): """Fixture to help with creating adhoc modules with a given source. @@ -160,7 +166,7 @@ def test_query_types(self, search_name, expected_name, expected_origin): ] ) def test_query_prefix(self, search_name, expected_name, expected_origin): - db = TypeMatcher(prefixes=self.type_prefixes.copy()) + db = TypeMatcher(type_prefixes=self.type_prefixes.copy()) type_name, type_origin = db.match(search_name) diff --git a/tests/test_config.py b/tests/test_config.py index 1cb0222..a2f0082 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -2,8 +2,8 @@ class Test_Config: - def test_from_default(self): - config = Config.from_default() + def test_numpy_config(self): + config = Config.from_toml(Config.NUMPY_PATH) assert len(config.types) > 0 assert len(config.type_prefixes) > 0 - assert len(config.type_aliases) > 0 + assert len(config.type_nicknames) > 0 diff --git a/tests/test_doc.py b/tests/test_doc.py new file mode 100644 index 0000000..35c83ee --- /dev/null +++ b/tests/test_doc.py @@ -0,0 +1,73 @@ +"""Test documentation in doc/.""" + +import re +from pathlib import Path + +import click +from click.testing import CliRunner + +from docstub._cli import main as docstub_main + +PROJECT_ROOT = Path(__file__).parent.parent + + +def test_getting_started_example(tmp_path): + # Load user guide + md_file = PROJECT_ROOT / "doc/user_guide.md" + with md_file.open("r") as io: + md_content = io.read() + + # Extract code block for example.py + regex_py = ( + r"" + r"\n+```python(.*)```\n+" + r"" + ) + matches_py = re.findall(regex_py, md_content, flags=re.DOTALL) + assert len(matches_py) == 1 + py_source = matches_py[0] + + # Create example.py and run docstub on it + py_file = tmp_path / "example.py" + with py_file.open("x") as io: + io.write(py_source) + runner = CliRunner() + run_result = runner.invoke(docstub_main, [str(py_file)]) # noqa: F841 + + # Load created PYI file, this is what we expect to find in the user guide's + # code block for example.pyi + pyi_file = py_file.with_suffix(".pyi") + assert pyi_file.is_file() + with pyi_file.open("r") as io: + expected_pyi = io.read().strip() + + # Extract code block for example.pyi from guide + regex_pyi = ( + r"" + r"\n+```python(.*)```\n+" + r"" + ) + matches_pyi = re.findall(regex_pyi, md_content, flags=re.DOTALL) + assert len(matches_pyi) == 1 + actual_pyi = matches_pyi[0].strip() + + assert expected_pyi == actual_pyi + + +def test_command_line_help(): + ctx = click.Context(docstub_main, info_name="docstub") + expected_help = f""" +```plain +{docstub_main.get_help(ctx)} +``` +""".strip() + md_file = PROJECT_ROOT / "doc/command_line.md" + with md_file.open("r") as io: + md_content = io.read() + + regex = r"(.*)" + matches = re.findall(regex, md_content, flags=re.DOTALL) + assert len(matches) == 1 + + actual_help = matches[0].strip() + assert actual_help == expected_help