-
Notifications
You must be signed in to change notification settings - Fork 517
Expand file tree
/
Copy pathmoddocs.py
More file actions
84 lines (53 loc) · 7.92 KB
/
Copy pathmoddocs.py
File metadata and controls
84 lines (53 loc) · 7.92 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
r"""# Module docs: creating them from existing notebooks
Module docs are the LLM-facing documentation channel of an nbdev project: module docstrings, the package docstring, and `llms.txt`, all generated by `nbdev-export`. They ship inside the code, so they are there for every user of the library with no separate network lookup, and they cannot be stale relative to the installed version. They also make learning uniform. One call, `doc(foo)`, teaches a package, a module, a class, or a function, and plain `help(foo)` shows the same text. A reader works progressively: `llms.txt` or the package docstring says which modules matter, `doc(module)` teaches one module in a single read, and the `Docs:` URL ending each docstring leads to the full page when needed.
This guide covers one task. A project's notebooks already exist, and you are creating the module docs from them. General notebook authoring, and the `#| export`/`#| exportd` docstring mechanics, are `doc(nbdev.skill)`'s job. Read it first.
## Creating a module's docs from its notebook
The notebook already contains the documentation. Its author made the content decisions when writing it: which ideas get prose, which examples demonstrate them, what matters. Creating module docs is a projection of that material, not new writing. The work is deciding which prose becomes the docstring, and reshaping it to stand alone.
Start with the baseline. Run `doc(module)` as-is and note what a reader gets today, usually the bare summary line. Then read the whole notebook, sorting its markdown into two kinds:
- Standalone prose, which teaches a concept without depending on the cells around it. These are tagging candidates.
- Cell-adjacent prose ("...to this:", "The code for `Int` is shown below:", six cells walking one function's variations). These cannot export as-is.
The mix decides between two options, and the user picks:
**Option 1: extract.** Build the docstring from the notebook's own narrative. When a cell stands alone, tag it `#| export` where it sits. When it nearly stands alone, reshape lightly first:
- Split the cell so only the standalone part is tagged.
- End a summary with a colon so it reads straight into a `#| exportd` demo.
- Fix wording that only works on the page, such as "the helpers below" or "as shown above".
Where a section's prose is all cell-adjacent, write one new summary cell under its heading, condensing what those cells teach: the idioms a reader could not guess from signatures, with the long tail left to the docs page. This improves the docs page at the same time, since a section that jumped straight into per-function detail was missing its overview anyway. When most of a module's docs can be tagged nearly as-is, the original notebook was authored well, and the docstring comes almost for free.
**Option 2: front block.** Write the docstring as a few fresh markdown cells right after the title, optionally with `#| exportd` demos between them, all tagged, forming one self-contained overview read top to bottom. Reach for this when extraction keeps producing fragments, when the notebook's structure doesn't match the story the docstring should tell, or when the user simply prefers it. It is the simplest safe default. Nothing the author wrote is touched, so nothing on the page changes, except that the page gains an overview at the top, which usually helps. Don't push for restructuring the whole notebook when a front block serves.
**Demos.** An existing example cell can often be tagged `#| exportd` as-is. A new demo must run after the definitions it uses, so its notebook position is constrained, but its docstring position is not. Only tagged cells assemble, so a summary and its demo sit adjacent in the docstring however many definition cells separate them in the notebook. Run every demo in notebook order and see it pass. Run code to check every prose claim too, since old prose can be stale, and a docstring stating something false is worse than one stating nothing.
**Iterate and review.** Preview with `nbdev.export.nb_mdoc(nbname)`, which shows the docstring the notebook would produce, until it reads well top to bottom. Then show the user every new and changed cell verbatim, as markdown and code with their directives. Cell ids and diffs are not reviewable. The actual text is. After export, read `doc(module)` the way the eventual reader will.
## Creating the package docs from index.ipynb
The same projection applies at package level. `index.ipynb` already carries the project intro, and tagging markdown there feeds both the package docstring (written into `__init__.py`) and `llms.txt`. The pieces:
- The summary line is `[project].description` from `pyproject.toml`.
- Tag the intro paragraphs `#| export`. Tips lists, curated links, and a short `#| exportd` quick-start demo all project well. Installation instructions and marketing positioning stay untagged.
- The `Modules:` list assembles itself: every module whose docstring now says more than the default summary-plus-link is listed with its summary line. Finishing a module's docs is what lists it, so do the modules first and the package list follows.
- `llms.txt` is the same parts in llms.txt shape, written into `nbs/` on export.
## Ask the user
- Which modules get docs at all, before starting. Internal, vendored, and niche modules can stay bare, which also keeps them out of the package listing. Leaving a module bare is itself a curation decision.
- The cut for each module, before writing: propose a summary paragraph and demo for the defining idioms, a sentence naming the supporting functions, and nothing for the rest, then wait for a steer.
- Whether existing page prose may be rewritten in place, since that changes the docs site too.
- Extract or front block. Present the choice with your read of how well the existing prose extracts, and let the user pick.
- Exact-output assertions in demos, when an exact string arguably is the contract, such as a rendered template or a signature.
- Anything discovered along the way. Report and propose stale claims in old prose, candidate deprecations, and missing features, and let the user decide. API changes are never part of a docs pass.
## Worked example: fastcore.script
`fastcore/nbs/06_script.ipynb` opens with a bare title cell, then a tagged overview whose first line becomes the module summary:
# Script - CLI
[cell] #| export
Creates a CLI from a Python function decorated with `call_parse`.
The function's parameters become the script's arguments, its docstring becomes the program description, and its [docments](https://fastcore.fast.ai/docments.html) comments become the help for each argument.
Deeper in the notebook, after `is_cli` is defined, an existing summary and demo pair was tagged where it stood:
[cell] #| export
A function usable both from the command line and from Python can call `is_cli` to tell which way it was invoked, e.g. returning a value to Python callers but printing it (or exiting with an error code) when run as a CLI:
[cell] #| exportd
@call_parse
def sum_args(a:int=0, b:int=0):
"Add `a` and `b`"
if is_cli(): print(a+b)
else: return a+b
test_eq(sum_args(1,2), 3) # Python call: returns the value, prints nothing
In the assembled docstring the pair renders as the summary followed by a fenced python block, and the package artifacts pick the module up automatically. Its line in fastcore's `llms.txt`:
- [fastcore.script](https://fastcore.fast.ai/script.html.md): Creates a CLI from a Python function decorated with `call_parse`.
Points to notice:
- The docstring is a small fraction of the docs page, yet answers most usage questions. That ratio is the target.
- The overview and the `is_cli` pair were existing cells, tagged in place with light tightening.
- The "Param types" summary in the same notebook was condensed from scattered examples into a new cell, improving the page and the docstring together.
- The demo asserts the behavior it teaches, and its assertion runs in the notebook's test suite from then on."""