docs(build): unified output subtree under docs/build/#4081
Conversation
|
reverting to draft till it builds on sid |
…ionally CI on PR LinuxCNC#4081 failed for two reasons exposed by --enable-build-documentation=pdf: * The asciidoctor PDF rule did not set the fixturedir attribute, so the mb2hal {fixturedir}/drivers/mb2hal_HOWTO.ini include resolved literally in every Master_Documentation*.pdf and asciidoctor reported it missing. Pass fixturedir=$(DOC_OUT_ADOC)/<lang> from the rule, where <lang> is the literal "en" for the English call site and $(firstword $(subst /, ,$*)) for the per-language call site (so the same recipe serves both). * docs/build/html/en/gcode.html only got copied as a side effect of .copy-asciidoc-stamp, which lives under htmldocs. debian/configure sets --enable-build-documentation=pdf for CI, so htmldocs never fires and the en/ copy never lands, breaking dh_installdocs --package= linuxcnc-doc-en. Promote the copy to its own rule, depended on unconditionally by `docs:`.
|
I tested your branch just for the english language.
Another thing we may could do in this rush: there is a folder |
…extractor
The mb2hal.adoc source contains `include::{fixturedir}/drivers/mb2hal_HOWTO.ini[]`,
where `{fixturedir}` is an asciidoctor attribute resolved per-language at
render time. asciideps does literal sed extraction and was emitting the
`{fixturedir}` placeholder as a phantom prereq path; the subsequent
recursive sed on that path could not find the file and produced
sed: can't read .../{fixturedir}/drivers/mb2hal_HOWTO.ini: No such file or directory
four times per build. Make went on (sed in a pipe does not propagate
non-zero exit), so the build still succeeded, but the noise was
confusing.
Wire FIXTUREDIR via env from the depends/%.d rules (one per language)
and let asciideps sed-substitute it in extracted include paths. Mirrors
the attribute asciidoctor itself receives in the HTML and PDF rules.
Reported by @hansu on PR LinuxCNC#4081.
Per @hansu's review of PR LinuxCNC#4081. Moving PDFs out from under the html subtree keeps the html zip free of PDF bloat and matches the rest of the docs/build/ layout convention (one top-level dir per format). `PDF_TARGETS_*` rules now build into $(DOC_OUT_PDF)/<lang>/, where $(DOC_OUT_PDF) := $(DOC_BUILD)/pdf. Per-page hardlink rules and the Manual_Pages.pdf rule follow. The pdf-index helper now lands docs/build/pdf/index.html (was docs/build/html/pdf-index.html), enumerated from the new tree with relative `<lang>/X.pdf` links. install-doc-pdf still flattens $(PDF_TARGETS) into the docsdir, so the debian package layout (single flat usr/share/doc/linuxcnc/X.pdf per .docs file) is unchanged.
Per @hansu's review of PR LinuxCNC#4081. gen_complist.py runs early (its output is a po4a master), well before the HTML manpage targets are rendered. Its broken-link check against $(DOC_OUT_HTML)/en/hal/<link> always failed at that stage, emitting gen_complist: Broken link: ../man/man1/axis.1.html once per linked component (~100 warnings per build) even though the links resolved correctly in the final tree. Match the link href against the `manN/<name>` capture and check membership in the man_files set (built from listdir of ../docs/man/man{1,9}). man_files is populated before gen_complist runs from its $(MAN_SRCS) prereq, so the check now reflects whether a manpage source actually exists. Real broken-link validation between rendered HTML pages is checkref's job; this script only needs to know whether a target manpage exists in the source set.
|
Just discovered: Running make docs the second time invokes Anyway, this script needs to be changed when the reorganisation moves stuff to other places. |
Aligns with the rest of the documentation source tree (docs/src/<topic>/ for everything po4a-translated and consumed by the docs build). Refs adjusted: - docs/po4a.cfg masters: help/rtfaults.adoc -> src/help/rtfaults.adoc (same for tklinuxcnc.adoc); translated outputs still land at docs/build/adoc/<lang>/help/. - src/Makefile install-docs DOCS_HELP glob. Debian package layout unchanged (install-docs flattens the .adoc into usr/share/doc/linuxcnc/). Suggested by @hansu on PR LinuxCNC#4081.
pdfdocs was phony with `../scripts/make-docs-pdf-index` in its recipe, so the script re-ran on every invocation and rewrote $(DOC_OUT_PDF)/index.html even when no PDF input changed. Make the pdf-index a real file target gated on $(PDF_TARGETS); the script fires only when a PDF changes or the script itself changes. Second `make pdfdocs` is now a no-op. Reported by @BsAtHome on PR LinuxCNC#4081.
|
Build is stable now. Next up is to review it all... |
Per @BsAtHome's review on PR LinuxCNC#4086. The old COMP_MANPAGES / COMP_DRIVER_MANPAGES rules ran halcompile in --document mode (troff straight out) and sed-edited the troff to escape .als / .URL groff directives the old dblatex-based PDF pipeline could not parse. Asciidoctor now reads the component adoc directly, so the sed hacks are obsolete. New chain: - halcompile --adoc -o objects/man/man9/<comp>.9.adoc <comp>.comp - asciidoctor --doctype=manpage --backend=manpage -> ../docs/man/man9/<comp>.9 The adoc extraction only needs Python + halcompile.py (yapps-generated, not the C build), so a fresh-tree docs-only build (make htmldocs / pdfdocs / docs) extracts and renders component manpages without first compiling the rest of LinuxCNC. Previously a docs-only build silently omitted them. Also adds homecomp.9 to COMP_MANPAGES alongside tpcomp.9 (both .comp files are filtered out of COMPS because they do not compile to .o modules, but their manpages are still worth shipping). Surfaces in @hansu's review of PR LinuxCNC#4081 as a 'gen_complist: Broken link: ../man/man9/homecomp.9.html' warning. .9.adoc suffix matches the convention the docs HTML manpage rule already expects under objects/man/. Tested: clean-tree make docs (English-only) builds 207 troff + 154 extracted component adocs, link checker clean. Standalone make ../docs/man/man9/abs.9 from a tree with no halcompile binary rebuilds halcompile and produces the manpage without touching C code.
|
I think there is some whitespace problem here. Seems to happen for each language: |
|
There are PDF build artifacts left in the docs/src directory. The languages at the top of the master index.html does not match the list made in the po4a.cfg. The list must be generated but the The PDF tree is outside the HTML tree. That means we cannot link the PDFs in one tree. The index.html generated needs to be rewritten. We need one index.html per language and it must be linked in the master HTML index to find the PDFs. A lot of the .gitignore files in the docs tree have become obsolete and should be removed entirely. The master .gitignore needs to be cleaned up too. I suggest that the ignores for the docs are actually in The man page tree still contains the cat[139] parts. I think we can get rid of the (old) catalog stuff. We only really need the man-page tree. That would also clean up some stuff in the makefiles again. FWIW, I don't really understand why Another issue seen is: This only seems to happen for the Ukranian (uk) translation. There must be something in there that trips it and it is a consistent error. (and probably more...) |
|
Thanks for the thorough review. Most of it I can just fix, one item needs a decision from you first. PDF tree location: On the master index language list not matching po4a.cfg: that one is handled in the topbar/lang-switcher PR, which drops the hardcoded |
For consistency, all PDFs should IMO be called I think the target directory
Check, great! |
da91abf expanded {fixturedir} but left `$DIR/` in the sed pattern, so an absolute expansion got doubled (`$DIR//abs/.../mb2hal_HOWTO.ini`), firing once per language. Reported by @BsAtHome on PR LinuxCNC#4081. Capture the bare target, expand attrs, then prepend DIR only when the result is relative. For includes this runs in the command substitution, before the loop, since DIR is global and the recursive call clobbers it.
|
Pushed a batch addressing the review. PDF layout (your call): PDFs now build into Also in this push: PDF artifacts out of Deferred to the topbar PR: the lang-list/ |
|
The real problem here is that you updated the .pot/.po outside weblate. That will give serious trouble. The way is to change the source file, then goto weblate and fix all the translations. Then, probably @smoe, should perform a pot/po update to make things consistent. It sucks this way, but otherwise we'll have a merge conflict on translations. Maybe @smoe can chime in here on the preferred sequence? |
|
You have not merged anything against master yet Weblate, right? Weblate is not in a good state at the moment. I suggest I bring this back to normal and independently you remove anything on the .pot/.po from your branch. My biggest concern is that we lose externally contributed translations. Everything else is fine with me. |
| # Generated component manpage list (gen_complist.py). | ||
| /src/hal/components_gen.adoc | ||
|
|
There was a problem hiding this comment.
Wouldn't it be better to run po4a on the build/adoc/en tree and not have to generate the components_get.adoc in the src tree?
There was a problem hiding this comment.
Agreed it's cleaner. The catch: components.adoc includes components_gen.adoc with a plain relative include, so they must sit in the same dir. To generate it into build/adoc/en/hal/, English has to render from build/adoc/en/ like the translations do, otherwise the src-tree components.adoc can't resolve the include without a {fixturedir}-style hack.
That's a deeper change (the en pipeline is special-cased throughout the Submakefile), so I'd do it as its own follow-up. Two ways to populate build/adoc/en/:
- Add
ento po4a as an identity passthrough (emptyen.po,--keep 0). Reuses po4a's tree generation, but touchespo/. - Copy the English
.adoc+ images intobuild/adoc/en/and point the render rules there. Nopo/touch.
Which do you prefer? The same move then lets the .dot SVGs generate into build/ too.
There was a problem hiding this comment.
grandixximo/linuxcnc@docs/unified-build-output...docs/render-en-from-build
Went for no po/ touching
Expand scope here? Wait for follow up PR?
No, and if it is up to me we shouldn't touch the pot/po in normal PRs at all. And in this case here it is actually not necessary.
My PR comments already suggest to remove all these pot/po changes so we keep the status quo. |
|
Hmm I get this, but doesn't seem to be a problem on the CI: |
We can move the |
|
Thanks, this is all addressed or routed.
On "keep
|
Emit all generated docs (HTML, PDF, adoc, man) under docs/build/ instead of
scattering them across docs/html, docs/src and docs/man; source trees stay
clean. PDFs land under build/html/<lang>/pdf/, each suffixed _<lang>.
Also: consolidate the per-directory .gitignores into docs/.gitignore;
inline mb2hal_HOWTO.ini into docs/src/drivers/ (drops the {fixturedir} hack,
no pot/po churn); gen_complist checks the manpage source set, not rendered
HTML; make pdfdocs idempotent; fix asciideps on absolute include paths.
Align with the rest of the doc source tree. Adjusts the po4a.cfg masters and the src/Makefile DOCS_HELP glob. Suggested by @hansu on PR LinuxCNC#4081.
Takes the cat[139]/index.db cruft and whatis-parse noise with it; the whatis database is a packaging concern (dpkg), not the doc build.
grandixximo
force-pushed
the
docs/unified-build-output
branch
from
e4b2922 to
44dcf44
Compare
4 participants
Summary
Move all rendered docs artefacts (HTML, PDF, po4a-translated
.adoc) under a singledocs/build/tree, drop asciidoc-py-era leftovers indocs/html/, and stop leaking build-time artefacts intodocs/src/.Builds on top of #4053 (asciidoctor migration). Implements Bertho's layout proposal agreed in the #4053 discussion.
Layout
English now lives under
en/like every other language, so the top-level redirect lands users on the right tree.Manpage troff stays at
docs/man/because halcompile output,install-man,mandb, andrip-environmentread it.Source-tree cleanup
Three build-time artefacts no longer leak into
docs/src/:docs/src/man/man1/linuxcnc.1.adoc(config.status output)docs/build/adoc/en/man/man1/linuxcnc.1.adocdocs/src/drivers/mb2hal_HOWTO.ini(include fixture)docs/build/adoc/en/drivers/mb2hal_HOWTO.inidocs/src/gui/gmoccapy_release_notes.txtmb2hal.adoc'sinclude::now uses the{fixturedir}attribute, set per-language by the asciidoctor HTML rule. The 8.pofiles are repointed at the new include path (mechanical msgid edit, no po4a regen).The manpage HTML rule grows a build-tree fallback so the English manpage build finds
linuxcnc.1.adoc; theLinuxCNC_Manual_Pages.pdfmaster generator grows the same fallback. An explicit troff rule forlinuxcnc.1replaces the generic pattern (which expects sources underdocs/src/man/).Static assets in the former output dir
docs/html/move todocs/src/(gcode.html,index.css, logo). asciidoc-py-era files (asciidoc.css,linuxcnc.css,minus.png,plus.png) are deleted.debian/*.docsanddocs/po4a.cfgare repointed at the new tree. Install destinations underusr/share/doc/are unchanged.Test plan
cd src && make -j$(nproc)clean full buildmb2hal.htmlcontains the example INI fixture in all 8 languagesLinuxCNC_Documentation.pdfcontains the mb2hal INI fixturelinuxcnc(1)troff renders todocs/man/man1/linuxcnc.1linuxcnc(1)HTML and PDF (inLinuxCNC_Manual_Pages.pdf) both contain the manpage bodymake doccleanleaves no generated.adoc/.ini/.txtindocs/src/git clean -dxn docs/src/reports onlyhal/components_gen.adoc(pre-existing, generated bygen_complist.py, not from this PR)