mirror of https://github.com/torvalds/linux.git
Merge branch 'build-script' into docs-mw
Quoth Mauro:
This series should probably be called:
"Move the trick-or-treat build hacks accumulated over time
into a single place and document them."
as this reflects its main goal. As such:
- it places the jobserver logic on a library;
- it removes sphinx/parallel-wrapper.sh;
- the code now properly implements a jobserver-aware logic
to do the parallelism when called via GNU make, failing back to
"-j" when there's no jobserver;
- converts check-variable-fonts.sh to Python and uses it via
function call;
- drops an extra script to generate man pages, adding a makefile
target for it;
- ensures that return code is 0 when PDF successfully builds;
- about half of the script is comments and documentation.
I tried to do my best to document all tricks that are inside the
script. This way, the docs build steps is now documented.
It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.
Such script was written in a way that it can be called either
directly or via a Makefile. Running outside Makefile is
interesting specially when debug is needed. The command line
interface replaces the need of having lots of env vars before
calling sphinx-build:
$ ./tools/docs/sphinx-build-wrapper --help
usage: sphinx-build-wrapper [-h]
[--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
[--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
[-j JOBS] [-i] [-V [VENV]]
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Kernel documentation builder
positional arguments:
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Documentation target to build
options:
-h, --help show this help message and exit
--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
Specific directories to build
--conf CONF Sphinx configuration file
--builddir BUILDDIR Sphinx configuration file
--theme THEME Sphinx theme to use
--css CSS Custom CSS file for HTML/EPUB
--paper {,a4,letter} Paper size for LaTeX/PDF output
-v, --verbose place build in verbose mode
-j, --jobs JOBS Sets number of jobs to use with sphinx-build
-i, --interactive Change latex default to run in interactive mode
-V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)
the only mandatory argument is the target, which is identical with
"make" targets.
The call inside Makefile doesn't use the last four arguments. They're
there to help identifying problems at the build:
-v makes the output verbose;
-j helps to test parallelism;
-i runs latexmk in interactive mode, allowing to debug PDF
build issues;
-V is useful when testing it with different venvs.
When used with GNU make (or some other make which implements jobserver),
a call like:
make -j <targets> htmldocs
will make the wrapper to automatically use POSIX jobserver to claim
the number of available job slots, calling sphinx-build with a
"-j" parameter reflecting it. ON such case, the default can be
overriden via SPHINXDIRS argument.
Visiable changes when compared with the old behavior:
When V=0, the only visible difference is that:
- pdfdocs target now returns 0 on success, 1 on failures.
This addresses an issue over the current process where we
it always return success even on failures;
- it will now print the name of PDF files that failed to build,
if any.
In verbose mode, sphinx-build-wrapper and sphinx-build command lines
are now displayed.
This commit is contained in:
Jonathan Corbet
commit
3df5affb4b
|
|
@ -23,164 +23,76 @@ SPHINXOPTS =
|
||||||
SPHINXDIRS = .
|
SPHINXDIRS = .
|
||||||
DOCS_THEME =
|
DOCS_THEME =
|
||||||
DOCS_CSS =
|
DOCS_CSS =
|
||||||
_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
|
RUSTDOC =
|
||||||
SPHINX_CONF = conf.py
|
|
||||||
PAPER =
|
PAPER =
|
||||||
BUILDDIR = $(obj)/output
|
BUILDDIR = $(obj)/output
|
||||||
PDFLATEX = xelatex
|
PDFLATEX = xelatex
|
||||||
LATEXOPTS = -interaction=batchmode -no-shell-escape
|
LATEXOPTS = -interaction=batchmode -no-shell-escape
|
||||||
|
|
||||||
|
PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__)
|
||||||
|
|
||||||
|
# Wrapper for sphinx-build
|
||||||
|
|
||||||
|
BUILD_WRAPPER = $(srctree)/tools/docs/sphinx-build-wrapper
|
||||||
|
|
||||||
# For denylisting "variable font" files
|
# For denylisting "variable font" files
|
||||||
# Can be overridden by setting as an env variable
|
# Can be overridden by setting as an env variable
|
||||||
FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf
|
FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf
|
||||||
|
|
||||||
ifeq ($(findstring 1, $(KBUILD_VERBOSE)),)
|
|
||||||
SPHINXOPTS += "-q"
|
|
||||||
endif
|
|
||||||
|
|
||||||
# User-friendly check for sphinx-build
|
# User-friendly check for sphinx-build
|
||||||
HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
||||||
|
|
||||||
|
ifneq ($(wildcard $(srctree)/.config),)
|
||||||
|
ifeq ($(CONFIG_RUST),y)
|
||||||
|
RUSTDOC=--rustdoc
|
||||||
|
endif
|
||||||
|
endif
|
||||||
|
|
||||||
ifeq ($(HAVE_SPHINX),0)
|
ifeq ($(HAVE_SPHINX),0)
|
||||||
|
|
||||||
.DEFAULT:
|
.DEFAULT:
|
||||||
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
|
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
|
||||||
@echo
|
@echo
|
||||||
@$(srctree)/scripts/sphinx-pre-install
|
@$(srctree)/tools/docs/sphinx-pre-install
|
||||||
@echo " SKIP Sphinx $@ target."
|
@echo " SKIP Sphinx $@ target."
|
||||||
|
|
||||||
else # HAVE_SPHINX
|
else # HAVE_SPHINX
|
||||||
|
|
||||||
# User-friendly check for pdflatex and latexmk
|
# Common documentation targets
|
||||||
HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
|
||||||
HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
$(Q)PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \
|
||||||
|
$(srctree)/tools/docs/sphinx-pre-install --version-check
|
||||||
|
+$(Q)PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \
|
||||||
|
$(PYTHON3) $(BUILD_WRAPPER) $@ \
|
||||||
|
--sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
|
||||||
|
--builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
|
||||||
|
--theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
|
||||||
|
|
||||||
ifeq ($(HAVE_LATEXMK),1)
|
|
||||||
PDFLATEX := latexmk -$(PDFLATEX)
|
|
||||||
endif #HAVE_LATEXMK
|
|
||||||
|
|
||||||
# Internal variables.
|
|
||||||
PAPEROPT_a4 = -D latex_elements.papersize=a4paper
|
|
||||||
PAPEROPT_letter = -D latex_elements.papersize=letterpaper
|
|
||||||
ALLSPHINXOPTS = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
|
|
||||||
ALLSPHINXOPTS += $(PAPEROPT_$(PAPER)) $(SPHINXOPTS)
|
|
||||||
ifneq ($(wildcard $(srctree)/.config),)
|
|
||||||
ifeq ($(CONFIG_RUST),y)
|
|
||||||
# Let Sphinx know we will include rustdoc
|
|
||||||
ALLSPHINXOPTS += -t rustdoc
|
|
||||||
endif
|
endif
|
||||||
endif
|
|
||||||
# the i18n builder cannot share the environment and doctrees with the others
|
|
||||||
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
|
||||||
|
|
||||||
# commands; the 'cmd' from scripts/Kbuild.include is not *loopable*
|
|
||||||
loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
|
|
||||||
|
|
||||||
# $2 sphinx builder e.g. "html"
|
|
||||||
# $3 name of the build subfolder / e.g. "userspace-api/media", used as:
|
|
||||||
# * dest folder relative to $(BUILDDIR) and
|
|
||||||
# * cache folder relative to $(BUILDDIR)/.doctrees
|
|
||||||
# $4 dest subfolder e.g. "man" for man pages at userspace-api/media/man
|
|
||||||
# $5 reST source folder relative to $(src),
|
|
||||||
# e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media
|
|
||||||
|
|
||||||
PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__)
|
|
||||||
|
|
||||||
quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
|
|
||||||
cmd_sphinx = \
|
|
||||||
PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \
|
|
||||||
BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(src)/$5/$(SPHINX_CONF)) \
|
|
||||||
$(PYTHON3) $(srctree)/scripts/jobserver-exec \
|
|
||||||
$(CONFIG_SHELL) $(srctree)/Documentation/sphinx/parallel-wrapper.sh \
|
|
||||||
$(SPHINXBUILD) \
|
|
||||||
-b $2 \
|
|
||||||
-c $(abspath $(src)) \
|
|
||||||
-d $(abspath $(BUILDDIR)/.doctrees/$3) \
|
|
||||||
-D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) \
|
|
||||||
$(ALLSPHINXOPTS) \
|
|
||||||
$(abspath $(src)/$5) \
|
|
||||||
$(abspath $(BUILDDIR)/$3/$4) && \
|
|
||||||
if [ "x$(DOCS_CSS)" != "x" ]; then \
|
|
||||||
cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_static/; \
|
|
||||||
fi
|
|
||||||
|
|
||||||
htmldocs:
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
|
|
||||||
|
|
||||||
htmldocs-redirects: $(srctree)/Documentation/.renames.txt
|
|
||||||
@tools/docs/gen-redirects.py --output $(BUILDDIR) < $<
|
|
||||||
|
|
||||||
# If Rust support is available and .config exists, add rustdoc generated contents.
|
|
||||||
# If there are any, the errors from this make rustdoc will be displayed but
|
|
||||||
# won't stop the execution of htmldocs
|
|
||||||
|
|
||||||
ifneq ($(wildcard $(srctree)/.config),)
|
|
||||||
ifeq ($(CONFIG_RUST),y)
|
|
||||||
$(Q)$(MAKE) rustdoc || true
|
|
||||||
endif
|
|
||||||
endif
|
|
||||||
|
|
||||||
texinfodocs:
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
|
|
||||||
|
|
||||||
# Note: the 'info' Make target is generated by sphinx itself when
|
|
||||||
# running the texinfodocs target define above.
|
|
||||||
infodocs: texinfodocs
|
|
||||||
$(MAKE) -C $(BUILDDIR)/texinfo info
|
|
||||||
|
|
||||||
linkcheckdocs:
|
|
||||||
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
|
|
||||||
|
|
||||||
latexdocs:
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
|
|
||||||
|
|
||||||
ifeq ($(HAVE_PDFLATEX),0)
|
|
||||||
|
|
||||||
pdfdocs:
|
|
||||||
$(warning The '$(PDFLATEX)' command was not found. Make sure you have it installed and in PATH to produce PDF output.)
|
|
||||||
@echo " SKIP Sphinx $@ target."
|
|
||||||
|
|
||||||
else # HAVE_PDFLATEX
|
|
||||||
|
|
||||||
pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
|
|
||||||
pdfdocs: latexdocs
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
$(foreach var,$(SPHINXDIRS), \
|
|
||||||
$(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
|
|
||||||
mkdir -p $(BUILDDIR)/$(var)/pdf; \
|
|
||||||
mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \
|
|
||||||
)
|
|
||||||
|
|
||||||
endif # HAVE_PDFLATEX
|
|
||||||
|
|
||||||
epubdocs:
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
|
|
||||||
|
|
||||||
xmldocs:
|
|
||||||
@$(srctree)/scripts/sphinx-pre-install --version-check
|
|
||||||
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
|
|
||||||
|
|
||||||
endif # HAVE_SPHINX
|
|
||||||
|
|
||||||
# The following targets are independent of HAVE_SPHINX, and the rules should
|
# The following targets are independent of HAVE_SPHINX, and the rules should
|
||||||
# work or silently pass without Sphinx.
|
# work or silently pass without Sphinx.
|
||||||
|
|
||||||
|
htmldocs-redirects: $(srctree)/Documentation/.renames.txt
|
||||||
|
@tools/docs/gen-redirects.py --output $(BUILDDIR) < $<
|
||||||
|
|
||||||
refcheckdocs:
|
refcheckdocs:
|
||||||
$(Q)cd $(srctree);scripts/documentation-file-ref-check
|
$(Q)cd $(srctree);scripts/documentation-file-ref-check
|
||||||
|
|
||||||
cleandocs:
|
cleandocs:
|
||||||
$(Q)rm -rf $(BUILDDIR)
|
$(Q)rm -rf $(BUILDDIR)
|
||||||
|
|
||||||
|
# Used only on help
|
||||||
|
_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
|
||||||
|
|
||||||
dochelp:
|
dochelp:
|
||||||
@echo ' Linux kernel internal documentation in different formats from ReST:'
|
@echo ' Linux kernel internal documentation in different formats from ReST:'
|
||||||
@echo ' htmldocs - HTML'
|
@echo ' htmldocs - HTML'
|
||||||
@echo ' htmldocs-redirects - generate HTML redirects for moved pages'
|
@echo ' htmldocs-redirects - generate HTML redirects for moved pages'
|
||||||
@echo ' texinfodocs - Texinfo'
|
@echo ' texinfodocs - Texinfo'
|
||||||
@echo ' infodocs - Info'
|
@echo ' infodocs - Info'
|
||||||
|
@echo ' mandocs - Man pages'
|
||||||
@echo ' latexdocs - LaTeX'
|
@echo ' latexdocs - LaTeX'
|
||||||
@echo ' pdfdocs - PDF'
|
@echo ' pdfdocs - PDF'
|
||||||
@echo ' epubdocs - EPUB'
|
@echo ' epubdocs - EPUB'
|
||||||
|
|
@ -194,11 +106,13 @@ dochelp:
|
||||||
@echo ' make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2'
|
@echo ' make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2'
|
||||||
@echo ' valid values for SPHINXDIRS are: $(_SPHINXDIRS)'
|
@echo ' valid values for SPHINXDIRS are: $(_SPHINXDIRS)'
|
||||||
@echo
|
@echo
|
||||||
@echo ' make SPHINX_CONF={conf-file} [target] use *additional* sphinx-build'
|
|
||||||
@echo ' configuration. This is e.g. useful to build with nit-picking config.'
|
|
||||||
@echo
|
|
||||||
@echo ' make DOCS_THEME={sphinx-theme} selects a different Sphinx theme.'
|
@echo ' make DOCS_THEME={sphinx-theme} selects a different Sphinx theme.'
|
||||||
@echo
|
@echo
|
||||||
@echo ' make DOCS_CSS={a .css file} adds a DOCS_CSS override file for html/epub output.'
|
@echo ' make DOCS_CSS={a .css file} adds a DOCS_CSS override file for html/epub output.'
|
||||||
@echo
|
@echo
|
||||||
|
@echo ' make PAPER={a4|letter} Specifies the paper size used for LaTeX/PDF output.'
|
||||||
|
@echo
|
||||||
|
@echo ' make FONTS_CONF_DENY_VF={path} sets a deny list to block variable Noto CJK fonts'
|
||||||
|
@echo ' for PDF build. See tools/docs/lib/latex_fonts.py for more details'
|
||||||
|
@echo
|
||||||
@echo ' Default location for the generated documents is Documentation/output'
|
@echo ' Default location for the generated documents is Documentation/output'
|
||||||
|
|
|
||||||
|
|
@ -18,8 +18,6 @@ import sphinx
|
||||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||||
sys.path.insert(0, os.path.abspath("sphinx"))
|
sys.path.insert(0, os.path.abspath("sphinx"))
|
||||||
|
|
||||||
from load_config import loadConfig # pylint: disable=C0413,E0401
|
|
||||||
|
|
||||||
# Minimal supported version
|
# Minimal supported version
|
||||||
needs_sphinx = "3.4.3"
|
needs_sphinx = "3.4.3"
|
||||||
|
|
||||||
|
|
@ -93,8 +91,12 @@ def config_init(app, config):
|
||||||
# LaTeX and PDF output require a list of documents with are dependent
|
# LaTeX and PDF output require a list of documents with are dependent
|
||||||
# of the app.srcdir. Add them here
|
# of the app.srcdir. Add them here
|
||||||
|
|
||||||
# When SPHINXDIRS is used, we just need to get index.rst, if it exists
|
# Handle the case where SPHINXDIRS is used
|
||||||
if not os.path.samefile(doctree, app.srcdir):
|
if not os.path.samefile(doctree, app.srcdir):
|
||||||
|
# Add a tag to mark that the build is actually a subproject
|
||||||
|
tags.add("subproject")
|
||||||
|
|
||||||
|
# get index.rst, if it exists
|
||||||
doc = os.path.basename(app.srcdir)
|
doc = os.path.basename(app.srcdir)
|
||||||
fname = "index"
|
fname = "index"
|
||||||
if os.path.exists(os.path.join(app.srcdir, fname + ".rst")):
|
if os.path.exists(os.path.join(app.srcdir, fname + ".rst")):
|
||||||
|
|
@ -583,13 +585,6 @@ pdf_documents = [
|
||||||
kerneldoc_bin = "../scripts/kernel-doc.py"
|
kerneldoc_bin = "../scripts/kernel-doc.py"
|
||||||
kerneldoc_srctree = ".."
|
kerneldoc_srctree = ".."
|
||||||
|
|
||||||
# ------------------------------------------------------------------------------
|
|
||||||
# Since loadConfig overwrites settings from the global namespace, it has to be
|
|
||||||
# the last statement in the conf.py file
|
|
||||||
# ------------------------------------------------------------------------------
|
|
||||||
loadConfig(globals())
|
|
||||||
|
|
||||||
|
|
||||||
def setup(app):
|
def setup(app):
|
||||||
"""Patterns need to be updated at init time on older Sphinx versions"""
|
"""Patterns need to be updated at init time on older Sphinx versions"""
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -579,20 +579,23 @@ source.
|
||||||
How to use kernel-doc to generate man pages
|
How to use kernel-doc to generate man pages
|
||||||
-------------------------------------------
|
-------------------------------------------
|
||||||
|
|
||||||
If you just want to use kernel-doc to generate man pages you can do this
|
To generate man pages for all files that contain kernel-doc markups, run::
|
||||||
from the kernel git tree::
|
|
||||||
|
|
||||||
$ scripts/kernel-doc -man \
|
$ make mandocs
|
||||||
$(git grep -l '/\*\*' -- :^Documentation :^tools) \
|
|
||||||
| scripts/split-man.pl /tmp/man
|
|
||||||
|
|
||||||
Some older versions of git do not support some of the variants of syntax for
|
Or calling ``script-build-wrapper`` directly::
|
||||||
path exclusion. One of the following commands may work for those versions::
|
|
||||||
|
|
||||||
$ scripts/kernel-doc -man \
|
$ ./tools/docs/sphinx-build-wrapper mandocs
|
||||||
$(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \
|
|
||||||
| scripts/split-man.pl /tmp/man
|
|
||||||
|
|
||||||
$ scripts/kernel-doc -man \
|
The output will be at ``/man`` directory inside the output directory
|
||||||
$(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \
|
(by default: ``Documentation/output``).
|
||||||
| scripts/split-man.pl /tmp/man
|
|
||||||
|
Optionally, it is possible to generate a partial set of man pages by
|
||||||
|
using SPHINXDIRS:
|
||||||
|
|
||||||
|
$ make SPHINXDIRS=driver-api/media mandocs
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
When SPHINXDIRS={subdir} is used, it will only generate man pages for
|
||||||
|
the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file.
|
||||||
|
|
|
||||||
|
|
@ -106,7 +106,7 @@ There's a script that automatically checks for Sphinx dependencies. If it can
|
||||||
recognize your distribution, it will also give a hint about the install
|
recognize your distribution, it will also give a hint about the install
|
||||||
command line options for your distro::
|
command line options for your distro::
|
||||||
|
|
||||||
$ ./scripts/sphinx-pre-install
|
$ ./tools/docs/sphinx-pre-install
|
||||||
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
||||||
Warning: better to also install "texlive-luatex85".
|
Warning: better to also install "texlive-luatex85".
|
||||||
You should run:
|
You should run:
|
||||||
|
|
@ -116,7 +116,7 @@ command line options for your distro::
|
||||||
. sphinx_2.4.4/bin/activate
|
. sphinx_2.4.4/bin/activate
|
||||||
pip install -r Documentation/sphinx/requirements.txt
|
pip install -r Documentation/sphinx/requirements.txt
|
||||||
|
|
||||||
Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
|
Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
|
||||||
|
|
||||||
By default, it checks all the requirements for both html and PDF, including
|
By default, it checks all the requirements for both html and PDF, including
|
||||||
the requirements for images, math expressions and LaTeX build, and assumes
|
the requirements for images, math expressions and LaTeX build, and assumes
|
||||||
|
|
|
||||||
|
|
@ -220,7 +220,7 @@
|
||||||
If you want them, please install non-variable ``Noto Sans CJK''
|
If you want them, please install non-variable ``Noto Sans CJK''
|
||||||
font families along with the texlive-xecjk package by following
|
font families along with the texlive-xecjk package by following
|
||||||
instructions from
|
instructions from
|
||||||
\sphinxcode{./scripts/sphinx-pre-install}.
|
\sphinxcode{./tools/docs/sphinx-pre-install}.
|
||||||
Having optional non-variable ``Noto Serif CJK'' font families will
|
Having optional non-variable ``Noto Serif CJK'' font families will
|
||||||
improve the looks of those translations.
|
improve the looks of those translations.
|
||||||
\end{sphinxadmonition}}
|
\end{sphinxadmonition}}
|
||||||
|
|
|
||||||
|
|
@ -1,60 +0,0 @@
|
||||||
# -*- coding: utf-8; mode: python -*-
|
|
||||||
# SPDX-License-Identifier: GPL-2.0
|
|
||||||
# pylint: disable=R0903, C0330, R0914, R0912, E0401
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from sphinx.util.osutil import fs_encoding
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------------------
|
|
||||||
def loadConfig(namespace):
|
|
||||||
# ------------------------------------------------------------------------------
|
|
||||||
|
|
||||||
"""Load an additional configuration file into *namespace*.
|
|
||||||
|
|
||||||
The name of the configuration file is taken from the environment
|
|
||||||
``SPHINX_CONF``. The external configuration file extends (or overwrites) the
|
|
||||||
configuration values from the origin ``conf.py``. With this you are able to
|
|
||||||
maintain *build themes*. """
|
|
||||||
|
|
||||||
config_file = os.environ.get("SPHINX_CONF", None)
|
|
||||||
if (config_file is not None
|
|
||||||
and os.path.normpath(namespace["__file__"]) != os.path.normpath(config_file) ):
|
|
||||||
config_file = os.path.abspath(config_file)
|
|
||||||
|
|
||||||
# Let's avoid one conf.py file just due to latex_documents
|
|
||||||
start = config_file.find('Documentation/')
|
|
||||||
if start >= 0:
|
|
||||||
start = config_file.find('/', start + 1)
|
|
||||||
|
|
||||||
end = config_file.rfind('/')
|
|
||||||
if start >= 0 and end > 0:
|
|
||||||
dir = config_file[start + 1:end]
|
|
||||||
|
|
||||||
print("source directory: %s" % dir)
|
|
||||||
new_latex_docs = []
|
|
||||||
latex_documents = namespace['latex_documents']
|
|
||||||
|
|
||||||
for l in latex_documents:
|
|
||||||
if l[0].find(dir + '/') == 0:
|
|
||||||
has = True
|
|
||||||
fn = l[0][len(dir) + 1:]
|
|
||||||
new_latex_docs.append((fn, l[1], l[2], l[3], l[4]))
|
|
||||||
break
|
|
||||||
|
|
||||||
namespace['latex_documents'] = new_latex_docs
|
|
||||||
|
|
||||||
# If there is an extra conf.py file, load it
|
|
||||||
if os.path.isfile(config_file):
|
|
||||||
sys.stdout.write("load additional sphinx-config: %s\n" % config_file)
|
|
||||||
config = namespace.copy()
|
|
||||||
config['__file__'] = config_file
|
|
||||||
with open(config_file, 'rb') as f:
|
|
||||||
code = compile(f.read(), fs_encoding, 'exec')
|
|
||||||
exec(code, config)
|
|
||||||
del config['__file__']
|
|
||||||
namespace.update(config)
|
|
||||||
else:
|
|
||||||
config = namespace.copy()
|
|
||||||
config['tags'].add("subproject")
|
|
||||||
namespace.update(config)
|
|
||||||
|
|
@ -1,33 +0,0 @@
|
||||||
#!/bin/sh
|
|
||||||
# SPDX-License-Identifier: GPL-2.0+
|
|
||||||
#
|
|
||||||
# Figure out if we should follow a specific parallelism from the make
|
|
||||||
# environment (as exported by scripts/jobserver-exec), or fall back to
|
|
||||||
# the "auto" parallelism when "-jN" is not specified at the top-level
|
|
||||||
# "make" invocation.
|
|
||||||
|
|
||||||
sphinx="$1"
|
|
||||||
shift || true
|
|
||||||
|
|
||||||
parallel="$PARALLELISM"
|
|
||||||
if [ -z "$parallel" ] ; then
|
|
||||||
# If no parallelism is specified at the top-level make, then
|
|
||||||
# fall back to the expected "-jauto" mode that the "htmldocs"
|
|
||||||
# target has had.
|
|
||||||
auto=$(perl -e 'open IN,"'"$sphinx"' --version 2>&1 |";
|
|
||||||
while (<IN>) {
|
|
||||||
if (m/([\d\.]+)/) {
|
|
||||||
print "auto" if ($1 >= "1.7")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
close IN')
|
|
||||||
if [ -n "$auto" ] ; then
|
|
||||||
parallel="$auto"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
# Only if some parallelism has been determined do we add the -jN option.
|
|
||||||
if [ -n "$parallel" ] ; then
|
|
||||||
parallel="-j$parallel"
|
|
||||||
fi
|
|
||||||
|
|
||||||
exec "$sphinx" $parallel "$@"
|
|
||||||
|
|
@ -109,7 +109,7 @@ Sphinx. Se lo script riesce a riconoscere la vostra distribuzione, allora
|
||||||
sarà in grado di darvi dei suggerimenti su come procedere per completare
|
sarà in grado di darvi dei suggerimenti su come procedere per completare
|
||||||
l'installazione::
|
l'installazione::
|
||||||
|
|
||||||
$ ./scripts/sphinx-pre-install
|
$ ./tools/docs/sphinx-pre-install
|
||||||
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
||||||
Warning: better to also install "texlive-luatex85".
|
Warning: better to also install "texlive-luatex85".
|
||||||
You should run:
|
You should run:
|
||||||
|
|
@ -119,7 +119,7 @@ l'installazione::
|
||||||
. sphinx_2.4.4/bin/activate
|
. sphinx_2.4.4/bin/activate
|
||||||
pip install -r Documentation/sphinx/requirements.txt
|
pip install -r Documentation/sphinx/requirements.txt
|
||||||
|
|
||||||
Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
|
Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
|
||||||
|
|
||||||
L'impostazione predefinita prevede il controllo dei requisiti per la generazione
|
L'impostazione predefinita prevede il controllo dei requisiti per la generazione
|
||||||
di documenti html e PDF, includendo anche il supporto per le immagini, le
|
di documenti html e PDF, includendo anche il supporto per le immagini, le
|
||||||
|
|
|
||||||
|
|
@ -84,7 +84,7 @@ PDF和LaTeX构建
|
||||||
这有一个脚本可以自动检查Sphinx依赖项。如果它认得您的发行版，还会提示您所用发行
|
这有一个脚本可以自动检查Sphinx依赖项。如果它认得您的发行版，还会提示您所用发行
|
||||||
版的安装命令::
|
版的安装命令::
|
||||||
|
|
||||||
$ ./scripts/sphinx-pre-install
|
$ ./tools/docs/sphinx-pre-install
|
||||||
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
|
||||||
Warning: better to also install "texlive-luatex85".
|
Warning: better to also install "texlive-luatex85".
|
||||||
You should run:
|
You should run:
|
||||||
|
|
@ -94,7 +94,7 @@ PDF和LaTeX构建
|
||||||
. sphinx_2.4.4/bin/activate
|
. sphinx_2.4.4/bin/activate
|
||||||
pip install -r Documentation/sphinx/requirements.txt
|
pip install -r Documentation/sphinx/requirements.txt
|
||||||
|
|
||||||
Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
|
Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
|
||||||
|
|
||||||
默认情况下，它会检查html和PDF的所有依赖项，包括图像、数学表达式和LaTeX构建的
|
默认情况下，它会检查html和PDF的所有依赖项，包括图像、数学表达式和LaTeX构建的
|
||||||
需求，并假设将使用虚拟Python环境。html构建所需的依赖项被认为是必需的，其他依
|
需求，并假设将使用虚拟Python环境。html构建所需的依赖项被认为是必需的，其他依
|
||||||
|
|
|
||||||
|
|
@ -64,7 +64,7 @@ Linux 发行版和简单地使用 Linux 命令行，那么可以迅速开始了
|
||||||
::
|
::
|
||||||
|
|
||||||
cd linux
|
cd linux
|
||||||
./scripts/sphinx-pre-install
|
./tools/docs/sphinx-pre-install
|
||||||
|
|
||||||
以 Fedora 为例，它的输出是这样的::
|
以 Fedora 为例，它的输出是这样的::
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -7411,7 +7411,6 @@ S: Maintained
|
||||||
P: Documentation/doc-guide/maintainer-profile.rst
|
P: Documentation/doc-guide/maintainer-profile.rst
|
||||||
T: git git://git.lwn.net/linux.git docs-next
|
T: git git://git.lwn.net/linux.git docs-next
|
||||||
F: Documentation/
|
F: Documentation/
|
||||||
F: scripts/check-variable-fonts.sh
|
|
||||||
F: scripts/checktransupdate.py
|
F: scripts/checktransupdate.py
|
||||||
F: scripts/documentation-file-ref-check
|
F: scripts/documentation-file-ref-check
|
||||||
F: scripts/get_abi.py
|
F: scripts/get_abi.py
|
||||||
|
|
@ -7420,7 +7419,6 @@ F: scripts/lib/abi/*
|
||||||
F: scripts/lib/kdoc/*
|
F: scripts/lib/kdoc/*
|
||||||
F: tools/docs/*
|
F: tools/docs/*
|
||||||
F: tools/net/ynl/pyynl/lib/doc_generator.py
|
F: tools/net/ynl/pyynl/lib/doc_generator.py
|
||||||
F: scripts/sphinx-pre-install
|
|
||||||
X: Documentation/ABI/
|
X: Documentation/ABI/
|
||||||
X: Documentation/admin-guide/media/
|
X: Documentation/admin-guide/media/
|
||||||
X: Documentation/devicetree/
|
X: Documentation/devicetree/
|
||||||
|
|
@ -7455,7 +7453,7 @@ L: linux-doc@vger.kernel.org
|
||||||
S: Maintained
|
S: Maintained
|
||||||
F: Documentation/sphinx/parse-headers.pl
|
F: Documentation/sphinx/parse-headers.pl
|
||||||
F: scripts/documentation-file-ref-check
|
F: scripts/documentation-file-ref-check
|
||||||
F: scripts/sphinx-pre-install
|
F: tools/docs/sphinx-pre-install
|
||||||
|
|
||||||
DOCUMENTATION/ITALIAN
|
DOCUMENTATION/ITALIAN
|
||||||
M: Federico Vaga <federico.vaga@vaga.pv.it>
|
M: Federico Vaga <federico.vaga@vaga.pv.it>
|
||||||
|
|
|
||||||
7
Makefile
7
Makefile
|
|
@ -1797,9 +1797,10 @@ $(help-board-dirs): help-%:
|
||||||
|
|
||||||
# Documentation targets
|
# Documentation targets
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs htmldocs-redirects \
|
DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
|
||||||
epubdocs cleandocs linkcheckdocs dochelp refcheckdocs \
|
linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs \
|
||||||
texinfodocs infodocs
|
htmldocs-redirects
|
||||||
|
|
||||||
PHONY += $(DOC_TARGETS)
|
PHONY += $(DOC_TARGETS)
|
||||||
$(DOC_TARGETS):
|
$(DOC_TARGETS):
|
||||||
$(Q)$(MAKE) $(build)=Documentation $@
|
$(Q)$(MAKE) $(build)=Documentation $@
|
||||||
|
|
|
||||||
|
|
@ -1,115 +0,0 @@
|
||||||
#!/bin/sh
|
|
||||||
# SPDX-License-Identifier: GPL-2.0-only
|
|
||||||
# Copyright (C) Akira Yokosawa, 2024
|
|
||||||
#
|
|
||||||
# For "make pdfdocs", reports of build errors of translations.pdf started
|
|
||||||
# arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE
|
|
||||||
# tumbleweed have started deploying variable-font [3] format of "Noto CJK"
|
|
||||||
# fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK
|
|
||||||
# (Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which
|
|
||||||
# does not (and likely never will) understand variable fonts for historical
|
|
||||||
# reasons.
|
|
||||||
#
|
|
||||||
# The build error happens even when both of variable- and non-variable-format
|
|
||||||
# fonts are found on the build system. To make matters worse, Fedora enlists
|
|
||||||
# variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN,
|
|
||||||
# -zh_TW, etc. Hence developers who have interest in CJK pages are more
|
|
||||||
# likely to encounter the build errors.
|
|
||||||
#
|
|
||||||
# This script is invoked from the error path of "make pdfdocs" and emits
|
|
||||||
# suggestions if variable-font files of "Noto CJK" fonts are in the list of
|
|
||||||
# fonts accessible from XeTeX.
|
|
||||||
#
|
|
||||||
# References:
|
|
||||||
# [1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
|
|
||||||
# [2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
|
|
||||||
# [3]: https://en.wikipedia.org/wiki/Variable_font
|
|
||||||
# [4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
|
|
||||||
# [5]: https://build.opensuse.org/request/show/1157217
|
|
||||||
#
|
|
||||||
#===========================================================================
|
|
||||||
# Workarounds for building translations.pdf
|
|
||||||
#===========================================================================
|
|
||||||
#
|
|
||||||
# * Denylist "variable font" Noto CJK fonts.
|
|
||||||
# - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with
|
|
||||||
# tweaks if necessary. Remove leading "# ".
|
|
||||||
# - Path of fontconfig/fonts.conf can be overridden by setting an env
|
|
||||||
# variable FONTS_CONF_DENY_VF.
|
|
||||||
#
|
|
||||||
# * Template:
|
|
||||||
# -----------------------------------------------------------------
|
|
||||||
# <?xml version="1.0"?>
|
|
||||||
# <!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
|
|
||||||
# <fontconfig>
|
|
||||||
# <!--
|
|
||||||
# Ignore variable-font glob (not to break xetex)
|
|
||||||
# -->
|
|
||||||
# <selectfont>
|
|
||||||
# <rejectfont>
|
|
||||||
# <!--
|
|
||||||
# for Fedora
|
|
||||||
# -->
|
|
||||||
# <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
|
|
||||||
# <!--
|
|
||||||
# for openSUSE tumbleweed
|
|
||||||
# -->
|
|
||||||
# <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
|
|
||||||
# </rejectfont>
|
|
||||||
# </selectfont>
|
|
||||||
# </fontconfig>
|
|
||||||
# -----------------------------------------------------------------
|
|
||||||
#
|
|
||||||
# The denylisting is activated for "make pdfdocs".
|
|
||||||
#
|
|
||||||
# * For skipping CJK pages in PDF
|
|
||||||
# - Uninstall texlive-xecjk.
|
|
||||||
# Denylisting is not needed in this case.
|
|
||||||
#
|
|
||||||
# * For printing CJK pages in PDF
|
|
||||||
# - Need non-variable "Noto CJK" fonts.
|
|
||||||
# * Fedora
|
|
||||||
# - google-noto-sans-cjk-fonts
|
|
||||||
# - google-noto-serif-cjk-fonts
|
|
||||||
# * openSUSE tumbleweed
|
|
||||||
# - Non-variable "Noto CJK" fonts are not available as distro packages
|
|
||||||
# as of April, 2024. Fetch a set of font files from upstream Noto
|
|
||||||
# CJK Font released at:
|
|
||||||
# https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc
|
|
||||||
# and at:
|
|
||||||
# https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc
|
|
||||||
# , then uncompress and deploy them.
|
|
||||||
# - Remember to update fontconfig cache by running fc-cache.
|
|
||||||
#
|
|
||||||
# !!! Caution !!!
|
|
||||||
# Uninstalling "variable font" packages can be dangerous.
|
|
||||||
# They might be depended upon by other packages important for your work.
|
|
||||||
# Denylisting should be less invasive, as it is effective only while
|
|
||||||
# XeLaTeX runs in "make pdfdocs".
|
|
||||||
|
|
||||||
# Default per-user fontconfig path (overridden by env variable)
|
|
||||||
: ${FONTS_CONF_DENY_VF:=$HOME/deny-vf}
|
|
||||||
|
|
||||||
export XDG_CONFIG_HOME=${FONTS_CONF_DENY_VF}
|
|
||||||
|
|
||||||
notocjkvffonts=`fc-list : file family variable | \
|
|
||||||
grep 'variable=True' | \
|
|
||||||
grep -E -e 'Noto (Sans|Sans Mono|Serif) CJK' | \
|
|
||||||
sed -e 's/^/ /' -e 's/: Noto S.*$//' | sort | uniq`
|
|
||||||
|
|
||||||
if [ "x$notocjkvffonts" != "x" ] ; then
|
|
||||||
echo '============================================================================='
|
|
||||||
echo 'XeTeX is confused by "variable font" files listed below:'
|
|
||||||
echo "$notocjkvffonts"
|
|
||||||
echo
|
|
||||||
echo 'For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.'
|
|
||||||
echo 'Or, CJK pages can be skipped by uninstalling texlive-xecjk.'
|
|
||||||
echo
|
|
||||||
echo 'For more info on denylisting, other options, and variable font, see header'
|
|
||||||
echo 'comments of scripts/check-variable-fonts.sh.'
|
|
||||||
echo '============================================================================='
|
|
||||||
fi
|
|
||||||
|
|
||||||
# As this script is invoked from Makefile's error path, always error exit
|
|
||||||
# regardless of whether any variable font is discovered or not.
|
|
||||||
exit 1
|
|
||||||
|
|
@ -1,77 +1,35 @@
|
||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
# SPDX-License-Identifier: GPL-2.0+
|
# SPDX-License-Identifier: GPL-2.0+
|
||||||
#
|
|
||||||
# This determines how many parallel tasks "make" is expecting, as it is
|
|
||||||
# not exposed via an special variables, reserves them all, runs a subprocess
|
|
||||||
# with PARALLELISM environment variable set, and releases the jobs back again.
|
|
||||||
#
|
|
||||||
# https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
|
|
||||||
from __future__ import print_function
|
|
||||||
import os, sys, errno
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
# Extract and prepare jobserver file descriptors from environment.
|
"""
|
||||||
claim = 0
|
Determines how many parallel tasks "make" is expecting, as it is
|
||||||
jobs = b""
|
not exposed via any special variables, reserves them all, runs a subprocess
|
||||||
try:
|
with PARALLELISM environment variable set, and releases the jobs back again.
|
||||||
# Fetch the make environment options.
|
|
||||||
flags = os.environ['MAKEFLAGS']
|
|
||||||
|
|
||||||
# Look for "--jobserver=R,W"
|
See:
|
||||||
# Note that GNU Make has used --jobserver-fds and --jobserver-auth
|
https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
|
||||||
# so this handles all of them.
|
"""
|
||||||
opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
|
|
||||||
|
|
||||||
# Parse out R,W file descriptor numbers and set them nonblocking.
|
import os
|
||||||
# If the MAKEFLAGS variable contains multiple instances of the
|
import sys
|
||||||
# --jobserver-auth= option, the last one is relevant.
|
|
||||||
fds = opts[-1].split("=", 1)[1]
|
|
||||||
|
|
||||||
# Starting with GNU Make 4.4, named pipes are used for reader and writer.
|
LIB_DIR = "lib"
|
||||||
# Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
|
SRC_DIR = os.path.dirname(os.path.realpath(__file__))
|
||||||
_, _, path = fds.partition('fifo:')
|
|
||||||
|
|
||||||
if path:
|
sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
|
||||||
reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
|
|
||||||
writer = os.open(path, os.O_WRONLY)
|
|
||||||
else:
|
|
||||||
reader, writer = [int(x) for x in fds.split(",", 1)]
|
|
||||||
# Open a private copy of reader to avoid setting nonblocking
|
|
||||||
# on an unexpecting process with the same reader fd.
|
|
||||||
reader = os.open("/proc/self/fd/%d" % (reader),
|
|
||||||
os.O_RDONLY | os.O_NONBLOCK)
|
|
||||||
|
|
||||||
# Read out as many jobserver slots as possible.
|
from jobserver import JobserverExec # pylint: disable=C0415
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
slot = os.read(reader, 8)
|
|
||||||
jobs += slot
|
|
||||||
except (OSError, IOError) as e:
|
|
||||||
if e.errno == errno.EWOULDBLOCK:
|
|
||||||
# Stop at the end of the jobserver queue.
|
|
||||||
break
|
|
||||||
# If something went wrong, give back the jobs.
|
|
||||||
if len(jobs):
|
|
||||||
os.write(writer, jobs)
|
|
||||||
raise e
|
|
||||||
# Add a bump for our caller's reserveration, since we're just going
|
|
||||||
# to sit here blocked on our child.
|
|
||||||
claim = len(jobs) + 1
|
|
||||||
except (KeyError, IndexError, ValueError, OSError, IOError) as e:
|
|
||||||
# Any missing environment strings or bad fds should result in just
|
|
||||||
# not being parallel.
|
|
||||||
pass
|
|
||||||
|
|
||||||
# We can only claim parallelism if there was a jobserver (i.e. a top-level
|
|
||||||
# "-jN" argument) and there were no other failures. Otherwise leave out the
|
|
||||||
# environment variable and let the child figure out what is best.
|
|
||||||
if claim > 0:
|
|
||||||
os.environ['PARALLELISM'] = '%d' % (claim)
|
|
||||||
|
|
||||||
rc = subprocess.call(sys.argv[1:])
|
def main():
|
||||||
|
"""Main program"""
|
||||||
|
if len(sys.argv) < 2:
|
||||||
|
name = os.path.basename(__file__)
|
||||||
|
sys.exit("usage: " + name +" command [args ...]\n" + __doc__)
|
||||||
|
|
||||||
# Return all the reserved slots.
|
with JobserverExec() as jobserver:
|
||||||
if len(jobs):
|
jobserver.run(sys.argv[1:])
|
||||||
os.write(writer, jobs)
|
|
||||||
|
|
||||||
sys.exit(rc)
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,149 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# SPDX-License-Identifier: GPL-2.0+
|
||||||
|
#
|
||||||
|
# pylint: disable=C0103,C0209
|
||||||
|
#
|
||||||
|
#
|
||||||
|
|
||||||
|
"""
|
||||||
|
Interacts with the POSIX jobserver during the Kernel build time.
|
||||||
|
|
||||||
|
A "normal" jobserver task, like the one initiated by a make subrocess would do:
|
||||||
|
|
||||||
|
- open read/write file descriptors to communicate with the job server;
|
||||||
|
- ask for one slot by calling:
|
||||||
|
claim = os.read(reader, 1)
|
||||||
|
- when the job finshes, call:
|
||||||
|
os.write(writer, b"+") # os.write(writer, claim)
|
||||||
|
|
||||||
|
Here, the goal is different: This script aims to get the remaining number
|
||||||
|
of slots available, using all of them to run a command which handle tasks in
|
||||||
|
parallel. To to that, it has a loop that ends only after there are no
|
||||||
|
slots left. It then increments the number by one, in order to allow a
|
||||||
|
call equivalent to make -j$((claim+1)), e.g. having a parent make creating
|
||||||
|
$claim child to do the actual work.
|
||||||
|
|
||||||
|
The end goal here is to keep the total number of build tasks under the
|
||||||
|
limit established by the initial make -j$n_proc call.
|
||||||
|
|
||||||
|
See:
|
||||||
|
https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
|
||||||
|
"""
|
||||||
|
|
||||||
|
import errno
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
class JobserverExec:
|
||||||
|
"""
|
||||||
|
Claim all slots from make using POSIX Jobserver.
|
||||||
|
|
||||||
|
The main methods here are:
|
||||||
|
- open(): reserves all slots;
|
||||||
|
- close(): method returns all used slots back to make;
|
||||||
|
- run(): executes a command setting PARALLELISM=<available slots jobs + 1>
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""Initialize internal vars"""
|
||||||
|
self.claim = 0
|
||||||
|
self.jobs = b""
|
||||||
|
self.reader = None
|
||||||
|
self.writer = None
|
||||||
|
self.is_open = False
|
||||||
|
|
||||||
|
def open(self):
|
||||||
|
"""Reserve all available slots to be claimed later on"""
|
||||||
|
|
||||||
|
if self.is_open:
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Fetch the make environment options.
|
||||||
|
flags = os.environ["MAKEFLAGS"]
|
||||||
|
# Look for "--jobserver=R,W"
|
||||||
|
# Note that GNU Make has used --jobserver-fds and --jobserver-auth
|
||||||
|
# so this handles all of them.
|
||||||
|
opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
|
||||||
|
|
||||||
|
# Parse out R,W file descriptor numbers and set them nonblocking.
|
||||||
|
# If the MAKEFLAGS variable contains multiple instances of the
|
||||||
|
# --jobserver-auth= option, the last one is relevant.
|
||||||
|
fds = opts[-1].split("=", 1)[1]
|
||||||
|
|
||||||
|
# Starting with GNU Make 4.4, named pipes are used for reader
|
||||||
|
# and writer.
|
||||||
|
# Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
|
||||||
|
_, _, path = fds.partition("fifo:")
|
||||||
|
|
||||||
|
if path:
|
||||||
|
self.reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
|
||||||
|
self.writer = os.open(path, os.O_WRONLY)
|
||||||
|
else:
|
||||||
|
self.reader, self.writer = [int(x) for x in fds.split(",", 1)]
|
||||||
|
# Open a private copy of reader to avoid setting nonblocking
|
||||||
|
# on an unexpecting process with the same reader fd.
|
||||||
|
self.reader = os.open("/proc/self/fd/%d" % (self.reader),
|
||||||
|
os.O_RDONLY | os.O_NONBLOCK)
|
||||||
|
|
||||||
|
# Read out as many jobserver slots as possible
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
slot = os.read(self.reader, 8)
|
||||||
|
self.jobs += slot
|
||||||
|
except (OSError, IOError) as e:
|
||||||
|
if e.errno == errno.EWOULDBLOCK:
|
||||||
|
# Stop at the end of the jobserver queue.
|
||||||
|
break
|
||||||
|
# If something went wrong, give back the jobs.
|
||||||
|
if self.jobs:
|
||||||
|
os.write(self.writer, self.jobs)
|
||||||
|
raise e
|
||||||
|
|
||||||
|
# Add a bump for our caller's reserveration, since we're just going
|
||||||
|
# to sit here blocked on our child.
|
||||||
|
self.claim = len(self.jobs) + 1
|
||||||
|
|
||||||
|
except (KeyError, IndexError, ValueError, OSError, IOError):
|
||||||
|
# Any missing environment strings or bad fds should result in just
|
||||||
|
# not being parallel.
|
||||||
|
self.claim = None
|
||||||
|
|
||||||
|
self.is_open = True
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
"""Return all reserved slots to Jobserver"""
|
||||||
|
|
||||||
|
if not self.is_open:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Return all the reserved slots.
|
||||||
|
if len(self.jobs):
|
||||||
|
os.write(self.writer, self.jobs)
|
||||||
|
|
||||||
|
self.is_open = False
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
self.open()
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc_value, exc_traceback):
|
||||||
|
self.close()
|
||||||
|
|
||||||
|
def run(self, cmd, *args, **pwargs):
|
||||||
|
"""
|
||||||
|
Run a command setting PARALLELISM env variable to the number of
|
||||||
|
available job slots (claim) + 1, e.g. it will reserve claim slots
|
||||||
|
to do the actual build work, plus one to monitor its children.
|
||||||
|
"""
|
||||||
|
self.open() # Ensure that self.claim is set
|
||||||
|
|
||||||
|
# We can only claim parallelism if there was a jobserver (i.e. a
|
||||||
|
# top-level "-jN" argument) and there were no other failures. Otherwise
|
||||||
|
# leave out the environment variable and let the child figure out what
|
||||||
|
# is best.
|
||||||
|
if self.claim:
|
||||||
|
os.environ["PARALLELISM"] = str(self.claim)
|
||||||
|
|
||||||
|
return subprocess.call(cmd, *args, **pwargs)
|
||||||
|
|
@ -275,7 +275,10 @@ class KernelFiles():
|
||||||
self.config.log.warning("No kernel-doc for file %s", fname)
|
self.config.log.warning("No kernel-doc for file %s", fname)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
for arg in self.results[fname]:
|
symbols = self.results[fname]
|
||||||
|
self.out_style.set_symbols(symbols)
|
||||||
|
|
||||||
|
for arg in symbols:
|
||||||
m = self.out_msg(fname, arg.name, arg)
|
m = self.out_msg(fname, arg.name, arg)
|
||||||
|
|
||||||
if m is None:
|
if m is None:
|
||||||
|
|
|
||||||
|
|
@ -5,8 +5,9 @@
|
||||||
#
|
#
|
||||||
|
|
||||||
class KdocItem:
|
class KdocItem:
|
||||||
def __init__(self, name, type, start_line, **other_stuff):
|
def __init__(self, name, fname, type, start_line, **other_stuff):
|
||||||
self.name = name
|
self.name = name
|
||||||
|
self.fname = fname
|
||||||
self.type = type
|
self.type = type
|
||||||
self.declaration_start_line = start_line
|
self.declaration_start_line = start_line
|
||||||
self.sections = {}
|
self.sections = {}
|
||||||
|
|
|
||||||
|
|
@ -215,6 +215,9 @@ class OutputFormat:
|
||||||
|
|
||||||
# Virtual methods to be overridden by inherited classes
|
# Virtual methods to be overridden by inherited classes
|
||||||
# At the base class, those do nothing.
|
# At the base class, those do nothing.
|
||||||
|
def set_symbols(self, symbols):
|
||||||
|
"""Get a list of all symbols from kernel_doc"""
|
||||||
|
|
||||||
def out_doc(self, fname, name, args):
|
def out_doc(self, fname, name, args):
|
||||||
"""Outputs a DOC block"""
|
"""Outputs a DOC block"""
|
||||||
|
|
||||||
|
|
@ -577,6 +580,7 @@ class ManFormat(OutputFormat):
|
||||||
|
|
||||||
super().__init__()
|
super().__init__()
|
||||||
self.modulename = modulename
|
self.modulename = modulename
|
||||||
|
self.symbols = []
|
||||||
|
|
||||||
dt = None
|
dt = None
|
||||||
tstamp = os.environ.get("KBUILD_BUILD_TIMESTAMP")
|
tstamp = os.environ.get("KBUILD_BUILD_TIMESTAMP")
|
||||||
|
|
@ -593,6 +597,69 @@ class ManFormat(OutputFormat):
|
||||||
|
|
||||||
self.man_date = dt.strftime("%B %Y")
|
self.man_date = dt.strftime("%B %Y")
|
||||||
|
|
||||||
|
def arg_name(self, args, name):
|
||||||
|
"""
|
||||||
|
Return the name that will be used for the man page.
|
||||||
|
|
||||||
|
As we may have the same name on different namespaces,
|
||||||
|
prepend the data type for all types except functions and typedefs.
|
||||||
|
|
||||||
|
The doc section is special: it uses the modulename.
|
||||||
|
"""
|
||||||
|
|
||||||
|
dtype = args.type
|
||||||
|
|
||||||
|
if dtype == "doc":
|
||||||
|
return self.modulename
|
||||||
|
|
||||||
|
if dtype in ["function", "typedef"]:
|
||||||
|
return name
|
||||||
|
|
||||||
|
return f"{dtype} {name}"
|
||||||
|
|
||||||
|
def set_symbols(self, symbols):
|
||||||
|
"""
|
||||||
|
Get a list of all symbols from kernel_doc.
|
||||||
|
|
||||||
|
Man pages will uses it to add a SEE ALSO section with other
|
||||||
|
symbols at the same file.
|
||||||
|
"""
|
||||||
|
self.symbols = symbols
|
||||||
|
|
||||||
|
def out_tail(self, fname, name, args):
|
||||||
|
"""Adds a tail for all man pages"""
|
||||||
|
|
||||||
|
# SEE ALSO section
|
||||||
|
self.data += f'.SH "SEE ALSO"' + "\n.PP\n"
|
||||||
|
self.data += (f"Kernel file \\fB{args.fname}\\fR\n")
|
||||||
|
if len(self.symbols) >= 2:
|
||||||
|
cur_name = self.arg_name(args, name)
|
||||||
|
|
||||||
|
related = []
|
||||||
|
for arg in self.symbols:
|
||||||
|
out_name = self.arg_name(arg, arg.name)
|
||||||
|
|
||||||
|
if cur_name == out_name:
|
||||||
|
continue
|
||||||
|
|
||||||
|
related.append(f"\\fB{out_name}\\fR(9)")
|
||||||
|
|
||||||
|
self.data += ",\n".join(related) + "\n"
|
||||||
|
|
||||||
|
# TODO: does it make sense to add other sections? Maybe
|
||||||
|
# REPORTING ISSUES? LICENSE?
|
||||||
|
|
||||||
|
def msg(self, fname, name, args):
|
||||||
|
"""
|
||||||
|
Handles a single entry from kernel-doc parser.
|
||||||
|
|
||||||
|
Add a tail at the end of man pages output.
|
||||||
|
"""
|
||||||
|
super().msg(fname, name, args)
|
||||||
|
self.out_tail(fname, name, args)
|
||||||
|
|
||||||
|
return self.data
|
||||||
|
|
||||||
def output_highlight(self, block):
|
def output_highlight(self, block):
|
||||||
"""
|
"""
|
||||||
Outputs a C symbol that may require being highlighted with
|
Outputs a C symbol that may require being highlighted with
|
||||||
|
|
@ -618,7 +685,9 @@ class ManFormat(OutputFormat):
|
||||||
if not self.check_doc(name, args):
|
if not self.check_doc(name, args):
|
||||||
return
|
return
|
||||||
|
|
||||||
self.data += f'.TH "{self.modulename}" 9 "{self.modulename}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
out_name = self.arg_name(args, name)
|
||||||
|
|
||||||
|
self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
||||||
|
|
||||||
for section, text in args.sections.items():
|
for section, text in args.sections.items():
|
||||||
self.data += f'.SH "{section}"' + "\n"
|
self.data += f'.SH "{section}"' + "\n"
|
||||||
|
|
@ -627,7 +696,9 @@ class ManFormat(OutputFormat):
|
||||||
def out_function(self, fname, name, args):
|
def out_function(self, fname, name, args):
|
||||||
"""output function in man"""
|
"""output function in man"""
|
||||||
|
|
||||||
self.data += f'.TH "{name}" 9 "{name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
|
out_name = self.arg_name(args, name)
|
||||||
|
|
||||||
|
self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
|
||||||
|
|
||||||
self.data += ".SH NAME\n"
|
self.data += ".SH NAME\n"
|
||||||
self.data += f"{name} \\- {args['purpose']}\n"
|
self.data += f"{name} \\- {args['purpose']}\n"
|
||||||
|
|
@ -671,7 +742,9 @@ class ManFormat(OutputFormat):
|
||||||
self.output_highlight(text)
|
self.output_highlight(text)
|
||||||
|
|
||||||
def out_enum(self, fname, name, args):
|
def out_enum(self, fname, name, args):
|
||||||
self.data += f'.TH "{self.modulename}" 9 "enum {name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
out_name = self.arg_name(args, name)
|
||||||
|
|
||||||
|
self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
||||||
|
|
||||||
self.data += ".SH NAME\n"
|
self.data += ".SH NAME\n"
|
||||||
self.data += f"enum {name} \\- {args['purpose']}\n"
|
self.data += f"enum {name} \\- {args['purpose']}\n"
|
||||||
|
|
@ -703,8 +776,9 @@ class ManFormat(OutputFormat):
|
||||||
def out_typedef(self, fname, name, args):
|
def out_typedef(self, fname, name, args):
|
||||||
module = self.modulename
|
module = self.modulename
|
||||||
purpose = args.get('purpose')
|
purpose = args.get('purpose')
|
||||||
|
out_name = self.arg_name(args, name)
|
||||||
|
|
||||||
self.data += f'.TH "{module}" 9 "{name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
||||||
|
|
||||||
self.data += ".SH NAME\n"
|
self.data += ".SH NAME\n"
|
||||||
self.data += f"typedef {name} \\- {purpose}\n"
|
self.data += f"typedef {name} \\- {purpose}\n"
|
||||||
|
|
@ -717,8 +791,9 @@ class ManFormat(OutputFormat):
|
||||||
module = self.modulename
|
module = self.modulename
|
||||||
purpose = args.get('purpose')
|
purpose = args.get('purpose')
|
||||||
definition = args.get('definition')
|
definition = args.get('definition')
|
||||||
|
out_name = self.arg_name(args, name)
|
||||||
|
|
||||||
self.data += f'.TH "{module}" 9 "{args.type} {name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
|
||||||
|
|
||||||
self.data += ".SH NAME\n"
|
self.data += ".SH NAME\n"
|
||||||
self.data += f"{args.type} {name} \\- {purpose}\n"
|
self.data += f"{args.type} {name} \\- {purpose}\n"
|
||||||
|
|
|
||||||
|
|
@ -254,8 +254,9 @@ SECTION_DEFAULT = "Description" # default section
|
||||||
|
|
||||||
class KernelEntry:
|
class KernelEntry:
|
||||||
|
|
||||||
def __init__(self, config, ln):
|
def __init__(self, config, fname, ln):
|
||||||
self.config = config
|
self.config = config
|
||||||
|
self.fname = fname
|
||||||
|
|
||||||
self._contents = []
|
self._contents = []
|
||||||
self.prototype = ""
|
self.prototype = ""
|
||||||
|
|
@ -350,6 +351,7 @@ class KernelEntry:
|
||||||
self.section = SECTION_DEFAULT
|
self.section = SECTION_DEFAULT
|
||||||
self._contents = []
|
self._contents = []
|
||||||
|
|
||||||
|
python_warning = False
|
||||||
|
|
||||||
class KernelDoc:
|
class KernelDoc:
|
||||||
"""
|
"""
|
||||||
|
|
@ -383,9 +385,13 @@ class KernelDoc:
|
||||||
# We need Python 3.7 for its "dicts remember the insertion
|
# We need Python 3.7 for its "dicts remember the insertion
|
||||||
# order" guarantee
|
# order" guarantee
|
||||||
#
|
#
|
||||||
if sys.version_info.major == 3 and sys.version_info.minor < 7:
|
global python_warning
|
||||||
|
if (not python_warning and
|
||||||
|
sys.version_info.major == 3 and sys.version_info.minor < 7):
|
||||||
|
|
||||||
self.emit_msg(0,
|
self.emit_msg(0,
|
||||||
'Python 3.7 or later is required for correct results')
|
'Python 3.7 or later is required for correct results')
|
||||||
|
python_warning = True
|
||||||
|
|
||||||
def emit_msg(self, ln, msg, warning=True):
|
def emit_msg(self, ln, msg, warning=True):
|
||||||
"""Emit a message"""
|
"""Emit a message"""
|
||||||
|
|
@ -417,7 +423,8 @@ class KernelDoc:
|
||||||
The actual output and output filters will be handled elsewhere
|
The actual output and output filters will be handled elsewhere
|
||||||
"""
|
"""
|
||||||
|
|
||||||
item = KdocItem(name, dtype, self.entry.declaration_start_line, **args)
|
item = KdocItem(name, self.fname, dtype,
|
||||||
|
self.entry.declaration_start_line, **args)
|
||||||
item.warnings = self.entry.warnings
|
item.warnings = self.entry.warnings
|
||||||
|
|
||||||
# Drop empty sections
|
# Drop empty sections
|
||||||
|
|
@ -440,7 +447,7 @@ class KernelDoc:
|
||||||
variables used by the state machine.
|
variables used by the state machine.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
self.entry = KernelEntry(self.config, ln)
|
self.entry = KernelEntry(self.config, self.fname, ln)
|
||||||
|
|
||||||
# State flags
|
# State flags
|
||||||
self.state = state.NORMAL
|
self.state = state.NORMAL
|
||||||
|
|
|
||||||
|
|
@ -1,28 +0,0 @@
|
||||||
#!/usr/bin/env perl
|
|
||||||
# SPDX-License-Identifier: GPL-2.0
|
|
||||||
#
|
|
||||||
# Author: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
|
|
||||||
#
|
|
||||||
# Produce manpages from kernel-doc.
|
|
||||||
# See Documentation/doc-guide/kernel-doc.rst for instructions
|
|
||||||
|
|
||||||
if ($#ARGV < 0) {
|
|
||||||
die "where do I put the results?\n";
|
|
||||||
}
|
|
||||||
|
|
||||||
mkdir $ARGV[0],0777;
|
|
||||||
$state = 0;
|
|
||||||
while (<STDIN>) {
|
|
||||||
if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) {
|
|
||||||
if ($state == 1) { close OUT }
|
|
||||||
$state = 1;
|
|
||||||
$fn = "$ARGV[0]/$1.9";
|
|
||||||
print STDERR "Creating $fn\n";
|
|
||||||
open OUT, ">$fn" or die "can't open $fn: $!\n";
|
|
||||||
print OUT $_;
|
|
||||||
} elsif ($state != 0) {
|
|
||||||
print OUT $_;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
close OUT;
|
|
||||||
|
|
@ -0,0 +1,33 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# SPDX-License-Identifier: GPL-2.0-only
|
||||||
|
# Copyright (C) Akira Yokosawa, 2024
|
||||||
|
#
|
||||||
|
# Ported to Python by (c) Mauro Carvalho Chehab, 2025
|
||||||
|
#
|
||||||
|
# pylint: disable=C0103
|
||||||
|
|
||||||
|
"""
|
||||||
|
Detect problematic Noto CJK variable fonts.
|
||||||
|
|
||||||
|
or more details, see lib/latex_fonts.py.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from lib.latex_fonts import LatexFontChecker
|
||||||
|
|
||||||
|
checker = LatexFontChecker()
|
||||||
|
|
||||||
|
parser=argparse.ArgumentParser(description=checker.description(),
|
||||||
|
formatter_class=argparse.RawTextHelpFormatter)
|
||||||
|
parser.add_argument("--deny-vf",
|
||||||
|
help="XDG_CONFIG_HOME dir containing fontconfig/fonts.conf file")
|
||||||
|
|
||||||
|
args=parser.parse_args()
|
||||||
|
|
||||||
|
msg = LatexFontChecker(args.deny_vf).check()
|
||||||
|
if msg:
|
||||||
|
print(msg)
|
||||||
|
|
||||||
|
sys.exit(1)
|
||||||
|
|
@ -0,0 +1,167 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# SPDX-License-Identifier: GPL-2.0-only
|
||||||
|
# Copyright (C) Akira Yokosawa, 2024
|
||||||
|
#
|
||||||
|
# Ported to Python by (c) Mauro Carvalho Chehab, 2025
|
||||||
|
|
||||||
|
"""
|
||||||
|
Detect problematic Noto CJK variable fonts.
|
||||||
|
|
||||||
|
For "make pdfdocs", reports of build errors of translations.pdf started
|
||||||
|
arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE
|
||||||
|
tumbleweed have started deploying variable-font [3] format of "Noto CJK"
|
||||||
|
fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK
|
||||||
|
(Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which
|
||||||
|
does not (and likely never will) understand variable fonts for historical
|
||||||
|
reasons.
|
||||||
|
|
||||||
|
The build error happens even when both of variable- and non-variable-format
|
||||||
|
fonts are found on the build system. To make matters worse, Fedora enlists
|
||||||
|
variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN,
|
||||||
|
-zh_TW, etc. Hence developers who have interest in CJK pages are more
|
||||||
|
likely to encounter the build errors.
|
||||||
|
|
||||||
|
This script is invoked from the error path of "make pdfdocs" and emits
|
||||||
|
suggestions if variable-font files of "Noto CJK" fonts are in the list of
|
||||||
|
fonts accessible from XeTeX.
|
||||||
|
|
||||||
|
References:
|
||||||
|
[1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
|
||||||
|
[2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
|
||||||
|
[3]: https://en.wikipedia.org/wiki/Variable_font
|
||||||
|
[4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
|
||||||
|
[5]: https://build.opensuse.org/request/show/1157217
|
||||||
|
|
||||||
|
#===========================================================================
|
||||||
|
Workarounds for building translations.pdf
|
||||||
|
#===========================================================================
|
||||||
|
|
||||||
|
* Denylist "variable font" Noto CJK fonts.
|
||||||
|
- Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with
|
||||||
|
tweaks if necessary. Remove leading "".
|
||||||
|
- Path of fontconfig/fonts.conf can be overridden by setting an env
|
||||||
|
variable FONTS_CONF_DENY_VF.
|
||||||
|
|
||||||
|
* Template:
|
||||||
|
-----------------------------------------------------------------
|
||||||
|
<?xml version="1.0"?>
|
||||||
|
<!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
|
||||||
|
<fontconfig>
|
||||||
|
<!--
|
||||||
|
Ignore variable-font glob (not to break xetex)
|
||||||
|
-->
|
||||||
|
<selectfont>
|
||||||
|
<rejectfont>
|
||||||
|
<!--
|
||||||
|
for Fedora
|
||||||
|
-->
|
||||||
|
<glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
|
||||||
|
<!--
|
||||||
|
for openSUSE tumbleweed
|
||||||
|
-->
|
||||||
|
<glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
|
||||||
|
</rejectfont>
|
||||||
|
</selectfont>
|
||||||
|
</fontconfig>
|
||||||
|
-----------------------------------------------------------------
|
||||||
|
|
||||||
|
The denylisting is activated for "make pdfdocs".
|
||||||
|
|
||||||
|
* For skipping CJK pages in PDF
|
||||||
|
- Uninstall texlive-xecjk.
|
||||||
|
Denylisting is not needed in this case.
|
||||||
|
|
||||||
|
* For printing CJK pages in PDF
|
||||||
|
- Need non-variable "Noto CJK" fonts.
|
||||||
|
* Fedora
|
||||||
|
- google-noto-sans-cjk-fonts
|
||||||
|
- google-noto-serif-cjk-fonts
|
||||||
|
* openSUSE tumbleweed
|
||||||
|
- Non-variable "Noto CJK" fonts are not available as distro packages
|
||||||
|
as of April, 2024. Fetch a set of font files from upstream Noto
|
||||||
|
CJK Font released at:
|
||||||
|
https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc
|
||||||
|
and at:
|
||||||
|
https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc
|
||||||
|
, then uncompress and deploy them.
|
||||||
|
- Remember to update fontconfig cache by running fc-cache.
|
||||||
|
|
||||||
|
!!! Caution !!!
|
||||||
|
Uninstalling "variable font" packages can be dangerous.
|
||||||
|
They might be depended upon by other packages important for your work.
|
||||||
|
Denylisting should be less invasive, as it is effective only while
|
||||||
|
XeLaTeX runs in "make pdfdocs".
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import textwrap
|
||||||
|
import sys
|
||||||
|
|
||||||
|
class LatexFontChecker:
|
||||||
|
"""
|
||||||
|
Detect problems with CJK variable fonts that affect PDF builds for
|
||||||
|
translations.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, deny_vf=None):
|
||||||
|
if not deny_vf:
|
||||||
|
deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf")
|
||||||
|
|
||||||
|
self.environ = os.environ.copy()
|
||||||
|
self.environ['XDG_CONFIG_HOME'] = os.path.expanduser(deny_vf)
|
||||||
|
|
||||||
|
self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK")
|
||||||
|
|
||||||
|
def description(self):
|
||||||
|
return __doc__
|
||||||
|
|
||||||
|
def get_noto_cjk_vf_fonts(self):
|
||||||
|
"""Get Noto CJK fonts"""
|
||||||
|
|
||||||
|
cjk_fonts = set()
|
||||||
|
cmd = ["fc-list", ":", "file", "family", "variable"]
|
||||||
|
try:
|
||||||
|
result = subprocess.run(cmd,stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE,
|
||||||
|
universal_newlines=True,
|
||||||
|
env=self.environ,
|
||||||
|
check=True)
|
||||||
|
|
||||||
|
except subprocess.CalledProcessError as exc:
|
||||||
|
sys.exit(f"Error running fc-list: {repr(exc)}")
|
||||||
|
|
||||||
|
for line in result.stdout.splitlines():
|
||||||
|
if 'variable=True' not in line:
|
||||||
|
continue
|
||||||
|
|
||||||
|
match = self.re_cjk.search(line)
|
||||||
|
if match:
|
||||||
|
cjk_fonts.add(match.group(1))
|
||||||
|
|
||||||
|
return sorted(cjk_fonts)
|
||||||
|
|
||||||
|
def check(self):
|
||||||
|
"""Check for problems with CJK fonts"""
|
||||||
|
|
||||||
|
fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), " ")
|
||||||
|
if not fonts:
|
||||||
|
return None
|
||||||
|
|
||||||
|
rel_file = os.path.relpath(__file__, os.getcwd())
|
||||||
|
|
||||||
|
msg = "=" * 77 + "\n"
|
||||||
|
msg += 'XeTeX is confused by "variable font" files listed below:\n'
|
||||||
|
msg += fonts + "\n"
|
||||||
|
msg += textwrap.dedent(f"""
|
||||||
|
For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
|
||||||
|
Or, CJK pages can be skipped by uninstalling texlive-xecjk.
|
||||||
|
|
||||||
|
For more info on denylisting, other options, and variable font, run:
|
||||||
|
|
||||||
|
tools/docs/check-variable-fonts.py -h
|
||||||
|
""")
|
||||||
|
msg += "=" * 77
|
||||||
|
|
||||||
|
return msg
|
||||||
|
|
@ -0,0 +1,178 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# SPDX-License-Identifier: GPL-2.0-or-later
|
||||||
|
# Copyright (c) 2017-2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
|
||||||
|
|
||||||
|
"""
|
||||||
|
Handle Python version check logic.
|
||||||
|
|
||||||
|
Not all Python versions are supported by scripts. Yet, on some cases,
|
||||||
|
like during documentation build, a newer version of python could be
|
||||||
|
available.
|
||||||
|
|
||||||
|
This class allows checking if the minimal requirements are followed.
|
||||||
|
|
||||||
|
Better than that, PythonVersion.check_python() not only checks the minimal
|
||||||
|
requirements, but it automatically switches to a the newest available
|
||||||
|
Python version if present.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import shlex
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from glob import glob
|
||||||
|
from textwrap import indent
|
||||||
|
|
||||||
|
class PythonVersion:
|
||||||
|
"""
|
||||||
|
Ancillary methods that checks for missing dependencies for different
|
||||||
|
types of types, like binaries, python modules, rpm deps, etc.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, version):
|
||||||
|
"""Ïnitialize self.version tuple from a version string"""
|
||||||
|
self.version = self.parse_version(version)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def parse_version(version):
|
||||||
|
"""Convert a major.minor.patch version into a tuple"""
|
||||||
|
return tuple(int(x) for x in version.split("."))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def ver_str(version):
|
||||||
|
"""Returns a version tuple as major.minor.patch"""
|
||||||
|
return ".".join([str(x) for x in version])
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def cmd_print(cmd, max_len=80):
|
||||||
|
cmd_line = []
|
||||||
|
|
||||||
|
for w in cmd:
|
||||||
|
w = shlex.quote(w)
|
||||||
|
|
||||||
|
if cmd_line:
|
||||||
|
if not max_len or len(cmd_line[-1]) + len(w) < max_len:
|
||||||
|
cmd_line[-1] += " " + w
|
||||||
|
continue
|
||||||
|
else:
|
||||||
|
cmd_line[-1] += " \\"
|
||||||
|
cmd_line.append(w)
|
||||||
|
else:
|
||||||
|
cmd_line.append(w)
|
||||||
|
|
||||||
|
return "\n ".join(cmd_line)
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
"""Returns a version tuple as major.minor.patch from self.version"""
|
||||||
|
return self.ver_str(self.version)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_python_version(cmd):
|
||||||
|
"""
|
||||||
|
Get python version from a Python binary. As we need to detect if
|
||||||
|
are out there newer python binaries, we can't rely on sys.release here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
kwargs = {}
|
||||||
|
if sys.version_info < (3, 7):
|
||||||
|
kwargs['universal_newlines'] = True
|
||||||
|
else:
|
||||||
|
kwargs['text'] = True
|
||||||
|
|
||||||
|
result = subprocess.run([cmd, "--version"],
|
||||||
|
stdout = subprocess.PIPE,
|
||||||
|
stderr = subprocess.PIPE,
|
||||||
|
**kwargs, check=False)
|
||||||
|
|
||||||
|
version = result.stdout.strip()
|
||||||
|
|
||||||
|
match = re.search(r"(\d+\.\d+\.\d+)", version)
|
||||||
|
if match:
|
||||||
|
return PythonVersion.parse_version(match.group(1))
|
||||||
|
|
||||||
|
print(f"Can't parse version {version}")
|
||||||
|
return (0, 0, 0)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def find_python(min_version):
|
||||||
|
"""
|
||||||
|
Detect if are out there any python 3.xy version newer than the
|
||||||
|
current one.
|
||||||
|
|
||||||
|
Note: this routine is limited to up to 2 digits for python3. We
|
||||||
|
may need to update it one day, hopefully on a distant future.
|
||||||
|
"""
|
||||||
|
patterns = [
|
||||||
|
"python3.[0-9][0-9]",
|
||||||
|
"python3.[0-9]",
|
||||||
|
]
|
||||||
|
|
||||||
|
python_cmd = []
|
||||||
|
|
||||||
|
# Seek for a python binary newer than min_version
|
||||||
|
for path in os.getenv("PATH", "").split(":"):
|
||||||
|
for pattern in patterns:
|
||||||
|
for cmd in glob(os.path.join(path, pattern)):
|
||||||
|
if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
|
||||||
|
version = PythonVersion.get_python_version(cmd)
|
||||||
|
if version >= min_version:
|
||||||
|
python_cmd.append((version, cmd))
|
||||||
|
|
||||||
|
return sorted(python_cmd, reverse=True)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def check_python(min_version, show_alternatives=False, bail_out=False,
|
||||||
|
success_on_error=False):
|
||||||
|
"""
|
||||||
|
Check if the current python binary satisfies our minimal requirement
|
||||||
|
for Sphinx build. If not, re-run with a newer version if found.
|
||||||
|
"""
|
||||||
|
cur_ver = sys.version_info[:3]
|
||||||
|
if cur_ver >= min_version:
|
||||||
|
ver = PythonVersion.ver_str(cur_ver)
|
||||||
|
return
|
||||||
|
|
||||||
|
python_ver = PythonVersion.ver_str(cur_ver)
|
||||||
|
|
||||||
|
available_versions = PythonVersion.find_python(min_version)
|
||||||
|
if not available_versions:
|
||||||
|
print(f"ERROR: Python version {python_ver} is not spported anymore\n")
|
||||||
|
print(" Can't find a new version. This script may fail")
|
||||||
|
return
|
||||||
|
|
||||||
|
script_path = os.path.abspath(sys.argv[0])
|
||||||
|
|
||||||
|
# Check possible alternatives
|
||||||
|
if available_versions:
|
||||||
|
new_python_cmd = available_versions[0][1]
|
||||||
|
else:
|
||||||
|
new_python_cmd = None
|
||||||
|
|
||||||
|
if show_alternatives and available_versions:
|
||||||
|
print("You could run, instead:")
|
||||||
|
for _, cmd in available_versions:
|
||||||
|
args = [cmd, script_path] + sys.argv[1:]
|
||||||
|
|
||||||
|
cmd_str = indent(PythonVersion.cmd_print(args), " ")
|
||||||
|
print(f"{cmd_str}\n")
|
||||||
|
|
||||||
|
if bail_out:
|
||||||
|
msg = f"Python {python_ver} not supported. Bailing out"
|
||||||
|
if success_on_error:
|
||||||
|
print(msg, file=sys.stderr)
|
||||||
|
sys.exit(0)
|
||||||
|
else:
|
||||||
|
sys.exit(msg)
|
||||||
|
|
||||||
|
print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
|
||||||
|
|
||||||
|
# Restart script using the newer version
|
||||||
|
args = [new_python_cmd, script_path] + sys.argv[1:]
|
||||||
|
|
||||||
|
try:
|
||||||
|
os.execv(new_python_cmd, args)
|
||||||
|
except OSError as e:
|
||||||
|
sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
|
||||||
|
|
@ -56,68 +56,40 @@ import sys
|
||||||
from concurrent import futures
|
from concurrent import futures
|
||||||
from glob import glob
|
from glob import glob
|
||||||
|
|
||||||
LIB_DIR = "lib"
|
from lib.python_version import PythonVersion
|
||||||
|
from lib.latex_fonts import LatexFontChecker
|
||||||
|
|
||||||
|
LIB_DIR = "../../scripts/lib"
|
||||||
SRC_DIR = os.path.dirname(os.path.realpath(__file__))
|
SRC_DIR = os.path.dirname(os.path.realpath(__file__))
|
||||||
|
|
||||||
sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
|
sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
|
||||||
|
|
||||||
from jobserver import JobserverExec # pylint: disable=C0413
|
from jobserver import JobserverExec # pylint: disable=C0413,C0411,E0401
|
||||||
|
|
||||||
|
#
|
||||||
def parse_version(version):
|
# Some constants
|
||||||
"""Convert a major.minor.patch version into a tuple"""
|
#
|
||||||
return tuple(int(x) for x in version.split("."))
|
|
||||||
|
|
||||||
def ver_str(version):
|
|
||||||
"""Returns a version tuple as major.minor.patch"""
|
|
||||||
|
|
||||||
return ".".join([str(x) for x in version])
|
|
||||||
|
|
||||||
# Minimal supported Python version needed by Sphinx and its extensions
|
|
||||||
MIN_PYTHON_VERSION = parse_version("3.7")
|
|
||||||
|
|
||||||
# Default value for --venv parameter
|
|
||||||
VENV_DEFAULT = "sphinx_latest"
|
VENV_DEFAULT = "sphinx_latest"
|
||||||
|
MIN_PYTHON_VERSION = PythonVersion("3.7").version
|
||||||
|
PAPER = ["", "a4", "letter"]
|
||||||
|
|
||||||
# List of make targets and its corresponding builder and output directory
|
|
||||||
TARGETS = {
|
TARGETS = {
|
||||||
"cleandocs": {
|
"cleandocs": { "builder": "clean" },
|
||||||
"builder": "clean",
|
"linkcheckdocs": { "builder": "linkcheck" },
|
||||||
},
|
"htmldocs": { "builder": "html" },
|
||||||
"htmldocs": {
|
"epubdocs": { "builder": "epub", "out_dir": "epub" },
|
||||||
"builder": "html",
|
"texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" },
|
||||||
},
|
"infodocs": { "builder": "texinfo", "out_dir": "texinfo" },
|
||||||
"epubdocs": {
|
"mandocs": { "builder": "man", "out_dir": "man" },
|
||||||
"builder": "epub",
|
"latexdocs": { "builder": "latex", "out_dir": "latex" },
|
||||||
"out_dir": "epub",
|
"pdfdocs": { "builder": "latex", "out_dir": "latex" },
|
||||||
},
|
"xmldocs": { "builder": "xml", "out_dir": "xml" },
|
||||||
"texinfodocs": {
|
|
||||||
"builder": "texinfo",
|
|
||||||
"out_dir": "texinfo",
|
|
||||||
},
|
|
||||||
"infodocs": {
|
|
||||||
"builder": "texinfo",
|
|
||||||
"out_dir": "texinfo",
|
|
||||||
},
|
|
||||||
"latexdocs": {
|
|
||||||
"builder": "latex",
|
|
||||||
"out_dir": "latex",
|
|
||||||
},
|
|
||||||
"pdfdocs": {
|
|
||||||
"builder": "latex",
|
|
||||||
"out_dir": "latex",
|
|
||||||
},
|
|
||||||
"xmldocs": {
|
|
||||||
"builder": "xml",
|
|
||||||
"out_dir": "xml",
|
|
||||||
},
|
|
||||||
"linkcheckdocs": {
|
|
||||||
"builder": "linkcheck"
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# Paper sizes. An empty value will pick the default
|
|
||||||
PAPER = ["", "a4", "letter"]
|
#
|
||||||
|
# SphinxBuilder class
|
||||||
|
#
|
||||||
|
|
||||||
class SphinxBuilder:
|
class SphinxBuilder:
|
||||||
"""
|
"""
|
||||||
|
|
@ -125,15 +97,7 @@ class SphinxBuilder:
|
||||||
with the Kernel.
|
with the Kernel.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def is_rust_enabled(self):
|
def get_path(self, path, use_cwd=False, abs_path=False):
|
||||||
"""Check if rust is enabled at .config"""
|
|
||||||
config_path = os.path.join(self.srctree, ".config")
|
|
||||||
if os.path.isfile(config_path):
|
|
||||||
with open(config_path, "r", encoding="utf-8") as f:
|
|
||||||
return "CONFIG_RUST=y" in f.read()
|
|
||||||
return False
|
|
||||||
|
|
||||||
def get_path(self, path, abs_path=False):
|
|
||||||
"""
|
"""
|
||||||
Ancillary routine to handle patches the right way, as shell does.
|
Ancillary routine to handle patches the right way, as shell does.
|
||||||
|
|
||||||
|
|
@ -143,23 +107,97 @@ class SphinxBuilder:
|
||||||
|
|
||||||
path = os.path.expanduser(path)
|
path = os.path.expanduser(path)
|
||||||
if not path.startswith("/"):
|
if not path.startswith("/"):
|
||||||
path = os.path.join(self.srctree, path)
|
if use_cwd:
|
||||||
|
base = os.getcwd()
|
||||||
|
else:
|
||||||
|
base = self.srctree
|
||||||
|
|
||||||
|
path = os.path.join(base, path)
|
||||||
|
|
||||||
if abs_path:
|
if abs_path:
|
||||||
return os.path.abspath(path)
|
return os.path.abspath(path)
|
||||||
|
|
||||||
return path
|
return path
|
||||||
|
|
||||||
def __init__(self, venv=None, verbose=False, n_jobs=None, interactive=None):
|
def get_sphinx_extra_opts(self, n_jobs):
|
||||||
|
"""
|
||||||
|
Get the number of jobs to be used for docs build passed via command
|
||||||
|
line and desired sphinx verbosity.
|
||||||
|
|
||||||
|
The number of jobs can be on different places:
|
||||||
|
|
||||||
|
1) It can be passed via "-j" argument;
|
||||||
|
2) The SPHINXOPTS="-j8" env var may have "-j";
|
||||||
|
3) if called via GNU make, -j specifies the desired number of jobs.
|
||||||
|
with GNU makefile, this number is available via POSIX jobserver;
|
||||||
|
4) if none of the above is available, it should default to "-jauto",
|
||||||
|
and let sphinx decide the best value.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#
|
||||||
|
# SPHINXOPTS env var, if used, contains extra arguments to be used
|
||||||
|
# by sphinx-build time. Among them, it may contain sphinx verbosity
|
||||||
|
# and desired number of parallel jobs.
|
||||||
|
#
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument('-j', '--jobs', type=int)
|
||||||
|
parser.add_argument('-q', '--quiet', action='store_true')
|
||||||
|
|
||||||
|
#
|
||||||
|
# Other sphinx-build arguments go as-is, so place them
|
||||||
|
# at self.sphinxopts, using shell parser
|
||||||
|
#
|
||||||
|
sphinxopts = shlex.split(os.environ.get("SPHINXOPTS", ""))
|
||||||
|
|
||||||
|
#
|
||||||
|
# Build a list of sphinx args, honoring verbosity here if specified
|
||||||
|
#
|
||||||
|
|
||||||
|
verbose = self.verbose
|
||||||
|
sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts)
|
||||||
|
if sphinx_args.quiet is True:
|
||||||
|
verbose = False
|
||||||
|
|
||||||
|
#
|
||||||
|
# If the user explicitly sets "-j" at command line, use it.
|
||||||
|
# Otherwise, pick it from SPHINXOPTS args
|
||||||
|
#
|
||||||
|
if n_jobs:
|
||||||
|
self.n_jobs = n_jobs
|
||||||
|
elif sphinx_args.jobs:
|
||||||
|
self.n_jobs = sphinx_args.jobs
|
||||||
|
else:
|
||||||
|
self.n_jobs = None
|
||||||
|
|
||||||
|
if not verbose:
|
||||||
|
self.sphinxopts += ["-q"]
|
||||||
|
|
||||||
|
def __init__(self, builddir, venv=None, verbose=False, n_jobs=None,
|
||||||
|
interactive=None):
|
||||||
"""Initialize internal variables"""
|
"""Initialize internal variables"""
|
||||||
self.venv = venv
|
self.venv = venv
|
||||||
self.verbose = None
|
self.verbose = None
|
||||||
|
|
||||||
|
#
|
||||||
# Normal variables passed from Kernel's makefile
|
# Normal variables passed from Kernel's makefile
|
||||||
|
#
|
||||||
self.kernelversion = os.environ.get("KERNELVERSION", "unknown")
|
self.kernelversion = os.environ.get("KERNELVERSION", "unknown")
|
||||||
self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown")
|
self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown")
|
||||||
self.pdflatex = os.environ.get("PDFLATEX", "xelatex")
|
self.pdflatex = os.environ.get("PDFLATEX", "xelatex")
|
||||||
|
|
||||||
|
#
|
||||||
|
# Kernel main Makefile defines a PYTHON3 variable whose default is
|
||||||
|
# "python3". When set to a different value, it allows running a
|
||||||
|
# diferent version than the default official python3 package.
|
||||||
|
# Several distros package python3xx-sphinx packages with newer
|
||||||
|
# versions of Python and sphinx-build.
|
||||||
|
#
|
||||||
|
# Honor such variable different than default
|
||||||
|
#
|
||||||
|
self.python = os.environ.get("PYTHON3")
|
||||||
|
if self.python == "python3":
|
||||||
|
self.python = None
|
||||||
|
|
||||||
if not interactive:
|
if not interactive:
|
||||||
self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape")
|
self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape")
|
||||||
else:
|
else:
|
||||||
|
|
@ -168,112 +206,107 @@ class SphinxBuilder:
|
||||||
if not verbose:
|
if not verbose:
|
||||||
verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "")
|
verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "")
|
||||||
|
|
||||||
# Handle SPHINXOPTS evironment
|
|
||||||
sphinxopts = shlex.split(os.environ.get("SPHINXOPTS", ""))
|
|
||||||
|
|
||||||
# As we handle number of jobs and quiet in separate, we need to pick
|
|
||||||
# it the same way as sphinx-build would pick, so let's use argparse
|
|
||||||
# do to the right argument expansion
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument('-j', '--jobs', type=int)
|
|
||||||
parser.add_argument('-q', '--quiet', type=int)
|
|
||||||
|
|
||||||
# Other sphinx-build arguments go as-is, so place them
|
|
||||||
# at self.sphinxopts
|
|
||||||
sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts)
|
|
||||||
if sphinx_args.quiet == True:
|
|
||||||
self.verbose = False
|
|
||||||
|
|
||||||
if sphinx_args.jobs:
|
|
||||||
self.n_jobs = sphinx_args.jobs
|
|
||||||
|
|
||||||
# Command line arguments was passed, override SPHINXOPTS
|
|
||||||
if verbose is not None:
|
if verbose is not None:
|
||||||
self.verbose = verbose
|
self.verbose = verbose
|
||||||
|
|
||||||
self.n_jobs = n_jobs
|
#
|
||||||
|
|
||||||
# Source tree directory. This needs to be at os.environ, as
|
# Source tree directory. This needs to be at os.environ, as
|
||||||
# Sphinx extensions and media uAPI makefile needs it
|
# Sphinx extensions use it
|
||||||
|
#
|
||||||
self.srctree = os.environ.get("srctree")
|
self.srctree = os.environ.get("srctree")
|
||||||
if not self.srctree:
|
if not self.srctree:
|
||||||
self.srctree = "."
|
self.srctree = "."
|
||||||
os.environ["srctree"] = self.srctree
|
os.environ["srctree"] = self.srctree
|
||||||
|
|
||||||
|
#
|
||||||
# Now that we can expand srctree, get other directories as well
|
# Now that we can expand srctree, get other directories as well
|
||||||
|
#
|
||||||
self.sphinxbuild = os.environ.get("SPHINXBUILD", "sphinx-build")
|
self.sphinxbuild = os.environ.get("SPHINXBUILD", "sphinx-build")
|
||||||
self.kerneldoc = self.get_path(os.environ.get("KERNELDOC",
|
self.kerneldoc = self.get_path(os.environ.get("KERNELDOC",
|
||||||
"scripts/kernel-doc.py"))
|
"scripts/kernel-doc.py"))
|
||||||
self.obj = os.environ.get("obj", "Documentation")
|
self.builddir = self.get_path(builddir, use_cwd=True, abs_path=True)
|
||||||
self.builddir = self.get_path(os.path.join(self.obj, "output"),
|
|
||||||
abs_path=True)
|
|
||||||
|
|
||||||
# Media uAPI needs it
|
|
||||||
os.environ["BUILDDIR"] = self.builddir
|
|
||||||
|
|
||||||
# Detect if rust is enabled
|
|
||||||
self.config_rust = self.is_rust_enabled()
|
|
||||||
|
|
||||||
|
#
|
||||||
# Get directory locations for LaTeX build toolchain
|
# Get directory locations for LaTeX build toolchain
|
||||||
|
#
|
||||||
self.pdflatex_cmd = shutil.which(self.pdflatex)
|
self.pdflatex_cmd = shutil.which(self.pdflatex)
|
||||||
self.latexmk_cmd = shutil.which("latexmk")
|
self.latexmk_cmd = shutil.which("latexmk")
|
||||||
|
|
||||||
self.env = os.environ.copy()
|
self.env = os.environ.copy()
|
||||||
|
|
||||||
# If venv parameter is specified, run Sphinx from venv
|
self.get_sphinx_extra_opts(n_jobs)
|
||||||
|
|
||||||
|
#
|
||||||
|
# If venv command line argument is specified, run Sphinx from venv
|
||||||
|
#
|
||||||
if venv:
|
if venv:
|
||||||
bin_dir = os.path.join(venv, "bin")
|
bin_dir = os.path.join(venv, "bin")
|
||||||
if os.path.isfile(os.path.join(bin_dir, "activate")):
|
if not os.path.isfile(os.path.join(bin_dir, "activate")):
|
||||||
|
sys.exit(f"Venv {venv} not found.")
|
||||||
|
|
||||||
# "activate" virtual env
|
# "activate" virtual env
|
||||||
self.env["PATH"] = bin_dir + ":" + self.env["PATH"]
|
self.env["PATH"] = bin_dir + ":" + self.env["PATH"]
|
||||||
self.env["VIRTUAL_ENV"] = venv
|
self.env["VIRTUAL_ENV"] = venv
|
||||||
if "PYTHONHOME" in self.env:
|
if "PYTHONHOME" in self.env:
|
||||||
del self.env["PYTHONHOME"]
|
del self.env["PYTHONHOME"]
|
||||||
print(f"Setting venv to {venv}")
|
print(f"Setting venv to {venv}")
|
||||||
else:
|
|
||||||
sys.exit(f"Venv {venv} not found.")
|
|
||||||
|
|
||||||
def run_sphinx(self, sphinx_build, build_args, *args, **pwargs):
|
def run_sphinx(self, sphinx_build, build_args, *args, **pwargs):
|
||||||
"""
|
"""
|
||||||
Executes sphinx-build using current python3 command and setting
|
Executes sphinx-build using current python3 command.
|
||||||
-j parameter if possible to run the build in parallel.
|
|
||||||
|
When calling via GNU make, POSIX jobserver is used to tell how
|
||||||
|
many jobs are still available from a job pool. claim all remaining
|
||||||
|
jobs, as we don't want sphinx-build to run in parallel with other
|
||||||
|
jobs.
|
||||||
|
|
||||||
|
Despite that, the user may actually force a different value than
|
||||||
|
the number of available jobs via command line.
|
||||||
|
|
||||||
|
The "with" logic here is used to ensure that the claimed jobs will
|
||||||
|
be freed once subprocess finishes
|
||||||
"""
|
"""
|
||||||
|
|
||||||
with JobserverExec() as jobserver:
|
with JobserverExec() as jobserver:
|
||||||
if jobserver.claim:
|
if jobserver.claim:
|
||||||
|
#
|
||||||
|
# when GNU make is used, claim available jobs from jobserver
|
||||||
|
#
|
||||||
n_jobs = str(jobserver.claim)
|
n_jobs = str(jobserver.claim)
|
||||||
else:
|
else:
|
||||||
n_jobs = "auto" # Supported since Sphinx 1.7
|
#
|
||||||
|
# Otherwise, let sphinx decide by default
|
||||||
|
#
|
||||||
|
n_jobs = "auto"
|
||||||
|
|
||||||
cmd = []
|
#
|
||||||
|
# If explicitly requested via command line, override default
|
||||||
if self.venv:
|
#
|
||||||
cmd.append("python")
|
|
||||||
else:
|
|
||||||
cmd.append(sys.executable)
|
|
||||||
|
|
||||||
cmd.append(sphinx_build)
|
|
||||||
|
|
||||||
# if present, SPHINXOPTS or command line --jobs overrides default
|
|
||||||
if self.n_jobs:
|
if self.n_jobs:
|
||||||
n_jobs = str(self.n_jobs)
|
n_jobs = str(self.n_jobs)
|
||||||
|
|
||||||
if n_jobs:
|
#
|
||||||
|
# We can't simply call python3 sphinx-build, as OpenSUSE
|
||||||
|
# Tumbleweed uses an ELF binary file (/usr/bin/alts) to switch
|
||||||
|
# between different versions of sphinx-build. So, only call it
|
||||||
|
# prepending "python3.xx" when PYTHON3 variable is not default.
|
||||||
|
#
|
||||||
|
if self.python:
|
||||||
|
cmd = [self.python]
|
||||||
|
else:
|
||||||
|
cmd = []
|
||||||
|
|
||||||
|
cmd += [sphinx_build]
|
||||||
cmd += [f"-j{n_jobs}"]
|
cmd += [f"-j{n_jobs}"]
|
||||||
|
|
||||||
if not self.verbose:
|
|
||||||
cmd.append("-q")
|
|
||||||
|
|
||||||
cmd += self.sphinxopts
|
cmd += self.sphinxopts
|
||||||
|
|
||||||
cmd += build_args
|
cmd += build_args
|
||||||
|
|
||||||
if self.verbose:
|
if self.verbose:
|
||||||
print(" ".join(cmd))
|
print(" ".join(cmd))
|
||||||
|
|
||||||
rc = subprocess.call(cmd, *args, **pwargs)
|
return subprocess.call(cmd, *args, **pwargs)
|
||||||
|
|
||||||
def handle_html(self, css, output_dir):
|
def handle_html(self, css, output_dir, rustdoc):
|
||||||
"""
|
"""
|
||||||
Extra steps for HTML and epub output.
|
Extra steps for HTML and epub output.
|
||||||
|
|
||||||
|
|
@ -281,9 +314,7 @@ class SphinxBuilder:
|
||||||
copied to the output _static directory
|
copied to the output _static directory
|
||||||
"""
|
"""
|
||||||
|
|
||||||
if not css:
|
if css:
|
||||||
return
|
|
||||||
|
|
||||||
css = os.path.expanduser(css)
|
css = os.path.expanduser(css)
|
||||||
if not css.startswith("/"):
|
if not css.startswith("/"):
|
||||||
css = os.path.join(self.srctree, css)
|
css = os.path.join(self.srctree, css)
|
||||||
|
|
@ -296,17 +327,30 @@ class SphinxBuilder:
|
||||||
except (OSError, IOError) as e:
|
except (OSError, IOError) as e:
|
||||||
print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr)
|
print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr)
|
||||||
|
|
||||||
|
if rustdoc:
|
||||||
|
if "MAKE" in self.env:
|
||||||
|
cmd = [self.env["MAKE"]]
|
||||||
|
else:
|
||||||
|
cmd = ["make", "LLVM=1"]
|
||||||
|
|
||||||
|
cmd += [ "rustdoc"]
|
||||||
|
if self.verbose:
|
||||||
|
print(" ".join(cmd))
|
||||||
|
|
||||||
|
try:
|
||||||
|
subprocess.run(cmd, check=True)
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
print(f"Ignored errors when building rustdoc: {e}. Is RUST enabled?",
|
||||||
|
file=sys.stderr)
|
||||||
|
|
||||||
def build_pdf_file(self, latex_cmd, from_dir, path):
|
def build_pdf_file(self, latex_cmd, from_dir, path):
|
||||||
"""Builds a single pdf file using latex_cmd"""
|
"""Builds a single pdf file using latex_cmd"""
|
||||||
try:
|
try:
|
||||||
subprocess.run(latex_cmd + [path],
|
subprocess.run(latex_cmd + [path],
|
||||||
cwd=from_dir, check=True)
|
cwd=from_dir, check=True, env=self.env)
|
||||||
|
|
||||||
return True
|
return True
|
||||||
except subprocess.CalledProcessError:
|
except subprocess.CalledProcessError:
|
||||||
# LaTeX PDF error code is almost useless: it returns
|
|
||||||
# error codes even when build succeeds but has warnings.
|
|
||||||
# So, we'll ignore the results
|
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def pdf_parallel_build(self, tex_suffix, latex_cmd, tex_files, n_jobs):
|
def pdf_parallel_build(self, tex_suffix, latex_cmd, tex_files, n_jobs):
|
||||||
|
|
@ -316,7 +360,14 @@ class SphinxBuilder:
|
||||||
max_len = 0
|
max_len = 0
|
||||||
has_tex = False
|
has_tex = False
|
||||||
|
|
||||||
# Process files in parallel
|
#
|
||||||
|
# LaTeX PDF error code is almost useless for us:
|
||||||
|
# any warning makes it non-zero. For kernel doc builds it always return
|
||||||
|
# non-zero even when build succeeds. So, let's do the best next thing:
|
||||||
|
# Ignore build errors. At the end, check if all PDF files were built,
|
||||||
|
# printing a summary with the built ones and returning 0 if all of
|
||||||
|
# them were actually built.
|
||||||
|
#
|
||||||
with futures.ThreadPoolExecutor(max_workers=n_jobs) as executor:
|
with futures.ThreadPoolExecutor(max_workers=n_jobs) as executor:
|
||||||
jobs = {}
|
jobs = {}
|
||||||
|
|
||||||
|
|
@ -327,46 +378,51 @@ class SphinxBuilder:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
name = name[:-len(tex_suffix)]
|
name = name[:-len(tex_suffix)]
|
||||||
|
|
||||||
max_len = max(max_len, len(name))
|
|
||||||
|
|
||||||
has_tex = True
|
has_tex = True
|
||||||
|
|
||||||
future = executor.submit(self.build_pdf_file, latex_cmd,
|
future = executor.submit(self.build_pdf_file, latex_cmd,
|
||||||
from_dir, entry.path)
|
from_dir, entry.path)
|
||||||
jobs[future] = (from_dir, name, entry.path)
|
jobs[future] = (from_dir, pdf_dir, name)
|
||||||
|
|
||||||
for future in futures.as_completed(jobs):
|
for future in futures.as_completed(jobs):
|
||||||
from_dir, name, path = jobs[future]
|
from_dir, pdf_dir, name = jobs[future]
|
||||||
|
|
||||||
pdf_name = name + ".pdf"
|
pdf_name = name + ".pdf"
|
||||||
pdf_from = os.path.join(from_dir, pdf_name)
|
pdf_from = os.path.join(from_dir, pdf_name)
|
||||||
|
pdf_to = os.path.join(pdf_dir, pdf_name)
|
||||||
|
out_name = os.path.relpath(pdf_to, self.builddir)
|
||||||
|
max_len = max(max_len, len(out_name))
|
||||||
|
|
||||||
try:
|
try:
|
||||||
success = future.result()
|
success = future.result()
|
||||||
|
|
||||||
if success and os.path.exists(pdf_from):
|
if success and os.path.exists(pdf_from):
|
||||||
pdf_to = os.path.join(pdf_dir, pdf_name)
|
|
||||||
|
|
||||||
os.rename(pdf_from, pdf_to)
|
os.rename(pdf_from, pdf_to)
|
||||||
builds[name] = os.path.relpath(pdf_to, self.builddir)
|
|
||||||
|
#
|
||||||
|
# if verbose, get the name of built PDF file
|
||||||
|
#
|
||||||
|
if self.verbose:
|
||||||
|
builds[out_name] = "SUCCESS"
|
||||||
else:
|
else:
|
||||||
builds[name] = "FAILED"
|
builds[out_name] = "FAILED"
|
||||||
build_failed = True
|
build_failed = True
|
||||||
except Exception as e:
|
except futures.Error as e:
|
||||||
builds[name] = f"FAILED ({str(e)})"
|
builds[out_name] = f"FAILED ({repr(e)})"
|
||||||
build_failed = True
|
build_failed = True
|
||||||
|
|
||||||
|
#
|
||||||
# Handle case where no .tex files were found
|
# Handle case where no .tex files were found
|
||||||
|
#
|
||||||
if not has_tex:
|
if not has_tex:
|
||||||
name = "Sphinx LaTeX builder"
|
out_name = "LaTeX files"
|
||||||
max_len = max(max_len, len(name))
|
max_len = max(max_len, len(out_name))
|
||||||
builds[name] = "FAILED (no .tex file was generated)"
|
builds[out_name] = "FAILED: no .tex files were generated"
|
||||||
build_failed = True
|
build_failed = True
|
||||||
|
|
||||||
return builds, build_failed, max_len
|
return builds, build_failed, max_len
|
||||||
|
|
||||||
def handle_pdf(self, output_dirs):
|
def handle_pdf(self, output_dirs, deny_vf):
|
||||||
"""
|
"""
|
||||||
Extra steps for PDF output.
|
Extra steps for PDF output.
|
||||||
|
|
||||||
|
|
@ -377,9 +433,22 @@ class SphinxBuilder:
|
||||||
builds = {}
|
builds = {}
|
||||||
max_len = 0
|
max_len = 0
|
||||||
tex_suffix = ".tex"
|
tex_suffix = ".tex"
|
||||||
|
|
||||||
# Get all tex files that will be used for PDF build
|
|
||||||
tex_files = []
|
tex_files = []
|
||||||
|
|
||||||
|
#
|
||||||
|
# Since early 2024, Fedora and openSUSE tumbleweed have started
|
||||||
|
# deploying variable-font format of "Noto CJK", causing LaTeX
|
||||||
|
# to break with CJK. Work around it, by denying the variable font
|
||||||
|
# usage during xelatex build by passing the location of a config
|
||||||
|
# file with a deny list.
|
||||||
|
#
|
||||||
|
# See tools/docs/lib/latex_fonts.py for more details.
|
||||||
|
#
|
||||||
|
if deny_vf:
|
||||||
|
deny_vf = os.path.expanduser(deny_vf)
|
||||||
|
if os.path.isdir(deny_vf):
|
||||||
|
self.env["XDG_CONFIG_HOME"] = deny_vf
|
||||||
|
|
||||||
for from_dir in output_dirs:
|
for from_dir in output_dirs:
|
||||||
pdf_dir = os.path.join(from_dir, "../pdf")
|
pdf_dir = os.path.join(from_dir, "../pdf")
|
||||||
os.makedirs(pdf_dir, exist_ok=True)
|
os.makedirs(pdf_dir, exist_ok=True)
|
||||||
|
|
@ -397,10 +466,12 @@ class SphinxBuilder:
|
||||||
if entry.name.endswith(tex_suffix):
|
if entry.name.endswith(tex_suffix):
|
||||||
tex_files.append((from_dir, pdf_dir, entry))
|
tex_files.append((from_dir, pdf_dir, entry))
|
||||||
|
|
||||||
|
#
|
||||||
# When using make, this won't be used, as the number of jobs comes
|
# When using make, this won't be used, as the number of jobs comes
|
||||||
# from POSIX jobserver. So, this covers the case where build comes
|
# from POSIX jobserver. So, this covers the case where build comes
|
||||||
# from command line. On such case, serialize by default, except if
|
# from command line. On such case, serialize by default, except if
|
||||||
# the user explicitly sets the number of jobs.
|
# the user explicitly sets the number of jobs.
|
||||||
|
#
|
||||||
n_jobs = 1
|
n_jobs = 1
|
||||||
|
|
||||||
# n_jobs is either an integer or "auto". Only use it if it is a number
|
# n_jobs is either an integer or "auto". Only use it if it is a number
|
||||||
|
|
@ -410,13 +481,17 @@ class SphinxBuilder:
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
#
|
||||||
# When using make, jobserver.claim is the number of jobs that were
|
# When using make, jobserver.claim is the number of jobs that were
|
||||||
# used with "-j" and that aren't used by other make targets
|
# used with "-j" and that aren't used by other make targets
|
||||||
|
#
|
||||||
with JobserverExec() as jobserver:
|
with JobserverExec() as jobserver:
|
||||||
n_jobs = 1
|
n_jobs = 1
|
||||||
|
|
||||||
|
#
|
||||||
# Handle the case when a parameter is passed via command line,
|
# Handle the case when a parameter is passed via command line,
|
||||||
# using it as default, if jobserver doesn't claim anything
|
# using it as default, if jobserver doesn't claim anything
|
||||||
|
#
|
||||||
if self.n_jobs:
|
if self.n_jobs:
|
||||||
try:
|
try:
|
||||||
n_jobs = int(self.n_jobs)
|
n_jobs = int(self.n_jobs)
|
||||||
|
|
@ -426,12 +501,17 @@ class SphinxBuilder:
|
||||||
if jobserver.claim:
|
if jobserver.claim:
|
||||||
n_jobs = jobserver.claim
|
n_jobs = jobserver.claim
|
||||||
|
|
||||||
# Build files in parallel
|
|
||||||
builds, build_failed, max_len = self.pdf_parallel_build(tex_suffix,
|
builds, build_failed, max_len = self.pdf_parallel_build(tex_suffix,
|
||||||
latex_cmd,
|
latex_cmd,
|
||||||
tex_files,
|
tex_files,
|
||||||
n_jobs)
|
n_jobs)
|
||||||
|
|
||||||
|
#
|
||||||
|
# In verbose mode, print a summary with the build results per file.
|
||||||
|
# Otherwise, print a single line with all failures, if any.
|
||||||
|
# On both cases, return code 1 indicates build failures,
|
||||||
|
#
|
||||||
|
if self.verbose:
|
||||||
msg = "Summary"
|
msg = "Summary"
|
||||||
msg += "\n" + "=" * len(msg)
|
msg += "\n" + "=" * len(msg)
|
||||||
print()
|
print()
|
||||||
|
|
@ -441,13 +521,22 @@ class SphinxBuilder:
|
||||||
print(f"{pdf_name:<{max_len}}: {pdf_file}")
|
print(f"{pdf_name:<{max_len}}: {pdf_file}")
|
||||||
|
|
||||||
print()
|
print()
|
||||||
|
|
||||||
# return an error if a PDF file is missing
|
|
||||||
|
|
||||||
if build_failed:
|
if build_failed:
|
||||||
sys.exit(f"PDF build failed: not all PDF files were created.")
|
msg = LatexFontChecker().check()
|
||||||
else:
|
if msg:
|
||||||
print("All PDF files were built.")
|
print(msg)
|
||||||
|
|
||||||
|
sys.exit("Error: not all PDF files were created.")
|
||||||
|
|
||||||
|
elif build_failed:
|
||||||
|
n_failures = len(builds)
|
||||||
|
failures = ", ".join(builds.keys())
|
||||||
|
|
||||||
|
msg = LatexFontChecker().check()
|
||||||
|
if msg:
|
||||||
|
print(msg)
|
||||||
|
|
||||||
|
sys.exit(f"Error: Can't build {n_failures} PDF file(s): {failures}")
|
||||||
|
|
||||||
def handle_info(self, output_dirs):
|
def handle_info(self, output_dirs):
|
||||||
"""
|
"""
|
||||||
|
|
@ -463,12 +552,78 @@ class SphinxBuilder:
|
||||||
except subprocess.CalledProcessError as e:
|
except subprocess.CalledProcessError as e:
|
||||||
sys.exit(f"Error generating info docs: {e}")
|
sys.exit(f"Error generating info docs: {e}")
|
||||||
|
|
||||||
def cleandocs(self, builder):
|
def handle_man(self, kerneldoc, docs_dir, src_dir, output_dir):
|
||||||
|
"""
|
||||||
|
Create man pages from kernel-doc output
|
||||||
|
"""
|
||||||
|
|
||||||
|
re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)")
|
||||||
|
re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"')
|
||||||
|
|
||||||
|
if docs_dir == src_dir:
|
||||||
|
#
|
||||||
|
# Pick the entire set of kernel-doc markups from the entire tree
|
||||||
|
#
|
||||||
|
kdoc_files = set([self.srctree])
|
||||||
|
else:
|
||||||
|
kdoc_files = set()
|
||||||
|
|
||||||
|
for fname in glob(os.path.join(src_dir, "**"), recursive=True):
|
||||||
|
if os.path.isfile(fname) and fname.endswith(".rst"):
|
||||||
|
with open(fname, "r", encoding="utf-8") as in_fp:
|
||||||
|
data = in_fp.read()
|
||||||
|
|
||||||
|
for line in data.split("\n"):
|
||||||
|
match = re_kernel_doc.match(line)
|
||||||
|
if match:
|
||||||
|
if os.path.isfile(match.group(1)):
|
||||||
|
kdoc_files.add(match.group(1))
|
||||||
|
|
||||||
|
if not kdoc_files:
|
||||||
|
sys.exit(f"Directory {src_dir} doesn't contain kernel-doc tags")
|
||||||
|
|
||||||
|
cmd = [ kerneldoc, "-m" ] + sorted(kdoc_files)
|
||||||
|
try:
|
||||||
|
if self.verbose:
|
||||||
|
print(" ".join(cmd))
|
||||||
|
|
||||||
|
result = subprocess.run(cmd, stdout=subprocess.PIPE, text= True)
|
||||||
|
|
||||||
|
if result.returncode:
|
||||||
|
print(f"Warning: kernel-doc returned {result.returncode} warnings")
|
||||||
|
|
||||||
|
except (OSError, ValueError, subprocess.SubprocessError) as e:
|
||||||
|
sys.exit(f"Failed to create man pages for {src_dir}: {repr(e)}")
|
||||||
|
|
||||||
|
fp = None
|
||||||
|
try:
|
||||||
|
for line in result.stdout.split("\n"):
|
||||||
|
match = re_man.match(line)
|
||||||
|
if not match:
|
||||||
|
if fp:
|
||||||
|
fp.write(line + '\n')
|
||||||
|
continue
|
||||||
|
|
||||||
|
if fp:
|
||||||
|
fp.close()
|
||||||
|
|
||||||
|
fname = f"{output_dir}/{match.group(2)}.{match.group(1)}"
|
||||||
|
|
||||||
|
if self.verbose:
|
||||||
|
print(f"Creating {fname}")
|
||||||
|
fp = open(fname, "w", encoding="utf-8")
|
||||||
|
fp.write(line + '\n')
|
||||||
|
finally:
|
||||||
|
if fp:
|
||||||
|
fp.close()
|
||||||
|
|
||||||
|
def cleandocs(self, builder): # pylint: disable=W0613
|
||||||
|
"""Remove documentation output directory"""
|
||||||
shutil.rmtree(self.builddir, ignore_errors=True)
|
shutil.rmtree(self.builddir, ignore_errors=True)
|
||||||
|
|
||||||
def build(self, target, sphinxdirs=None, conf="conf.py",
|
def build(self, target, sphinxdirs=None,
|
||||||
theme=None, css=None, paper=None):
|
theme=None, css=None, paper=None, deny_vf=None, rustdoc=False,
|
||||||
|
skip_sphinx=False):
|
||||||
"""
|
"""
|
||||||
Build documentation using Sphinx. This is the core function of this
|
Build documentation using Sphinx. This is the core function of this
|
||||||
module. It prepares all arguments required by sphinx-build.
|
module. It prepares all arguments required by sphinx-build.
|
||||||
|
|
@ -477,32 +632,38 @@ class SphinxBuilder:
|
||||||
builder = TARGETS[target]["builder"]
|
builder = TARGETS[target]["builder"]
|
||||||
out_dir = TARGETS[target].get("out_dir", "")
|
out_dir = TARGETS[target].get("out_dir", "")
|
||||||
|
|
||||||
|
#
|
||||||
# Cleandocs doesn't require sphinx-build
|
# Cleandocs doesn't require sphinx-build
|
||||||
|
#
|
||||||
if target == "cleandocs":
|
if target == "cleandocs":
|
||||||
self.cleandocs(builder)
|
self.cleandocs(builder)
|
||||||
return
|
return
|
||||||
|
|
||||||
# Other targets require sphinx-build
|
if theme:
|
||||||
|
os.environ["DOCS_THEME"] = theme
|
||||||
|
|
||||||
|
#
|
||||||
|
# Other targets require sphinx-build, so check if it exists
|
||||||
|
#
|
||||||
|
if not skip_sphinx:
|
||||||
sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"])
|
sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"])
|
||||||
if not sphinxbuild:
|
if not sphinxbuild and target != "mandocs":
|
||||||
sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n")
|
sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n")
|
||||||
|
|
||||||
if builder == "latex":
|
if target == "pdfdocs":
|
||||||
if not self.pdflatex_cmd and not self.latexmk_cmd:
|
if not self.pdflatex_cmd and not self.latexmk_cmd:
|
||||||
sys.exit("Error: pdflatex or latexmk required for PDF generation")
|
sys.exit("Error: pdflatex or latexmk required for PDF generation")
|
||||||
|
|
||||||
docs_dir = os.path.abspath(os.path.join(self.srctree, "Documentation"))
|
docs_dir = os.path.abspath(os.path.join(self.srctree, "Documentation"))
|
||||||
|
|
||||||
# Prepare base arguments for Sphinx build
|
#
|
||||||
|
# Fill in base arguments for Sphinx build
|
||||||
|
#
|
||||||
kerneldoc = self.kerneldoc
|
kerneldoc = self.kerneldoc
|
||||||
if kerneldoc.startswith(self.srctree):
|
if kerneldoc.startswith(self.srctree):
|
||||||
kerneldoc = os.path.relpath(kerneldoc, self.srctree)
|
kerneldoc = os.path.relpath(kerneldoc, self.srctree)
|
||||||
|
|
||||||
# Prepare common Sphinx options
|
args = [ "-b", builder, "-c", docs_dir ]
|
||||||
args = [
|
|
||||||
"-b", builder,
|
|
||||||
"-c", docs_dir,
|
|
||||||
]
|
|
||||||
|
|
||||||
if builder == "latex":
|
if builder == "latex":
|
||||||
if not paper:
|
if not paper:
|
||||||
|
|
@ -510,41 +671,48 @@ class SphinxBuilder:
|
||||||
|
|
||||||
args.extend(["-D", f"latex_elements.papersize={paper}paper"])
|
args.extend(["-D", f"latex_elements.papersize={paper}paper"])
|
||||||
|
|
||||||
if self.config_rust:
|
if rustdoc:
|
||||||
args.extend(["-t", "rustdoc"])
|
args.extend(["-t", "rustdoc"])
|
||||||
|
|
||||||
if conf:
|
|
||||||
self.env["SPHINX_CONF"] = self.get_path(conf, abs_path=True)
|
|
||||||
|
|
||||||
if not sphinxdirs:
|
if not sphinxdirs:
|
||||||
sphinxdirs = os.environ.get("SPHINXDIRS", ".")
|
sphinxdirs = os.environ.get("SPHINXDIRS", ".")
|
||||||
|
|
||||||
|
#
|
||||||
# The sphinx-build tool has a bug: internally, it tries to set
|
# The sphinx-build tool has a bug: internally, it tries to set
|
||||||
# locale with locale.setlocale(locale.LC_ALL, ''). This causes a
|
# locale with locale.setlocale(locale.LC_ALL, ''). This causes a
|
||||||
# crash if language is not set. Detect and fix it.
|
# crash if language is not set. Detect and fix it.
|
||||||
|
#
|
||||||
try:
|
try:
|
||||||
locale.setlocale(locale.LC_ALL, '')
|
locale.setlocale(locale.LC_ALL, '')
|
||||||
except Exception:
|
except locale.Error:
|
||||||
self.env["LC_ALL"] = "C"
|
self.env["LC_ALL"] = "C"
|
||||||
self.env["LANG"] = "C"
|
|
||||||
|
|
||||||
|
#
|
||||||
# sphinxdirs can be a list or a whitespace-separated string
|
# sphinxdirs can be a list or a whitespace-separated string
|
||||||
|
#
|
||||||
sphinxdirs_list = []
|
sphinxdirs_list = []
|
||||||
for sphinxdir in sphinxdirs:
|
for sphinxdir in sphinxdirs:
|
||||||
if isinstance(sphinxdir, list):
|
if isinstance(sphinxdir, list):
|
||||||
sphinxdirs_list += sphinxdir
|
sphinxdirs_list += sphinxdir
|
||||||
else:
|
else:
|
||||||
for name in sphinxdir.split(" "):
|
sphinxdirs_list += sphinxdir.split()
|
||||||
sphinxdirs_list.append(name)
|
|
||||||
|
|
||||||
# Build each directory
|
#
|
||||||
|
# Step 1: Build each directory in separate.
|
||||||
|
#
|
||||||
|
# This is not the best way of handling it, as cross-references between
|
||||||
|
# them will be broken, but this is what we've been doing since
|
||||||
|
# the beginning.
|
||||||
|
#
|
||||||
output_dirs = []
|
output_dirs = []
|
||||||
for sphinxdir in sphinxdirs_list:
|
for sphinxdir in sphinxdirs_list:
|
||||||
src_dir = os.path.join(docs_dir, sphinxdir)
|
src_dir = os.path.join(docs_dir, sphinxdir)
|
||||||
doctree_dir = os.path.join(self.builddir, ".doctrees")
|
doctree_dir = os.path.join(self.builddir, ".doctrees")
|
||||||
output_dir = os.path.join(self.builddir, sphinxdir, out_dir)
|
output_dir = os.path.join(self.builddir, sphinxdir, out_dir)
|
||||||
|
|
||||||
|
#
|
||||||
# Make directory names canonical
|
# Make directory names canonical
|
||||||
|
#
|
||||||
src_dir = os.path.normpath(src_dir)
|
src_dir = os.path.normpath(src_dir)
|
||||||
doctree_dir = os.path.normpath(doctree_dir)
|
doctree_dir = os.path.normpath(doctree_dir)
|
||||||
output_dir = os.path.normpath(output_dir)
|
output_dir = os.path.normpath(output_dir)
|
||||||
|
|
@ -564,93 +732,34 @@ class SphinxBuilder:
|
||||||
output_dir,
|
output_dir,
|
||||||
]
|
]
|
||||||
|
|
||||||
# Execute sphinx-build
|
if target == "mandocs":
|
||||||
|
self.handle_man(kerneldoc, docs_dir, src_dir, output_dir)
|
||||||
|
elif not skip_sphinx:
|
||||||
try:
|
try:
|
||||||
self.run_sphinx(sphinxbuild, build_args, env=self.env)
|
result = self.run_sphinx(sphinxbuild, build_args,
|
||||||
except Exception as e:
|
env=self.env)
|
||||||
sys.exit(f"Build failed: {e}")
|
|
||||||
|
|
||||||
# Ensure that html/epub will have needed static files
|
if result:
|
||||||
|
sys.exit(f"Build failed: return code: {result}")
|
||||||
|
|
||||||
|
except (OSError, ValueError, subprocess.SubprocessError) as e:
|
||||||
|
sys.exit(f"Build failed: {repr(e)}")
|
||||||
|
|
||||||
|
#
|
||||||
|
# Ensure that each html/epub output will have needed static files
|
||||||
|
#
|
||||||
if target in ["htmldocs", "epubdocs"]:
|
if target in ["htmldocs", "epubdocs"]:
|
||||||
self.handle_html(css, output_dir)
|
self.handle_html(css, output_dir, rustdoc)
|
||||||
|
|
||||||
# PDF and Info require a second build step
|
#
|
||||||
|
# Step 2: Some targets (PDF and info) require an extra step once
|
||||||
|
# sphinx-build finishes
|
||||||
|
#
|
||||||
if target == "pdfdocs":
|
if target == "pdfdocs":
|
||||||
self.handle_pdf(output_dirs)
|
self.handle_pdf(output_dirs, deny_vf)
|
||||||
elif target == "infodocs":
|
elif target == "infodocs":
|
||||||
self.handle_info(output_dirs)
|
self.handle_info(output_dirs)
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def get_python_version(cmd):
|
|
||||||
"""
|
|
||||||
Get python version from a Python binary. As we need to detect if
|
|
||||||
are out there newer python binaries, we can't rely on sys.release here.
|
|
||||||
"""
|
|
||||||
|
|
||||||
result = subprocess.run([cmd, "--version"], check=True,
|
|
||||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
|
||||||
universal_newlines=True)
|
|
||||||
version = result.stdout.strip()
|
|
||||||
|
|
||||||
match = re.search(r"(\d+\.\d+\.\d+)", version)
|
|
||||||
if match:
|
|
||||||
return parse_version(match.group(1))
|
|
||||||
|
|
||||||
print(f"Can't parse version {version}")
|
|
||||||
return (0, 0, 0)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def find_python():
|
|
||||||
"""
|
|
||||||
Detect if are out there any python 3.xy version newer than the
|
|
||||||
current one.
|
|
||||||
|
|
||||||
Note: this routine is limited to up to 2 digits for python3. We
|
|
||||||
may need to update it one day, hopefully on a distant future.
|
|
||||||
"""
|
|
||||||
patterns = [
|
|
||||||
"python3.[0-9]",
|
|
||||||
"python3.[0-9][0-9]",
|
|
||||||
]
|
|
||||||
|
|
||||||
# Seek for a python binary newer than MIN_PYTHON_VERSION
|
|
||||||
for path in os.getenv("PATH", "").split(":"):
|
|
||||||
for pattern in patterns:
|
|
||||||
for cmd in glob(os.path.join(path, pattern)):
|
|
||||||
if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
|
|
||||||
version = SphinxBuilder.get_python_version(cmd)
|
|
||||||
if version >= MIN_PYTHON_VERSION:
|
|
||||||
return cmd
|
|
||||||
|
|
||||||
return None
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def check_python():
|
|
||||||
"""
|
|
||||||
Check if the current python binary satisfies our minimal requirement
|
|
||||||
for Sphinx build. If not, re-run with a newer version if found.
|
|
||||||
"""
|
|
||||||
cur_ver = sys.version_info[:3]
|
|
||||||
if cur_ver >= MIN_PYTHON_VERSION:
|
|
||||||
return
|
|
||||||
|
|
||||||
python_ver = ver_str(cur_ver)
|
|
||||||
|
|
||||||
new_python_cmd = SphinxBuilder.find_python()
|
|
||||||
if not new_python_cmd:
|
|
||||||
sys.exit(f"Python version {python_ver} is not supported anymore.")
|
|
||||||
|
|
||||||
# Restart script using the newer version
|
|
||||||
script_path = os.path.abspath(sys.argv[0])
|
|
||||||
args = [new_python_cmd, script_path] + sys.argv[1:]
|
|
||||||
|
|
||||||
print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
os.execv(new_python_cmd, args)
|
|
||||||
except OSError as e:
|
|
||||||
sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
|
|
||||||
|
|
||||||
def jobs_type(value):
|
def jobs_type(value):
|
||||||
"""
|
"""
|
||||||
Handle valid values for -j. Accepts Sphinx "-jauto", plus a number
|
Handle valid values for -j. Accepts Sphinx "-jauto", plus a number
|
||||||
|
|
@ -668,7 +777,7 @@ def jobs_type(value):
|
||||||
|
|
||||||
raise argparse.ArgumentTypeError(f"Minimum jobs is 1, got {value}")
|
raise argparse.ArgumentTypeError(f"Minimum jobs is 1, got {value}")
|
||||||
except ValueError:
|
except ValueError:
|
||||||
raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}")
|
raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}") # pylint: disable=W0707
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
"""
|
"""
|
||||||
|
|
@ -682,7 +791,7 @@ def main():
|
||||||
help="Documentation target to build")
|
help="Documentation target to build")
|
||||||
parser.add_argument("--sphinxdirs", nargs="+",
|
parser.add_argument("--sphinxdirs", nargs="+",
|
||||||
help="Specific directories to build")
|
help="Specific directories to build")
|
||||||
parser.add_argument("--conf", default="conf.py",
|
parser.add_argument("--builddir", default="output",
|
||||||
help="Sphinx configuration file")
|
help="Sphinx configuration file")
|
||||||
|
|
||||||
parser.add_argument("--theme", help="Sphinx theme to use")
|
parser.add_argument("--theme", help="Sphinx theme to use")
|
||||||
|
|
@ -692,6 +801,12 @@ def main():
|
||||||
parser.add_argument("--paper", choices=PAPER, default=PAPER[0],
|
parser.add_argument("--paper", choices=PAPER, default=PAPER[0],
|
||||||
help="Paper size for LaTeX/PDF output")
|
help="Paper size for LaTeX/PDF output")
|
||||||
|
|
||||||
|
parser.add_argument('--deny-vf',
|
||||||
|
help="Configuration to deny variable fonts on pdf builds")
|
||||||
|
|
||||||
|
parser.add_argument('--rustdoc', action="store_true",
|
||||||
|
help="Enable rustdoc build. Requires CONFIG_RUST")
|
||||||
|
|
||||||
parser.add_argument("-v", "--verbose", action='store_true',
|
parser.add_argument("-v", "--verbose", action='store_true',
|
||||||
help="place build in verbose mode")
|
help="place build in verbose mode")
|
||||||
|
|
||||||
|
|
@ -701,19 +816,26 @@ def main():
|
||||||
parser.add_argument('-i', '--interactive', action='store_true',
|
parser.add_argument('-i', '--interactive', action='store_true',
|
||||||
help="Change latex default to run in interactive mode")
|
help="Change latex default to run in interactive mode")
|
||||||
|
|
||||||
|
parser.add_argument('-s', '--skip-sphinx-build', action='store_true',
|
||||||
|
help="Skip sphinx-build step")
|
||||||
|
|
||||||
parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}',
|
parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}',
|
||||||
default=None,
|
default=None,
|
||||||
help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})')
|
help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})')
|
||||||
|
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
SphinxBuilder.check_python()
|
PythonVersion.check_python(MIN_PYTHON_VERSION, show_alternatives=True,
|
||||||
|
bail_out=True)
|
||||||
|
|
||||||
builder = SphinxBuilder(venv=args.venv, verbose=args.verbose,
|
builder = SphinxBuilder(builddir=args.builddir, venv=args.venv,
|
||||||
n_jobs=args.jobs, interactive=args.interactive)
|
verbose=args.verbose, n_jobs=args.jobs,
|
||||||
|
interactive=args.interactive)
|
||||||
|
|
||||||
builder.build(args.target, sphinxdirs=args.sphinxdirs, conf=args.conf,
|
builder.build(args.target, sphinxdirs=args.sphinxdirs,
|
||||||
theme=args.theme, css=args.css, paper=args.paper)
|
theme=args.theme, css=args.css, paper=args.paper,
|
||||||
|
rustdoc=args.rustdoc, deny_vf=args.deny_vf,
|
||||||
|
skip_sphinx=args.skip_sphinx_build)
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
main()
|
||||||
|
|
@ -26,26 +26,17 @@ system pacage install is recommended.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import locale
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
from glob import glob
|
from glob import glob
|
||||||
|
|
||||||
|
from lib.python_version import PythonVersion
|
||||||
|
|
||||||
def parse_version(version):
|
RECOMMENDED_VERSION = PythonVersion("3.4.3").version
|
||||||
"""Convert a major.minor.patch version into a tuple"""
|
MIN_PYTHON_VERSION = PythonVersion("3.7").version
|
||||||
return tuple(int(x) for x in version.split("."))
|
|
||||||
|
|
||||||
|
|
||||||
def ver_str(version):
|
|
||||||
"""Returns a version tuple as major.minor.patch"""
|
|
||||||
|
|
||||||
return ".".join([str(x) for x in version])
|
|
||||||
|
|
||||||
|
|
||||||
RECOMMENDED_VERSION = parse_version("3.4.3")
|
|
||||||
MIN_PYTHON_VERSION = parse_version("3.7")
|
|
||||||
|
|
||||||
|
|
||||||
class DepManager:
|
class DepManager:
|
||||||
|
|
@ -235,95 +226,11 @@ class AncillaryMethods:
|
||||||
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def get_python_version(cmd):
|
|
||||||
"""
|
|
||||||
Get python version from a Python binary. As we need to detect if
|
|
||||||
are out there newer python binaries, we can't rely on sys.release here.
|
|
||||||
"""
|
|
||||||
|
|
||||||
result = SphinxDependencyChecker.run([cmd, "--version"],
|
|
||||||
capture_output=True, text=True)
|
|
||||||
version = result.stdout.strip()
|
|
||||||
|
|
||||||
match = re.search(r"(\d+\.\d+\.\d+)", version)
|
|
||||||
if match:
|
|
||||||
return parse_version(match.group(1))
|
|
||||||
|
|
||||||
print(f"Can't parse version {version}")
|
|
||||||
return (0, 0, 0)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def find_python():
|
|
||||||
"""
|
|
||||||
Detect if are out there any python 3.xy version newer than the
|
|
||||||
current one.
|
|
||||||
|
|
||||||
Note: this routine is limited to up to 2 digits for python3. We
|
|
||||||
may need to update it one day, hopefully on a distant future.
|
|
||||||
"""
|
|
||||||
patterns = [
|
|
||||||
"python3.[0-9]",
|
|
||||||
"python3.[0-9][0-9]",
|
|
||||||
]
|
|
||||||
|
|
||||||
# Seek for a python binary newer than MIN_PYTHON_VERSION
|
|
||||||
for path in os.getenv("PATH", "").split(":"):
|
|
||||||
for pattern in patterns:
|
|
||||||
for cmd in glob(os.path.join(path, pattern)):
|
|
||||||
if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
|
|
||||||
version = SphinxDependencyChecker.get_python_version(cmd)
|
|
||||||
if version >= MIN_PYTHON_VERSION:
|
|
||||||
return cmd
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def check_python():
|
|
||||||
"""
|
|
||||||
Check if the current python binary satisfies our minimal requirement
|
|
||||||
for Sphinx build. If not, re-run with a newer version if found.
|
|
||||||
"""
|
|
||||||
cur_ver = sys.version_info[:3]
|
|
||||||
if cur_ver >= MIN_PYTHON_VERSION:
|
|
||||||
ver = ver_str(cur_ver)
|
|
||||||
print(f"Python version: {ver}")
|
|
||||||
|
|
||||||
# This could be useful for debugging purposes
|
|
||||||
if SphinxDependencyChecker.which("docutils"):
|
|
||||||
result = SphinxDependencyChecker.run(["docutils", "--version"],
|
|
||||||
capture_output=True, text=True)
|
|
||||||
ver = result.stdout.strip()
|
|
||||||
match = re.search(r"(\d+\.\d+\.\d+)", ver)
|
|
||||||
if match:
|
|
||||||
ver = match.group(1)
|
|
||||||
|
|
||||||
print(f"Docutils version: {ver}")
|
|
||||||
|
|
||||||
return
|
|
||||||
|
|
||||||
python_ver = ver_str(cur_ver)
|
|
||||||
|
|
||||||
new_python_cmd = SphinxDependencyChecker.find_python()
|
|
||||||
if not new_python_cmd:
|
|
||||||
print(f"ERROR: Python version {python_ver} is not spported anymore\n")
|
|
||||||
print(" Can't find a new version. This script may fail")
|
|
||||||
return
|
|
||||||
|
|
||||||
# Restart script using the newer version
|
|
||||||
script_path = os.path.abspath(sys.argv[0])
|
|
||||||
args = [new_python_cmd, script_path] + sys.argv[1:]
|
|
||||||
|
|
||||||
print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
os.execv(new_python_cmd, args)
|
|
||||||
except OSError as e:
|
|
||||||
sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def run(*args, **kwargs):
|
def run(*args, **kwargs):
|
||||||
"""
|
"""
|
||||||
Excecute a command, hiding its output by default.
|
Excecute a command, hiding its output by default.
|
||||||
Preserve comatibility with older Python versions.
|
Preserve compatibility with older Python versions.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
capture_output = kwargs.pop('capture_output', False)
|
capture_output = kwargs.pop('capture_output', False)
|
||||||
|
|
@ -516,8 +423,19 @@ class MissingCheckers(AncillaryMethods):
|
||||||
"""
|
"""
|
||||||
Gets sphinx-build version.
|
Gets sphinx-build version.
|
||||||
"""
|
"""
|
||||||
|
env = os.environ.copy()
|
||||||
|
|
||||||
|
# The sphinx-build tool has a bug: internally, it tries to set
|
||||||
|
# locale with locale.setlocale(locale.LC_ALL, ''). This causes a
|
||||||
|
# crash if language is not set. Detect and fix it.
|
||||||
try:
|
try:
|
||||||
result = self.run([cmd, "--version"],
|
locale.setlocale(locale.LC_ALL, '')
|
||||||
|
except Exception:
|
||||||
|
env["LC_ALL"] = "C"
|
||||||
|
env["LANG"] = "C"
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = self.run([cmd, "--version"], env=env,
|
||||||
stdout=subprocess.PIPE,
|
stdout=subprocess.PIPE,
|
||||||
stderr=subprocess.STDOUT,
|
stderr=subprocess.STDOUT,
|
||||||
text=True, check=True)
|
text=True, check=True)
|
||||||
|
|
@ -527,11 +445,11 @@ class MissingCheckers(AncillaryMethods):
|
||||||
for line in result.stdout.split("\n"):
|
for line in result.stdout.split("\n"):
|
||||||
match = re.match(r"^sphinx-build\s+([\d\.]+)(?:\+(?:/[\da-f]+)|b\d+)?\s*$", line)
|
match = re.match(r"^sphinx-build\s+([\d\.]+)(?:\+(?:/[\da-f]+)|b\d+)?\s*$", line)
|
||||||
if match:
|
if match:
|
||||||
return parse_version(match.group(1))
|
return PythonVersion.parse_version(match.group(1))
|
||||||
|
|
||||||
match = re.match(r"^Sphinx.*\s+([\d\.]+)\s*$", line)
|
match = re.match(r"^Sphinx.*\s+([\d\.]+)\s*$", line)
|
||||||
if match:
|
if match:
|
||||||
return parse_version(match.group(1))
|
return PythonVersion.parse_version(match.group(1))
|
||||||
|
|
||||||
def check_sphinx(self, conf):
|
def check_sphinx(self, conf):
|
||||||
"""
|
"""
|
||||||
|
|
@ -542,7 +460,7 @@ class MissingCheckers(AncillaryMethods):
|
||||||
for line in f:
|
for line in f:
|
||||||
match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line)
|
match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line)
|
||||||
if match:
|
if match:
|
||||||
self.min_version = parse_version(match.group(1))
|
self.min_version = PythonVersion.parse_version(match.group(1))
|
||||||
break
|
break
|
||||||
except IOError:
|
except IOError:
|
||||||
sys.exit(f"Can't open {conf}")
|
sys.exit(f"Can't open {conf}")
|
||||||
|
|
@ -562,8 +480,8 @@ class MissingCheckers(AncillaryMethods):
|
||||||
sys.exit(f"{sphinx} didn't return its version")
|
sys.exit(f"{sphinx} didn't return its version")
|
||||||
|
|
||||||
if self.cur_version < self.min_version:
|
if self.cur_version < self.min_version:
|
||||||
curver = ver_str(self.cur_version)
|
curver = PythonVersion.ver_str(self.cur_version)
|
||||||
minver = ver_str(self.min_version)
|
minver = PythonVersion.ver_str(self.min_version)
|
||||||
|
|
||||||
print(f"ERROR: Sphinx version is {curver}. It should be >= {minver}")
|
print(f"ERROR: Sphinx version is {curver}. It should be >= {minver}")
|
||||||
self.need_sphinx = 1
|
self.need_sphinx = 1
|
||||||
|
|
@ -1304,7 +1222,7 @@ class SphinxDependencyChecker(MissingCheckers):
|
||||||
else:
|
else:
|
||||||
if self.need_sphinx and ver >= self.min_version:
|
if self.need_sphinx and ver >= self.min_version:
|
||||||
return (f, ver)
|
return (f, ver)
|
||||||
elif parse_version(ver) > self.cur_version:
|
elif PythonVersion.parse_version(ver) > self.cur_version:
|
||||||
return (f, ver)
|
return (f, ver)
|
||||||
|
|
||||||
return ("", ver)
|
return ("", ver)
|
||||||
|
|
@ -1411,7 +1329,7 @@ class SphinxDependencyChecker(MissingCheckers):
|
||||||
return
|
return
|
||||||
|
|
||||||
if self.latest_avail_ver:
|
if self.latest_avail_ver:
|
||||||
latest_avail_ver = ver_str(self.latest_avail_ver)
|
latest_avail_ver = PythonVersion.ver_str(self.latest_avail_ver)
|
||||||
|
|
||||||
if not self.need_sphinx:
|
if not self.need_sphinx:
|
||||||
# sphinx-build is present and its version is >= $min_version
|
# sphinx-build is present and its version is >= $min_version
|
||||||
|
|
@ -1507,7 +1425,7 @@ class SphinxDependencyChecker(MissingCheckers):
|
||||||
else:
|
else:
|
||||||
print("Unknown OS")
|
print("Unknown OS")
|
||||||
if self.cur_version != (0, 0, 0):
|
if self.cur_version != (0, 0, 0):
|
||||||
ver = ver_str(self.cur_version)
|
ver = PythonVersion.ver_str(self.cur_version)
|
||||||
print(f"Sphinx version: {ver}\n")
|
print(f"Sphinx version: {ver}\n")
|
||||||
|
|
||||||
# Check the type of virtual env, depending on Python version
|
# Check the type of virtual env, depending on Python version
|
||||||
|
|
@ -1613,7 +1531,8 @@ def main():
|
||||||
|
|
||||||
checker = SphinxDependencyChecker(args)
|
checker = SphinxDependencyChecker(args)
|
||||||
|
|
||||||
checker.check_python()
|
PythonVersion.check_python(MIN_PYTHON_VERSION,
|
||||||
|
bail_out=True, success_on_error=True)
|
||||||
checker.check_needs()
|
checker.check_needs()
|
||||||
|
|
||||||
# Call main if not used as module
|
# Call main if not used as module
|
||||||
Loading…
Reference in New Issue