ArrayNest AKA indexed-array support currently skips inner type
validation. We count the attributes and then we parse them,
make sure we call validate, too. Otherwise buggy / unexpected
kernel response may lead to crashes.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20250414211851.602096-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When user calls request_attrA_set() multiple times (for the same
attribute), and attrA is of type which allocates memory -
we try to free the previously associated values. For array
types (including multi-attr) we have only freed the array,
but the array may have contained pointers.
Refactor the code generation for free attr and reuse the generated
lines in setters to flush out the previous state. Since setters
are static inlines in the header we need to add forward declarations
for the free helpers of pure nested structs. Track which types get
used by arrays and include the right forwad declarations.
At least ethtool string set and bit set would not be freed without
this. Tho, admittedly, overriding already set attribute twice is likely
a very very rare thing to do.
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20250414211851.602096-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The "function writing helper" tries to put local variables
between prototype and the opening bracket. Clearly wrong,
but up until now nothing actually uses it to write local
vars so it wasn't noticed.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20250414211851.602096-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The codegen tries to follow the "old" C style and declare loop
iterators at the start of the block / function. Only nested
request handling breaks this style, so adjust it.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20250414211851.602096-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Turn attribute names with leading digits into valid C names by
prepending an underscore, e.g. 5ghz -> _5ghz
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250211120127.84858-7-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The ynl tool uses display-hint to know when to format IP addresses in
printed output, but not to parse IP addresses from --json input. Add
support for parsing ipv4 and ipv6 strings.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250211120127.84858-5-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The nl80211 family encodes the list of supported ciphers as a C array of
u32 values. Add support for translating arrays of scalars into strings
for enum names and display hints.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250211120127.84858-4-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When decoding an indexed-array with a scalar subtype, it is currently
only possible to add a display-hint. Add support for decoding each value
as an enum.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250211120127.84858-3-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
_decode_array_attr() uses variable subattrs in every branch when only
one branch decodes more than a single attribute.
Change the variable name to subattr in the branches that only decode a
single attribute so that the intent is more obvious.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250211120127.84858-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The Makefile.deps lists uAPI headers to make the build work when
system headers are older than in-tree headers. The problem doesn't
occur for new headers, because system headers are not there at all.
But out-of-tree YNL clone on GH also uses this header to identify
header dependencies, and one day the system headers will exist,
and will get out of date. So let's add the headers we missed.
I don't think this is a fix, but FWIW the commits which added
the missing headers are:
commit 04e65df94b ("netlink: spec: add shaper YAML spec")
commit 49922401c2 ("ethtool: separate definitions that are gonna be generated")
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250205173352.446704-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
A definition with a "header" property is an "external" definition
for C code, as in it is defined already in another C header file.
Other languages will need the exact value but C codegen should
not recreate it. So don't output those definitions in the uAPI
header.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Link: https://patch.msgid.link/20250203215510.1288728-1-kuba@kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
netlink reports which attribute was incorrect by sending back
an attribute offset. Offset points to the address of struct nlattr,
but to interpret the type we also need the nesting path.
Attribute IDs have different meaning in different nests
of the same message.
Correct the condition for "is the offset within current attribute".
ynl_attr_data_len() does not include the attribute header,
so the end offset was off by 4 bytes.
This means that we'd always skip over flags and empty nests.
The devmem tests, for example, issues an invalid request with
empty queue nests, resulting in the following error:
YNL failed: Kernel error: missing attribute: .queues.ifindex
The message is incorrect, "queues" nest does not have an "ifindex"
attribute defined. With this fix we decend correctly into the nest:
YNL failed: Kernel error: missing attribute: .queues.id
Fixes: 86878f14d7 ("tools: ynl: user space helpers")
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250124012130.1121227-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Replace hard-coded paths for spec and schema with lookup functions so
that ethtool.py will work in-tree or when installed.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250111154803.7496-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a --family option to ynl to specify the spec by family name instead
of file path, with support for searching in-tree and system install
location and a --list-families option to show the available families.
./tools/net/ynl/pyynl/cli.py --family rt_addr --dump getaddr
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20250111154803.7496-1-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Generate docs using ynl_gen_rst and add install target for
headers, specs and generates rst files.
Factor out SPECS_DIR since it's repeated many times.
Signed-off-by: Jan Stancek <jstancek@redhat.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/645c68e3d201f1ef4276e3daddfe06262a0c2804.1736343575.git.jstancek@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Move python code to a separate directory so it can be
packaged as a python module. Updates existing references
in selftests and docs.
Also rename ynl-gen-[c|rst] to ynl_gen_[c|rst], avoid
dashes as these prevent easy imports for entrypoints.
Signed-off-by: Jan Stancek <jstancek@redhat.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/a4151bad0e6984e7164d395125ce87fd2e048bf1.1736343575.git.jstancek@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Empty nests are the same size as a flag at the netlink level
(just a 4 byte nlattr without a payload). They are sometimes
useful in case we want to only communicate a presence of
something but may want to add more details later.
This may be the case in the upcoming io_uring ZC patches,
for example.
Improve handling of nested empty structs. We already support
empty structs since a lot of netlink replies are empty, but
for nested ones we need minor tweaks to avoid pointless empty
lines and unused variables.
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://patch.msgid.link/20250108200758.2693155-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When parsing throws an exception one often has to figure out which
attribute couldn't be parsed from first principles. For families
with large message parsing trees like rtnetlink guessing the
attribute can be hard.
Print a bit of information as the exception travels out, e.g.:
# when dumping rt links
Error decoding 'flags' from 'linkinfo-ip6tnl-attrs'
Error decoding 'data' from 'linkinfo-attrs'
Error decoding 'linkinfo' from 'link-attrs'
Traceback (most recent call last):
File "/home/kicinski/linux/./tools/net/ynl/cli.py", line 119, in <module>
main()
File "/home/kicinski/linux/./tools/net/ynl/cli.py", line 100, in main
reply = ynl.dump(args.dump, attrs)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1064, in dump
return self._op(method, vals, dump=True)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1058, in _op
return self._ops(ops)[0]
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 1045, in _ops
rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 738, in _decode
subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'], search_attrs)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 763, in _decode
decoded = self._decode_sub_msg(attr, attr_spec, search_attrs)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 714, in _decode_sub_msg
subdict = self._decode(NlAttrs(attr.raw, offset), msg_format.attr_set)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 749, in _decode
decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order)
File "/home/kicinski/linux/tools/net/ynl/lib/ynl.py", line 147, in as_scalar
return format.unpack(self.raw)[0]
struct.error: unpack requires a buffer of 2 bytes
The Traceback is what we would previously see, the "Error..."
messages are new. We print a message per level (in the stack
order). Printing single combined message gets tricky quickly
given sub-messages etc.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20250107022820.2087101-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Use the correct attribute space for sub-message key lookup in nested
attributes when adding attributes. This fixes rt_link where the "kind"
key and "data" sub-message are nested attributes in "linkinfo".
For example:
./tools/net/ynl/cli.py \
--create \
--spec Documentation/netlink/specs/rt_link.yaml \
--do newlink \
--json '{"link": 99,
"linkinfo": { "kind": "vlan", "data": {"id": 4 } }
}'
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Fixes: ab463c4342 ("tools/net/ynl: Add support for encoding sub-messages")
Link: https://patch.msgid.link/20241213130711.40267-1-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Without -o the tool currently crashes, but it's not marked
as required. The only thing we can't do without it is to
generate the correct #include for user source files, but
we can put a placeholder instead.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Link: https://patch.msgid.link/20241206113100.89d35bf124d6.I9228fb704e6d5c9d8e046ef15025a47a48439c1e@changeid
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Essentially reverse the order of headers for userspace generated files.
Before (make -C tools/net/ynl/; cat tools/net/ynl/ethtool-user.h):
#include <linux/ethtool_netlink_generated.h>
#include <linux/ethtool.h>
#include <linux/ethtool.h>
#include <linux/ethtool.h>
After:
#include <linux/ethtool.h>
#include <linux/ethtool_netlink_generated.h>
While at it, make sure we track which headers we've already included
and include the headers only once.
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20241204155549.641348-6-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The intent is to generate ethtool uapi headers. For now, some of the
things are hard-coded:
- <FAMILY>_MSG_{USER,KERNEL}_MAX
- the split between USER and KERNEL messages
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20241204155549.641348-4-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This is similar to existing attr-cnt-name in the attributes
to allow changing the name of the 'count' enum entry.
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20241204155549.641348-2-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Jeff Layton contributed a scalability improvement to NFSD's NFSv4
backchannel session implementation. This improvement is intended to
increase the rate at which NFSD can safely recall NFSv4 delegations
from clients, to avoid the need to revoke them. Revoking requires
a slow state recovery process.
A wide variety of bug fixes and other incremental improvements make
up the bulk of commits in this series. As always I am grateful to
the NFSD contributors, reviewers, testers, and bug reporters who
participated during this cycle.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEKLLlsBKG3yQ88j7+M2qzM29mf5cFAmdEgLQACgkQM2qzM29m
f5cwmg/9HcfG7blepU/2qNHopzSYRO5vZw1YNJQ5/Wi3bmqIea83lf8OcCY1G/aj
6K+jnenzHrwfhaA4u7N2FPXPVl8sPSMuOrJXY5zC4yE5QnIbranjcyEW5l5zlj3n
ukkTYQgjUsKre3pHlvn3JmDHfUhNPEfzirsJeorP7DS3omne+OFA1LNncNP6emRu
h0aEC6EJ43zUkYiz9nZYqPwIAwrUIA0WOrvVnq7vsi6gR4/Muk7nS+X/y4qFjli3
9enVskEv8sFmmOAIMK3CHJq+exEeKtKEKUuYkD23QgPt2R4+IwqS70o9IM/S1ypf
APiv958BIhxm/SwUn1IjoxIckTB5EdksMxU5/4qGr1ZxprPG4/ruKO80BkrxLzW2
n1HmJ4ZNnpWPQvHN7RQ0WOsPNzL8byxJbGr1bpNgU4AGXnTFWPrAnB6juiyX4xb+
YNfgkQGDY79o7r1OJ5UUdCyx0QBSnaLNACTGm2u2FpI/ukMFPdrWIE99QbBgSe1p
MgWaiPwSY+9crFfGPJeQ4t6/siRAec6L3RO9KT9Epcd2S7/Uts3NXYRdJfwZ+Qza
TkPY2bm7T/WCcMhW7DN372hqgfRHPWOf4tacJ1Tob+As1d6p6qXEX2zi6piCCOLj
dmTVDSVPClRXt8YigF9WqosyWv1jUzSnh9ne+eYPBpj93Ag2YBY=
=wBvS
-----END PGP SIGNATURE-----
Merge tag 'nfsd-6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/cel/linux
Pull nfsd updates from Chuck Lever:
"Jeff Layton contributed a scalability improvement to NFSD's NFSv4
backchannel session implementation. This improvement is intended to
increase the rate at which NFSD can safely recall NFSv4 delegations
from clients, to avoid the need to revoke them. Revoking requires a
slow state recovery process.
A wide variety of bug fixes and other incremental improvements make up
the bulk of commits in this series. As always I am grateful to the
NFSD contributors, reviewers, testers, and bug reporters who
participated during this cycle"
* tag 'nfsd-6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/cel/linux: (72 commits)
nfsd: allow for up to 32 callback session slots
nfs_common: must not hold RCU while calling nfsd_file_put_local
nfsd: get rid of include ../internal.h
nfsd: fix nfs4_openowner leak when concurrent nfsd4_open occur
NFSD: Add nfsd4_copy time-to-live
NFSD: Add a laundromat reaper for async copy state
NFSD: Block DESTROY_CLIENTID only when there are ongoing async COPY operations
NFSD: Handle an NFS4ERR_DELAY response to CB_OFFLOAD
NFSD: Free async copy information in nfsd4_cb_offload_release()
NFSD: Fix nfsd4_shutdown_copy()
NFSD: Add a tracepoint to record canceled async COPY operations
nfsd: make nfsd4_session->se_flags a bool
nfsd: remove nfsd4_session->se_bchannel
nfsd: make use of warning provided by refcount_t
nfsd: Don't fail OP_SETCLIENTID when there are too many clients.
svcrdma: fix miss destroy percpu_counter in svc_rdma_proc_init()
xdrgen: Remove program_stat_to_errno() call sites
xdrgen: Update the files included in client-side source code
xdrgen: Remove check for "nfs_ok" in C templates
xdrgen: Remove tracepoint call site
...
Binder places its headers under include/uapi/linux/android/
Make sure replace / with _ in the uAPI header guard, the c_upper()
is more strict and only converts - to _. This is likely a good
constraint to have, to enforce sane naming in enums etc.
But paths may include /.
Signed-off-by: Li Li <dualli@google.com>
Link: https://patch.msgid.link/20241113193239.2113577-2-dualli@chromium.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Refactor: Translating an on-the-wire value to a local host errno is
architecturally a job for the proc function, not the XDR decoder.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
In particular, client-side source code needs the definition of
"struct rpc_procinfo" and does not want header files that pull
in "struct svc_rqst". Otherwise, the source does not compile.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Obviously, "nfs_ok" is defined only for NFS protocols. Other XDR
protocols won't know "nfs_ok" from Adam.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
This tracepoint was a "note to self" and is not operational. It is
added only to client-side code, which so far we haven't needed. It
will cause immediate breakage once we start generating client code,
though, so remove it now.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
For convenience, copy the XDR extraction script from RFC
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
The notification handling in ynl is currently very simple, using sleep()
to wait a period of time and then handling all the buffered messages in
a single batch.
This patch adds async notification handling so that messages can be
processed as they are received. This makes it possible to use ynl as a
library that supplies notifications in a timely manner.
- Add poll_ntf() to be a generator that yields 1 notification at a
time and blocks until a notification is available.
- Add a --duration parameter to the CLI, with --sleep as an alias.
./tools/net/ynl/cli.py \
--spec <SPEC> --subscribe <TOPIC> [ --duration <SECS> ]
The cli will report any notifications for duration seconds and then
exit. If duration is not specified, then it will poll forever, until
interrupted.
Here is an example python snippet that shows how to use ynl as a library
for receiving notifications:
ynl = YnlFamily(f"{dir}/rt_route.yaml")
ynl.ntf_subscribe('rtnlgrp-ipv4-route')
for event in ynl.poll_ntf():
handle(event)
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20241113090843.72917-3-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This reverts commit 1bf70e6c3a.
This modification to check_ntf() is being reverted so that its behaviour
remains equivalent to ynl_ntf_check() in the C YNL. Instead a new
poll_ntf() will be added in a separate patch.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20241113090843.72917-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Make a minor change to eliminate a static checker warning. The type
of s->ifc is unsigned int, so the correct format specifier should be
%u instead of %d.
Signed-off-by: Luo Yifan <luoyifan@cmss.chinamobile.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20241113011142.290474-1-luoyifan@cmss.chinamobile.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Package build environments like Fedora rpmbuild introduced hardening
options (e.g. -pie -Wl,-z,now) by passing a -spec option to CFLAGS
and LDFLAGS.
ynl Makefiles currently override CFLAGS but not LDFLAGS, which leads
to a mismatch and build failure:
CC sample devlink
/usr/bin/ld: devlink.o: relocation R_X86_64_32 against symbol `ynl_devlink_family' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
Extend CFLAGS to support hardening options set by build environment.
Signed-off-by: Jan Stancek <jstancek@redhat.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/265b2d5d3a6d4721a161219f081058ed47dc846a.1731399562.git.jstancek@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Introduce logic in the code generators to emit maxsize (XDR
width) definitions. In C, these are pre-processor macros.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Not yet complete.
The tool doesn't do any math yet. Thus, even though the maximum XDR
width of a union is the width of the union enumerator plus the width
of its largest arm, we're using the sum of all the elements of the
union for the moment.
This means that buffer size requirements are overestimated, and that
the generated maxsize macro cannot yet be used for determining data
element alignment in the XDR buffer.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Jakub Kicinski
Donald Hunter
Jan Stancek
Johannes Berg
Stanislav Fomichev
Linus Torvalds
Chuck Lever
Luo Yifan