Problem with SAAS while compiling 9.2.12

[duckhawk]

Trying update DRBD to 9.2.12
Geting error while getting patch from SAAS

     SPATCH         e5b4538b952d02aa28acb205221181bc  5.15.0-107-generic
     drbd-kernel-compat/cocci_cache/e5b4538b952d02aa28acb205221181bc/.compat.cocci
    : Fatal error: exception Not_found

In SAAS logs there is nothing interesting. Switching to last saas can't resolve situation.

[rck]

from which source tarball do you start? for me compiling works perfectly fine with the tarball from pkg.linbit.com in a docker container. that said, in this case it even does not run into SPATCH as far as I saw.

Skipping BTF generation for /root/drbd-9.2.12/drbd/build-current/drbd_transport_tcp.ko due to unavailability of vmlinux
.======================================================================.
#  KDIR=/usr/src/linux-headers-5.15.0-107-generic                      #
#  KERNELVERSION=5.15.149                                              #
#  KERNELRELEASE=5.15.0-107-generic                                    #
#                                                                      #
#  /root/drbd-9.2.12/drbd/build-5.15.0-107-generic/                    #
#    drbd-kernel-compat/handshake/handshake.ko                         #
#    drbd.ko                                                           #
#    drbd_transport_lb-tcp.ko                                          #
#    drbd_transport_rdma.ko                                            #
#    drbd_transport_tcp.ko                                             #
'======================================================================'
make[1]: Leaving directory '/root/drbd-9.2.12/drbd'

        Module build was successful.

[duckhawk]

I'm using actual git repo with tag
https://github.com/LINBIT/drbd/tree/drbd-9.2.12
And in my case in any of os/server it wants spatch, but can't get it.

make[1]: Entering directory '/opt/drbd/drbd'
rm -rf build-current/.tmp_versions build-current/Module.markers build-current/Module.symvers build-current/modules.order build-current/.compat_test.* build-current/.cache.mk
rm -f build-current/*.[oas] build-current/*.ko build-current/.*.cmd build-current/.*.d build-current/.*.tmp build-current/*.mod.c build-current/.*.flags build-current/.depend build-current/.kernel* build-current/drbd-kernel-compat/*.[oas] build-current/drbd-kernel-compat/*.ko build-current/drbd-kernel-compat/.*.cmd build-current/drbd-kernel-compat/.*.d build-current/drbd-kernel-compat/.*.tmp build-current/drbd-kernel-compat/*.mod.c build-current/drbd-kernel-compat/.*.flags build-current/drbd-kernel-compat/.depend build-current/drbd-kernel-compat/.kernel* build-current/drbd-kernel-compat/handshake/*.[oas] build-current/drbd-kernel-compat/handshake/*.ko build-current/drbd-kernel-compat/handshake/.*.cmd build-current/drbd-kernel-compat/handshake/.*.d build-current/drbd-kernel-compat/handshake/.*.tmp build-current/drbd-kernel-compat/handshake/*.mod.c build-current/drbd-kernel-compat/handshake/.*.flags build-current/drbd-kernel-compat/handshake/.depend build-current/drbd-kernel-compat/handshake/.kernel* build-current/drbd-kernel-compat/compat.patch build-current/drbd-kernel-compat/.compat.cocci build-current/.tmp_[0-9]*
rm -f dummy-for-compat-h.c dummy-for-patch.c
make[1]: Leaving directory '/opt/drbd/drbd'
rm -f *~ ; rm -rf tmp.km-deb.*
make[1]: Entering directory '/opt/drbd/drbd'
     Calling toplevel makefile of kernel source tree, which I believe is in
     KDIR=/lib/modules/5.15.0-107-generic/build
make -C /lib/modules/5.15.0-107-generic/build    "PRE_CFLAGS=" M=/opt/drbd/drbd obj-m=dummy-for-compat.o dummy-for-compat-h.o
make -C /lib/modules/5.15.0-107-generic/build    "PRE_CFLAGS=" M=/opt/drbd/drbd obj-m=dummy-for-prep.o dummy-for-patch.o
make -C /opt/drbd/drbd -f Makefile.spatch /opt/drbd/drbd/build-5.15.0-107-generic/compat.patch
Fatal error: exception Not_found
Fatal error: exception Not_found
GENPATCHNAMES   5.15.0-107-generic
SPATCH         e5b4538b952d02aa28acb205221181bc  5.15.0-107-generic
          drbd-kernel-compat/cocci_cache/e5b4538b952d02aa28acb205221181bc/.compat.cocci
        : Fatal error: exception Not_found
make[4]: *** [Makefile.spatch:52: drbd-kernel-compat/cocci_cache/e5b4538b952d02aa28acb205221181bc/compat.patch] Error 2
make[3]: *** [/opt/drbd/drbd/Kbuild:147: /opt/drbd/drbd/build-5.15.0-107-generic/compat.patch] Error 2
make[2]: *** [Makefile:1911: /opt/drbd/drbd] Error 2
make[1]: *** [Makefile:244: prep] Error 2
make[1]: Leaving directory '/opt/drbd/drbd'
make: *** [Makefile:129: module] Error 2

[rck]

there seems to be some confusion here: if you use a git checkout, then the local spatch/coccinelle is used (not the online spatch as a service we provide). In this case the spatch/coccinelle shipped with Jammy is too old, it will fail. You would need to get a newer spatch from "somewhere". or you might be lucky if you generate a proper release tarball from the checkout and then run into the online spatch as a service.

building DRBD is complex, why not just use the tarballs we provide that have prebuilt patches for more than 100 kernels (or run into the online spatch as a service which generates a patch on the fly) and be happy?

[duckhawk]

We use it in several kubernetes clusters, saas created as pod and used locally in each of them. There can be different versions of OS, so we need to build it to kernel used in each OS. With 9.2.10 and 9.2.11 it works fine, problems started with 9.2.12. I don't sure, if problem with new version of drbd, or there some incorrect behavior of local saas service, because of very poor logging in saas.

[rck]

I now started from a tarball and removed the pre-generated patches so that it runs into the online version, and:

make -C /root/drbd-9.2.12/drbd -f Makefile.spatch /root/drbd-9.2.12/drbd/build-5.15.0-107-generic/compat.patch
./drbd-kernel-compat/gen_compat_patch.sh: line 17: spatch: command not found
./drbd-kernel-compat/gen_compat_patch.sh: line 44: hash: spatch: not found
  INFO: no suitable spatch found; trying spatch-as-a-service;
  be patient, may take up to 10 minutes
  if it is in the server side cache it might only take a second
  SPAAS    dd7e7afa0bfb8b8404de9beb508d49b0
Successfully connected to SPAAS ('b22d7c3dbb8554af6739245d936a0dc5be01c748')
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 47923    0 39116    0  8807  78804  17742 --:--:-- --:--:-- --:--:-- 96424
  You can create a new .tgz including this pre-computed compat patch
  by calling "echo drbd/drbd-kernel-compat/cocci_cache/dd7e7afa0bfb8b8404de9beb508d49b0/compat.patch >>.filelist ; make tgz"
...
'======================================================================'
make[1]: Leaving directory '/root/drbd-9.2.12/drbd'

        Module build was successful.

I find it interesting that not even the hashes match 🥴. You really did follow the recent changes to https://github.com/linbit/saas ? especially the last 2 commits switching to fedora and installing diffutils? other than that: works for me...

[duckhawk]

Ok, looks like moving to ubuntu 24.04 solved this problem with new spatch
But it changed to compilation error

 In file included from /opt/drbd/drbd/drbd-headers/linux/genl_magic_func.h:292,
                  from /opt/drbd/drbd/drbd_nl.c:90:
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h: In function 'drbd_genl_multicast_events':
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:21:16: error: too few arguments to function 'genlmsg_multicast_allns'
    21 |         return genlmsg_multicast_allns(&ZZZ_genl_family, skb, 0,        \
       |                ^~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 In file included from /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:20,
                  from /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:40,
                  from /opt/drbd/drbd/drbd_int.h:34,
                  from /opt/drbd/drbd/drbd_nl.c:25:
 ./include/net/genetlink.h:526:5: note: declared here
   526 | int genlmsg_multicast_allns(const struct genl_family *family,
       |     ^~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_resource_state':
 /opt/drbd/drbd/drbd_nl.c:7202:23: error: too many arguments to function 'drbd_genl_multicast_events'
  7202 |                 err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_device_state':
 /opt/drbd/drbd/drbd_nl.c:7252:23: error: too many arguments to function 'drbd_genl_multicast_events'
  7252 |                 err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_connection_state':
 /opt/drbd/drbd/drbd_nl.c:7304:23: error: too many arguments to function 'drbd_genl_multicast_events'
  7304 |                 err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_peer_device_state':
 /opt/drbd/drbd/drbd_nl.c:7355:23: error: too many arguments to function 'drbd_genl_multicast_events'
  7355 |                 err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_path_state':
 /opt/drbd/drbd/drbd_nl.c:7414:23: error: too many arguments to function 'drbd_genl_multicast_events'
  7414 |                 err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd_nl.c: In function 'notify_helper':
 /opt/drbd/drbd/drbd_nl.c:7475:15: error: too many arguments to function 'drbd_genl_multicast_events'
  7475 |         err = drbd_genl_multicast_events(skb, GFP_NOWAIT);
       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl_api.h:37:33: note: declared here
    37 | #define GENL_MAGIC_FAMILY       drbd
       |                                 ^~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_struct.h:29:25: note: in definition of macro 'CONCAT__'
    29 | #define CONCAT__(a,b)   a ## b
       |                         ^
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:12: note: in expansion of macro 'CONCAT_'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |            ^~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:16:20: note: in expansion of macro 'GENL_MAGIC_FAMILY'
    16 | static int CONCAT_(GENL_MAGIC_FAMILY, _genl_multicast_ ## group)(       \
       |                    ^~~~~~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h: In function 'drbd_genl_multicast_events':
 /opt/drbd/drbd/drbd-headers/linux/genl_magic_func-genl_register_family_with_ops_groups.h:23:1: error: control reaches end of non-void function [-Werror=return-type]
    23 | }
       | ^
 /opt/drbd/drbd/drbd-headers/linux/drbd_genl.h:385:1: note: in expansion of macro 'GENL_mc_group'
   385 | GENL_mc_group(events)
       | ^~~~~~~~~~~~~
 cc1: some warnings being treated as errors
 *** [scripts/Makefile.build:243: /opt/drbd/drbd/build-current/drbd_nl.o] Error 1
 *** [Makefile:1925: /opt/drbd/drbd/build-current] Error 2
 Leaving directory '/opt/drbd/drbd'
 *** [Makefile:248: kbuild] Error 2
 make: *** [Makefile:129: module] Error 2