kmod: eliminate duplication in calls to modinfo

[BroadlyWhitaker]

The original guard when building the new todo is:

todo += [m for m in depinfo.modules if m not in mods and m in nametofile]

mods only accumulates modules from previous iterations of the outer while loop. Within a single iteration, modules in the current batch are added to mods one at a time as moddep.items() is iterated. This means two distinct failure modes:

1. Cross-dependency within a batch: if modules A and B are both in the current todo and both depend on X, X passes the m not in mods check when processing A's deps and again when processing B's (since X hasn't been added to mods yet). X ends up in todo twice.

2. Intra-batch back-edge: if A depends on B and both are already in the current todo/moddep, B still passes m not in mods when processing A's dep list, queuing B for a redundant second modinfo call in the next iteration.

Both boil down to the same root cause: the check doesn't account for modules currently being processed.

The fix addresses both issues:

1. m not in moddep.keys() prevents modules from the current batch being re-queued into the next iteration's todo.

2. Turningtodo into a set prevents a module from appearing multiple times in the same next todo, when two modules in the current batch share a dependency

Because these duplicates frequently occur in practice, eliminating them provides an incremental speedup on top of the original optimization made in GH4092/GH4017.

[behrmann]

Thanks for the contribution! I will only have time to look at this later this week, but I started the CI run to see whether it works.

[behrmann]

Thank you! Sorry for the lag on my end.

[behrmann]

I've sat down an ran a tiny bit of benchmarking on this and running this against my host system, with empty inputs as

required_kernel_modules(kver, modules_include=[], modules_exclude=[], firmware_include=[], firmware_exclude=[], resolve_func=resolve_module_dependencies)

after making the function to resolve module dependencies pluggable to easily compare this, I get roughly the same run time for both functions, with the version from this PR actually a tiny bit, but consistently, slower.

At first I thought it was the chunking and I changed that to

        ittodo = iter(todo)
        while chunk := tuple(itertools.islice(ittodo, 8500)):
            moddep |= modinfo(kver, chunk)

which is roughly itertools.batch, which we don't yet have because it was only added in 3.12.

That helps a bit, but doesn't get one all the way, but the real trick is in lifting the todo update

        todo = set()
        mods |= moddep.keys()

        for name, depinfo in moddep.items():
            for d in depinfo.modules:
                if d not in nametofile and d not in builtin:
                    logging.warning(f"{d} is a dependency of {name} but is not installed, ignoring ")

            firmware.update(depinfo.firmware)
            todo.update(m for m in depinfo.modules if m not in mods and m in nametofile)

Since mods just gets the names from moddep added we can just add all keys from moddep to mods in one go. This then also makes the mods | moddep.keys() unnecessary. This shaves of 30% of the runtime off of required_kernel_modules for me.

Could you check that this makes sense and works in your usecase?

[BroadlyWhitaker]

Hi and thanks for looking at my PR.

How much benefit this does in practice depends both on the specifics of your KernelModules and on how fast your SSD is and It's difficult to give a meaningful general number. It's most impact comes when you specify a subset of modules, that have many common deps, and modinfo is called multiple times to resolve children-of-children. This isn't a common case, and the benefits are generally modest.

With your change I see smaller 0-10% faster here, and your version is also cleaner so I'm happy to include your suggestion. I also added a Co-Authored-By.

[martinpitt]

Thanks! I am like 80% sure the islice() is wrong. There are also a fair amount of test failures, I didn't go through them (slooow internet in the train), but they might be related

[behrmann]

I think the failures are something different, because, the opensuse/centos pairing notwithstanding, they're all Debian. I've only quickly looked at debian/debian, but there it's

2026-07-18T13:12:58.3668431Z ‣ losetup not found.
2026-07-18T13:12:58.4373304Z FAILED

which is a message from run not finding losetup during test_initrd_lvm and test_initrd_lvm_luks. That seems unrelated to me.

[martinpitt]

Thanks! I cannot resolve my own threads, but all good now.

[behrmann]

One last thing: Could you rebase against main so that we can run CI again? There was a change in Debian that broke CI (unrelated to your change).

[behrmann]

Thanks @BroadlyWhitaker!