Integration failure with python-pkcs11

[oli-ben]

Hello,

Thank you for this great tooling, the codebase is super easy to navigate, and the docs are good!

However, I have been experiencing a small issue when trying to use NXP's Secure Provisioning SDK (SPSDK):

.venv/lib64/python3.12/site-packages/pkcs11/__init__.py", line 31, in lib
    _lib = _pkcs11.lib(so)
           ^^^^^^^^^^^^^^^
  File "pkcs11/_pkcs11.pyx", line 2034, in pkcs11._pkcs11.lib.__init__
  File "pkcs11/_pkcs11.pyx", line 2011, in pkcs11._pkcs11.lib.initialize
  File "pkcs11/_pkcs11.pyx", line 47, in pkcs11._pkcs11.assertRV
pkcs11.exceptions.ArgumentsBad

I believe I have found the issue, namely that python's pkcs-11 module calls C_Initialize with a NULL pointer.

This is explicitly considered an error case in the yubihsm_pkcs11 module, and has been since the start.

However, when looking at the specification that you yourself link to, I found the following in section 5.4 General-purpose functions:

A call to C_Initialize with pInitArgs set to NULL_PTR is treated like a call to C_Initialize with pInitArgs pointing to a CK_C_INITIALIZE_ARGS which has the CreateMutex, DestroyMutex, LockMutex, UnlockMutex, and pReserved fields set to NULL_PTR, and has the flags field set to 0.

As far as I can tell, this would mean falling back to this:

If the flag isn’t set, and the function pointer fields aren’t supplied (i.e., they all have the value NULL_PTR), that means that the application won’t be accessing the Cryptoki library from multiple threads simultaneously.

which is handled here in yubihsm_pkcs11.

My impression from this is, that in this specific case, yubihsm_pkcs11 is breaking the PKCS#11 interface, and will not work for any python-pkcs11 users.
I am, however, a bit surprised to be the first one to stumble upon this bug, so I'd like to confirm my understanding with you before potentially sending a PR your way.

[oli-ben]

Oops. sorry about this: I misread the else, and it appears that the CRK_AGUMENTS_BAD error is coming from somewhere else.

[qpernil]

Calling C_Initialize with NULL is in fact allowed, per the spec. See the test for that on line 123:
if (pInitArgs != NULL) {...}

What is tripping you up is probably that it is not finding the configuration file that is required in order to specify a connector. If that happens it errors out a line 261:

if (args_info.connector_given == 0) {
DBG_ERR("No connector defined");
rv = CKR_ARGUMENTS_BAD;
goto c_i_failure;
}

The default configuration file name is ./yubihsm_pkcs11.conf, i.e. relative to the working directory of the calling application.

As can be seen in line 242 it will check for an environment variable (YUBIHSM_PKCS11_CONF) specifying a different file if the default one can't be found:

if (stat(config_file, &sb) == -1) {
config_file = getenv("YUBIHSM_PKCS11_CONF");
}

So I recommend either putting a file named yubihsm_pkcs11.conf in the working directory, or to point a different file out using the environment variable YUBIHSM_PKCS11_CONF.

[qpernil]

Also note that yubihsm_pkcs11 has another method to specify configuration without the need for a config file on environment variable - using init_args->pReserved, which supports openssl:s pkcs11 provider's ability to forward configuration from it's own config file to a pkcs11 module. (This was also supported for the now deprecated openssl engine interface)

[oli-ben]

Hello Per,

that was indeed exactly my problem!

I noticed that as well once I realized that the } else { clause returning CRK_AGUMENTS_BAD was one level deeper than I originally thought.

Thank you for your help, sorry about the noise!

[oli-ben]

Hello @qpernil !
Sorry to bother you on this again.

You mentioned

Also note that yubihsm_pkcs11 has another method to specify configuration without the need for a config file on environment variable - using init_args->pReserved, which supports openssl:s pkcs11 provider's ability to forward configuration from it's own config file to a pkcs11 module. (This was also supported for the now deprecated openssl engine interface)

I am now trying to interface with OpenSSL's PKCS#11 engine interface, and it is failing to load the module, probably for the same reason.
I am trying to figure out how to take advantage of this forwarding feature, but am struggling to find this information in the openSSL documentation.
Would you happen to know the syntax, or where I should look for it?

[qpernil]

Here is an article on using the pkcs11 provider (the recommended way) https://support.yubico.com/hc/en-us/articles/18706910948380-Generating-a-CSR-using-OpenSSL-PKCS-11-provider-and-the-YubiHSM2 but that article doesn't really show the config file, so for a clearer example here is an article about openssl pkcs11 engine https://docs.yubico.com/hardware/yubihsm-2/hsm-2-user-guide/hsm2-openssl-yubihsm2.html. Look for INIT_ARGS in openssl.conf (engine.conf in some examples). You can use that with providers as well. Note that the way the pkcs11 engine and provider send this config to a pkcs11 module doesn't follow the pkcs11 spec, but anyway it is the de-facto way it is done.