rknnlite inference crashes if numpy inputs are float64 (image_generator.py)

[54601]

I was generating images with lcm-segmind-stable-diffusion:

requests.post("http://<server_ip>:8080/v1/images/generations", {
        "model": "lcm-segmind-stable-diffusion",
        "prompt": "a female, photorealistic, dramatic lighting, ultra‑detailed",
        "size": "1024x1024",
        "n": 1
    })

But the request failed with 500. The server's log looks like this:

2026-01-30 20:48:41,121 - rkllama.api.image_generator -
I - Load Done. Took 5.1 seconds.
INPUT 0: type=<class 'numpy.ndarray'>, shape=(1, 4, 128, 128)
INPUT 1: type=<class 'numpy.ndarray'>, shape=(1,)
INPUT 2: type=<class 'numpy.ndarray'>, shape=(1, 77, 2048)
INPUT 3: type=<class 'numpy.ndarray'>, shape=(1, 256)
INPUT 4: type=<class 'torch.Tensor'>, shape=torch.Size([1, 6])
** TRANSFORMED** -> INPUT 4: type=<class 'numpy.ndarray'>, shape=(1, 6)
INPUT 5: type=<class 'numpy.ndarray'>, shape=(1, 1280)
E Catch exception when setting inputs.
E Traceback (most recent call last):
File "/home/user/.local/lib/python3.12/site-packages/rknnlite/api/rknn_lite.py", line 209, in inference self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
File "rknnlite/api/rknn_runtime.py", line 1035, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
Exception: E Unsupport inputs[0].dtype: float64
0%| | 0/4 [00:05<?, ?it/s] Process Process-1: Traceback (most recent call last):
File "/home/user/miniforge3/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap self.run()
File "/home/user/miniforge3/lib/python3.12/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs)
File "/home/user/miniforge3/lib/python3.12/site-packages/rkllama/api/worker.py", line 72, in run_image_generator image = generate_image(model_name, prompt, size, seed, num_inference_steps, guidance_scale) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniforge3/lib/python3.12/site-packages/rkllama/api/image_generator.py", line 1077, in generate_image image = pipe( ^^^^^
File "/home/user/miniforge3/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniforge3/lib/python3.12/site-packages/rkllama/api/image_generator.py", line 918, in call model_pred = self.unet( ^^^^^^^^^^
File "/home/user/miniforge3/lib/python3.12/site-packages/rkllama/api/image_generator.py", line 196, in call for res in results: ^^^^^^^
TypeError: 'NoneType' object is not iterable

It seems that the first error is: 'E Unsupport inputs[0].dtype: float64'.

After I changed the image_generator.py, the problem was solved and image was successfully generated.

       # Construct the list of input and transform them to Numpy if needed
        input_list = [value for key, value in kwargs.items()]
        input_list_np = []
        for i, input in enumerate(input_list):
            print(f"INPUT {i}: type={type(input)}, shape={getattr(input, 'shape', None)}, dtype={getattr(input, 'dtype', None)}") # first change
            # print(f"INPUT {i}: type={type(input)}, shape={None if not hasattr(input, 'shape') else input.shape}") # This was the original
            if isinstance(input, np.ndarray):
                # input_list_np.append(input) # This was the original
                input_list_np.append(input.astype(np.float32, copy=False)) # second change
            else:
                input_np = input.detach().cpu().numpy().astype(np.float32)
                input_list_np.append(input_np)
                print(f"** TRANSFORMED** -> INPUT {i}: type={type(input_np)}, shape={None if not hasattr(input_np, 'shape') else input_np.shape}")

Does this look like a bug in rkllama’s input handling?

[danielferr85]

The image generation works. Maybe the version of a package or library that you are using is failing.

[54601]

The image generation works. Maybe the version of a package or library that you are using is failing.

Thank you for reply! I find out that my numpy version is >2:

numpy==2.2.6

I think this may be the root cause. However, even though numpy 2 isn't officially supported yet, could a dtype check be added to avoid such errors?

[jaylfc]

This is the same root cause as #76. Fixed in my fork: jaylfc@27af1b5

The fix is exactly what @54601 suggested — cast numpy inputs to float32 in RKNN2Model.__call__:

input_list_np.append(input.astype(np.float32, copy=False))

Also added None checking in handle_complete so it returns a proper error instead of crashing when inference fails.