GPU support & a GUI would be super nice

[sarutobiumon]

Great work @imartinez and other contributors, please add a GUI interface and enable support for GPU if possible, as it is super slow right now.

This would also be an amazing addition to Endless OS, a linux distro made for education/school kids, please kindly consider integrating it with EndlessOs's echo system as a flatpak package. Thank you very much!

https://community.endlessos.com/c/en/feedback/22
https://www.endlessos.org/

[pseudotensor]

FYI: https://github.com/h2oai/h2ogpt has UI+GPU. It's focused on GPU support at moment, as opposed to cpu that privateGPT focused on.

[Simplegram]

I can use GPU on Windows with a fresh privateGPT install, albeit not 100%. I can only use 40 layers of GPU with a VRAM usage of ~9 GB. Speed is much faster compared to only using CPU. Follow maozdemir's or thekit's instruction at #217.

[Juusseli]

I managed to get working it with my old 1050 gpu. And it's much faster. Now its usable in my 6 years old laptop. Query respond time is 20-30s.