I had HAOS with Conbee II on raspberry pi 5 and I remember no issues.

Then I changed to run HA as a docker container on raspberry pi os + k2s and on such setup usb was not accessible and I had to assign access rights to the usb port in my kubernetes deployment of HA.

Mostly pv provisioning and db setups. I use my external hard drive as k3s storage (mnt/k3s) and e.g. for immich I failed to provide custom path (mnt/k3s/immich-media), it always complaints about some access rights.

K3s