Forza Horizon 4 - Game changes between controller and keyboard control?

In Forza Horizon 4, the game constantly changes between controller and keyboard control. This makes steering or playing impossible.

I'm using an Xbox One controller connected to the PC via USB wireless adapter.

If you already press a button, then it can happen. Look again, if your controller is really connected to the pc

If you use the keyboard, the keys are displayed, if you use the controller, the control is displayed.

It constantly changes back and forth without me pushing anything.

For other games, the controller works fine