Play together on the console and PC if only one of them owns the game?

I recently bought Sea of Thieves for the Xbox and thanks to Xbox Play I can play it anywhere on my PC. My brother has a second account on the console where he can also play the games I bought, but when I start Sea of Thieves on my account, I can't play with him. If he buys the game on his account, he can play with me.

If he buys it himself, then of course he can play with you