I have this issue too. It happens on both my work desktop and my home desktop. Both Win11 with Start11.
A reboot seems to always fix it. After a reboot, it will work fine for an undetermined amount of time, but eventually I'll stop seeing the thumbnail previews of my multiple open but grouped windows on the primary monitor.
This happens often, and while it is behaving oddly, I can not see the previews for the windows on my primary monitor. I can hover over the taskbar icon on my secondary display and see previews of the various Chrome or FireFox or Explorer windows while they're on the secondary displays. If take said windows from secondary monitor (where I can see the thumbnail previews) and drag them over to my primary monitor, then I can no longer get the thumbnail preview. If I drag it back over to a secondary monitor then I can see the see the thumbnail preview again.
If I reboot, I'll be able to see the thumbnail preview on the primary monitor again for some number of whiles but then it go weird again within a day or two.