Right click correspondence is correct: right clicking on the icon shows the context menu for the icon as shown, and right clicking on the hover windows also shows the context menu for the windows.
And yes, the taskbar is set to always combine icons.
That being said, I have a feeling that Modern apps (i.e. things like Settings and Microsoft Store apps) seem to be a causative factor - sometimes clicking on their taskbar icons does not bring them to focus even if they have active windows, and their presence/absence from the taskbar might be related to the icon/hover offset.
In fact, right now I have icons to the left of where the Settings app icon used to be when it was open where the icon/hover correspondence is correct, but to the right of that they are either off by 1 or 2.