You can uncheck "Allow Navigation", but then you can't navigate folders. I had this same issue and checked #3 in this troubleshooting FAQ and it solved the issue while still allowing navigation in portals. https://forums.stardock.com/486084/fences-support-faq#PortalNavFix
For some reason, everytime I do a windows update, this setting resets to the checked state & for Fences to work properly, it needs to be unchecked. See step #3 https://forums.stardock.com/486084/fences-support-faq#PortalNavFix
1 from hamburger menu on folder portal title — select/check "Allow navigation"
2 under folder options "General" tab — select/check "Open each folder in the same window" & click Apply
3 under folder options "View" tab — unselect/uncheck "Launch folder windows in a separate process" & click Apply