Let me start by saying that I love Fences!
This has happened three times in the last two weeks on two different PCs. It never happened before that. It may be related to the Windows 20H2 update.
All three times, it happened while I was rearranging (dragging and dropping) desktop icons. It has happened with individual icons (outside any fence) and it has happened when moving an entire fence. When it happens, all icons (or fences) disappear except for the one I'm dragging. Double clicking the desktop only hides/reveals that single icon or fence. The only way I have found to restore the entire desktop is to restart the PC.
I suspect fences only because hiding desktop icons is a fences feature.