I can't add non-armor equipment to units until that equipment has been research, so that type of gear can only be added to brand new units. I guess wearing a belt requires special barracks-only training that can't be learned in the field. Some of my units also won't upgrade their armor at all, such as the aforementioned archers. Level 4 is a paltry amount of health relative to level 10 or even 14 like my troops are at right now, so I'm looking at taking a massive hit to health for this required rebuild.
That does answer my question though: I haven't missed anything, I do indeed need to scrap all of my troops to give my army minor gear upgrades.