I create a unit of each weapon type that uses zero crystal but maximizes all other aspects. There are trinkets that require zero crystal (ie. First aid kit, etc.)   I give these units the prefix "Town" or Tower" in front of their name (ie. Town archer") . The melee units come in the mounted/ non mounted variety. (ie. "mounted tower sword" or just "Town Sword"). Unmounted units sit in towns. Mounted units sit in towers or unique resources (ie. resources that give unrest reduction/research boost). These are the units I typically fight other factions with. They make the bulk of my general army.

 Of course my sovereign and top 2-3 heroes run with crystal laden troops using the prefix "King's"  (ie. Mounted King's Axe" or "Mounted King's Archer"). They also run with a Henchman specializing in healing if the hero cannot do the healing himself. These are the monster killers. They roam at will, solving quests  and killing dragons with little worry or regard toward rushing home to defend the capital.  The general "metal" army takes care of the wars. The one exception is that mages require crystal and are necessary for the wars, but I keep the crystal to a minimum. Typically used in just the staffs.

I do this because there is simply not enough crystal to wield a huge elite army.

In the power level sense....yes. Many also do elemental type damage which is better against many creature types....though there are those that are resistant to cetain elemental damage types....so build your armies with care.

The problem, or I would argue...balance, to the puzze is that there is only about half the amount of crystal on the map compared to iron. Combine that with the fact that you harvest crystal at half the rates as well, you in effect have only one quarter the amount. You'll most often be short this valuable resource. (which is how it should be.)

Currently the game outputs an overabundance of metal in the game so three shouldn't be too much trouble to trade AI factions for it.

Several of us in the community...and I'm sure Stardock as well....is currently looking to scale back iron ore in some fashion....or make it more important in some way.

I'm confident there will be a patch one day that will address this concern. 

I've never checked this myself, do crystal weapons cost the same labor as iron or is it more?