It turns out that it was likely caused by a mod to increase trait and ability points for custom races. Once I disabled mods I was able to start with a custom faction.
The only changes in GalCiv3GlobalDefs.xml were the following (was experimenting with different values to see what was breaking the game):
As it turns out, with that mod enabled, even an unaltered copy of a base faction (albeit with extra unspent trait points) will crash on starting a new game. That is what threw me off and didn't make me suspect the mod immediately.
I understand that you can't be expected to troubleshoot mod problems, but just in case I'll post the "bad" copy of the Terran Alliance. Here