I don't know if its Entrenchment, or the mod that made them smarter
It's Entrenchment, though I wouldn't put it past the DS guys to further improve upon that behaviour. You'll need to keep your scouts in play. If the enemy shows up in force to secure the neutrals, let them have it. You can always come back in five minutes once they've left. In any case, this still doesn't excuse not having the map explored.
Once I set them on auto-explore, the majority of them were getting killed within 10 minutes, or 5-6 phase jumps, after creating them.
On larger maps like this, auto-explore on its own does not work. The auto-explore AI has certain inefficient behaviour quirks that can be corrected with at least some manual scouting. These quirks unfortunately get worse on larger maps, so on one of this size you cannot rely on auto-explore for the more distant areas in the solar system.
But a few minutes later they'd typically just take them for themselves with a colony frigate.
You should probably have a pack of scouts (4-6) roaming about. This is enough scouts that they can gang up and kill an unguarded colony frigate. You can always come back later when the enemy gets around to retaking the extractors. It's an ongoing thing, but Vasari need to keep coming back and hounding the enemy.
should I only research how to capture ice/volcanic planets
That's certainly one way to do it. Your problem is that you're getting too many civic techs; a couple on the side isn't a bad thing, but too many can leave you without money for more important pursuits. Early on, that's primarily the task of acquiring planets.
But I heard that the most important ones to get in the civic tree were Scavenging, and Enslaved labor for the Vasari in the early game. Is this true, or only true in certain scenarios?
I haven't actually done a breakdown on scavenging, but unless you're destroying enemy structures on a very regular basis I'd avoid it early on. The problem with this tech is that it's rather expensive to research, and at the same time requires you to be actively fighting the enemy and destroying structures.
As far as Enslaved Labour goes, some people swear by this tech, others are more reserved. I'm in the reserved category; it's a great tech whose savings can add up, but that's a steep cost to pay in the early game. It's definitely a worthwhile tech if you have the time and money on hand to go for it, but I wouldn't pursue it at the cost of my military or empire.