I had an idea similar to this awhile back. I had thought diplomacy would add this type of thing, and suggested that you could essentially be offensive without using fleets/military tree by getting culture going early and using it to acquire neutral worlds. Also, the envoys would either enhance the allegiance affect of your culture or deter the allegiance affect of an enemy. Culture wouldn't be the only thing you used to acquire neutrals, you'd need envoys to get you some diplomatic influence.

Lastly, I wanted the option to ally with neutral worlds. With some diplomatic research, you wouldn't have to wipe out a planet to get past it. Then later, your diplomatic efforts might get them to join you.

In general, I felt a the culture system should be strongly connected to the diplomatic system; both together should be a viable way to expand

I kinda felt that military starts would aquire the first few systems before the diplomatic method would, but when the diplomatic method had got going you would be acquiring multiple systems simultaneously, which would make it stronger at that point. On the other hand, your planets would be fairly unprotected, so a military incursion could be quite disruptive.

We'll see what they do.

If you follow Galactic Civilization's explanation, population is actually representative of citizens and not actual planetary populations.  That explains why a single colony frigate can get a colony up and running.  As you invest in civilian infrastructure you're basically building the sorts of things you'd see on worlds in Gal Civs.  If you read the message in Sins before you lose a planet is says something like 'Cultural rebels threaten to overthrow our government.'  That's basically a confirmation that your 'population' isn't the planet's actual population.  In Gal Civs you were only interested in the workings of the government.  The private sector was left to do its own thing, though you'd see income from setting up trade routes and tourism(calculated by total galactic citizenry and divided based on each faction's influence percentage).  This government focus is carried through into Sins.  It is just that Sins greatly simplifies the planet management, and focuses on the space warfare part heavily.  Several planetary structures from Gal Civs were adapted to be orbital structures instead.  E.g. factories and starports became frigate factories and capital ship factories, entertainment centers became culture buildings, markets were rolled together with trade freighters into trade ports, etc.