there are several issues that appear to need clarification.
a. 'in public' is not the same as 'public school' or a 'public park'. when you walk out of your door, you're 'in public' but perhaps not on public property. the space on a public thoroughfare in front of your home isn't yours to rent or occupy exclusively (as you'll discover should you try to block others from parking there or if you yourself park there on the wrong day or at the wrong time). while everyone has the right to express themselves individually in public (and nobody is arguing that isn't or shouldn't be the case), you are unlikely to be permitted to interfere with or block access to a public area such as a street.
if you own property that is open to the public--a mall or store--you have the right to express yourself or your opinions publicly in compliance with zoning regulations, commerce codes, etc. if you engage in interstate commerce, other regulations also apply. otherwise, if you wish to erect a creche, permit members of the hare krisna sect to solicit donations or chant to the solstice on your premises, you're free to do so.
b. facilities, buildings, parks, parking lots, etc., owned in common by the taxpayers are subject to regulations by agencies operating or administering them on our behalf. those regulations--and their application--must conform to local, state and federal laws.
c. permitting use government facilities or property owned in common by the public clearly benefits the user at the expense of taxpayers. it also creates a possibility--if not the probability--of situations requiring administrators of those facilities or properties to choose one potential user over another. clearly they are not to favor one group over the other on the basis of religion or their own spiritual beliefs. madison claims that government should have no cognizance of religion. that may be too much to expect from humans who act as government proxies. it is possible to provide no basis for such cognizance--thus eliminating any possibility of administerial malfeasance or misfeasance--by merely prohibiting use by any group that requiring of its members any profession of religiosity or lack thereof.
d. the constitution specifically forbids such a requirement of government officials or representatives in article vi: The Senators and Representatives before mentioned, and the Members of the several State Legislatures, and all executive and judicial Officers, both of the United States and of the several States, shall be bound by Oath or Affirmation, to support this Constitution; but no religious Test shall ever be required as a Qualification to any Office or public Trust under the United States.
note there is no mention of god or a higher power in the oath of office mandated in article ii, section 2 of the constitution: "I do solemnly swear (or affirm) that I will faithfully execute the Office of President of the United States, and will to the best of my Ability, preserve, protect and defend the Constitution of the United States."
if the boy scouts of america, or any other organization, feels it necessary to require its members to take an oath invoking or asserting a belief in god, they disqualify themselves from use of publicly-owned facilities. if parents wish their children to receive a religious education, they need only to enroll them in an accredited church-run school.
as madison stated in his 'Memorial and Remonstrance Against Religious Assessments' : Whilst we assert for ourselves a freedom to embrace, to profess and to observe the Religion which we believe to be of divine origin, we cannot deny an equal freedom to those whose minds have not yet yielded to the evidence which has convinced us.