The trick of this trick question is on you my friend. You (like many others) are looking at the Costitution backwards. The document does not address itself from the Federal Government to We, the People, but just the opposite. It addresses itself from We, the People TO the Federal government.
Therefore, it doesn't have to say anything about "privacy" for We the People to have that right... What it would have to do for the Federal Government to be able to infringe on the right to privacy that We the People retain for ourselves is enumerate to the Federal Government the circumstances for which We the People allow privacy to be infringed. In that, the Constitution is very clear.
Amendment 3 states that the Federal Government cannot infringe on the security of our persons, houses, papers and effects except for reasonable searches and seizures. Reasonable searches and seizures being those which where a warrant is issued.
So, in answer to your question... Yes, we do have the right to privacy and also, yes, there are circumstances where we have given the Government the authority to infringe upon it.