That is exactly what America was built upon.
You and I have the right to do whatever we please, whenever we please, where ever we please. The only caveat is that our actions to do not adversely impact another's freedoms. When we ignore this fact and act in an irresponsible manner that has a measurable adverse effect on another, then we can expect to be held accountable for our actions.
The quintessential example often used is: I have the right, if I so desire to swing my fists back and forth all day long and not expect any interference. However, once my fist comes in contact with your nose, then I can expect to suffer the consequence of my irresponsibility to the fullest extent of the law.