As to where the fees go, something like 85% of the money collected is supposed to stay in the project area. The other 15 % is either overhead to administer or is sent back to Washington.
The budgets for recreation and support of recreation (trails, campgrounds, etc) is not a static number. Every year the Forests and BLM Districts get less and less $, and very little discretion as to where it can be used. Then you add on that every year the fire budget gets blown away in the first half of the season, then they raid every closet and dog house they can find to pay for the rest of the season.
I was very much anti-fee when it first came out, but over the years have come around somewhat, and knowing what kind of money they have to work with every year and what is needed (as we get briefed at the RAC every year), I have come to support it in most cases. They can't keep up with the rest of what they need to do with the reduction in people now. The alternative is a whole lot of gates and locks, which would please the green side to the extent you would not believe.
In the end it will be a personal decision for everyone when they talk/write the knotheads in DC, or they get with the agencies and RAC members, either you do or you don't support fees in the area you want to play in. We are going to have to pay to play in some way, either in a fee that most of it stays in the area, or in higher taxes that Congress may appropriate to the agencies or not.
Personally I figure that a group is more than 6 people, but it can vary by area, type of recreation in that area and the agency.