I am a pretty confident traveler and enjoy meeting new people, particularly non-Americans. It is fun to learn about where they are from, their experiences and thoughts. I have had people hang onto me if they are not confident travelers. Usually it is no problem if the person is interesting to be around but I have ditched one or two in the past if they became too annoyinng.
You can only let others ruin your vacation if you let them. I will spend time with those I find interesting and pretty much ignore those who aren't. I am pretty good at tuning out people when I want to. As my mother told me once, when I want to be around people I will be around people and when I don't want to be around people I prefer to be my myself.
Some of my most interesting conversations of been with people I came across by accident and ended up having really interesting conversations. I was in the UK during the attempted coup in Russia. Before I headed home I came across two different people who where in Moscow as the attempted coup took place. I was able to hear two different first hand accounts about what happened and their experiences. I met an Australian girl who was passing through Zurich on the way back to Hong Kong (just before the Brits gave Hong Kong back to China) and had breakfast with her and learned a lot about what was going on in Hong Kong at that time. I was the only person who booked an afternoon rock and roll tour in London and had a total blast. The guide and I talked the whole time about the London music scene and so much more. When I travel overseas I am far more interested in talking to the locals than Americans.