I dont think i had this problem, but then again i had been learning a few years by watching movies and listening to songs before i met some actual English speakers. The benefit of a two way conversation is you can always ask them to repeat.