I wasn't convinced that we didn't. Apparently we do. But what's more is the gene flow only seems to go into the human population and not the other way around. My previous speculation on this topic was largely concerning the fact that it would be human women rather than neandertal women who would be having sex with the other group. If that were the case then all the mitochondrial DNA would track to human populations in Africa. Allow me to offer a novel theory as to what happened to the Neanderthals... human girls were prettier.