shauk, how many decades have you been living on this planet?
if you're in your 20s or 30s and havent had sex many times (regardless of # of distinct women), it's not too unusal.
now if you're in your 40s or 50s or 60s, there might be an underlying psychological or medical issue.
talk w/ another doctor who specializes in sex or a sex therapist. there are things that you can do to gain some *control* over your body and your reactions.
all is not lost ... dont even go down that road.
hang in there!