A common usage of 我去 is 我了个去 or 哎呀我去. It's just used to express surprise，ridiculousness, astonishment, etc..
It's not relevant to "damn, shit, fxxk" or anything. I think it's more like "what the heck/hell" in English.
I also disagree that it's short for 我去你的. 我去你的 is a dismissal phrase while 我去 doesn't connote that sense.
去你的 : (verb)
"我去你的" = "I (am saying this): damn!"
去你的 literally mean "go yours" (implying 'go to hell')
我去 is a proclamation. It is similar to "Damn!/ Shit!/ FXXK!" " in English.