Everyone fixates on this, it's just normal where he comes from. Entire neighborhoods leveled from The Incident.

"Yeah, I guess. They call 'em infinity stones, I think? A blue thing called the tesseract Captain America tried to bury. It fell into the wrong hands. At least, that's the gist I got from the news and people telling me when I got home." He was overseas for that whole thing, just another way he couldn't protect his family in the end.