No, please don't worry about it. I'm not offended and I was the one who said it first.

[The question makes him pause though. How does he explain this stuff uhhh]

Well, statements that may sound too critical or make people uncomfortable are usually avoided. Also, questions that may pry too deeply into someone's background aren't considered acceptable until you know them a little better.