They're not seriously going to walk out just because you show up. Tell them what you just told me, that you're an idiot and you've been an utter asshole and whatever else.

If they put up with me and half of them tolerate Xander, they'll at least coexist with you.