I'm guessing it's because your memories were so messed up, back then.

[ He frowns. He remembers what it was like at the beginning of his program. He didn't have it nearly as bad as Ardyn, but that's not particularly surprising. Noctis sighs and sips at his tea. ]

Yuuri was the glue that kept you together, huh? I didn't realize she was that important to you.