Yeh, for me, it's the eye strain from my eyes trying to "focus" on an image that's already focussed (and impossible to resolve the way my brain thinks it should be). This is similar to the Buscemi-with-four-eyes picture[1] - it tricks your eyes into continually changing focal distance in an attempt bring the picture into focus. It is physically straining your eye muscles.
Oh, now I know why I hate that ladder E: your sight tells you it's an E, but it has a bit more up, so it analyses it, and it's still an E, and the process repeats until your sight tells you that can't possibly be an E anymore.
https://cdn.q-dance.com/l6hfsc63q612/6q72WINFR6Elorje1a49Nt/...