I'm going to go with it's a deliberate obstruction. If Penny ever gets into Bucky's brain, it's like an endless labyrinth, with dead ends and blind corners, and even some ways that are not what they seem. So... yeah, it'd be interesting.