Playing a Phone Message

I have a 4S. When I attempt to play a message that someone left me, Siri pops up as I lift the phone towards my ear. If I leave the phone flat, it doesn't happen. I have to play a game with lifting it slowly in order to hear the recorded message without interference from Siri.

Help!