There are many very good reasons why someone takes an elevator one floor. But sometimes, it looks like people who should have no trouble with stairs (young, fit people) are taking space in the elevator and possibly making others wait, only to go one floor up or down. For someone fit and able and not encumbered with things, it can even be easier and faster to take the stairs, leaving the elevator's premium space for people like you, who need it.
That said, I blame our culture of locking stairwells in many buildings, as that deters people from using the stairs, thus making the elevator the habitual option. If people stopped to think about it, they might use the stairs, but they can't at work or at the doctor's office, so they don't in other places, either.