The literal meaning of "stalks" is the following one:
Moves in an arrogant and menacing way.
It is clear in the poem that the bird is angry due to being locked up in a cage. The figurative expression "bars of rage" shows the reader that. Therefore, when it is said that the bird "stalks down his narrow cage", it means the animal moves in a stiff, haughty manner, as well as threatening.
