Pretty much ALL PRNGs have this feature, but nobody ever implements the go-to-previous state method.
Consider:
Since a PRNG has a finite-sized state, calling next() repeatedly will eventually get into a state you been in before.
If you start off in a state that will eventually be reached again by making N calls to next(), then you can always find a predecessor state by making N-1 calls to next().
Of course, in real life you would want a much more efficient way to implement prev(). How to do it would depend on the RNG, but it's generally not too hard.
Note that, in order to make the best use of their state, PRNGs are designed to have cycles that are as big as possible. Usually all accessible states are in the same cycle.
EDIT:
I should add that when people want to do this in real life, they often just use a counter as the internal state, and hash it (or encrypt it with a fixed key) to generate each random number. In a cryptographic context this is called "CTR MODE" encryption, which is googlable.
Of course if you do this, then you get random access to every state.