No, that's not quite right. If you blur the same picture twice using the same blur filter, then yes, you end up with the same final image. But that's not the same as saying that if you blur two different pictures you end up with two different images. Two different pictures blurred with the same filter can end up being identical blurred images.

You can think of a blur filter as a set of vectors that move pixel information around, but the step you're missing is that the pixel information isn't moved to just one other pixel. It's spread around over several neighbouring pixels, and then added together with the information spread from other pixels that overlaps it. That adding together operation muddies the waters, so to speak - once the blurred pixel info is added together to form the final blurred image, you can't work out how to un-add them to separate them again. There will be multiple possible solutions to the problem of going backwards to an unblurred image, and no way to decide which is the correct solution.