Problem is solved.
The installed rstpvapgdecoder.dll was not good, this file was installed as a ActiveX file
Because the camera works fine on my Vista PC and because I could not find any blocking from port 556 I try to copy the rstpvapgdecoder.dll from the Vista PC to the XP PC and I was lucky, it works fine after that.
Thank you for helping me.