Look into the Kinect Toolbox. It offers a recorder and playback functionality which may match your needs as is, or provide you with a starting point:
http://kinecttoolbox.codeplex.com/
If you role your own, I'm not sure why you would need to map it to a color or depth frame, unless I'm missing a requirement of what you are doing.
Have a look at the SkeletonBasics example in the Microsoft Kinect for Windows SDK Toolkit examples. It will show you have to draw a skeleton manually based on skeleton data. From there, you could look into doing the following for you application:
- Set up your skeleton tracking callback
- At each skeleton frame, or less (if you don't need so many) save the joint positions
- Also save a 0-based timestamp
- Save data to format of choice when complete
During a playback, read in your recorded data and start a timer. When the timer hits the next skeleton frame's stored timestamp update your drawn skeleton on the screen (using the SkeletonBasics example app as guidance).