You should not expect to recover the larger image exactly. If this were possible, we could losslessly compress images by storing downsampled images and the upscaling them. However, some information is lost when the image is downsample which simply cannot be recovered. If you need to downsample the image for runtime reasons, make sure to keep a copy of the original image if needed.
The matlab imresize function provides a number of ways to perform interpolations. It's possible to test the error resulting from a downsample/upsample operation for a particular kernel:
I = int8(imread('rice.png'));
J = imresize(imresize(I,0.5,'box'),2,'box');
mean(abs(J(:) - I(:)))
Running this code with different kernel types yields:
box
4.132904052734375
triangle
4.843124389648438
cubic
4.094940185546875
lanczos2
4.088195800781250
lanczos3
3.948623657226562
Lanczos3 seems to have the smallest error for this image, which isn't too surprising. It's also likely to be the slowest.