Since this is not normal hex conversion, normal tricks won't apply.
The hexadecimal value for the decimal value 3533 is 0xDCD and that's not what you want.
Instead you want each digit of the decimal value separated out into its own byte, so basically you want this:
DEC = abcd
"HEX" = 0a0b0c0d
You can do this with this simple calculation:
DECLARE @VALUE INT = 1234
SELECT CONVERT(VARBINARY(8),
(@VALUE % 10) +
(@VALUE / 10 % 10) * 16*16 +
(@VALUE / 100 % 10) * 16*16*16*16 +
(@VALUE / 1000 % 10) * 16*16*16*16*16*16
) AS X
Output:
01020304
Note that the integer value can be found without the conversion, the conversion above is just to get it formatted like a hex value when testing. "Hex" is just a visual representation, the underlying number is the same.
In other words, the numeric value that corresponds to 0x0a0b0c0d can be found by just:
DECLARE @VALUE INT = 1234
SELECT
((@VALUE % 10) +
(@VALUE / 10 % 10) * 16*16 +
(@VALUE / 100 % 10) * 16*16*16*16 +
(@VALUE / 1000 % 10) * 16*16*16*16*16*16) AS X
The hex output is just a formatted representation.