I just tried to use box_value, but it seems that the non-parameterized constructor will be always called. After I deleted the default constructor, XamlUserType::ActivateInstance() just crashed with null pointer exception.
How to navigate to Parametrised Constructor in UWP?
I found a C# version of solution to this problem, but I'm not sure what is the equivalent way in C++/WinRT.