Deserializing an optional struct field with a generic type leads to semantic weirdness

Question

I am attempting to deserialize JSON into a struct that contains an optional field authorization. The JSON may or may not include this field. If it does include the field, I'm doing a custom deserialization into a hyper::header::Authorization<hyper::header::Scheme>. Because Authorization requires a generic type for Scheme, I'm required (as I've written it) to include the generic type on my struct.

All of the tests pass, but the last one (de_json_none, the one for JSON without the authorization field) is semantically weird because I have to target a variable with a definite Scheme type (either Bearer as shown or Basic), neither of which makes any sense for that data, despite being perfectly valid from Rust's perspective.

It's clear why that is the case, but it's something I don't want and something I'm not sure how to fix.

I want to write a Rocket handler that only matches data that contains the authorization field of type Authorization<Bearer> by setting the data type to Headers<Bearer>. At the moment, it would also match data that doesn't have the field at all. I'm also stuck without a clear way to call out the data with the missing field specifically by type.

I'm looking for suggestions on how to refactor this code to reflect the fact that Headers really has three distinct, mutually-exclusive incarnations (Basic, Bearer and None). Perhaps I should be looking to do something with an enum here?

extern crate hyper;
extern crate serde;
extern crate serde_json;
#[macro_use]
extern crate serde_derive;

use hyper::header::{Authorization, Header, Raw, Scheme};
use serde::{Deserialize, Deserializer};

#[derive(Debug, Deserialize, PartialEq)]
struct Headers<S>
where
    S: Scheme + 'static,
{
    #[serde(deserialize_with = "auth_header", default = "no_auth")]
    authorization: Option<Authorization<S>>,
    #[serde(rename = ":path")]
    path: String,
}

fn auth_header<'de, D, S>(deserializer: D) -> Result<Option<Authorization<S>>, D::Error>
where
    D: Deserializer<'de>,
    S: Scheme + 'static,
{
    let s = String::deserialize(deserializer)?;
    let auth = Authorization::parse_header(&Raw::from(s.into_bytes()));
    auth.map(|a| Some(a)).map_err(serde::de::Error::custom)
}

fn no_auth<S>() -> Option<Authorization<S>>
where
    S: Scheme + 'static,
{
    None
}

#[cfg(test)]
mod test {
    use hyper::header::{Basic, Bearer};
    use serde_json;
    use super::*;

    #[test]
    fn de_json_basic() {
        let data = r#"{
                        "authorization": "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==",
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: Some(Authorization(Basic {
                username: "Aladdin".to_owned(),
                password: Some("open sesame".to_owned()),
            })),
            path: "/service/".to_owned(),
        };

        let h: Headers<Basic> = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }

    #[test]
    fn de_json_bearer() {
        let data = r#"{
                        "authorization": "Bearer fpKL54jvWmEGVoRdCNjG",
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: Some(Authorization(
                Bearer { token: "fpKL54jvWmEGVoRdCNjG".to_owned() },
            )),
            path: "/service/".to_owned(),
        };

        let h: Headers<Bearer> = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }

    #[test]
    fn de_json_none() {
        let data = r#"{
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: None,
            path: "/service/".to_owned(),
        };

        let h: Headers<Bearer> = serde_json::from_str(data).unwrap();
        // this also works, though neither should ideally
        // let h: Headers<Basic> = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }
}

Answer 1

There's no concept of a None without a corresponding Some type. The compiler needs to know how much space to allocate for the value for either case:

struct ReallyBig([u8; 1024]);
struct ReallySmall(u8);

fn main() {
    let mut choice = None; // How much space to allocate?
}

In your code, the size of Authorization can depend on the value chosen for S. Since Headers contains an Option<Authorization<S>>, the size of Headers also can depend on the choice of S.

Even when you get no value, you must choose to parse into some specific type. Perhaps you will later manually change it from a None to a Some by building the appropriate values — if it wasn't allocated with enough space, that would be trouble!

Because of this, I can't see how your solution will work. Types are static — you need to know at compile time if decoding that JSON is going to result in Authorization or Bearer, and that's simply not possible.

Normally, I'd suggest you use dynamic dispatch with a Box<Scheme>. This won't work here because Scheme isn't object-safe.

Then, I would suggest you implement your own enum wrapping either Basic or Box and implement Scheme for that. This doesn't easily work because Scheme::scheme has to return a single keyword, but you actually support two keywords!

The next step up is to implement our own Header:

extern crate hyper;
extern crate serde;
extern crate serde_json;
#[macro_use]
extern crate serde_derive;

use hyper::header::{Authorization, Header, Raw, Basic, Bearer};
use serde::{Deserialize, Deserializer};
use std::fmt;

#[derive(Debug, Clone, PartialEq)]
enum MyAuthorization {
    Basic(Authorization<Basic>),
    Bearer(Authorization<Bearer>),
}

impl Header for MyAuthorization {
    fn header_name() -> &'static str {
        // Should always be the same header name, right?
        Authorization::<Basic>::header_name()
    }

    fn parse_header(raw: &Raw) -> hyper::error::Result<Self> {
        Authorization::<Basic>::parse_header(raw)
            .map(MyAuthorization::Basic)
            .or_else(|_| {
                Authorization::<Bearer>::parse_header(raw).map(MyAuthorization::Bearer)
            })
    }

    fn fmt_header(&self, f: &mut hyper::header::Formatter) -> fmt::Result {
        match *self {
            MyAuthorization::Basic(ref a) => a.fmt_header(f),
            MyAuthorization::Bearer(ref a) => a.fmt_header(f),
        }
    }
}

#[derive(Debug, Deserialize, PartialEq)]
struct Headers {
    #[serde(deserialize_with = "auth_header", default)]
    authorization: Option<MyAuthorization>,
    #[serde(rename = ":path")]
    path: String,
}

fn auth_header<'de, D>(deserializer: D) -> Result<Option<MyAuthorization>, D::Error>
where
    D: Deserializer<'de>,
{
    let s = String::deserialize(deserializer)?;
    let auth = MyAuthorization::parse_header(&Raw::from(s.into_bytes()));
    auth.map(Some).map_err(serde::de::Error::custom)
}

#[cfg(test)]
mod test {
    use hyper::header::{Basic, Bearer};
    use serde_json;
    use super::*;

    #[test]
    fn de_json_basic() {
        let data = r#"{
                        "authorization": "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==",
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: Some(MyAuthorization::Basic(Authorization(Basic {
                username: "Aladdin".to_owned(),
                password: Some("open sesame".to_owned()),
            }))),
            path: "/service/".to_owned(),
        };

        let h: Headers = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }

    #[test]
    fn de_json_bearer() {
        let data = r#"{
                        "authorization": "Bearer fpKL54jvWmEGVoRdCNjG",
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: Some(MyAuthorization::Bearer(Authorization(
                Bearer { token: "fpKL54jvWmEGVoRdCNjG".to_owned() },
            ))),
            path: "/service/".to_owned(),
        };

        let h: Headers = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }

    #[test]
    fn de_json_none() {
        let data = r#"{
                        ":path": "/service/",
                        ":method": "GET"
                      }"#;

        let message = Headers {
            authorization: None,
            path: "/service/".to_owned(),
        };

        let h: Headers = serde_json::from_str(data).unwrap();

        assert_eq!(message, h);
    }
}

You may wish to check with the Hyper maintainers to see if this is the expected way of doing such a thing.