How can I ignore extra tuple items when deserializing with Serde? ("trailing characters" error)

Question

Serde ignores unknown named fields when deserializing into regular structs. How can I similarly ignore extra items when deserializing into tuple structs (e.g. from a heterogeneous JSON array)?

For example, this code ignores the extra "c" field just fine:

#[derive(Serialize, Deserialize, Debug)]
pub struct MyStruct { a: String, b: i32 }

fn test_deserialize() -> MyStruct {
    ::serde_json::from_str::<MyStruct>(r#"
    {
        "a": "foo",
        "b": 123,
        "c": "ignore me"
    }
    "#).unwrap()
}
// => MyStruct { a: "foo", b: 123 }

By contrast, this fails on the extra item in the tuple:

#[derive(Serialize, Deserialize, Debug)]
pub struct MyTuple(String, i32);

fn test_deserialize_tuple() -> MyTuple {
    ::serde_json::from_str::<MyTuple>(r#"
        [
            "foo",
            123,
            "ignore me"
        ]
    "#).unwrap()
}
// => Error("trailing characters", line: 5, column: 13)

I'd like to allow extra items for forward compatibility in my data format. What's the easiest way to get Serde to ignore extra tuple items when deserializing?

score 9 · Accepted Answer · edited Aug 19 '19 at 14:29

You can implement a custom Visitor which ignores rest of the sequence. Be aware that the whole sequence must be consumed. This is an important part (try to remove it and you'll get same error):

// This is very important!
while let Some(IgnoredAny) = seq.next_element()? {
    // Ignore rest
}

Here's a working example:

use std::fmt;

use serde::de::{self, Deserialize, Deserializer, IgnoredAny, SeqAccess, Visitor};
use serde::Serialize;

#[derive(Serialize, Debug)]
pub struct MyTuple(String, i32);

impl<'de> Deserialize<'de> for MyTuple {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        struct MyTupleVisitor;

        impl<'de> Visitor<'de> for MyTupleVisitor {
            type Value = MyTuple;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("struct MyTuple")
            }

            fn visit_seq<V>(self, mut seq: V) -> Result<Self::Value, V::Error>
            where
                V: SeqAccess<'de>,
            {
                let s = seq
                    .next_element()?
                    .ok_or_else(|| de::Error::invalid_length(0, &self))?;
                let n = seq
                    .next_element()?
                    .ok_or_else(|| de::Error::invalid_length(1, &self))?;

                // This is very important!
                while let Some(IgnoredAny) = seq.next_element()? {
                    // Ignore rest
                }

                Ok(MyTuple(s, n))
            }
        }

        deserializer.deserialize_seq(MyTupleVisitor)
    }
}

fn main() {
    let two_elements = r#"["foo", 123]"#;
    let three_elements = r#"["foo", 123, "bar"]"#;

    let tuple: MyTuple = serde_json::from_str(two_elements).unwrap();
    assert_eq!(tuple.0, "foo");
    assert_eq!(tuple.1, 123);

    let tuple: MyTuple = serde_json::from_str(three_elements).unwrap();
    assert_eq!(tuple.0, "foo");
    assert_eq!(tuple.1, 123);
}

score 0 · Answer 2 · answered Aug 28 '19 at 16:01

For JSON, I'd combine RawValue and a custom deserialization:

use serde::{Deserialize, Deserializer};

#[derive(Debug)]
struct MyTuple(String, i32);

#[derive(Deserialize, Debug)]
struct MyTupleFutureCompat<'a>(
    String,
    i32,
    #[serde(default, borrow)] Option<&'a serde_json::value::RawValue>,
);

impl<'de> Deserialize<'de> for MyTuple {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let t: MyTupleFutureCompat = Deserialize::deserialize(deserializer)?;
        Ok(MyTuple(t.0, t.1))
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let json = r#"[
        "foo",
        123,
        "ignore me"
    ]"#;
    let d: MyTuple = serde_json::from_str(json)?;
    println!("{:?}", d);
    Ok(())
}

How can I ignore extra tuple items when deserializing with Serde? ("trailing characters" error)

2 Answers2

Linked