How can I serialize a near-sdk-rs LookUpMap that uses a String as a key, or is there a better way?

Question

I am adding a feature to the Sputnik DAO v2 near contract to map tokens to a number of votes.

Here is my constructor:


#[ext_contract(ext_self)]
pub trait Contract {
    fn exchange_callback_post_withdraw(&mut self, sender_id: AccountId, token_id: String, amount: U128);
}

#[near_bindgen]
impl Contract {
    #[init]
    pub fn new(
        owner_id: AccountId,
        token_ids: UnorderedSet<String>,
        unstake_period: U64,
        token_vote_weights: LookupMap<String, U128>,
    ) -> Self {
        Self {
            owner_id: owner_id.into(),
            vote_token_ids: token_ids,
            users: LookupMap::new(StorageKeys::Users),
            total_amount: UnorderedMap::new(StorageKeys::ValidNFTs),
            unstake_period: unstake_period.0,
            token_vote_weights,
        }
    }

From: https://github.com/roshkins/sputnik-dao-contract/blob/nft-tokensv4/sputnik-nft-staking/src/lib.rs#L59

I get this error:

error[E0277]: the trait bound `LookupMap<std::string::String, near_sdk::json_types::U128>: Serialize` is not satisfied
   --> sputnik-nft-staking/src/lib.rs:67:1
    |
67  | #[near_bindgen]
    | ^^^^^^^^^^^^^^^ the trait `Serialize` is not implemented for `LookupMap<std::string::String, near_sdk::json_types::U128>`
    | 
   ::: /home/rashi/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.64/src/value/mod.rs:965:8
    |
965 |     T: Serialize,
    |        --------- required by this bound in `to_value`
    |
    = note: required because of the requirements on the impl of `Serialize` for `&LookupMap<std::string::String, near_sdk::json_types::U128>`

How might I serialize the string, or is there a code change that might better help achieve my goal of associating a voting weight with a token?

score 2 · Answer 1 · answered Sep 08 '21 at 13:45

So there are two main points about why this can't be serialized (assuming you're expecting it to be serialized as a map):

LookupMap does not store all keys in storage, so it is not iterable and is not aware of what it contains. If you want an iterable map, you can use UnorderedMap or TreeMap from the SDK.
Even if it was iterable, like in the case of the data structures mentioned above, it is very expensive to load all elements if the data structure grows large. This serialization implementation isn't done by default because it should generally be avoided.

If you want to return the whole contract state as JSON, you would have to manually implement serialize or put values into a structure that can be serialized. If you do want to load the storage values from storage to return from the function, consider either limiting the max amount of values or having some sort of pagination.

Can limit doing something like this. Not limited to this but just an option:

let map: HashMap<K, V> = storage_map
                           .iter()
                           .skip(<skip first x elements for pagination>)
                           .take(<Max amount of elements to iterate>)
                           .collect();

I changed the type to Hash, and it still didn't work. I noticed that it looked for Serde serialization in the error, shouldn't it be using Borsh? — Rashi Abramson, Sep 08 '21 at 19:24
I assumed you wanted serde serialization for this. You can do borsh, but it shouldn't matter which you pick. The code I indicated was for when you need to manually implement `serde::Serialize` for a struct that contains the thing you want to serialize. I'm just now noticing that you are trying to pass these storage structures as parameters, which you probably don't want to do, but I will address it in the answer below. — Austin, Sep 10 '21 at 22:29

score 1 · Answer 2 · answered Sep 09 '21 at 00:00

The solution is to override the serialization default, since serialization is implemented for Borsh, not JSON/serde: https://www.near-sdk.io/contract-interface/serialization-interface#overriding-serialization-protocol-default


#[near_bindgen]
impl Contract {
    #[init]
    pub fn new(
        #[serializer(borsh)] owner_id: AccountId,
        #[serializer(borsh)] token_ids: UnorderedSet<String>,
        #[serializer(borsh)] unstake_period: U64,
        #[serializer(borsh)] token_vote_weights: LookupMap<String, U128>,
    ) -> Self {
        Self {
            owner_id: owner_id.into(),
            vote_token_ids: token_ids,
            users: LookupMap::new(StorageKeys::Users),
            total_amount: UnorderedMap::new(StorageKeys::ValidNFTs),
            unstake_period: unstake_period.0,
            token_vote_weights,
        }
    }

    /// Total number of tokens staked in this contract.
    pub fn nft_total_supply(&self) -> U128 {
        let sum = 0;
        for i in self.total_amount.iter() {
            sum += i.1;
        }
        U128(sum)
    }

So I think I might have misunderstood your issue. These storage structures shouldn't be passed as parameters because they will only deserialize metadata of the structure (in this case the prefix, which should always be constant). Instead of this, you'll want to just use a `std` structure to deserialize the data in the format you want, then fill the storage structure before constructing the state. I'll write a new answer with a suggestion. — Austin, Sep 10 '21 at 22:31
It was easier to just modify the code, here is the commit: https://github.com/austinabell/sputnik-dao-contract/commit/500c300773214478caedfca5bb60f1714ecb82b4 — Austin, Sep 10 '21 at 22:44

How can I serialize a near-sdk-rs LookUpMap that uses a String as a key, or is there a better way?

2 Answers2

Linked