3

We have ran into an issue with serialisation over JSON when using Firestore as our database.

When writing dates to Firestore it's recommended that we use the TimeStamp object. We're happy with that. I have a converter that converts all my DateTime's to TimeStamps when doing the toJson and fromJson. So reading and writing to Firestore from my dart client works fine.

The problem comes in when we use an api endpoint and send the data over http using json encoding. We've realised that we cannot send a TimeStamp object over json. We want to avoid as much as possible the usage of multiple models to represent the same data. So I'm trying to figure out how can we write raw time stamps to firestore.

Now leading to my question.

If I wanted to write a TimeStamp to firestore, without using the TimeStamp type how would I do that? It must be stored in some way, at the moment I can't find that through firestore UI or local emulator UI.

I'd rather serialise to and from TimeStampRaw than rely on the TimeStamp object because of the lack of fromJson / toJson in it.

Edit: For Clarity

  1. toJson() => Writing to Firestore directly from app we use a firestore.TimeStamp type and Firestore understands it great

  2. fromJson() => Reading from firestore directly works great

  3. toJson() => Post to our backend. There's no TimeStamp type encoding so we can't send it as a timestamp. This is the problem 1

  4. fromJson() => Reading a response from the backend. Can't serialise a timestamp from a string. This is problem 2

We want to use the same toJson and fromJson functions. We can't use millisecondsSinceEpoch as a String because then we lose accurate orderBy? As mentioned here . Hence me asking for the way to store TimeStamp as a raw value that Firestore will interpret as a TimeStamp

Filled Stacks
  • 4,116
  • 1
  • 23
  • 36

2 Answers2

1

You can try writing it as an epoch.

  DateTime.now().millisecondsSinceEpoch

which will result in an int that you can then turn into DateTime again using

DateTime.fromMillisecondsSinceEpoch(millisecondsSinceEpoch)

Note that you will lose the microseconds in that time so the accuracy is going to be 99.9999%

Alternatively, you can store the date-time, sample code: create a day module

   class DayModule {
  DateTime? date;
  String? day; // derived from .toString then substring

  DayModule({
    this.date,
    this.day,
  });
  factory DayModule.fromJSON(map) {
    return DayModule(
        date: DateTime.fromMillisecondsSinceEpoch(map['date'].seconds * 1000),
        day: map['day']);
  }

  toJSON() {
    return {
      'date': date,
      'day': day,
    };
    }
    }

Storing and exporting data :

var tempDay = DayModule(date : DateTime.now(), day : DateTime.now().toString().subString(0,10))

await FirebaseFirestore.instance.collection("days").doc("123").set(tempDay.toJSON())

// retrieving it 

    DayModule day;
    await FirebaseFirestore.instance
        .collection('days')
        .doc("123")
        .get()
        .then((value) => day = DayModule.fromJSON(value.data()));
YaDev
  • 33
  • 1
  • 6
  • And this will convert it to a TimeStamp type in Firstore if I write it as an int? – Filled Stacks Mar 17 '22 at 12:53
  • no, it will not, it will be stored as an int there. which then you have to convert it on your client accordingly as I showed you above. But if I may suggest something, Try storing DateTime directly to FirebaseFirestore it will automatically convert it into TimeStamp and then what you can do on your client side is to decode it like : DateTime.fromMillisecondsSinceEpoch(value.data()['time'].seconds) which will give you more flexibility and control – YaDev Mar 17 '22 at 13:21
  • it doesn't convert to a TimeStamp. It writes out a DateTime string. Then the order by query is not accurate on the backend. There's no automatic conversion from DateTime to TimeStamp on firestore. When I write it out it's just a DateTime String. – Filled Stacks Mar 17 '22 at 13:35
  • How are you handling your conversion? from and to firestore – YaDev Mar 17 '22 at 13:38
  • using a JsonConverter on the property that does toIso when writing toJson and converts from String. Remember this has to work reading from firestore and from our http json api endpoint. We want to use the same data model. When I send it to the backend it has to be in a format they can understand, which is anything really, when I get it in a request from json it has to be in a format I can understand. The problem comes in when I have to write date time to firestore and I use the toJson method, we're currently using TimeStamp so orderby works, but that doesnt work for http json model. – Filled Stacks Mar 17 '22 at 13:42
  • Added more in the post. I suspect that what I want doesn't work so we'll probably have to store an int for miilisenconds and order by that. I just have to double check that the orderBy is reliable when using an int instead of a timestamp. – Filled Stacks Mar 17 '22 at 13:50
  • I don't think you understand the question. I'm not struggling with the conversion. The problem is that I want to write to firebase and backend with the same model. Firebase requires TimeStamp, there's no native json encoding for a timestamp. This means my toJson won't work for one of those scenarios if I apply a strict conversion. The same goes for reading from firestore. I cant depend on a string, firestore returns a TimeStamp, I can't serialise from a TimeStamp, the api returns a string, int or map. I wrote my answer below that works fine for now. I'll update it if I get something better. – Filled Stacks Mar 21 '22 at 09:41
0

There doesn't seem to be a raw TimeStamp format that would make Firestore automatically update the type of the date to a timestamp. So what we decided for now is to write a custom converter for the fromJson and write a second toJsonApi call that converts a TimeStamp into a known format for the Api. This way I can call my normal toJson for writing directly to Firestore and the custom toJson function when posting to the Api. It's an extra method per object, but at least we can use the same models across firestore and our api.

From Json

I created a custom fromJson function that will check for all keys ending in _at which is our convention for any property that reflects time. I will then check the value of that property and:

  • If it's a String I'll assume it's a DateTime.iso8601() and parse it to a DateTime
  • If it's a TimeStamp I'll call .toDate on it.

To Json

I do the normal toJson call that's generated. I then inspect that map for keys ending in _at and if the type is TimeStamp I will call toDate.toIso8601 on it and send that to the api.

This is not ideal but I'd rather have it work and then improve on it later if there's a better way.

Filled Stacks
  • 4,116
  • 1
  • 23
  • 36