0

I would like to converter a Parquet TIMESTAMP_MICROS_STRINGIFIER in Java date format. My code below return a wrong date:

 private static void writeGroup(BufferedWriter w, Group g, MessageType schema, String csvDelimiter)
        throws IOException {

    SimpleDateFormat dt = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss");
    for (int j = 0; j < schema.getFieldCount(); j++) {
        if (j > 0) {
            w.write(csvDelimiter);
        }
        String valueToString;

        try {
            if (String.valueOf(g.getType().getType(schema.getFieldName(j)).asPrimitiveType().stringifier()).equals("TIMESTAMP_MICROS_STRINGIFIER")) {
                Instant instant = Instant.ofEpochMilli(g.getLong(j, 0));
                Date date = Date.from(instant);
                valueToString = dt.format(date);
            } else {
                valueToString = g.getValueToString(j, 0);
            }

        } catch (RuntimeException e) {
            valueToString = "";
        }

        w.write(valueToString);
    }
    w.write('\n');
}

Output Date: 53971-26-19 03:26:40

Expected Date: 2022-01-01 00:35:40

TIMESTAMP String Value: 1640997340000000

Schema: optional int64 tpep_pickup_datetime (TIMESTAMP(MICROS,false));

Thanks

Ole V.V.
  • 81,772
  • 15
  • 137
  • 161
user3582583
  • 57
  • 1
  • 1
  • 6
  • `.getLong(j,0)` returns what? – g00se Jan 30 '23 at 23:09
  • Could you share an example value of input - TIMESTAMP_MICROS_STRINGIFIER ? – Mio Jan 30 '23 at 23:19
  • return this value: 1640997340000000 – user3582583 Jan 30 '23 at 23:19
  • 1
    Too large - looks like nanoseconds maybe – g00se Jan 30 '23 at 23:21
  • You should probably divide your value by 1000: 1640997340000000 /1000 = 1640997340000 – Mio Jan 30 '23 at 23:22
  • I strongly recommend you don’t use `SimpleDateFormat` and `Date` since they are troublesome, poorly designed and long outdated. Stick to `Instant` and other classes of [java.time, the modern Java date and time API](https://docs.oracle.com/javase/tutorial/datetime/index.html). They give you all the functionality you need. Which, by the way, the old classes don’t. – Ole V.V. Jan 30 '23 at 23:22
  • 1
    I suggest `Instant.EPOCH.plus(1640997340000000L, ChronoUnit.MICROS)`. It yields 2022-01-01T00:35:40Z. And for formatting: `instantFromMicros.atZone(ZoneId.systemDefault()).format(DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss"))`. In my time zone (Europe/Copenhagen) it gave `2022-01-01 01:35:40`. – Ole V.V. Jan 30 '23 at 23:24

0 Answers0