1

I have an incoming JSON data that looks like below:

{"id":"1000","premium":29999,"eventTime":"2021-12-22 00:00:00"}

Now, I have created a class that will accept this record and will check whether the data type of the incoming record is according to the data types defined in the case class. However, when I am calling the method it is always calling the Failure part of the match case.

    case class Premium(id: String, premium: Long, eventTime: String)
    
    class Splitter extends ProcessFunction[String, Premium] {
      val outputTag = new OutputTag[String]("failed")
    
      def fromJson[T](json: String)(implicit m: Manifest[T]): Either[String, T] = {
        Try {
          println("inside")
          lazy val mapper = new ObjectMapper() with ScalaObjectMapper
          mapper.registerModule(DefaultScalaModule)
          mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
          mapper.readValue[T](json)
        } match {
          case Success(x) => {
            Right(x)
          }
          case Failure(err) => {
            Left(json)
          }
        }
      }
override def processElement(i: String, context: ProcessFunction[String, Premium]#Context, collector: Collector[Premium]): Unit = {
    fromJson(i) match {
      case Right(data) => {
        collector.collect(data)
        println("Good Records: " + data)
      }
      case Left(json) => {
        context.output(outputTag, json)
        println("Bad Records: " + json)
      }
    }
  }
    }

Based on the sample record above, it should pass the Success value but no matter what I pass, it always enters the Failure part. What else is missing?

I am using Scala 2.11.12 and I tried examples from this link and this link but no luck.

  • Seems to be working [fine](https://scastie.scala-lang.org/tPkvOQHiRcaaaJ6jBGrkTQ). What error are you getting. – Guru Stron Dec 30 '21 at 19:44
  • Just a sec check my other method in the code. There I am passing the value from the `Failure` part and printing it and when I am passing the data it is always printing the `Failure` part. Why is it so? Or there is some gap in my understanding? – user17775951 Dec 30 '21 at 19:47
  • If I pass this data `"{\n \"id\":\"asd\",\n \"premium\":\"123\",\n \"eventTime\":\"2021-20-12 00:00:00\"\n}"` it is giving me as the `Success` part but it should have been failed since the second attribute is of `String` type and not `Long` – user17775951 Dec 30 '21 at 20:00
  • `mapper.disable(MapperFeature.ALLOW_COERCION_OF_SCALARS)` – Guru Stron Dec 30 '21 at 20:17
  • Where should I add this? – user17775951 Dec 30 '21 at 20:22
  • Somewhere around `mapper.configure` – Guru Stron Dec 30 '21 at 20:23
  • I added below it and also checked without `configure` but still the same – user17775951 Dec 30 '21 at 20:25
  • It is to prevent handling of number strings as [numbers](https://scastie.scala-lang.org/pjaQxBB8SAetVoiN1fQsXA) – Guru Stron Dec 30 '21 at 20:26
  • Now, if I pass `"{\"id\":1000,\"premium\":29999,\"eventTime\":\"2021-12-22 00:00:00\"}"` it gives me as the correct record instead of bad record because the first attribute is of Int type and not String. So,here the expected result is Bad record – user17775951 Dec 30 '21 at 20:34
  • Any suggestions pls? – user17775951 Dec 31 '21 at 21:23
  • After a lot of research, I found out that this Parser doesn't have a strict mode within it so what is happening is if I pass some string value for the first attribute, still it is giving me a good record instead of a bad one. Any other suggestions pls? – user17775951 Jan 04 '22 at 16:16

0 Answers0