4

I have built an API using FastAPI and am trying to send data to it from a client.

Both the API and the client use a similar Pydantic model for the data that I want to submit. This includes a field that contains a file path, which I store in a field of type pathlib.path.

However, FastAPI does not accept the submission because it apparently cannot handle the path object:

TypeError: Object of type PosixPath is not JSON serializable

Here's a minimal test file that shows the problem:

import pathlib
from pydantic import BaseModel
from fastapi import FastAPI
from fastapi.testclient import TestClient


api = FastAPI()
client = TestClient(api)

class Submission(BaseModel):
    file_path: pathlib.Path

@api.post("/", response_model=Submission)
async def add_submission(subm: Submission):
    print(subm)
    # add submission to database
    return subm


def test_add_submission():
    data = {"file_path": "/my/path/to/file.csv"}
    print("original data:", data)

    # create a Submission object, which casts filePath to pathlib.Path:
    submission = Submission(**data)  
    print("submission object:", submission)

    payload = submission.dict()
    print("payload:", payload)

    response = client.post("/", json=payload)  # this throws the error
    assert response.ok

test_add_submission()

When I change the model on the client side to use a string instead of a Path for file_path, things go through. But then I lose the pydantic power of casting the input to a Path when a Submission object is created, and then having a Path attribute with all its possibilities. Surely, there must be better way?

What is the correct way to send a pathlib.PosixPath object to a FastAPI API as part of the payload?

(This is Python 3.8.9, fastapi 0.68.1, pydantic 1.8.2 on Ubuntu)

CodingCat
  • 4,999
  • 10
  • 37
  • 59
  • The error message means that JSON doesn't know how to turn the data into a JSON object like a string, boolean, dictionary, or list etc. You have to specify a serialization format, or just turn it into a structure which JSON understands. Trivially, `str(pathlib.Path(...))` produces the file name as a string; presumably that's what you actually want here. It's not clear where in your `submission` data you have a `pathlib.Path` so you will have to figure that out yourself, or provide a [mre] which shows how this structure looks so that we can help you with that. – tripleee Dec 01 '21 at 12:43
  • Possible duplicate of https://stackoverflow.com/questions/3768895/how-to-make-a-class-json-serializable – tripleee Dec 01 '21 at 12:54
  • @tripleee The above is an MRE. It's a valid pytest file that recreates the error... I have added the call to `test_add_submission()`, so now it runs as a normal python file. – CodingCat Dec 01 '21 at 12:59
  • Yes, I want to keep it as a Path object. That's why I stated in the question that I know casting it to string would solve the error but dows not help me. – CodingCat Dec 01 '21 at 13:01
  • Without information about what else is in `submission` we would have to investigate how `pydantic` defines it, etc. But the straightforward solution is probably to accept this as a duplicate. This unfortunately requires you to override some of the convenient functionality from `pydantic`. – tripleee Dec 01 '21 at 13:01
  • @tripleee The rest of `submission` does not have any impact on the issue. The code above is a complete Python file and throws the described error. It is complete as it is. Adding other information to `submission` would delute the MRE without adding anything of value. Yes, pydantic models are that simple. As FastAPI is built on pydantic, I'm hoping there is a more elegant solution than having to build a `.toJSON()` method into a class that already provides a `.json()` out of the box (but that produces a string, and `post` does not accept it, it wants a dict). – CodingCat Dec 01 '21 at 13:09
  • Maybe also look at https://stackoverflow.com/questions/66687244/what-is-the-best-practice-to-pass-a-class-to-json/66692548#66692548 ... though I agree that the ideal would be if `pydantic` offered a solution; I'm not familiar enough with it to tell you whether that is actually the case. – tripleee Dec 01 '21 at 13:12
  • @CodingCat The example is fine. Have you seen how pydantic handles JSON encoding? https://github.com/samuelcolvin/pydantic/blob/9f654a1fb2475d78731b76adf6c83c954df6d232/pydantic/json.py - this lead me to this question about extending the pydantic json encoding functionality:https://stackoverflow.com/questions/62311401/override-default-encoders-for-jsonable-encoder-in-fastapi .. you should be able to modify the ENCODERS_BY_TYPE and insert the path class there. – MatsLindh Dec 01 '21 at 17:04
  • @MatsLindh thanks for the pointer. The pydantic-code you linked looks like `pathlib.Path` objects should already be encoded as `str`, though, doesn't it? – CodingCat Dec 02 '21 at 08:29

1 Answers1

2

The problem with you code is, that you first transform the pydantic model into a dict, which you then pass to the client, which uses its own json serializer and not the one provided by pydantic.

submission.dict() converts any pydantic model into a dict but keeps any other datatype.

With client.post("/", json=payload) requests json serializer is used, which cannot handle pathlib.Path.

The solution is, not to convert the pydantic model into dict first, but use the json() method of the pydantic model itself and pass it to the client.

response = client.post("/", data=submission.json())

Notice that you have to change the parameter json to data.

finswimmer
  • 10,896
  • 3
  • 34
  • 44