1

Given the following API response from pinecone (https://www.pinecone.io/docs/api/operation/query/)

results = {'matches': [{'id': 'yral5m',
              'metadata': {'subreddit': '2qkq6',
                           'text': 'Black Friday SaaS Deals - 2022'},
              'score': 0.772717535,
              'sparseValues': {},
              'values': []},
             {'id': 'yqypa5',
              'metadata': {'subreddit': '2qkq6',
                           'text': 'B2B sales interface.'},
              'score': 0.74192214,
              'sparseValues': {},
              'values': []}],
 'namespace': ''}

i am simply trying to return the JSON results from a function. (from a service to a controller) and getting a range of errors:

doing so with:

return results yields RecursionError: maximum recursion depth exceeded in comparison

return json.dumps(results) yeilds TypeError: Object of type QueryResponse is not JSON serializable

*QueryResponse is the type returned from pinecone

return jsonpickle.encode(query_results) yeilds "null"

Very lost any advice would be appreciated!!

full code example:

Controller:

@router.post("/query/")
async def semantic_search(query: str, ):
result = await TopicQueryService.query(query)
return result

Service method:

    @staticmethod
    async def query(query) -> str:
    index = PineConeService.get_or_create_index("openai")
    embed = GPT3Service.get_embedding(query)
    query_results = index.query(
        vector=embed,
        top_k=2,
        include_metadata=True
    )
    return json.dumps(query_results)

Replacing Service Method query results with the logged response from index. query works fine eg below. Leading me to believe it is due to the QueryResponse object pinecone returns.

    @staticmethod
async def query(query) -> str:
    index = PineConeService.get_or_create_index("openai")
    embed = GPT3Service.get_embedding(query)
    logger.info(embed)
    query_results = {'matches': [{'id': 'yral5m',
                                  'metadata': {'subreddit': '2qkq6',
                                               'text': 'Black Friday SaaS Deals - 2022'},
                                  'score': 0.772717535,
                                  'sparseValues': {},
                                  'values': []},
                                 {'id': 'yqypa5',
                                  'metadata': {'subreddit': '2qkq6',
                                               'text': 'B2B sales interface.'},
                                  'score': 0.74192214,
                                  'sparseValues': {},
                                  'values': []}],
                     'namespace': ''}
    return json.dumps(query_results)
buildingstuff123
  • 11
  • 2
  • Could you give an example of your code? To better understand why this errors could be occured – puf Nov 10 '22 at 13:51
  • going to need some code samples. its not clear what issue you are having. for example the object you shared is json serializable. – NoPlaceLike127.0.0.1 Nov 10 '22 at 14:30
  • Hey i haver added some further examples! i am sure it must have something to do with the response type index.query returns. if i print it out, its exactly what i have shown above. if i then copy and past the json object it prints and paste it in place of the api call it all works fine. i added this example aswell – buildingstuff123 Nov 11 '22 at 09:19
  • @Chris agree i have just deleted that as its already causing me issues... i have gone through that post, it wasnt quite it, `return query_results` gives `RecursionError: maximum recursion depth exceeded in comparison` new solution is just iterating over and building my own object like so, does that seem reasonable? – buildingstuff123 Nov 11 '22 at 10:52
  • Please have a closer look at [this answer](https://stackoverflow.com/a/73974946/17865804), as well as [this answer](https://stackoverflow.com/a/71205127/17865804). If your object is already serialised, you can return a custom `Response` directly. No need to have FastAPI serialise it again; hence, in this case, you don't have to use `return json.dumps(blah)`. It is ok to loop over the results and create a `list`, but it might also be worth trying to find out what is causing the [`RecursionError`](https://stackoverflow.com/q/53786145) (see [this](https://stackoverflow.com/q/52873067) as well). – Chris Nov 11 '22 at 11:34
  • As the object is already been serialized. Simply use: matches = query_response['matches'] for match in matches: match_id = match['id'] score = match['score'] values = match['values'] # Do something with the match information print(f"Match ID: {match_id}, Score: {score}, Values: {values}") And you can parse it smoothly ~ – Zhongyi Jun 27 '23 at 22:27

2 Answers2

3

You can use the method .to_dict() from the QueryResponse object:

  @staticmethod
    async def query(query) -> str:
    index = PineConeService.get_or_create_index("openai")
    embed = GPT3Service.get_embedding(query)
    query_results = index.query(
        vector=embed,
        top_k=2,
        include_metadata=True
    )
    return json.dumps(query_results.to_dict())
dimirc
  • 6,347
  • 3
  • 23
  • 34
  • I wouldn't suggest doing that (i.e., `return json.dumps(query_results.to_dict())`). Please have a look at [this answer](https://stackoverflow.com/a/73974946/17865804), as well as the links provided in the comments section above, for more details and examples. – Chris May 14 '23 at 08:18
  • As the object is already been serialized. Simply use: matches = query_response['matches'] for match in matches: match_id = match['id'] score = match['score'] values = match['values'] # Do something with the match information print(f"Match ID: {match_id}, Score: {score}, Values: {values}") And you can parse it smoothly ~ – Zhongyi Jun 27 '23 at 22:26
0

Somewhat of a solution -> just iterate over and build an object to return, not ideal though

``blah = []
  for x in query_results.matches:
    blah.append({
    "id": x.id,
    "metadata": x.metadata,
     "score": x.score
    })
 json.dumps(blah)``
buildingstuff123
  • 11
  • 2