0

I am writing a Chrome extension where I need to stream responses from OpenAI endpoints in the content script. The content script runs when the context menu is clicked. I am using Craco to build the react file as the content script of the extension. I can't call the API from the background script since I am using eventsource-parser library. That's the main reason I am using Craco so that I can import that lib inside content script.

Here is the code to extract text from the stream:

const getStream = async (data) => {
    const nextTokens = await getNextTokens(data);
    const reader = nextTokens.getReader();
   

    const decoder = new TextDecoder();
    let done = false;
    let text = '';

    while (!done) {
        const { value, done: doneReading } = await reader.read();
        done = doneReading;
        const chunkValue = decoder.decode(value);
        console.log(chunkValue)

        text += chunkValue;
    }    
}

Here is the code for getNextTokens:

const getNextTokens = async (prompt) => {
    const url = "https://api.openai.com/v1/chat/completions";

    // Get config from storage
    const {
        apiKey,
        model,
        temperature,
        maxTokens,
        topP,
        frequencyPenalty,
        presencePenalty,
    } = await getConfig();

    // Create request body
    //const data = { model: model, messages: [{role: "user", content: prompt }], stream: true } 

    const encoder = new TextEncoder()
    const decoder = new TextDecoder()
  
    let counter = 0

    // Create headers
    const requestHeaders = {
        'Content-Type': 'application/json',
        Authorization: "Bearer " + apiKey,
      }

    const res = await fetch(url, {
        headers: requestHeaders,
        method: 'POST',
        body: JSON.stringify({ 
            model: model,
            messages: [{role: "user", content: prompt }],
            max_tokens: 1000,
            temperature: 1,
            stream: true,
        }),
    })

    const stream = new ReadableStream({
        async start(controller) {
          // callback
          function onParse(event) {
            if (event.type === 'event') {
              const data = event.data
              // https://beta.openai.com/docs/api-reference/completions/create#completions/create-stream
              if (data === '[DONE]') {
                console.log('DONE')
                controller.close()
                return
              }
              try {
                const json = JSON.parse(data)
                const text = json.choices[0].delta?.content || ''
                if (counter < 2 && (text.match(/\n/) || []).length) {
                  // this is a prefix character (i.e., "\n\n"), do nothing
                  return
                }
                const queue = encoder.encode(text)
                controller.enqueue(queue)
                counter++
              } catch (e) {
                // maybe parse error
                controller.error(e)
              }
            }
          }
    
          // stream response (SSE) from OpenAI may be fragmented into multiple chunks
          // this ensures we properly read chunks and invoke an event for each SSE event stream
          const parser = createParser(onParse)
          for await (const chunk of res.body) {
            parser.feed(decoder.decode(chunk))
          }
        },
      })
    
      return stream
};

This is always returning the Object is not async iterable error. Did anyone face this issue before? I'd appreciate some help. Thanks.

Nithur
  • 111
  • 2
  • 10

0 Answers0