sending audio file to open ai whisper model

Question

I am converting my recorded audio file to a blob object and then reading it with file reader to make a post request to open ai whisper model It expects a audio file and model name i.e whisper-1

The error i am getting is

1 validation error for Request\nbody -> file\n field required (type=value_error.missing)

my code is below

  const handleUpload = () => {
    const audioBlob = new Blob(audioChunks, { type: "audio/webm" });

    const reader = new FileReader();

    reader.readAsDataURL(audioBlob);

    reader.onload = async () => {
      const audioBase64 = await reader.result.split(",")[1];
      const payload = {
        audio: audioBase64,
        model: "whisper-1",
      };

      const payloadString =  JSON.stringify(payload);

      fetch("https://api.openai.com/v1/audio/transcriptions", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          Authorization: `Bearer ${key}`,
        },
        body: payloadString,
      })
        .then((response) => response.json())
        .then((data) => console.log(data))
        .catch((error) => console.error(error));
    };
  };

Answer 1

Once you get the audio file, pass it to the below function, you'll get the transcription.

It uses openai's transcription api.

import FormData from "form-data";
import axios from 'axios'

const transcribe = async (file) => {
  let data = new FormData();

  data.append("file", fs.createReadStream(file));
  data.append("model", "whisper-1");
  data.append("language", "en");

  let config = {
    method: "post",
    maxBodyLength: Infinity,
    url: "https://api.openai.com/v1/audio/transcriptions",
    headers: {
      Authorization:
        `Bearer ${process.env.OPENAI_API_KEY}`,
      "Content-Type": "multipart/form-data",
      ...data.getHeaders(),
    },
    data: data,
  };

  try {
    const response = await axios.request(config);
    const data = response.data;

    return { data };
  } catch (error) {
    return {};
  }
};

I think the problem that you are getting here is:

const payload = {
        audio: audioBase64,
        model: "whisper-1",
      };

Replace audio to file as key in the payload object. Like this:

const payload = {
        file: audioBase64,
        model: "whisper-1",
      };

Answer 2

https://api.openai.com/v1/audio/transcriptions - create transcription api takes multipart/form-data.

curl https://api.openai.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F file="@/path/to/file/audio.mp3" \
  -F model="whisper-1"

either you can use the fs module or send data as form type using form-data