1

I need to plot a spectrogram fast as possible. The spectrogram is calculated with librosa and then ploted with matplotlib. This needs now around 50ms, is it possible to do this faster?

I'm very new to matplotlib and librosa.

This is my code:

mel_spec = lb.melspectrogram(y=samples, sr=44100, fmax=8000, n_mels=256)

plt.figure()
plt.subplots(figsize=(0.5, 5))
librosa.display.specshow(librosa.power_to_db(mel_spec, ref=np.max), fmax=8000)
plt.axis('off')
plt.subplots_adjust(top=1, bottom=0, right=1, left=0, hspace=0, wspace=0)
plt.margins(0, 0)

predict_img = io.BytesIO()
plt.savefig(predict_img, format="jpeg")
predict_img.seek(0)

UPDATE:

More details about the use case.

This images are created with 100ms audio snippets and returned as base64 encoded string, this string is forwarded to a ML model for prediction. I need this setup with the lowest possible latency. I'm fine with the latency of the prediction but the latency of the image creation is a little bit slow.

Aron
  • 1,179
  • 15
  • 29
  • try to remove `plt.figure()` and see if it makes it faster. Also, `plt.savefig` takes some time – seralouk Aug 04 '19 at 14:45
  • You cannot make this faster, at least not if you want to get the same plot. You can of course change the parameters of the spectrogram; because plotting less pixels takes less time. – ImportanceOfBeingErnest Aug 04 '19 at 14:58
  • No idea what librosa or Mel spec are but if you have the data you can use pcolormesh and just update the data before each draw. See the set_array method of the quad mesh that pcolormesh returns – Jody Klymak Aug 04 '19 at 15:12
  • @serafeim removing plt.figure() and plt.savefig() reduce the latency. But how can get the ploted image when i remove savefig? – Aron Aug 04 '19 at 15:42
  • The first comment is not helpful. Of course you need to draw the figure (either via `show` or `savefig`) and that is what takes the time. @JodyKlymak's comment can be helpful, if the aim is to produce several figures - in that respect the question could go more into detail about the use case. But mostly it's really about not plotting more pixels than needed.. e.g. if you create a picture of 500 time 50 pixels, it may not make sense to plot an image with more than 50 frequency bins and more than 500 time bins. – ImportanceOfBeingErnest Aug 04 '19 at 15:56
  • @ImportanceOfBeingErnest i have update my question with more information about the use case. Thanks for the input about the image size. – Aron Aug 04 '19 at 16:11
  • Possibly this is a stupid question: Why do you feed an image into the algorithm and not the raw numpy array? – ImportanceOfBeingErnest Aug 04 '19 at 16:26
  • @ImportanceOfBeingErnest The model supports only images. I know the better way is to feed the array directly. – Aron Aug 04 '19 at 18:00

1 Answers1

0

You do not need, and should not use, matplotlib to create an image representation of your spectrogram. It will be much faster to convert the spectrogram array into an image without plotting. See the code in https://stackoverflow.com/a/57204349

Jon Nordby
  • 5,494
  • 1
  • 21
  • 50