Django REST Framework change rederer class based on response

Question

I'm using Django 2.2 and Django REST Framework.

I have an APIView which will download Zip file on success response.

class MultiDownloadCode(APIView):

    serializer_class = MultiDownloadSerializer
    permission_classes = (
        IsAuthenticated,
        CodeCanExport,
    )
    renderer_classes = (ZipRenderer,)

    def post(self, request):
        ...
        mime_type = 'application/zip'

        file_name = '{}.zip'.format(random_string(20))

        response = Response(data=in_memory_zip.read(), content_type=mime_type)
        response['Content-Disposition'] = 'attachment; filename=%s' % "-".join(file_name.split())

        return response

I have created a custom Renderer class ZipRenderer

class ZipRenderer(BaseRenderer):
    media_type = 'application/zip'
    format = 'zip'
    charset = 'utf-8'
    render_style = 'binary'

    def render(self, data, accepted_media_type=None, renderer_context=None):
        return data

This is working fine in case of a successful response. But in case of permission denied exception, the error messages are also binary encoded and not proper JSON rendered.

When I add JSONRenderer to the renderer_classes

renderer_classes = (ZipRenderer, JSONRenderer)

This works fine in case of exception but then gives error in case of a success response.

How can I change the renderer based on the response?