What is the best way to store a video frame by frame for video editing?

Question

I am trying to create a video editor, where obviously, you will be able to remove and add frames. My thinking, was to convert the video file itself into an array of frames which can then be manipulated.

Using this answers code, I did that. This works fine for small video files, but with big video files, a memory error can quickly occur - because, of course, memory is storing hundreds of uncompressed images.

This is the exact code I am using:

import numpy
import cv2

def video_to_frames(file):
    """Splits a video file into a numpy array of frames"""
    
    video = cv2.VideoCapture(file)
    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
    frame_width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
    frame_height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
    
    buffer = numpy.empty((frame_count, frame_height, frame_width, 3), numpy.dtype("uint8"))
    index_count = 0
    running = True

    while(index_count < frame_count and running): #Reads each frame to the array
        running, buffer[index_count] = video.read()
        index_count += 1
    video.release()
    
    return buffer #Returns the numpy array of frames


print(video_to_frames("Video.mp4"))

Finally, here is the exact memory error I got: MemoryError: Unable to allocate 249. GiB for an array with shape (46491, 1000, 1920, 3) and data type uint8

So I have two questions really:

1. Is this the most efficient way to go about manipulating a video?
2. If it is, how can I go about storing all those frames without running into a memory error?

Thank you.