0

I have about 10,000 short videos that I am trying to make several longer videos from.

I've created these videos using MoviePy, but keep getting memory errors when trying to concatenate them back together.

In my code I have an outer loop going through each letter of the alphabet and getting the files that start with that letter.

From the returned video clips I get the length and and strip off the last 3.5 seconds (outro_clip_duration) of the video, and then add it into a Python list clips.

Where I'm stuck is I then want to take this list of trimmed videos and make one long long video from it.

I have all the files, I just need to trim them, concatenate them, and then export them as one.

I've tried many times with different MoviePy attempts and keep getting MemoryErrors so I gave up and took a swing at an ffmpeg solution seen here but it is not working out either.

The latest version of the main part of my code is here:

clips = []
outro_clip = mpy.VideoFileClip('Logo_Intro_w_Stinger_Large.mp4')
outro_clip_duration = outro_clip.duration
for def_image in vid_list_long:
    video_item = mpy.VideoFileClip('F:/sm_My_Video/sm_%s.mp4' % def_image)
    video_item_duration = video_item.duration
    clips.append(ffmpeg_extract_subclip(video_item,0,(video_item_duration - outro_clip_duration), targetname = def_image))

# #Append the outro_clip to the end 
clips.append(mpy.VideoFileClip('Logo_Intro_w_Stinger_Large.mp4',target_resolution = (h,w),audio=True))
slided_clips = [CompositeVideoClip([clip.fx( transfx.crossfadein, transition_seconds)]) for clip in clips]
#added 'method = compose' NEED TO TEST - supposedly removes the weird glitches.
c = concatenate_videoclips(slided_clips, method = 'compose')
c.write_videofile('F:/Extended_Play/%s_Extended_Play_vid.mp4' % letter,fps=23.98)

My PC is Windows 10 and I have 32 GB of RAM running Anaconda and Python 3.

Programming_Learner_DK
  • 1,509
  • 4
  • 23
  • 49

3 Answers3

0

If it is a memory problem, perhaps you could try making a concatenated video for each letter of the alphabet and then in a separate script try to concatenate these videos? I know that many video formats include headers at the top that can take up a lot of space, so maybe if you combine them in steps you can have less of these headers loaded into memory. Perhaps (purely speculating because I don't really know) there is a limit on the file format of the video you are trying to produce and you are exceeding that, so what you desire can't be achieved at all in that specific format? If those suggestions don't work, perhaps you can try just concatenating the videos one letter at at time (after you already concatenated them by letter) and see where it breaks? As a last resort you could always lower resolution, or post to YouTube (or whatever you are doing) as 2 or 3 videos instead of 1. That's still a massive improvement over 10,000 videos!

Reedinationer
  • 5,661
  • 1
  • 12
  • 33
0

You should try closing the clips once you're done using them by adding close_clip(video_item) at the end of your for loop, something like

for def_image in vid_list_long:
    video_item = mpy.VideoFileClip('F:/sm_My_Video/sm_%s.mp4' % def_image)
    video_item_duration = video_item.duration
    clips.append(ffmpeg_extract_subclip(video_item,0,(video_item_duration - outro_clip_duration), targetname = def_image))
    close_clip(video_item)

Where the close_clip() would look something like

def close_clip(clip):
  try:
    clip.reader.close()
    if clip.audio != None:
      clip.audio.reader.close_proc()
      del clip.audio
    del clip
  except Exception as e:
    print("Error in close_clip() ", e)
Tristo
  • 2,328
  • 4
  • 17
  • 28
0
def concatenate():
    stringa = "ffmpeg -i \"concat:"
    elenco_video = glob.glob("*.mp4")
    elenco_file_temp = []
    for f in elenco_video:
        file = "temp" + str(elenco_video.index(f) + 1) + ".ts"
        os.system("ffmpeg -i " + f + " -c copy -bsf:v h264_mp4toannexb -f mpegts " + file)
        elenco_file_temp.append(file)
    print(elenco_file_temp)
    for f in elenco_file_temp:
        stringa += f
        if elenco_file_temp.index(f) != len(elenco_file_temp)-1:
            stringa += "|"
        else:
            stringa += "\" -c copy  -bsf:a aac_adtstoasc output.mp4"
    print(stringa)
    os.system(stringa)

# call the method
concatenate()
kakasuarez
  • 62
  • 9
Piyush
  • 1