FFmpegWriter.release() does not wait for ffmpeg process completion

[vlad-nn]

This causes race condition, when video file is still being written by running ffmpeg process while FFmpegWriter object is released.

Details:

run_async() in video_info.py actually returns the PID of the shell, not the PID of ffmpeg process.
So release_process() waits for shell process completion, not ffmpeg process completion.
On some environments (like in Google Colab) it takes quite a while for ffmpeg to finish.

The following code demonstrates the problem:

import numpy as np, ffmpegcv
import psutil

img = np.zeros((100, 100, 3))

fn = 'test.mp4'
w, h = 100, 100
writer = ffmpegcv.VideoWriter(fn, None, 30, (w, h))
for i in range(1000):
    if i % 10 == 0:
        img = np.random.randint(0,255,(w,h), dtype='uint8')
    writer.write(img)

for p in psutil.process_iter(['name']):
    print(p)

print(">>>>>", writer.process.pid)

writer.release()

for p in psutil.process_iter(['name']):
    print(p)

The sample output is like this:

psutil.Process(pid=25371, name='sh', status='sleeping', started='02:05:35')
psutil.Process(pid=25372, name='ffmpeg', status='sleeping', started='02:05:35')
psutil.Process(pid=25383, name='sleep', status='sleeping', started='02:05:36')
>>>>> 25371

As you can see, the writer.process.pid == 25371, which corresponds to pid=25371, name='sh', not ffmpeg.

[vlad-nn]

This is because shell argument in Popen() call in run_async evaluates to True.

[vlad-nn]

Workaround found:

for p in psutil.process_iter([]):
    if writer.process.pid == p.ppid():
        p.wait()

[chenxinfeng4]

Sorry for replying late. I agree with you that there is a bug of video release. Very thanks for your pointing it out.

shell argument in Popen() call in run_async evaluates to True.

To fix that bug, it's not a good idea to use psutil. First, the psutil didn't cross platform, seems not available in linux-ARM64. Second, I want to keep ffmpegcv simple to install, no other package if possible.

I believe the bug matters in specifical cases, however I didn't meet the bug in most my cases. In my opinion, I would add a time.sleep(xx) to skip that issue. Or I prefer to change the Popen as Popen(xxxx, shell=False), something like this.

[chenxinfeng4]

I would appreciate if you can create a pull request to revise Popen(xxxx, shell=False).

[vlad-nn]

Yes, you right: psutil is not supported on ARM64 (giampaolo/psutil#1782), so the most desired fix would be to get rid of psutil. BTW, why don't you use subprocess.Popen? (https://docs.python.org/3/library/subprocess.html)

[chenxinfeng4]

I do use subprocess.Popen for all kinds of ffmpegcv.VideoCaputureXX and VideoWriterXX.

#ffmpegcv/video_info.py
import subprocess
from subprocess import Popen, PIPE
...
def run_async(args):
    quiet = True
    stderr_stream = subprocess.DEVNULL if quiet else None
    bufsize = -1
    return Popen(
        args,
        stdin=PIPE,
        stdout=PIPE,
        stderr=stderr_stream,
        shell=isinstance(args, str),
        bufsize=bufsize,
    )

[MHRosenberg]

This is nother important issue for me. What is the current status?