Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Frame count not matching with input video to output video #197

Open
Codewithteju opened this issue Oct 17, 2024 · 0 comments
Open

Frame count not matching with input video to output video #197

Codewithteju opened this issue Oct 17, 2024 · 0 comments

Comments

@Codewithteju
Copy link

Hey , I am trying to decode a video and encode it back into a video file, but it is not working as expected. Please help me to resolve the errors or provide some insights where I'm doing wrong.
Here are my code files:
Extract_frames:use crate::decoder::decode_packet;
use crate::encoder::flush_encoder;
use rsmpeg::avcodec::{AVCodec, AVCodecContext};
use rsmpeg::avformat::{AVFormatContextInput, AVFormatContextOutput};
use rsmpeg::avutil::av_inv_q;
use rsmpeg::ffi::{self};
use std::error::Error;
use std::ffi::CString;

pub fn extract_frames(video_path: &str, total_decoded: &mut usize, total_encoded: &mut usize) -> Result<(), Box> {
let mut input_format_context =
AVFormatContextInput::open(&CString::new(video_path)?, None, &mut None)?;

let best_stream_index = input_format_context.find_best_stream(ffi::AVMEDIA_TYPE_VIDEO)?.unwrap();
let codecparameters = input_format_context.streams()[best_stream_index.0].codecpar().clone(); 

let input_codec =
    AVCodec::find_decoder(codecparameters.codec_id).ok_or("Failed to find codec")?;
let mut input_codec_ctx = AVCodecContext::new(&input_codec);

input_codec_ctx.apply_codecpar(&codecparameters)?;

input_codec_ctx.set_pkt_timebase(input_format_context.streams()[best_stream_index.0].time_base);

input_codec_ctx.set_framerate(input_format_context.streams()[best_stream_index.0].guess_framerate().unwrap());
input_codec_ctx.open(None)?;

let height = codecparameters.height;
let width = codecparameters.width;

let mut output_format_ctx = AVFormatContextOutput::create(
    &CString::new("/home/teju/rust/project/output/result.mp4")?,
    None,
)?;
let output_codec =
    AVCodec::find_encoder(input_codec_ctx.codec_id).ok_or("Failed to find output codec")?;

let mut output_codec_ctx = AVCodecContext::new(&output_codec);

output_codec_ctx.set_height(input_codec_ctx.height);
output_codec_ctx.set_width(input_codec_ctx.width);
output_codec_ctx.set_sample_aspect_ratio(input_codec_ctx.sample_aspect_ratio);
output_codec_ctx.set_pix_fmt(output_codec.pix_fmts().unwrap()[0]);
output_codec_ctx.set_time_base(av_inv_q(input_codec_ctx.framerate));

output_codec_ctx.set_pkt_timebase(input_codec_ctx.pkt_timebase);
if output_format_ctx.oformat().flags & ffi::AVFMT_GLOBALHEADER as i32 != 0 {
    output_codec_ctx.set_flags(output_codec_ctx.flags | ffi::AV_CODEC_FLAG_GLOBAL_HEADER as i32);
}
output_codec_ctx.open(None)?;

{
let mut stream = output_format_ctx.new_stream();
stream.set_codecpar(output_codec_ctx.extract_codecpar());
stream.set_time_base(output_codec_ctx.time_base);
}
output_format_ctx.write_header(&mut None)?;

let mut frame_count = 0;
loop {
    match input_format_context.read_packet() {
        Ok(Some(pkt))if pkt.stream_index == best_stream_index.0 as i32 => {
            let stream_index= pkt.stream_index;
            
            decode_packet(
                &mut input_codec_ctx,
                &pkt,
                stream_index,
                &mut output_codec_ctx,
                &mut output_format_ctx,
                total_decoded,
                total_encoded,
            )?;
            frame_count += 1;
        }
        Ok(Some(_)) => {}
        Ok(None) => break,
        Err(e) => {
            eprintln!("Error reading packet: {}", e);
            break;
        }
    }
}

flush_encoder(&mut output_codec_ctx, &mut output_format_ctx, best_stream_index.0 as i32 )?;
output_format_ctx.write_trailer()?;
println!("Frames processed: {}", frame_count);
Ok(())

}
Decoder:use crate::encoder::encode_frame;
use crate::yuv_to_rgb::yuv_rgb_sws; // Assume you have this conversion function defined
use rsmpeg::avcodec::{AVCodecContext, AVPacket};
use rsmpeg::avformat::AVFormatContextOutput;
use rsmpeg::avutil::av_rescale_q;
use rsmpeg::ffi::AV_PICTURE_TYPE_NONE;
use std::error::Error;

pub fn decode_packet(
input_codec_ctx: &mut AVCodecContext,
pkt: &AVPacket,
stream_index: i32,

output_codec_ctx: &mut AVCodecContext,
output_ctx: &mut AVFormatContextOutput,
total_decoded: &mut usize, // Add parameter for total decoded frames
total_encoded: &mut usize,

) -> Result<(), Box> {

// Send the packet to the decoder
if input_codec_ctx.send_packet(Some(pkt)).is_err() {
    return Err("Failed to send packet to decoder".into());
}

while let Ok(mut frame) = input_codec_ctx.receive_frame() {
   frame.set_pts(frame.best_effort_timestamp);

   frame.set_pict_type(AV_PICTURE_TYPE_NONE);

   if frame.pts != rsmpeg::ffi::AV_NOPTS_VALUE {
    frame.set_pts(av_rescale_q(frame.pts, frame.time_base, output_codec_ctx.time_base));
   }

    // Convert YUV frame to RGB (if needed)
    // let rgb_frame = yuv_rgb_sws(&frame); 
    // let mut yuv_frame = crate::yuv_to_rgb::rgb_yuv_sws(&rgb_frame); 
    // yuv_frame.set_pts(frame.pts);
    *total_decoded+=1;

    // Encode the frame
    encode_frame(output_codec_ctx, output_ctx, Some(&frame), stream_index)?;
    *total_encoded+=1;
}

Ok(())

}
Encoder:use anyhow::Context;
use rsmpeg::avcodec::AVCodecContext;
use rsmpeg::avformat::AVFormatContextOutput;
use rsmpeg::avutil:: AVFrame;
use std::error::Error;

pub fn encode_frame(
output_codec_ctx: &mut AVCodecContext,
output_ctx: &mut AVFormatContextOutput,
frame: Option<&AVFrame>,
stream_index:i32

) -> Result<(), Box> {

output_codec_ctx.send_frame(frame)?;
while let Ok(mut packet) = output_codec_ctx.receive_packet() {
    if stream_index < 0 || stream_index as usize >= output_ctx.streams().len() {
        return Err(format!("Invalid stream index: {}", stream_index).into());
    }

    packet.set_stream_index(stream_index);
    packet.rescale_ts(output_codec_ctx.time_base, output_ctx.streams()[stream_index as usize].time_base);
    output_ctx.interleaved_write_frame(&mut packet).context("Interleaved write frame failed")?;
}

Ok(())

}

pub fn flush_encoder(
output_codec_ctx: &mut AVCodecContext,
output_format_ctx: &mut AVFormatContextOutput,
stream_index:i32
) -> Result<(), Box> {

if output_codec_ctx.codec().capabilities & rsmpeg::ffi::AV_CODEC_CAP_DELAY as i32 ==0 {
    return Ok(());
}
encode_frame(output_codec_ctx, output_format_ctx, None, stream_index)?;
Ok(())

}

Now I am getting this error : [libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] non-strictly-monotonic PTS
[libx264 @ 0x55c07dfca240] frame I:1 Avg QP:26.14 size: 34194
[libx264 @ 0x55c07dfca240] mb I I16..4: 33.0% 59.5% 7.5%
[libx264 @ 0x55c07dfca240] 8x8 transform intra:59.5%
[libx264 @ 0x55c07dfca240] coded y,uvDC,uvAC intra: 23.7% 34.4% 6.2%
[libx264 @ 0x55c07dfca240] i16 v,h,dc,p: 42% 32% 6% 20%
[libx264 @ 0x55c07dfca240] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 39% 21% 18% 3% 4% 4% 4% 4% 3%
[libx264 @ 0x55c07dfca240] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 50% 21% 6% 3% 4% 7% 3% 5% 2%
[libx264 @ 0x55c07dfca240] i8c dc,h,v,p: 55% 19% 22% 4%
[libx264 @ 0x55c07dfca240] kb/s:inf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant