Solve the problem of recording file missing frame, eliminates the process of manual operation
When I do live video recording, I often find that the beginning of the recorded video file is missing frames. The specific performance is:
- On PotPlayer, the picture at the beginning of the video is still, but the sound continues.
- On MPV, the time frame automatically jumps to a few seconds later.
- When contributing to bilibili, if there is no secondary suppression which should be fully suppressed twice now, bilibili will remove the part without the picture and advance the subsequent pictures, resulting in a serious out of sync of the sound and picture. (The main impact. it does not affect the viewing experience much, because the player can automatically check the time-line.)
Since I did not go to a large number of live broadcasts of other anchors to verify the universality of the problem, this project is not universal. According to my personal experience, this kind of problem is relatively small. It is very likely that only this one anchor often has this kind of problem, at least it hasn't happened in another anchor I recorded. My personal guess is that there is a problem with the host’s start-up operation, which has something to do with her not turning on the microphone every time she starts the broadcast, maybe. Anyway, it doesn’t look like the server’s or BililiveRecorder's problem.
- Anchor: 一只萌豌豆
- Recording tools: BililiveRecorder，Version: 1.1.18
To solve this problem, the more brainless solution is to recode and suppress (Format Factory/Maruko's Toolbox), but this solution takes a long time and consumes a lot of resources, which is not suitable for my thin and light notebook. Non-destructive cutting is the solution that suits me. Using the -ss and -c copy parameters of FFmpeg, the missing frame can be quickly cut off (just cut the missing frame at the beginning, if there is a missing frame at the end, it will not cause the video, audio and picture of station B to be out of sync, and generally there is no), then the value of the -ss parameter is the time when the screen just started moving.
The above sentence is easy to understand. Start wherever there is no problem, but this will involve another problem, that is, why the video will start to move at this value position, which needs to explain the principle of key frames. The video frame is offset according to the data of the previous frame, and the source is traced forward one by one, and the previous key frame will be found. If the key frame is not found, the player will not know how to deal with the frame data before the next key frame, which causes frame loss. The key frame here refers to the key frame of the original video, not the key frame (file header) automatically marked after the video is cut (the player will not recognize the key frame without the key frame at the beginning of the video). Then a cut on the non-key frames will inevitably cause the missing frames of the video, which refers to cv4373238.
In the beginning, I saw it with the naked eye, adding and subtracting in units of 1s (a few tenths of a second is difficult to estimate with the naked eye). Sometimes it will be corrected by FFmpeg (The -ss parameter is to let FFmpeg find (seeking) keyframes around here, which refers to FFmpeg official documentation), most of the time it will cause new missing frames, that is, FFmpeg did not find the key frame in a limited range. If it is earlier than the actual key frame, the duration of the missing frame will be a little shorter; if it is cut after the key frame, then the generated video will miss the frame to the next key frame. If you fine-tune it at this time, it will undoubtedly be a waste of time. So, how to get the accurate key frame position? With Genteure's fib (see Project Dependence), you can find the location of the second key frame in the parsed XML file, and then use FFmpeg to cut this location (Assign the second key frame time to the -ss parameter).
If things go on like this, I gradually feel that it is a bit cumbersome to perform such operations every day. Let me count the steps in detail:
- Use fib to analyze FLV and get XML. command-line e.g.：
fib parse test.flv out.xml - Open the XML, search for the second keyframe attribute, find the time value of the same row
- Video cutting of the resulting time using FFmpeg. command-line e.g.：
ffmpeg -i test.flv -ss 8.3 -c copy test-cut.flv
It contains two questions:
- Enter similar commands repeatedly every day, and if the tool does not add environment variables, you need to drag and drop the path into the command line, and the recorded file name is usually very long, and you need to drag and drop.
- The size of the XML file depends on the size of the FLV file. For reasons of convenient management and fault tolerance, I set the automatic slicing duration of BililiveRecorder to one hour. A live video recording with a bit rate of 4Mbps is about 1.78GB in an hour. In contrast, XML is at least about 28.5MB. Therefore, when using Sublime to open XML, there will be a longer loading time (Large IDEs can be loaded in read-only mode in seconds, but my use case is not suitable for opening an IDE, just look at an XML), and the screen is also very stuck when scrolling.
So, how to combine these two tools through programming, avoid the tedious operation of repeatedly typing commands, hide the intermediate steps, and improve the reading speed of large-volume XML? This led to the birth of this project.
Most (almost all) operating environments of this project have been packaged into compressed packages and Setup, just pay attention to the operating system.
- Windows (Currently)
- JDK (Optional)
1.8(jar) /1.6+(exe) ，See Assets description here for details
FirstKeyFrameFilter.exe
FirstKeyFrameFilter.jar
FirstKeyFrameFilter{version-code}-Full.zip
FirstKeyFrameFilter{version-code}-Full_Setup.exe
FirstKeyFrameFilter{version-code}-Without_JRE.zip
-
Because the project relies on two exe executable files and the JDK environment, the program is too large, and the actual core code only occupies a relatively small part. But basically, neither the dependent
exenor the JDK environment will change. So please directly downloadFirstKeyFrameFilter.exein subsequent updates to overwrite the old version. -
The single
FirstKeyFrameFilter.execannot work normally, and at least thelib-appsfolder and twoexes under it must exist in the same directory. The error message is redirected to theerror.login the current directory. If the program does not output or the black box flashes by, or other abnormalities, please refer to thelogfile first. -
The
jarfile is uploaded for possible cross-platform programs in the future. How to use (command line):java -jar FirstKeyFrameFilter.jar {flvFile}。 Note that you must runjarwith the same JDK version as the compiled version (1.8), otherwise it will appear error:java.lang.UnsupportedClassVersionError: Main : Unsupported major.minor version 52.0. Theexeprogram does not have this problem. -
The files with the word
Fullcontain the complete dependencyexeand JDK environment. -
If there is already a JDK environment (>=
1.6) on the computer, that is, the environment variables are accessible, you can delete thejre1.8.0_161folder in the program directory, or download~Without_JRE.zip.
- Download the
zipcompressed package or~Setup.exe, if the JDK version is greater than1.6, you can download~Without_JRE.zip - Unzip directly or follow the installation wizard to install
- Directly download
FirstKeyFrameFilter.exeto cover when updating - If you patiently read the above Assets Description, you will find that this is just a summary
The program must be placed in the following structure to ensure the relative position, otherwise, it cannot run.
FirstKeyFrameFilter
│ FirstKeyFrameFilter.exe
│
├─jre1.8.0_161 (Optional)
│
└─lib-apps
ffmpeg.exe
FlvInteractiveRebase.exe
-
Drag File(s)
Drag one or multiple FLV files into
FirstKeyFrameFilter.exeto run, which is the easiest and direct method -
Command Line
FirstKeyFrameFilter.exe test.flv。Among them,test.flvis the pending missing frame FLV video file
-
FlvInteractiveRebase (fib，FLV File editing tool)
- BililiveRecorder QQ Group internal beta 1
- By genteure ( [email protected] )
- Software core functions:
- Tag level editing of FLV can be used to solve all kinds of strange problems
- Use a simple text file as the command format for manual editing...
- ...At the same time, it is convenient to write scripts and realize automatic processing
- The command file is relatively small and easy to transfer, which is convenient for people who do not understand the details of FLV to repair the file
- Main software usage (Assuming the software file name is fib) ：
- Read FLV files and generate command files (This project depends on)
./fib parse problem.flv command.xml - Read other FLV files for reference or copy data
./fib parse other.flv Other-commands.xml - Manually edit command text files
- Use the modified command file to generate a new FLV file
./fib build command.xml after-repair.flv
- See help please run
./fib help
- Read FLV files and generate command files (This project depends on)
-
- Mainly used for FLV lossless cutting
- Go to Issues to post a new question.
- Email me: [email protected].
Before development, you need:
- IntelliJ IDEA 2019.3.3 or other Java IDE
- JDK 1.6+，Preferably 1.8
Three modules are called by src/Main.java to complete the whole work, they are
- Call
fib: read FLV and generate XML - StAX parsing XML: time to read the second key frame
- Call
FFmpeg: Cut the FLV with the received time as the start time, generate a new file in the same folder, and the source file will not be deleted
If you want to study the source code of this project or modify the functions, you can start from the entry of the above three modules:
src/XMLGenerator/FlvInteractiveRebase.javasrc/XMLParsing/StAX/XMLReader.javasrc/FLVCutter/FastForwardMpeg.java
- Packaged into
jar: Each IDE has corresponding functions, and the specific methods are self-checked. - Package into
exe: use exe4j Wizard, see the configuration file/exe4jConfigFile.exe4j - Packaged into Setup: Use Inno Setup, see the configuration file
/InnoSetupConfig.iss
- repair: Because of
indexOf, the file name with multiple English periods cannot be handled correctly. See issue#2 - optimization: Extract the method of obtaining the file name without extension into
Utilsfor easy calling - repair: Calling FFmpeg to cut the video is recognized by PotPlayer as an audio problem, which is relatively compatible, so the version number is increased to 1.1.0. See issue#3
- optimization: The method of converting timestamp to seconds is transferred to
Utils, which is convenient to call and conforms to the specification - optimization. Remove useless
import
- repair: It cannot handle the case where the missing frame duration is within 1s, such as 0.15s.
The beginning of everything
- If the environment variables of the program in lib-apps exist on the computer, this folder still cannot be deleted, see issue#1 for details.
- Baidu Net Disk, Extraction code: 7w1w
- OneDrive
TypicalFLVis the recorded video file in question. It is unprocessed and will be exported directly from BililiveRecorder and will be continuously updated. A small number of files that do not have missing frames but have other problems are also included and have been specially marked.TypicalXMLare files generated by fib (see Project Dependency) analysis, which is a sample left over from the development of XML parsing and is no longer updated. The files with the wordTypicalCasein them come from the missing frame video (i.e.TypicalFLV).DanmakuXMLare newly added danmaku files in BililiveRecorder 1.1.20. The danmaku files that have not been suppressed and submitted are archived here, which has nothing to do with this project...