Accelerating H264 decoding on iOS with FFMPEG and VideoToolbox

Accelerating H264 decoding on iOS with FFMPEG and VideoToolbox

7 years ago
Anonymous $hM_jrxqbr-

https://medium.com/liveop-x-team/accelerating-h264-decoding-on-ios-with-ffmpeg-and-videotoolbox-1f000cb6c549

At LIVEOP, we focus on providing first responders with the most relevant information in a concise manner, while at the same time not compromising on our seamless user experience. When we partnered up with Zepcam, a leading provider of wireless (body-worn) camera systems around the world, we wanted to make sure that we delivered an experience that conforms to our high standards, not compromising on performance or efficiency.

Camera streams hosted by Zepcam come in several different formats, most importantly, HTTP Live Streaming (HLS), a first class citizen in the iOS ecosystem with built-in support in AVFoundation, and RTSP, the Real Time Streaming Protocol. HLS streams are commonly used for live television and news broadcasts. It focuses on a seamless experience for the viewer: frame drops are disallowed, out-of-order playback of frames is not allowed, and a small buffer of upcoming frames is maintained to ensure a smooth playback experience. The situations during which Zepcam streams are activated are often life-threatening. Officers could be live-streaming from their body-worn camera’s while attempting to contain a riot, or a ladder engine with a camera mounted on top could be providing a birds eye view of a large building fire including the position of firefighters on the ground. Our definition of a seamless user experience is different from the one prescribed by HTTP Live Streaming: in our case, it is important that the frames displayed to the user are as realtime as possible. They could arrive out-of-order, and a couple of frames could be dropped, as long as this benefits the realtime-ness of the stream. Adding our requirements up, we arrived at using RTSP over UDP.