What Is Video Processing?

Playing and recording videos on a computer, you can copy home movies to your computer, use video and audio scraping tools to edit, edit, and add some very common special effects to enhance the viewability of videos.

Video processing

Play and record videos on your computer, copy home movies to your computer, use video and

The aspect ratio of traditional television (green) and common movie picture aspect ratio is used to describe the ratio of video picture to picture elements. Traditional TV screens have an aspect ratio of 4: 3 (1.33: 1). The aspect ratio of HDTV is 16: 9 (1.78: 1). The 35mm film negative has an aspect ratio of about 1.37: 1.

Although most of the pixels on a computer screen are square, this is not usually the case for digital video. For example, CCIR 601, a digital preservation format for PAL and NTSC signals, and its corresponding non-equivalent widescreen format. Therefore, NTSC DV images recorded at 720x480 pixels may be thin

AVI --- AVI file

Video processing requires the use of a large amount of data as a test and training set, such as: Visual Object Classes Challenge 2011 (VOC2011) (PASCAL Visual Target Classification Challenge 2011), BEHAVE-Crowds (crowd behavior video data set), traffic video database (2010- Data for 2011), IR Marks video data set (University of California San Diego Facial Sports Video Database), The Honda / UCSD Video Database (University of California San Diego Facial Tracking Video Database), VIRAT Video Dataset (U.S. Department of Defense Advanced Research Planning Bureau VIRAT Video Database) and so on.

Bit rate (for digital signals only), bit rate (also translated as bit rate or

Most consumer video products are distinguished by video processing techniques and algorithms used during and after decompression. Some video processing technologies may be implemented in different ways than listed below, and the following technologies may have different names in different applications.

Video processing mainstream video processor

To create a successful digital video product, you need to choose the right processor. It sounds simple, but of course it's not simple. One of the big problems is that there are too many processor styles for you to choose from: general-purpose CPUs, FPGAs, DSPs, configurable processors, fixed-function chips, and other types of processors.

To complicate matters, digital video is a rapidly evolving field, with standards constantly evolving and changing. Therefore, in the field of digital video, the processor's ability to adapt to changes is more important than in other applications, but this flexibility usually comes at the cost of reduced efficiency.

When choosing a processor, it is inevitable to make some compromises, but the key is to know how to choose a processor so as not to compromise the success of the product.

Video processing required

Because there are so many processors to choose from, it's actually impossible to go through them one by one-you can't even understand the major categories in detail, you can use a hierarchical method: use the most important screening criteria for you to exclude the unsuitable Candidate.

Commonly used criteria in primary elections include:

* Speed. Digital video tasks, like many other types of signal processing tasks, require processors to perform heavy computing tasks. For the target application, carefully analyze whether the processor has sufficient speed, it is best to use video-oriented test benchmarks such as BDTI Video Benchmarks.

* Price. Although chip price is important, cost per channel or overall system cost may be more important.

* energy efficiency. In most cases, evaluating energy efficiency is more meaningful than power consumption, because energy usage determines battery life.

* Flexibility. Some types of processors are more flexible than other types of processors, and can adapt to changes in future product features, or allow field upgrades, such as adding support for new compression algorithms. However, in general, the more flexible the processor, the lower its cost and energy efficiency.

* Development tool quality. Whether the processor has the tools to support the development of signal processing applications (or further, video applications) may have a significant impact on development time, and then affect time to market.

* Compatibility with earlier processor models. This is often important if you expect to reuse software from earlier products.

* Supplier roadmap. Does the supplier's product roadmap fit well with your subsequent product development plan? Will the processor be supported or upgraded during your product life?

* Sold as a chip or licensable core. Some processors are sold as packaged off-the-shelf chips; others are sold as licensable intellectual property-often referred to as licensable cores, used to make custom chips. Most of the processor types discussed in this article include both packaged chips and licensable cores.

Pros and cons of various processors

It focuses on the six types of processors commonly used in digital video: fixed-function engines, application-specific standard products (ASSP), media processors, DSPs, embedded RISC processors, and FPGAs. These ranges cover the most professional and flexible products, discuss the pros and cons of each type of processor, and analyze a specific product in each category.

Let's start with the fixed function engine. It uses hard-wired processor architectures for maximum efficiency; they do not use instruction streams and cannot be programmed. Hard-wired logic sacrifices flexibility for extraordinary processing speed, energy efficiency, and often cost-effectiveness.

The use of fixed-function engines can simplify system design and testing. Because the fixed-function engine is not programmable, product developers do not have to learn programming tools or integrate multiple software modules. And they do not need to consider whether the multiple tasks performed by the processor may affect each other in undesired ways, and whether it will interfere with the real-time characteristics of the system.

Fixed-function engines are typically provided in a form of licensable intellectual property (IP) to facilitate integration into custom chips. In this form, fixed-function engines are best suited for high-volume applications such as mobile phones. Fixed-function engines are sometimes also available in chip form. Fixed-function video chips such as MPEG-2 decoding

Features and representative products of mainstream video processors

Processor chip, which can add functionality to existing products at a lower cost, especially when the product has a host processor capable of handling demand control and user interface functions.

Take, for example, the 5150 MPEG-4 video decoder from Hantro, which is an example of a fixed-function engine sold as IP. The engine is intended to be used as a coprocessor, attached to a general-purpose processor, which handles less demanding sub-tasks required for MPEG-4 decoding.

The main disadvantage of fixed-function hardware is its lack of flexibility. Because it is not programmable, product developers cannot easily modify fixed-function hardware to support new standards or different functions. This is a cause for concern, as many video applications are still immature and standards and features change rapidly.

Therefore, fixed-function engines are often used as part of specialized standard products. Let's talk about ASSPs below.

Application-specific standard products (ASSP) are specialized chips with a high degree of integration. Can be compared with application specific integrated circuits (ASIC). ASICs are designed by systems companies and used in their own products. The ASSP was designed by a chip company and provided to many system developers as off-the-shelf chips. Because developing a complex chip is costly and time-consuming, ASSPs are often used for already large-volume, finalized products, or higher volume products are expected.

For example, Zoran's Vaddis 5R is a highly specialized chip for audio and video processing in DVD recorders. The key algorithms required have been identified: the most prominent is MPEG-2 video compression and decompression.

Although Vaddis 5R includes two RISC processors, it uses fixed-function hardware accelerators such as MPEG-2 video decoding and color space conversion when performing the most computationally intensive tasks. For these reasons, Vaddis 5R (and other similar ASSPs) also have the advantages and disadvantages of fixed-function engines: good performance and energy efficiency, but limited flexibility.

Limited flexibility means that when designing products, system designers can easily make their products similar to other products that use the same ASSP. It also means that system designers are highly dependent on chip vendors' roadmaps because new chips are required to support significantly different functions.

The other is an ASSP that relies heavily on a programmable processor to perform heavy video tasks, sacrificing energy efficiency and cost efficiency for flexibility. This type of ASSP is often bundled with critical software such as video decoders and hardware drivers, freeing system developers from having to do much low-level software development. However, compared to ASSPs based on fixed-function hardware, significant effort may still be required in software development and integration.

The media processor is between ASSP and digital signal processor (DSP) in terms of specialization / flexibility. Media processors are optimized for tasks related to audio and video processing, and are not optimized for a wide range of signal processing tasks like DSPs. Media processors typically use multiple processors, including a DSP-like main processing engine, two or three dedicated coprocessors, and audio and video-specific peripherals.

Philips' PNX1500 is a media processor. Like a typical media processor, the PNX1500 is based on a powerful, highly parallel processor core that is highly efficient when performing video processing tasks. The PNX1500 also includes several fixed-function hardware accelerators and dedicated peripheral devices, which are also typical of typical media processors. The main processor core handles complex video tasks such as compression, and system designers can program the core.

Similar to Zoran's Vaddis 5R, the PNX1500 is well-suited for MPEG-2 decoding. But unlike Zoran's ASSP, the PNX1500 is flexible enough to work with other video compression standards such as H.264. Of course, this flexibility comes at a price: software video decoders are generally less energy and cost-efficient than fixed-function hardware. ^[2]

Because the media processor uses multiple different processors, compared to other programmable processors, it causes great difficulties in software development. For example, to perform a particular video task, two or more processing units must generally be programmed and coordinated. To help compensate for this shortcoming, media processor vendors often provide optimized software component libraries.

Media processor vendors often emphasize the use of C or C ++ when developing software and do not recommend or support assembly language. The emphasis on developing software in high-level languages is to keep programmers away from many complex issues in processor architecture. The downside is that programmers must rely on the compiler to generate valid code, which is not always realistic. Developers may need to invest a lot of effort to manually adjust their high-level language code for best performance.

Digital signal processors (DSPs) are designed for a range of signal processing applications. Compared with media processors, the specifications used by the DSP are less focused on video processing and have lower parallelism. In order to compensate for the weakness of lower parallelism, in a given application, the instruction execution speed of the DSP must usually be higher than the media processor. Higher instruction speeds can complicate system design and increase energy consumption. On the other hand, compared to embedded RISC processors (described below), DSPs require lower clock speeds for video tasks. The key advantages of a DSP are its flexibility and powerful application development tools.