What Is Visual Control?

Vision control means that the robot receives and processes images through the vision system, and performs corresponding operations through feedback information from the vision system.

Machine vision

Machine vision arises from industrial automation. In modern industrial automation production, a variety of inspection, measurement and part identification applications are involved, such as automobile parts size inspection and automatic assembly integrity inspection, automatic assembly component positioning of electronic assembly lines, and printing quality inspection of beverage bottle caps , Barcode and character recognition on product packaging, etc. The common features of this type of application are continuous high-volume production and very high demands on appearance quality. Usually this kind of highly repetitive and intelligent work can only be completed by manual inspection. We often see hundreds or even more than thousands of inspection workers to perform this process behind the modernization lines of some factories. At the same time that the factory adds huge labor and management costs, it still cannot guarantee a 100% inspection pass rate. Today's competition between companies has not allowed even 0.1% defects. Sometimes, such as accurate and fast measurement of tiny size, shape matching, color recognition, etc., cannot be performed continuously and steadily by the human eye, and other physical quantity sensors are also difficult to use. At this time, people began to consider combining the fastness, reliability, and repeatability of computers with the highly intelligent and abstract capabilities of human vision, and gradually formed a new discipline-machine vision.

Machine vision is the science and technology that studies the use of computers to simulate biological macroscopic vision functions. In layman's terms, machines are used to replace human eyes for measurement and judgment. First, a CCD camera is used to convert the captured target into an image signal, which is transmitted to a dedicated image processing system, which is converted into a digital signal based on information such as pixel distribution, brightness, and color; the image system performs various operations on these signals to extract the characteristics of the target , Such as: area, length, number, position, etc .: Finally, output results according to preset tolerances and other conditions, such as: size, angle, offset, number, pass / fail, presence / absence, etc. Machine vision is characterized by automation, objectivity, and non-contact. Compared with image processing systems in the general sense, machine vision emphasizes identification and judgment, as well as reliability in the industrial field environment.

Machine vision is a fairly new and rapidly developing research area. People began to study the statistical pattern recognition of two-dimensional images from the 1950s. Roberts began to research on 3D machine vision in the 1960s. In the 1970s, the MIT artificial intelligence laboratory officially opened a "machine vision" course. In the 1980s, it began to be global Research boom, machine vision has developed vigorously, and new concepts and new theories continue to emerge. Nowadays, machine vision is still a very active research field, and its related disciplines include: image processing, computer graphics, pattern recognition, artificial intelligence, artificial neural network, etc.

The emergence and development of computer vision

Computer vision began in the 1950s with statistical pattern recognition. At that time, work focused on two-dimensional image analysis, recognition and understanding, such as the analysis and interpretation of optical character recognition, workpiece surfaces, micrographs, and aerial photographs. . In the 1960s, Roberts restricted the environment to the so-called "building block world", that is, the surrounding objects are composed of polyhedrons. Objects that need to be identified can be represented by a combination of simple points, lines, and planes. A computer program is used to extract the three-dimensional structure of a polyhedron, such as a cube, a wedge, or a prism, from a digital image, and describe the shape of the object and its spatial relationship. Roberts' research work pioneered the research of 3D machine vision with the goal of understanding 3D scenes. By the 1970s, some visual application systems had appeared. In the mid-to-late 1970s, the maturity of television camera technology and the development of computers provided advanced technological means for the study of computer vision. During this period, the Computer Vision Research Group of the MIT Artificial Intelligence (AI Laboratory) established, and Opened "Machine Vision" course, which attracted many well-known scholars to participate in the research of machine vision theory, algorithms, and system design. In 1977, Marr proposed a computer vision theory different from the "building block world" analysis method ( Computational Vision), the theory became a very important theoretical framework in the field of computer vision research in the 1980s. Marr proposed that the study of visual information processing should be divided into three levels, namely the theory level of calculation and the algorithm level , Hardware implementation layer. The three answered the input and output and the constraints between them, the input and output representations and corresponding algorithms, and how to physically implement such representations and algorithms. Although there are incomplete aspects of the framework in details and even in the dominant ideology, many parties There are still many controversies, but it is still the basic framework of current computer vision research. Marr theory provides us with many precious philosophical ideas and research methods for studying machine vision, and also creates many research starting points for the field of computer vision research.

Since the 1980s, computer vision has developed vigorously, new concepts, new methods, and new theories have emerged. More and more computer vision researchers have challenged the traditional general vision based on the Marr framework. The most representative is the emergence of The vision vision school led by Aloimonos Y of the Computer Vision Research Laboratory of the University of Maryland, USA; the active vision school of Bajcsy, Department of Computer Science, University of Pennsylvania, USA; the active vision school of Ballade and Brown, University of Rochester, etc. Objective vision and active vision have been the research hotspots of computer vision in recent years. Different from the general vision theory based on Marr, active vision emphasizes two points. One is that the visual system should have the ability of active perception; the other is that the visual system should be based on a certain task (Purposive Directed) or purpose. At the same time, active vision believes that the process of vision based on no purpose is meaningless, and the vision system must be connected with specific purposes (such as navigation, recognition, operation, etc.) to form a perception / action cycle. . Purpose vision believes that vision has a purpose, and the purpose is behavior. Aiming at specific objects and applications, the purpose vision has been widely used in industry and agriculture and other industries. The research of general vision is more focused on basic theory, and the goal vision is more application-oriented. The research of general vision should draw on the results of active perception and feedback control in target vision. The research of target vision seeks new growth points for the research of general vision.

A typical machine vision control system generally includes the following parts: light source, lens, CCD camera, image processing unit (or frame grabber), image processing software, monitor, communication / input / output unit, etc. The output of the vision system is not an image video signal, but a detection result after arithmetic processing, such as size data or judgment classification. After the upper computer such as PC and PLC obtain the detection results in real time, they instruct the motion system or I / 0 system to perform corresponding control actions, such as positioning and sorting. The basic building blocks are shown in the figure.

The research content of visual control is relatively extensive, mainly including camera calibration, image processing, feature extraction, visual measurement and control algorithms, etc .:

Camera calibration

The process of obtaining the internal parameters and external parameters of the camera. The vision system starts from the image information obtained by the camera, calculates geometric information such as the position and shape of the object in the three-dimensional environment, and reconstructs the three-dimensional object from it. The position of each point on the image is related to the geometric position of the corresponding point on the surface of the space object. The relationship between these positions is determined by the camera imaging geometric model. The parameters of the geometric model are called camera parameters, which mainly include internal parameters and external parameters. The internal parameters mainly include the image coordinates of the center point of the optical axis, the magnification coefficient from the imaging plane coordinates to the image coordinates, and the lens distortion coefficient. The external parameter is the representation of the camera coordinate system in the reference coordinate system. Camera calibration provides the link between non-measurement cameras and professional cameras. The so-called non-measurement camera refers to a camera of this type, whose internal parameters are completely unknown, partially unknown, or in principle uncertain. Camera calibration is to obtain the internal and external parameters of the camera through calibration experiments.

Visual measurement

Measurement of the position and attitude of the target based on the visual information obtained by the camera. Visual measurement mainly studies the mapping from two-dimensional image information to three-dimensional Cartesian spatial information and the composition of a visual measurement system. The brightness of each point on the image reflects the intensity of the reflected light at a point on the surface of the space object, and the position of the point on the image is related to the geometric position of the corresponding point on the surface of the space object. The research of visual measurement mainly lies in measurement speed and accuracy.

Structure and Algorithm of Vision Control

The robot vision control essentially uses the two-dimensional image information collected by the camera to control the robot's motion. Different uses of the visual information will result in different control effects. The closed-loop control system formed in Cartesian space can only ensure that the position and attitude of the visually measured target in Cartesian space reach the desired value. Due to the model error of the camera and the matching error of the feature points, the visual measurement itself has a large Error, plus the model error of the robot, so the target in the Cartesian space will sometimes have a larger error between the actual position and attitude, and the control accuracy is lower. A closed-loop system is formed in the image space. Although the accuracy can be improved, the stability of the control is difficult to guarantee. ^[2]