HONG KONG, July 28, 2022 (GLOBE NEWSWIRE) — WIMI Hologram Academy, working in partnership with the Holographic Science Innovation Center, has written a new technical article describing their exploration of VR-based remote vehicle driving technology. This article follows below:

Driverless as well as artificial intelligence are the current research hotspots in the field of vehicles. Considering the cost of experiments, how to conduct unmanned vehicle road tests in a virtual environment has become a new research direction. In driverless experiments, a driver is required to maintain control of the vehicle in the test vehicle, which undoubtedly increases the risk of driver injury. Remote driving not only retains the driver’s control of the vehicle, but also reduces the risk of driver injury. At the same time, remote driving has good prospects in the field of mobile robotics. Scientists from WIMI Hologram Academy of WIMI Hologram Cloud Inc.(NASDAQ: WIMI), discussed in detail the development of VR-based remote vehicle driving technology.

With the continuous progress of sensors and machine learning, many domestic and foreign passenger car manufacturers are aiming at the development direction of unmanned and intelligent vehicle driving. The rapid rise of autonomous vehicle driving technology has pushed the driverless industry to heat up rapidly. But if the realization of completely driverless, and put into actual operation, there are still many technical difficulties, can not yet reach all-weather and unstructured road driving, driverless technology from its initial exploration, to the development of mature technology to the full promotion of popularization will take a long time.

Driverless technology mainly faces many technical limitations such as road condition recognition, vehicle control and vehicle information interaction, and a series of social issues such as ethics, legal responsibilities and regulatory policies are gradually emerging. Road condition recognition and environmental perception based on artificial intelligence is one of the most important technologies for driverless vehicles, which need to independently identify pedestrians, oncoming vehicles and traffic lights and make corresponding judgments. However, in recent years, there have been a number of fatal accidents involving self-driving cars, posing greater questions and challenges to the current road condition recognition technology.

For unmanned vehicles, the information interaction between vehicles and modern transportation systems, and vehicles and vehicles is also particularly important. In addition, OTA as the future development trend of driverless vehicles to upgrade the on-board program, the download speed, coverage and signal stability of the network also play a vital role in the driving safety and ease of use of driverless vehicles. From this point of view, the development of driverless vehicles will have a long way to go.

Remote control technology was first proposed in the early 20th century, and with the emergence of new technologies such as computer technology, wireless communication technology and sensor technology, it is now widely used in various fields such as sea, land and air, affecting all aspects of human life. The use of remote control technology can enable machines or equipment to reach difficult environments that are difficult for people to reach, effectively reducing work costs and avoiding casualties. Reliable navigation, effective motion control and real-time acquisition of positioning sensor data are the keys to vehicle remote control technology. The traditional vehicle remote control technology is through the picture returned from the vehicle installed sensors and cameras, the operator analyzes the current environment in which the vehicle is located and makes a decision to adjust the attitude of the vehicle to realize the vehicle control. Due to the limited number of sensors and cameras installed in the vehicle, blind spots may arise. In addition, the operator’s sense of presence is not strong, and the operation accuracy is not guaranteed.

The communication technology also limits the real-time data transmission, which makes the remote operator’s control of vehicle motion posture unable to make timely adjustment according to the environmental changes. Since remote vehicles can only rely on human behavioral decisions for movement, but the operator cannot fully access vehicle and environmental information, when the vehicle faces unexpected conditions, the lack of certain autonomy, such as obstacle judgment and environmental perception, leads to vehicle remote control failure. With the vigorous development of machine learning, artificial intelligence and other emerging disciplines, vehicles are becoming more intelligent and autonomous, providing a new starting point for vehicle remote driving. In addition, with the development of 5G network, the large bandwidth, ultra-low latency and ultra-high reliability make the real-time transmission of data effectively guaranteed, and also provide reliable communication guarantee for vehicle remote driving.

Virtual reality technology has strong immersion, interactivity and imagination, the so-called 3I characteristics, and has also gradually penetrated into various fields of science and technology in recent years. China, because of its late research on driving technology in virtual reality environment, needs further development to gradually narrow the gap with developing countries in the new historical environment, so as to achieve stronger and more immersive virtual simulation technology. Relevant enterprises in China are also committed to the research of VR remote control. So far, there has been the emergence of head enterprises for unmanned vehicles virtual driving research, in a relatively leading position in the country.

Domestic and foreign research and a series of results achieved through the VR remote control robot, robotic arm, so that VR remote control vehicles become possible. Combining the movement of human and remote machinery, the operator manipulates the remote machinery while the remote machinery has a certain degree of autonomy, thus realizing the training and remote control of the machinery. Therefore, combined with virtual reality technology, integrated with the development of modern communication, and improve a certain degree of autonomy of the vehicle, through the virtual environment to complete the full range of observation and control of unmanned vehicles, to achieve virtual remote driving of vehicles, not only can effectively solve the current unmanned vehicles facing the lack of environmental recognition, vehicle autonomy control inaccuracy and other problems, but also to ensure vehicle safety, but also to enable the operator to have an immersive driving experience.

1 Framework for remote driving technology

As digital technology continues to evolve, virtual driving technology based on virtual reality continues to advance. Virtual driving technology has also gone from research on car simulators to today’s applications in road testing of unmanned vehicles and driver training. A virtual driving system mainly consists of a control system, vision simulation system, driving simulator and other components such as audio equipment and display. Among them, the control system is physical, including the steering wheel, gas pedal, brake pedal and gearshift lever, and the driver changes the motion of the vehicle in the virtual environment through the control system, and the vision simulation system solves the vehicle posture and transmits the vehicle information to the driver through the monitor, so that the driver can perceive the vehicle motion posture and road environment changes in time. The driving simulator adjusts the vehicle motion posture according to the changes of the vehicle’s posture, so that the driver’s posture is consistent with the body posture, enhancing the experience and authenticity.

Vehicle remote control system mainly includes two major parts: monitoring terminal and vehicle terminal. The main position and posture information of the vehicle is collected through satellite positioning and sensors, and transmitted to the operation interface of the monitoring personnel by network, and the operator analyzes and judges the information and issues relevant instructions to realize the adjustment of the vehicle’s movement posture. The remote control system mainly transmits information through wireless network, and the emergence of 5G network provides a good solution to the communication delay.

The virtual vehicle is realized in a virtual environment by vehicle dynamics model, road model, tire model, etc., while the real vehicle is driven on the actual road. Due to the influence of modeling accuracy, in order to ensure the virtual vehicle and the real vehicle as consistent as possible with the bit attitude state, based on the analysis of the virtual driving system and the vehicle remote control system, the structure of the vehicle remote driving technology based on virtual reality is established. Moreover, the information between the real and virtual vehicles uses 5G technology to achieve shop floor communication. The driver controls the motion state of the virtual vehicle by manipulating the steering wheel, gas pedal, brake pedal and shift handle and other manipulation systems, and the virtual vehicle adjusts the manipulation of the actual vehicle through attitude precession to make the motion state of the real vehicle change. If the real vehicle state is not consistent with the virtual vehicle state, the data is transmitted to the attitude prediction module through the attitude correction module between the two vehicles to realize the vehicle attitude adjustment of the real vehicle. At the same time, the surrounding environment information collected by the real vehicle sensor generates the virtual environment of the virtual vehicle in real time and passes it to the visual simulation system, and feeds the information to the driver through some helmet display devices, and the posture information generated by the vehicle due to the change of the motion posture is fed back through the driving seat to ensure that the remote driver has the feeling of driving the actual vehicle, further enhancing the remote vehicle The safety of driving.

The operator wears a VR headset, and by observing the movement state of the vehicle and the surrounding environment in the virtual environment within the headset, the operator can make corresponding operation commands to the vehicle, such as manipulating the gearshift handle to shift gears, controlling the steering wheel to make the vehicle steering, etc., so that the movement state of the vehicle in the real environment can be changed accordingly to achieve remote control of the real vehicle. At the same time, in order to eliminate blind spots in the field of view, the vehicle has a certain degree of autonomy by installing radar and sensors on the real vehicle. Compared with the unmanned technology, the vehicle virtual remote driving technology does not really exclude people in the vehicle control, but through the operator remotely issued a series of control actions, and through the virtual environment to monitor and control the actual vehicle, both to achieve the vehicle “unmanned” driving, and effectively improve the safety of remote vehicle driving performance, so that the driver and the vehicle can be separated. The driver is separated from the vehicle, and the vehicle can be truly unmanned.

2 Key technology analysis

1)Environment real-time generation technology

Virtual driving environment can provide a simulation environment similar to the actual driving for autonomous driving and virtual driving, and can also construct a typical and complex driving environment. Due to the specificity of its function, the use of virtual reality technology to achieve real-time simulation of the driving environment and to constitute a multidimensional, perceptible, measurable and realistic virtual driving environment is particularly important for remote driving of vehicles. In the virtual world, the physical model of the virtual world focuses on geometric representation and lacks realistic physical and behavioral models, thus making a big difference between virtual objects and real objects. Moreover, virtual reality technology currently focuses on visual aspects, and research in auditory, haptic, and force feedback is still relatively backward, which may lead to some limitations in human-computer interaction and reduce realism and real-time. The complexity and uncertainty of the actual driving environment makes the real-time rendering of the virtual driving environment a great challenge.

2) Vehicle dynamics simulation technology based on artificial intelligence

Artificial intelligence is one of the branches of computer science, which contains more fields, covering robotics, language recognition, image recognition, etc. With the deepening of artificial intelligence research, artificial intelligence is gradually applied to various fields. At present, the application of artificial intelligence in vehicles is mainly used in the automatic driving of vehicles, to complete the recognition of visual and language information of vehicles. The structure of the vehicle dynamics system is more complex, with more degrees of freedom in the system and more connections between components, which increases the difficulty of accurately describing the vehicle dynamics performance in the virtual environment. Artificial intelligence based on deep learning, applied to the study of vehicle dynamics simulation model, the further development of vehicle dynamics simulation technology in virtual environment is of great significance.

3) Collision detection technology

The models in the virtual scene, including the sky, ground, vehicles and other objects, are independent of each other, in the process of interactive control, the viewpoint (virtual vehicle) can pass directly from each object. In real life, it is impossible for two impenetrable objects to have the same spatial area, so collision detection can be used to determine whether a collision has occurred between different objects in the virtual scene at a certain moment. However, the current collision detection technology does not meet the needs of arbitrary object detection and is still limited to regular geometric convex bodies, so it is important to study the collision detection technology to meet the virtual driving environment.

4) Wireless communication technology

In the past decade, mobile communication technology has become the most widely used and rapidly developing communication technology today. The rapid development of mobile communication technology has led to the emergence of new technologies and new services. To realize remote control driving of real-world vehicles in a virtual environment, it is necessary to transmit the relevant data of the virtual world and real vehicles in real time and send instructions in time. The wireless communication technology is like a bridge that builds the road of data communication between the vehicle remote control system and the monitoring center. The use of wireless communication network to establish vehicle remote control system has the characteristics of always online, wide network coverage, low communication cost, fast data transmission, and good real-time. The real-time and rapid nature of communication technology is important for adjusting the real-time posture of the actual vehicle as well as responding to unexpected situations. For remote driving vehicles, the real-time transmission of data and drawings is especially important. Therefore, the problem of delay and lag of wireless communication technology is a key technology. The development of 5G wireless communication technology also provides greater possibility for the realization of remote driving of vehicles.

3 Conclusion

This paper analyzed the characteristics of driverless, virtual driving and vehicle remote control at home and abroad, combined with the variability of driving environment, the simulation accuracy of vehicle dynamics model and the needs of immersion and interaction in actual driving to improve vehicle driving safety, introduced the framework of remote driving of vehicles in virtual reality environment, analyzed the related key technologies, and described the future development brought by virtual remote driving based on it.

Founded in August 2020, WIMI Hologram Academy is dedicated to holographic AI vision exploration, and conducts research on basic science and innovative technologies, driven by human vision. The Holographic Science Innovation Center, in partnership with WIMI Hologram Academy, is committed to exploring the unknown technology of holographic AI vision, attracting, gathering and integrating relevant global resources and superior forces, promoting comprehensive innovation with scientific and technological innovation as the core, and carrying out basic science and innovative technology research.

Contacts

Holographic Science Innovation Center

Email: pr@holo-science. com