Wenjie Song

A unified framework for multi-agent task assignment and distributed trajectory planning that can autonomously adapt to complex interactive environments and multi-task constraints has always been the bottleneck of unmanned cluster task. In multi-agent systems such as smart parking lots and smart intersections, tasks are time-varying with dynamically interactive agents, which cause the challenges such as inefficient assignment, spatiotemporal conflict and poor adaptive ability. Focusing on them, this paper proposes a hierarchical framework that combines centralized task assignment module with a multi-agent reinforcement learning based distributed trajectory planning module, which has a good scalability and task adaptability. The benefit of the proposed hierarchical framework is that it utilizes behavioral heuristic of congestion level and planning cost during assignment to motivate the appropriate assignment, which achieve an organic integration of multiple objectives. In terms of spatiotemporal conflicts for multi-agent, a weighted network structure is designed to capture dynamic obstacle information while introducing conflict constraints for collision avoidance policy optimization. Furthermore, in order to cope with changing tasks, re-assignment and re-planning mechanisms are incorporated into the framework, as well as the graph encoding layer which is adaptive to the uncertain tasks. Extensive experiments are conducted in autonomous parking scenarios to validate the effectiveness of the approach in task assignment and path conflict mitigation. Compared with other state of art methods, the task assignment and overall success rates have increased by 12.1% and 6.8%, respectively, with negligible computing time.

Autonomous driving in urban scenarios faces uncertain dynamic changes, especially in China, where a dense mixture of cars, cyclists and pedestrians travel together on roads with random uncertain behaviors and high-risk road crossing. This paper proposes a Multi-step Continuous Decision Making and Spatio-temporal Trajectory Planning framework to achieve stable continuous decision making and high-quality trajectory planning in such uncertain and highly dynamic environments. Firstly, a 3D spatio-temporal probabilistic map is constructed to represent the uncertain future driving environment. Based on the map, parallel spatio-temporal trajectory search is performed to obtain multi-strategy feasible spatio-temporal trajectories that satisfy the short-term deterministic and long-term uncertain environmental constraints. Then considering the continuity and consistency of decision making, risk-aware rolling-fusion of trajectory sequences is proposed, achieving efficient and exploratory far-end planning with a stable and safe near-end driving trajectory. To validate the proposed framework, we collected the Hard Case data from real Chinese urban roads, containing challenging scenarios such as dense traffic flows, mixed vehicle-pedestrian roads, and complex intersections, which are widely recognized barriers to the successful real-world deployment of autonomous driving. Moreover, the SMARTS simulator is used to build closed-loop simulation scenarios to verify the effectiveness of the framework. Experimental results show the superior performance of our proposed framework in complex uncertain dynamic scenarios.

When applied in building rescue, forest search and other missions, UAV swarms often encounter challenges such as narrow space constraints and dynamic changes of obstacles, which leads to internal conflicts in the swarm or cause system chaos due to avoiding dynamic obstacles during the execution of tasks. These rigorous characteristics of the actual application scenarios and tasks always make global searching difficult to be optimized and local obstacle avoidance into deadlock. To solve the above problems, this paper introduces a hierarchical online collaborative planning framework inspired by global spatio-temporal paths. In order to reduce the solution complexity while dissolving the overall conflict, global rough searching is conducted for initial spatio-temporal guidance path. To overcome dynamic obstacles without affecting the system internal harmony, online conflict-trend-based clustering co-optimization inspired by the initial path is performed to achieve obstacle avoidance while improving scalability. Experimental tests were conducted in simulation environments with narrow pipelines and dynamic obstacles with comparison to current mainstream centralized and distributed methods. The results demonstrate that our work can generate smoother and safer trajectories with higher success rate in the constraint-rich dynamic environments.

Unknown area exploration is a crucial but challenging task for autonomous driving of unmanned ground vehicles (UGV) in unknown off-road environments. However, the exploration efficiency of a single UGV is low due to its limited sensing range. To solve this problem, this paper proposes a risk-inspired aerial active exploration system, which utilizes the flexibility and field of view advantages of Unmanned Aerial Vehicles (UAV) to guide the UGV in unknown off-road environments. Firstly, a fast terrain risk mapping method that can be used for both UAV and UGV is developed. This method efficiently combines quadtree and hash table data structure to enable UAV to analyze large scale terrain point cloud in real time. Based on the risk mapping result, a risk-inspired active exploration method is proposed to actively search a safe reference path for the UGV, which introduces terrain risk information into the process of travel point selection. Finally, the reference path is gradually generated and optimized, so that the UGV can safely and smoothly follow the path to the target location. Compared with single UGV exploration system, our approach reduces the overall path risk by 26.8% in simulated experiments, showing that the proposed system can enhance autonomous driving of the UGV and help it effectively avoid high-risk areas in unknown off-road environments.

All-day self-supervised monocular depth estimation has strong practical significance for autonomous systems to continuously perceive the 3D information of the world. However, night-time scenes pose challenges of weak texture and violating the brightness consistency assumption due to low illumination and varying lighting, respectively, which easily leads to most existing self-supervised models only being able to handle day-time scenes. To address this problem, we propose a self-supervised monocular depth estimation unified framework that can handle all-day scenarios, which has three features: (1) an Illumination Compensation PoseNet (ICP) is designed, which is based on the classic Phong illumination theory and compensates for lighting changes in adjacent frames by estimating per-pixel transformations; (2) a Dual-Axis Transformer (DAT) block is proposed as the backbone network of the depth encoder, which infers the depth of local low-illumination areas through spatial-channel dual-dimensional global context information of night-time images; (3) a cross-layer Adaptive Fusion Module (AFM) is introduced between multiple DAT blocks, which learns attention weights between different layer features and adaptively fuses cross-layer features using the learned weights, enhancing the complementarity of different layer features. This work was evaluated on multiple datasets, including: RobotCar, Waymo and KITTI datasets, achieving state-of-the-art results in both day-time and night-time scenarios.

The accurate and stable estimation of traversability is a crucial task for unmanned ground vehicle (UGV) driving in off-road environments. However, the complexity of off-road environments increases the difficulty of the task. Moreover, the stability of traversability estimation is adversely affected by ego-motion uncertainty caused by the violent jolts when the UGV is traveling fast on uneven surfaces. The lack of prior information also poses a significant challenge to the task. To address these challenges, this article proposes a novel framework for cost map (CM) generation, which uses light detection and ranging (LiDAR)-inertial odometry (LIO) and historically observed frames to generate local CMs with no need for any prior information. To describe the traversability of complex environments, we design a cost calculation method that includes a variety of factors affecting UGV driving. Not only terrain features such as slope and roughness, but also potential slip risks are considered in it. In consideration of ego-motion uncertainty, terrain continuity is modeled as a spatial constraint to enhance sparse laser scans, and historical observations are fused to filter out noises in the temporal dimension. Real-world experimental results demonstrate that the proposed method can generate stable CM with detailed traversability descriptions even with violent UGV jolts.

In the past decade, reinforcement learning (RL) has achieved encouraging results in autonomous driving, especially in well-structured and regulated highway environments. However, few researches pay attention to RL-based multiple-vehicles cooperative driving, which is much more challenging because of dynamic real-time interactions and transient scenarios. This article proposes a Multi-Agent Reinforcement Learning (MARL) based twin-vehicles cooperative driving decision making method which achieves the generalization adaptation of the RL method in highly dynamic highway environments and enhances the flexibility and effectiveness of collaborative decision making system. The proposed fair cooperative MARL method pays equal attention to the individual intelligence and the cooperative performance, and employs a stable estimation method to reduce the propagation of overestimated joint Q -values between agents. Thus, the twin-vehicles system strikes a balance between maintaining formation and free overtaking in dynamic highway environments, to intelligently adapt to different scenarios, such as heavy traffic, loose traffic, even some emergency. Targeted experiments show that our method has strong cooperative performance, also further increases the possibility of creating a harmonious driving environment.

Terrain risk estimation has always been a crucial but challenging task for autonomous driving of unmanned ground vehicles (UGV) in off-road environments. The map with dense and accurate risk information can help UGV avoid special obstacles such as steep slopes, pits, and ditches while path planning. To solve this problem, this article proposes an aerial-ground collaboration system for continuous risk mapping in off-road environments, utilizing the flexibility and perspective advantages of unmanned aerial vehicles (UAVs). In the system, UAV carries a downward 2-D laser and a visual inertial system to construct a bird’s eye map, while UGV is equipped with a 3-D LiDAR for local mapping. In detail, our system focuses on two aspects, which are cross-view map matching and collaborative continuous risk mapping. For cross-view map matching, risk values for point clouds from aerial and ground platforms are calculated and used for terrain segmentation. Then, point-to-voxel residual is built for terrain point sets to accelerate matching. For collaborative continuous risk mapping, an entropy based probabilistic fusion is proposed for accurate risk fusion of overlapping cells. Then, a Bayesian generalized kernel inference algorithm is adapted to predict risk values of the remaining unknown areas. Simulation and real-world experimental results show that the dense and continuous global risk map constructed by the proposed system can effectively assist UGV for path planning in off-road environments.

Automated Valet Parking (AVP) has been exten-sively researched as an important application of autonomous driving. Considering the high dynamics and density of real parking lots, a system that considers multiple vehicles simultaneously is more robust and efficient than a single vehicle setting as in most studies. In this paper, we propose a dis-tributed Multi-agent Reinforcement Learning(MARL) method for coordinating multiple vehicles in the framework of an AVP system. This method utilizes traditional trajectory planning to accelerate the learning process and introduces collision conflict constraints for policy optimization to mitigate the path conflict problem. In contrast to other centralized multi-agent path finding methods, the proposed approach is scalable, distributed, and adapts to dynamic stochastic scenarios. We train the models in random scenarios and validate in several artificially designed complex parking scenarios where vehicles are always disturbed by dynamic and static obstacles. Experimental results show that our approach mitigates path conflicts and excels in terms of success rate and efficiency.

Self-supervised monocular depth estimation methods have achieved remarkable results on natural clear images. However, it is still a serious challenge to directly recover depth information from blurred images caused by long-time exposure while camera fast moving. To address this issue, we propose a unified framework for simultaneous deblurring and depth estimation (SDDE), which has higher coupling performance and flexibility compared with the simple concatenation strategy of deblurring model and depth estimation model. This framework mainly benefits from three features: 1) a novel Task-aware Fusion Module (TFM) to adaptively select the most relevant intermediate shared features for the dual decoder network by aggregating multi-scale features, 2) a unique Spatial Interaction Module (SIM) to learn higher-order representation in the encoder stage to better describe complex boundaries of different classes in high-dimensional space, and focuses on the task-related region by modeling the pairwise spatial correlation of the holistic tensor, 3) a Priors-Based Composite Regularization term to jointly optimize the shared encoder-dual decoder network. This work was evaluated on multiple datasets, including: Stereo blur, KITTI,NYUv2, REDS and our own large-scale stereo blur dataset, resulting in state-of-the-art results for depth estimation and image deblurring, respectively.

As the vital factor of vehicle behavior understanding and prediction, vehicle taillight recognition is an important technology for autonomous driving, especially in diverse actual traffic scenes full of dynamic interactive traffic participants. However, in practical application, it always faces many challenges, such as ‘variable lighting conditions’, ‘non-uniform taillight standards’ and ‘random relative observation pose’, which lead to few mature solutions in current common autopilot systems. This work proposes an action-state joint learning-based vehicle taillight recognition method on the basis of vehicles detection and tracking, which takes both taillight state features and time series features into account, consequently getting practicable results even in complex actual scenes. In detail, vehicle tracking sequence is used as input and split into pieces through a sliding window. Then, a CNN-LSTM model is applied to simultaneously identify the action features of brake lights and turn signals, dividing taillight actions into five categories: None, Brake_on, Brake_off, Left_turn, Right_turn. Next, the brightness of high-position brake light is extracted through semantic segmentation and combined with taillight actions to form higher-level features for taillight state sequence analysis. Finally, an undirected graph model is used to establish the long-term dependence between successive pieces by analysing the higher-level features, thus inferring the continuous taillight state into: off , brake , left , right . Datasets including daytime, nighttime, congested road, highway, etc. were collected, tested and published in our work to demonstrate its effectiveness and practicability.

In order to serve humanity better in the disabled assisting, transportation, rescue, and other aspects, a human-sized bionic bipedal structure with low inertia and high-load capacity is presented. The designed robot named XingT adopts a non-coaxial five-link leg structure, after kinematic modeling analysis and structural comparison, being able to bear a load beyond its weight. Besides, bionic tibia modules and passive compliant units are adopted, effectively absorbing motion impact in high-load and strong-impact scenarios. In addition, with structural design and simulation analysis, the proposed five-link structure can realize multi-mode switching, which can adapt to high load application in the convex five-link form and switch to concave five-link mode for light and flexible walking effect. Results of prototype and simulation experiments showed that it can reduce impacts by 67% and increase the load performance by 22.3%, which verifies its structural performance for future practical application.

Intelligent decision making and efficient trajectory planning are closely related in autonomous driving technology, especially in highway environment full of dynamic interactive traffic participants. This work integrates them into a unified hierarchical framework with long-term behavior planning (LTBP) and short-term dynamic planning (STDP) running in two parallel threads with different horizon, consequently forming a closed-loop maneuver and trajectory planning system that can react to the dynamic environment effectively and efficiently. In LTBP, a novel voxel structure and the ‘voxel expansion’ algorithm are proposed for the generation of driving corridors in 3D configuration, which involves the prediction states of surrounding vehicles. By using Dijkstra search, the maneuver with minimal cost is determined in form of voxel sequences, then a quadratic programming (QP) problem is constructed for solving the optimal trajectory. And in STDP, another small-scaled QP problem is performed to track or adjust the reference trajectory from LTBP in response to the dynamic obstacles. Meanwhile, a Responsibility-Sensitive Safety (RSS) Checker keeps running at high frequency for real-time feedback to ensure security. Experiments on real data collected in different highway scenarios demonstrate the effectiveness and efficiency of our work.

Autonomous driving, including intelligent decision-making and path planning, in dynamic environments (like highway) is significantly more difficult than the navigation in static scenarios because of the additional time dimension. Therefore, correlating the time dimension and the space dimension through prediction to create a spatio-temporal navigation map can make decision-making and path planning in such kinds of environment much easier. In this article, NGSIM data is analysed and processed from the perspective of the ego-vehicle (using the data as an ego-vehicle’s perception results). Based on the data, we develop an LSTM (Long-Short Term Memory)-based framework to predict possible trajectories of multiple surrounding vehicles within a certain range of the ego-vehicle. Then, the multiple predicted trajectories in a series of continuous dynamic highway scenes are projected into a spatio-temporal domain to create an octree map. Thus, dynamic targets and static obstacles can be unified into the same domain or map so that the dynamic disturbance problem for autonomous driving in highway environments can be resolved. Experimental results show that the proposed model is capable of predicting all the future trajectories around the ego-vehicle efficiently and the corresponding spatio-temporal map can be generated accurately in different dynamic scenarios.

Trajectory planning for the unmanned vehicle in the complex environment has always been a challenging task. Planned trajectory with the corresponding target velocity or acceleration sequence must be collision-free guaranteed and as comfortable as possible on the premise of obeying the traffic rules and interaction with other dynamic social vehicles. To meet this requirement, this paper proposes a framework for trajectory planning based on spatio-temporal map. Due to the time layer architecture in the map, the trajectory can be generated with velocity and acceleration simultaneously, and the whole trajectory is constrained within a ‘safety strip’, resulting in an efficient and safety guaranteed trajectory. The framework is composed of three sections: rough search, fine optimization and safety strip-based collision avoidance. For rough search, we propose an improved A* algorithm implemented in the discrete time layer to find out the suboptimal states efficiently. In fine optimization, the B-spline curve is exploited to connect the searched states into a continuous trajectory. And the optimal control points of B-spline are further grouped into several segments, forming the safety strip which is actually the distribution space of the planned trajectory. If necessary, an adjustment will be applied to keep the strip away from the collision zone, making the entire trajectory completely collision-free. Experiments on both public dataset and self-driving simulator show that the proposed framework can adapt to different kinds of complex traffic scenes well.

Intersections are quite important and complex traffic scenarios, where the future motion of surrounding vehicles is an indispensable reference factor for the decision-making or path planning of autonomous vehicles. Considering that the motion trajectory of a vehicle at an intersection partly obeys the statistical law of historical data once its driving intention is determined, this paper proposes a long short-term memory based (LSTM-based) framework that combines intention prediction and trajectory prediction together. First, we build an intersection prior trajectories model (IPTM) by clustering and statistically analyzing a large number of prior traffic flow trajectories. The prior trajectories model with fitted probabilistic density is used to approximate the distribution of the predicted trajectory, and also serves as a reference for credibility evaluation. Second, we conduct the intention prediction through another LSTM model and regard it as a crucial cue for a trajectory forecast at the early stage. Furthermore, the predicted intention is also a key that is associated with the prior trajectories model. The proposed framework is validated on two publically released datasets, next generation simulation (NGSIM) and INTERACTION. Compared with other prediction methods, our framework is able to sample a trajectory from the estimated distribution, with its accuracy improved by about 20%. Finally, the credibility evaluation, which is based on the prior trajectories model, makes the framework more practical in the real-world applications.

Deep reinforcement learning (DRL) has been successfully applied to end-to-end autonomous driving, especially in simulation environments. However, common DRL approaches used in complex autonomous driving scenarios sometimes are unstable or difficult to converge. This paper proposes two approaches to improve the stability of the policy model training with as few manual data as possible. For the first approach, reinforcement learning is combined with imitation learning to train a feature network with a small amount of manual data for parameters initialization. For the second approach, an auxiliary network is added to the reinforcement learning framework, which can leverage the real-time measurement information to deepen the understanding of environment, without any guide of demonstrators. To verify the effectiveness of these two approaches, simulations in image information-based and lidar information-based end-to-end autonomous driving systems are conducted, respectively. These approaches are not only tested in the virtual game world, but also applied in Gazebo, in which we build a 3D world based on the real vehicle model of Ranger XP900 platform, the real 3D obstacle model, and the real motion constraints with inertial characteristics, so as to ensure that the trained end-to-end autonomous driving model is more suitable for the real world. Experimental results show that the performance is increased by over 45% in the virtual game world, and can converge quickly and stably in Gazebo in which previous methods can hardly converge.