DOI:http://doi.org/10.65613/742416

CECILIA GARCÍA-HUIDOBRO

Ecuador Latin America University

Abstract

To address the real-time and robust nature of robot obstacle avoidance strategies in dynamic environments, this paper proposes a DRL-based action decision and dynamic obstacle avoidance optimization method. This approach constructs an action policy network centered on the PPO algorithm, introducing a multi-objective reward function to jointly constrain safety, path efficiency, and trajectory smoothness. State inputs fuse LiDAR and inertial navigation data to achieve spatial modeling and velocity estimation for static and dynamic obstacles.The action space employs a continuous control model, outputting linear and angular velocity commands. Simultaneously, an obstacle prediction and fallback mechanism is integrated into the policy layer to enhance obstacle avoidance success rates in highly dynamic environments. The system constructs multi-scenario simulation test environments on the ROS+Gazebo platform and conducts comparative experiments with traditional DWA and non-predictive PPO strategies.Results demonstrate that in high-density dynamic obstacle environments, the proposed method achieves a collision rate below 2.3%, elevates target arrival rate to 96.7%, reduces average path length by 12.5%, and maintains average decision latency under 112 ms, exhibiting strong generalization capability and decision stability. This research provides theoretical support and algorithmic pathways for autonomous behavior planning in mobile robots navigating dynamic complex scenarios, offering broad engineering application potential.

Keywords

Deep reinforcement learning; Dynamic obstacle avoidance; Robot action decision-making; State modeling; Trajectory optimization

1. Introduction

Enhancing robotic motion decision efficiency and obstacle avoidance robustness in highly dynamic environments has become a key research direction for intelligent mobile platforms.Xiao et al. [1] constructed an obstacle avoidance architecture integrating trajectory tracking and policy update mechanisms, validating deep reinforcement learning’s decision-making capabilities in continuous control scenarios. Yang et al. [2] introduced a risk-constrained optimization model to dynamically balance path efficiency and safety redundancy. Hu et al. [3] focused on path planning in unknown environments, proposing a perception-action mapping method based on deep policy iteration to enhance navigation generalization.Lyu et al. [4] optimized action output continuity by refining the DDPG algorithm, enabling robots to achieve higher trajectory stability on complex paths. Park et al. [5] embedded deep reinforcement learning within a digital twin platform, successfully deploying a sensor fusion-based action strategy system in highly enclosed nuclear power scenarios. These studies demonstrate the significant potential of deep reinforcement learning frameworks in multi-source perception fusion, action strategy generation, and complex trajectory planning.

Despite progress in static environments or under regular obstacle conditions, existing methods exhibit three limitations: first, inadequate modeling capabilities for high-density dynamic disturbances, leading to strategy instability under behavioral changes; second, insufficient resolution in action output structural control, hindering adaptation to practical robotic control interfaces; third, the absence of a systematic training framework integrating perception prediction, dynamic obstacle avoidance, and trajectory optimization.To address these challenges, developing a deep policy model for continuous action spaces with trajectory prediction capabilities and multi-objective reward coordination mechanisms is crucial. By adopting a dual-network architecture based on PPO and incorporating velocity prior constraints with path smoothing penalties, the proposed approach enhances control stability while enabling feedforward adaptation to dynamic obstacle behaviors.Within the constructed ROS+Gazebo multi-scenario simulation platform, the proposed system maintains a low collision rate and optimal path efficiency even under highly complex interference scenarios, demonstrating strong deployment versatility and system engineering value.

• Machine Learning-Based Action Decision Methods

In robotic motion control research, rule-based path generation methods like DWA and APF are commonly used for near-field obstacle avoidance. However, their decision-making processes heavily rely on local information and fixed thresholds, making them ill-suited for scenarios with dynamic uncertainties. Building upon this, machine learning techniques have been progressively integrated into the action decision chain. By enhancing state representation capabilities and strategy generalization, these methods significantly improve robotic autonomy in unstructured environments [6].As shown in Table 1, while DWA offers computational efficiency advantages, its local velocity window model lacks dynamic trajectory prediction capabilities. Imitation learning can achieve human-like control strategies with sufficient training data, but its robustness significantly degrades under abnormal conditions due to the strategy’s non-updatability.In contrast, deep reinforcement learning (DRL)-based methods such as PPO and SAC construct state-action mapping functions ( ). This enables dynamic optimization of policy parameters ( ) during interactions and maximizes the value of long-term control behavior through cumulative discounted return functions.

Table 1

Comparative Analysis of Action Decision Model Structures and Performance

Decision Method	State Modeling Approach	Control Output Type	Policy Update Capability	Dynamic Adaptive
Dynamic Weighted Average (DWA)	Local velocity window-based	Discrete velocity pairs	× (Fixed Rules)	Weak (Prone to Local Optima)
Imitative Learning (BC)	Expert trajectory mapping	Discrete or low-dimensional continuous control	× (Offline Learning)	Medium (Depends on coverage)
DRL (PPO/SAC)	Fusion of historical states and sensor inputs	High-dimensional continuous actions (v, ω)	✓ (Continuous Optimization)	Strong (policy generalizability)

• System Models and Methods
  • Environment Modeling

In deep reinforcement learning-based robotic action decision systems, environment modeling not only determines the structural complexity of the state space but also directly influences the perceptual boundaries and generalization capabilities of the policy network. In this study, the environment space is constructed as a two-dimensional discrete-continuous hybrid domain containing static structural obstacles and dynamic moving entities. Here, denotes the set of immovable obstacle regions, represents the set of dynamic obstacles with velocity vectors, indicates the target point, and signifies the boundary topology[7]. To enhance the training stability of reinforcement learning strategies in complex scenarios, the velocity vectors of all dynamic obstacles in the environment are modeled as smooth trajectory functions with Poisson disturbance terms, defined as:

                                                                                                                                                                                                                                                   (1)

where denotes the initial velocity of the obstacle, controls the perturbation amplitude, represents the perturbation frequency, and is the Gaussian noise perturbation term, simulating trajectory deviations and non-periodic disturbances of dynamic obstacles in real environments. Figure 1 illustrates the spatial element composition within the constructed training environment, comprising multi-source obstacles, target points, and path constraint sets for dynamic moving bodies. The motion states of dynamic elements serve as critical input variables during the state modeling phase [8].This modeling approach enables the policy network to learn trajectory prediction and risk avoidance capabilities in dynamic scenarios during training, providing a necessary foundation for subsequent state perception fusion and policy optimization.

Figure 1 Training Space Construction Diagram in Dynamic Environments

• State and Observation Modeling

Before constructing the robot’s action policy network, the environmental state must undergo structured modeling and information fusion. The state vector design must simultaneously incorporate the environment’s static configuration information, dynamic obstacle behavior characteristics, and the robot’s own state feedback to meet the policy network’s requirements for expressing spatiotemporal dynamic features.As shown in Figure 2, the state input comprises three components: ① The two-dimensional distance vector from the LiDAR, representing local spatial boundaries; ② The robot’s velocity and attitude information obtained via IMU and encoders, characterizing its own motion state [9]; ③ The set of dynamic obstacle motion vectors extracted using the Kalman filter fusion module, where denotes obstacle coordinates, represents velocity, and indicates the direction angle of motion.

Figure 2 Structural Framework Diagram of State Modeling and Perception Fusion

This state modeling structure ultimately combines the aforementioned multi-source observations into a unified state vector for input to the deep reinforcement learning policy network, where denotes the length after dimensionality reduction. To ensure the state representation maintains temporal dependency and captures trajectory trends, a sequential state update mechanism is introduced:

                                                                                                                                                                                                                                                             (2)

Here, denotes the time window length, and represents the historical sequence of states at the current time step. This sequence is utilized within the policy network to model state evolution trends via LSTM or Transformer modules [10]. Table 2 lists the components of each state, their source modules, and temporal update characteristics, facilitating precise binding of perception channels during policy architecture design.

Table 2

Correspondence Table of Multi-Source State Component Composition and Perception Sources

State Component Symbol	Source Module	Type	Time-Dependent Update
	LiDAR	Local Spatial Representation	Yes
	IMU + Encoder	Self-motion feedback	Yes
	Trajectory Fusion Module	Dynamic Obstacle Modeling	Yes
	Encoder Attitude Data	Absolute Heading Angle	Yes
	State Cache Structure	Timing history state sequence	Yes

• Action Space and Control Interface

The design of the action space directly determines the control accuracy of the policy output and the executability of the trajectory. Especially when facing dynamic obstacles and irregular spatial layouts, discrete action modes struggle to meet higher-order control requirements such as trajectory continuity and turning flexibility.This system employs a two-dimensional differential-drive mobile robot platform as the execution entity. Its action space is modeled as a continuous control space , jointly defined by linear velocity and angular velocity , enabling higher-resolution strategy network outputs and adjustable control behaviors [11]. Within the strategy architecture, control outputs are generated through mapping functions

                                                                                                                                                                                                                                                            (3)

maps the current state to a continuous action vector . Here, represents the parameterized output of the policy network, while denotes the Gaussian policy perturbation used to enhance exploration capabilities. Figure 3 constructs the structural mapping diagram of the control output channel. In this structure, the policy output directly interfaces with the differential control command issuance interface after passing through a normalization layer and a velocity mapping module, ensuring protocol consistency between the policy output and the lower-level execution module.

Figure 3 Structural Map of Continuous Action Policy Output and Control Interface Mapping

To enhance the controllability of the physical platform, the system also incorporates a dynamic velocity constraint function:

                                                                                                                                                                                                                                                                   (4)

Here, and represent velocity adjustment functions based on obstacle distance and angular deviation, respectively. These ensure automatic velocity amplitude limitation during dynamic obstacle approaches or sharp turning requirements, reinforcing the strategy’s safety boundary control capability. This action control structure provides the physical execution foundation for defining dynamic smoothness and obstacle avoidance metrics in subsequent reward function design. It also interacts with the behavior pruning mechanism during training to form a control–strategy–feedback closed-loop structure [12].

• Reward Function Design

As the cornerstone for deep reinforcement learning strategy convergence and behavioral regulation, reward function design must simultaneously balance multiple performance objectives including navigation efficiency, path safety, and trajectory smoothness. In the current task, the action strategy outputs continuous velocity commands, while the state space encompasses static obstacle distances, dynamic target relative displacements, and the robot’s own velocity information. Consequently, the reward function structure must achieve joint modeling of target guidance and risk penalties [13].The overall reward function is constructed as a weighted composite structure:

                                                                                                                                                                                                                                                         (5)

where denotes the instantaneous reward value at time step; represents the reward for reducing distance toward the target point; indicates the reward for distance to the nearest obstacle, which becomes a penalty if below the threshold; reflects the smoothness of action change rates, encouraging velocity continuity; and signifies the target approach rate per unit path length. The weighting coefficient controls the relative importance of each objective during optimization [14].The mathematical definitions, constraint boundaries, and strategic roles of each reward subfunction are shown in Table 3.

Table 3

Structural Definition and Policy Effect Analysis of Multi-Objective Reward Subfunctions

Reward Term Symbol	Mathematical Definition	Boundary Constraint	Strategic Regulation Effect
			Incentivizes sustained approach toward target
			Suppression of Approach to Obstacles
			Enhancing trajectory control continuity
			Optimizing path efficiency and energy consumption ratio

To prevent policy oscillations during multi-objective trade-offs, a dynamic weight recalibration mechanism is employed during training. This mechanism adaptively adjusts the weighting of the reward function ( ) based on the trends of evaluation metrics during the training phase, enhancing policy convergence stability and generalization robustness [15]. This reward function structure will be directly input as the policy optimization objective function in subsequent training, forming a complete closed-loop optimization path in conjunction with the continuous action space and temporal states.

• DRL Algorithm and Network Architecture

To achieve action policy learning and trajectory optimization control in highly dynamic environments, the policy training employs the Proximal Policy Optimization (PPO) algorithm framework. A dual-network architecture is constructed to separately model policy output and state value estimation (see Figure 4). The policy network takes the sequence state vector as input, extracts temporal features through multi-layer perceptron modules and LSTM structures, and outputs continuous action pairs[16].The PPO algorithm employs a clipped ratio loss function to enhance policy update stability. Its core objective function is defined as:

                                                                                                                                                                                                                                            (6)

Where represents the probability ratio between the old and new policies, denotes the estimated value function, serves as the policy update clipping threshold to suppress excessive policy drift, and indicates the current policy parameters. To enhance value function learning, an additional value network loss term is introduced:

                                                                                                                                                                                                                                                                 (7)

where is the output of the value function network with parameters , and is the target cumulative reward. The final training loss function combines policy loss, value loss, and an entropy regularization term [17]:

                                                                                                                                                                                                                                                            (8)

where represents the policy entropy, enhancing exploratory behavior, and denote the respective weighting factors for the loss terms. This optimization structure demonstrated strong convergence stability and generalization capability in high-dimensional action spaces during experiments, providing a foundational policy for real-time decision-making in dynamic obstacle avoidance mechanisms.

Figure 4 PPO Deep Reinforcement Learning Dual-Network Architecture Diagram

• Dynamic Obstacle Avoidance Mechanism Design

Obstacle behavior in dynamic environments exhibits strong nonlinearity, uncertainty, and temporal coupling characteristics. Relying solely on the static action output from the policy network cannot guarantee robot safety and timely responsiveness. To enhance the policy’s environmental adaptability, the obstacle avoidance mechanism introduces a behavior correction module and an obstacle prediction structure into the policy output layer.By expanding the state space to incorporate the obstacle’s relative velocity vector ( ) and position change rate ( ), and integrating Kalman filter trajectory prediction results, the system constructs a short-term obstacle position estimation function [18]:

                                                                                                                                                                                                                                                       (9)

where represents the predicted obstacle position at time , and denotes the estimated covariance matrix. The action pruning mechanism removes the action subspace corresponding to the collision probability threshold based on the policy confidence distribution , ensuring behavioral decision convergence within the safety domain [19]. Collision confidence is calculated using an obstacle position confidence ellipse:

                                                                                                                                                                                                                                                (10)

where denotes the robot’s current position, and represent the robot’s and obstacle’s radii, respectively, and is the minimum safety distance redundancy factor.

• Simulation Platform and System Implementation
  • Simulation Platform Configuration

To validate the effectiveness of the deep reinforcement learning obstacle avoidance strategy in a virtual environment, the simulation platform was built based on ROS Melodic and Gazebo 9, running on an Ubuntu 18.04 system environment.The Gazebo physics engine employs ODE with a 10 ms control cycle and fixed-step integration mode to ensure simulation stability [20]. For plugin configuration, the system integrates the Hokuyo URG-04LX-UG01 LiDAR plugin with a resolution of 0.35° and a scanning frequency of 10 Hz. The visual sensor utilizes a simulated RGB camera with a resolution of 640×480 andframe rate of 30 FPS, and equipped with the YOLO-Lite inference plugin for object recognition and occlusion detection. Obstacle behavior modeling employs Python control scripts, utilizing a multithreaded asynchronous control interface to implement pseudo-random path switching, velocity perturbation simulation, and position repositioning for obstacles. Combined with a Kalman prediction module for path memory, this forms a dynamic behavior replay system [21].Key physical parameter configurations in the simulation environment are listed in Table 4. The entire simulation system synchronizes timing with the ROS core scheduler, and all data publishing/subscribing nodes are managed via the ROS topic mechanism. This maximizes simulation fidelity to the real robot’s control-perception-execution closed-loop structure, ensuring policy training transferability for real-world engineering deployment.

Table 4

Key Parameter Configuration for Gazebo Simulation Environment

Parameter Name	Value	Description
Gravity acceleration	9.81 m/s²	Simulates Earth’s standard gravitational acceleration
Robot chassis mass	14.3 kg	Differential robot chassis mass
Joint friction coefficient	0.025	Control power loss caused by friction between joints
Wheel damping torque	0.03 N·m·s/rad	Suppresses torque oscillations caused by high-speed rotation

• Robot Platform Configuration

The simulated robot model is constructed based on a differential drive structure, employing dual-wheel independent drive with a tail wheel support configuration. The motion model represents a planar motion-constrained structure of an incomplete system, suitable for 2D navigation and obstacle avoidance tasks.As shown in Figure 5, the robot chassis utilizes the simulated version of the TurtleBot3 Burger platform, measuring 138 × 178 mm with a total weight of 1.6 kg. It supports a maximum angular velocity of 4.0 rad/s and a linear velocity upper limit of 0.26 m/s [22].The sensor system integrates LiDAR, RGB cameras, and an IMU module, centrally mounted on the chassis top to minimize structural bias effects. Sensor error models are defined through parameter files and simulation plugin interfaces, with specific parameters detailed in Table 5.

Table 5

Key Sensor Parameters and Error Model Settings for Differential Drive Robot

Sensor Type	Resolution/Accuracy	Field of View/Sampling Frequency	Error Model Type	Start-up Disturbance Range
LiDAR	0.35°/±10 mm	270°/10 Hz	Gaussian + White Noise	±5 mm
RGB Camera	640×480/8-bit	60°/30FPS	Fixed Frame Offset + Amplitude Jitter	±2 pixels
IMU Module	±0.2° (Angle)	100Hz	Zero drift offset + low-frequency jitter	±0.5° (Cumulative Drift)

Control interfaces are implemented via ROS control nodes. Differential control commands are published using the Twist message type. Gazebo synchronizes command execution with wheel speed calculations through a plugin bridge. The control frequency is set to 10 Hz, matching the physics engine’s calculation cycle to ensure temporal consistency in the perception-action loop [23].

Figure 5 Differential Robot Structure and Control Mapping Diagram

• Scenario and Experimental Setup

To comprehensively evaluate the robustness and adaptability of the action strategy under varying obstacle dynamics, four typical simulation environments were designed, modeling static interference, single-object dynamic perturbations, multi-object nonlinear trajectory interference, and high-complexity dynamic adversarial conditions [24].

Scenario 1 constructs a dense static obstacle zone with at least 20 obstacles ( ) randomly distributed within a 5×5 m space at an average spacing <0.8 m to test path planning’s spatial compactness adaptation. Scenario 2 introduces a single dynamic obstacle traversing the primary navigation path at a constant speed of 1.0 m/s with randomly injected disturbance events to examine the strategy’s dynamic response stability.Scenario 3 constructs disturbance groups comprising 3 to 5 dynamic obstacles. Trajectory variations are driven by a multi-source normal distribution generation function, with irregular switching of movement speed and direction, testing the stability of obstacle avoidance strategies against multi-source random disturbances.Scenario 4 combines multiple complex factors: obstacles exceed 8 in number, movement speeds surpass 2.0 m/s, and asymmetrical shapes (L-shaped, irregular ring-shaped, etc.) with random turning behaviors are introduced. This simulates the bottlenecks in strategy generalization when encountering sudden disturbances in real-world high-density crowds or automated transport environments.All scenarios employ uniform target zones, starting points, and collision detection mechanisms. Obstacle behaviors incorporate repeatable trajectory scripts to ensure performance consistency.

• Experiments and Results Analysis
  • Baseline Comparison Experiments

To evaluate the performance advantages of multi-objective deep reinforcement learning obstacle avoidance strategies in highly dynamic obstacle environments, comparative experiments were conducted against traditional and mainstream methods. Four comparison groups were established: Dynamic Window Approach (DWA), A* with dynamic replanning, standard PPO strategy, and PPO strategy optimized with trajectory prediction. Experiments were conducted in a unified control environment with consistent obstacle behavior and task settings. Each strategy was run 50 times to eliminate random fluctuations.Key performance metrics are presented in Table 6.

Table 6 data indicates that the PPO + Prediction strategy outperforms other methods across multiple key metrics. Its collision rate of 2.3% is significantly lower than DWA (7.3%) and A* re-planning (5.9%), demonstrating the prediction mechanism’s effectiveness in avoiding potential conflicts within highly dynamic scenarios. The goal reach rate improved to 96.7%, reflecting the strategy’s path stability in interference fields.Path length is controlled at 7.82m, shortening by 1.64m compared to DWA, saving travel distance while reducing energy consumption by 13.4%. Decision latency remains within 112ms, comparable to classical methods but with superior performance. Overall, the tabular results demonstrate that the prediction-enhanced strategy achieves a balanced advantage in safety, efficiency, and real-time capability within complex dynamic environments, making it suitable for deployment on resource-constrained mobile platforms.

Table 6

Comparison of Baseline Methods in Dynamic Obstacle Scenarios

Method Type	Collision Rate (%)	Arrival Rate (%)	Average Path Length (m)	Decision Latency (ms)	Average Energy Consumption (J)
DWA	7.3	83.2	9.46	107	18.2
A* + Replanning	5.9	87.5	8.92	244	16.9
PPO (No Prediction)	4.5	91.6	8.24	119	16.3
PPO+Prediction	2.3	96.7	7.82	112	14.1

As shown in Figure 6, the trajectory performance differences among the four strategies in Scenarios 3 and 4 reveal that the PPO and A* schemes without prediction mechanisms exhibit significant path deviation and local oscillations in multi-objective disturbance fields. The PPO + Prediction strategy generates overall smoother trajectories with minimal deviation, navigating close to obstacle edges, significantly reducing total travel distance and avoiding redundant looping behavior.In contrast, the DWA trajectory exhibits severe oscillations and frequent backtracking, resulting in high path complexity. This indicates its limited ability to recognize global structural patterns within the local dynamic window. While the PPO (without prediction) achieves relatively shorter paths, it experiences short pauses and repeated planning during multi-object interference. The A* strategy suffers from excessively long paths and instances of entering enclosed spaces, reflecting its delayed response in global path replanning.The figure clearly demonstrates that introducing trajectory prediction yields the optimal strategy in terms of path quality and collision avoidance, enhancing its feedforward control capabilities for local dynamic changes.

Figure 6 Trajectory Comparison in High-Density Dynamic Scenarios

• Ablation Studies

To validate the contribution of core modules to overall strategy performance, three ablation experiments were designed. Components were isolated across three dimensions—reward function constraints, trajectory prediction structure, and action space encoding—to assess system degradation characteristics.As shown in Table 7, in the reward function structure test, removing the trajectory smoothness term or weakening the safety weight (from 0.5 to 0.1) both increased the collision rate. Notably, without the smoothness penalty term, the robot exhibited significant backtracking behavior in dense obstacle scenarios, with path length increasing from 7.82 m to 9.04 m.Second, in trajectory prediction module tests, removing the prediction mechanism exacerbated system response latency. The average collision rate rebounded to 4.6%, while target arrival rate dropped by over 4 percentage points, indicating that local planning cannot compensate for insufficient perception of future disturbance trends. Finally, in action space representation comparisons, continuous action strategies outperformed discrete output structures in path smoothness and control stability. The former enables finer speed adjustments to conform to obstacle boundaries.Comprehensive comparisons reveal that trajectory prediction structures and continuous control outputs make structural contributions in high-complexity scenarios and cannot be directly separated. These experimental results demonstrate that strategy performance highly depends on the integrity of reward design structures and the quality of feedforward information. The system should prioritize preserving the structure and ensuring efficient coordination between trajectory prediction and continuous control components.

Table 7

Module-Wise Ablation Results Under Complex Scenarios

Ablation Configuration	Collision Rate (%)	Arrival Rate (%)	Average Path Length (m)	Average Latency (ms)
Full Structure (Complete Policy)	2.3	96.7	7.82	112
No smoothing penalty	4.4	92.5	9.04	113
Safety weight reduction	3.7	94.2	8.86	114
Trajectory-Free Prediction Module	4.6	91.8	8.94	117
Discrete Action Space (5-class discrete)	3.9	93.7	8.48	111
Continuous Action Space	2.3	96.7	7.82	112

• Robustness and Generalization Testing

To evaluate the strategy’s adaptability under non-ideal conditions, multiple test environments with significant disturbance factors were constructed, including unseen dynamic trajectories, obstacle field structures of varying shapes, and sensor measurement errors.During experiments, the model’s training parameters remained frozen and were directly deployed to new scenarios. Test scenarios included: ① Dynamic obstacle trajectories generated by a bivariate normal disturbance function, with no overlap in trajectory patterns with training data; ② Obstacle layouts transitioning from regular grid topologies to density distribution gradient maps; ③ Laser radar and IMU sensors injected with zero-mean Gaussian noise (σ=0.08) to simulate signal distortion.

As shown in Table 8, under trajectory perturbation, the PPO+ prediction strategy maintained a 91.3% arrival rate. The collision rate increased by 2.6%, and latency grew by 8 ms, indicating the strategy possesses some feedforward adjustment redundancy. In the map topology change test, path length increased by 11.9% due to locally altered structures causing longer routes, yet the task completion rate remained at 92.4%.Under Gaussian noise disturbance, control decisions were significantly affected by radar interference, resulting in a collision rate of 4.9%. However, complete failure did not occur, indicating that the perception fusion mechanism retained fault tolerance. Overall, this strategy demonstrated strong generalization capabilities beyond the training scenario and structural robustness under multi-source interference.

Table 8

Robustness and Generalization Evaluation Results Under Disturbance Conditions

Interference Condition Type	Arrival Rate(%)	Collision Rate(%)	Average Path Length(m)	Decision Latency(ms)
No dynamic trajectory observed	91.3	4.9	8.21	120
Topological structure mutation map	92.4	3.1	8.75	118
Sensor Measurement Noise (σ=0.08)	89.6	5.4	8.52	122

• Conclusion

The comprehensive experimental results demonstrate that the dynamic obstacle avoidance strategy built upon the Proximal Policy Optimization (PPO) algorithm exhibits stable performance in high-density, strongly perturbed environments. The sustained superiority in key performance metrics stems from the synergistic effects of three key factors:First, the multi-objective design of the reward function effectively imposes joint constraints on collision avoidance, path efficiency, and control smoothness, ensuring a dynamic equilibrium between goal-oriented behavior and risk mitigation in complex scenarios. Second, the introduction of a feedforward module based on trajectory trend prediction significantly enhances the strategy’s perception of dynamic obstacle behavior changes. This improves identification of future risk zones and response redundancy, thereby reducing collision risks caused by decision delays.Third, adopting a continuous action space modeling approach establishes an optimal trade-off between control resolution and motion smoothness, effectively improving path quality and execution fluidity. Ablation experiments and robustness testing further validate the system’s performance dependency on key components and its adaptability range under non-training distributions, demonstrating the strategy’s cross-scenario transferability and a degree of perceptual fault tolerance.However, under scenarios with severe sensor errors, highly non-stationary obstacle behavior, or severely constrained control frequencies, the system exhibits predictable performance degradation. Future work may integrate attention mechanisms with multi-source heterogeneous input fusion strategies to enhance decision robustness and perceptual resilience in extreme conditions. Overall, this strategy structure demonstrates high deployability and real-world adaptability, providing an engineering-viable algorithmic pathway for autonomous decision-making in mobile robots navigating unknown dynamic environments.

References

• Xiao H, Chen C, Zhang G, et al. Reinforcement learning-driven dynamic obstacle avoidance for mobile robot trajectory tracking[J]. Knowledge-Based Systems, 2024, 297: 111974.
• Yang Z, Xiang H. Risk-Constrained Multi-Objective Deep Reinforcement Learning for AGV Path Planning in Rail Transit[J]. Applied System Innovation, 2025, 8(5): 145.
• Hu H, Wang Y, Tong W, et al. Path planning for autonomous vehicles in unknown dynamic environment based on deep reinforcement learning[J]. Applied Sciences, 2023, 13(18): 10056.
• Lyu P. Robot path planning algorithm with improved DDPG algorithm[J]. International Journal on Interactive Design and Manufacturing (IJIDeM), 2025, 19(2): 1123-1133.
• Park S Y, Lee C, Jeong S, et al. Digital twin and deep reinforcement learning-driven robotic automation system for confined workspaces: A nozzle dam replacement case study in nuclear power plants[J]. International Journal of Precision Engineering and Manufacturing-Green Technology, 2024, 11(3): 939-962.
• Malathi V, Sreedharan P, PR R, et al. Decision-Making for Path Planning of Mobile Robots Under Uncertainty: A Review of Belief-Space Planning Simplifications[J]. Robotics, 2025, 14(9): 127.
• Ranga J, PRASATH A A, Kumar N, et al. UrbanDrivePathway: A Decision-Making Framework for Navigating Urban Autonomous Vehicles in Complex Traffic Systems[C]//2025 8th International Conference on Trends in Electronics and Informatics (ICOEI). IEEE, 2025: 1575-1582.
• Bingol M C. A Safe Navigation Algorithm for Differential-Drive Mobile Robots by Using Fuzzy Logic Reward Function-Based Deep Reinforcement Learning[J]. Electronics, 2025, 14(8): 1593.
• Zhuang X, Li D, Li H, et al. A dynamic control decision approach for fixed-wing aircraft games via hybrid action reinforcement learning[J]. Science China Information Sciences, 2025, 68(3): 132201.
• Xiong L, Li Z, Zhong D, et al. Rule-Guidance Reinforcement Learning for Lane Change Decision-making: A Risk Assessment Approach[J]. Chinese Journal of Mechanical Engineering, 2025, 38(1): 30.
• Ywet N L, Maw A A, Lee J W. Deep Learning-driven DAA System for UAVs With Frenet Trajectory Optimization[J]. International Journal of Control, Automation and Systems, 2025, 23(9): 2780-2796.
• Sucuoglu H S. Development of Topologically Optimized Mobile Robotic System with Machine Learning-Based Energy-Efficient Path Planning Structure[J]. Machines, 2025, 13(8): 638.
• Vasquez-Jalpa C, Nakano M, Velasco-Villa M, et al. NRNH-AR: A Small Robotic Agent Using Tri-Fold Learning for Navigation and Obstacle Avoidance[J]. Applied Sciences, 2025, 15(15): 8149.
• Ait Dahmad H, Ayad H, García Cerezo A, et al. Adaptive Model Predictive Control for 4WD-4WS Mobile Robot: A Multivariate Gaussian Mixture Model-Ant Colony Optimization for Robust Trajectory Tracking and Obstacle Avoidance[J]. Sensors, 2025, 25(12): 3805.
• Huang Z, Tian Y. Three Dimensional Path Planning System for Unmanned Aerial Vehicles Based on Reinforcement Learning Algorithm[J]. Academic Journal of Engineering and Technology Science, 2024, 7(6): 155-161.
• Musham N K, Hemnath R. Real-time path planning for IoT-enabled autonomous vehicle robotics using RRT and A* algorithms[J]. International Journal of Multidisciplinary Research and Explorer, 2025, 5(1): 66-84.
• Kumar D. Reinforcement imitation learning for reliable and efficient autonomous navigation in complex environments[J]. Neural Computing and Applications, 2024, 36(20): 11945-11961.
• Brad S, Balog B. Integrated Rover Path Planning and Validation on Real Outdoor Terrain Scenarios Using Satellite Information to Conduct a Real Achievable Trajectory[J]. Electronics, 2025, 14(5): 921.
• Alcayaga J M, Menéndez O A, Torres-Torriti M A, et al. LSTM-Enhanced Deep Reinforcement Learning for Robust Trajectory Tracking Control of Skid-Steer Mobile Robots Under Terra-Mechanical Constraints[J]. Robotics, 2025, 14(6): 74.
• Behzadfar M, Karimpourfard A, Feng Y. A Bio-Inspired Data-Driven Locomotion Optimization Framework for Adaptive Soft Inchworm Robots[J]. Biomimetics, 2025, 10(5): 325.
• Huang Q, Luo Y, Liu Z, et al. A Hierarchical Reinforcement Learning Framework for Multi-Agent Cooperative Maneuver Interception in Dynamic Environments[J]. Journal of Marine Science and Engineering, 2025, 13(7): 1271.
• Benmachiche A, Derdour M, Kahil M S, et al. Adaptive hybrid PSO–APF algorithm for advanced path planning in next-generation autonomous robots[J]. Sensors, 2025, 25(18): 5742.
• Mushtaq M U, Venter H, Nxasana P, et al. Hybrid Evolutionary Reinforcement Learning for UAV Path Planning: Genetic Programming and Soft Actor Critic Integrations[J]. Journal of Intelligent Communication, 2025, 4(2): 40-58.
• Wang S, Du Y, Lin J, et al. Research on mobile robot path planning in complex environment based on DRQN algorithm[J]. Physica Scripta, 2024, 99(7): 076012.