Software:General Optimal control Problem Solver

From HandWiki
Short description: Open-source package for reinforcement learning


General Optimal control Problems Solver (GOPS) is an open-source reinforcement learning (RL) package that aims to address optimal control problems in industrial fields.[1] GOPS is developed by iDLab (Intelligent Driving Laboratory)[2] at Tsinghua University. It is built with a modular structure, enabling the creation of controllers for diverse industrial tasks.

Overview

Addressing optimal control problems is essential for meeting the basic requirements of industrial control tasks. Traditional approaches such as model predictive control often encounter significant computational burdens during real-time execution. GOPS is developed for building real-time controllers in industrial applications using RL techniques. GOPS has a modular architecture, which provides flexibility for further development, catering to the diverse needs of industrial control tasks. GOPS includes a conversion tool that enables integration with Matlab/Simulink, facilitating environment construction, controller design, and performance validation. GOPS also incorporate both serial and parallel trainers with embedded buffers and samplers to tackle large-scale control problems. Moreover, GOPS offers a range of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, and convolutional neural network models.

Features

GOPS presents a set of features specifically designed for industrial control applications:

  • Modular Configuration: GOPS is built with a modular structure, allowing for customization and development of environments and algorithms.
  • Diverse Training Modes: GOPS supports different training modes, including serial and parallel setups, on-policy and off-policy approaches, as well as model-free and model-based algorithms.
  • Compatibility with Matlab/Simulink: GOPS provides a conversion tool for Matlab/Simulink, which converts Simulink models into GOPS-compatible environments and sends learned policies back to Simulink for further integration and evaluation.

Applications

Applications of GOPS in industrial control scenarios include: coupled velocity and energy management optimization,[3] travel pattern analysis and demand prediction,[4] design of reward functions in vehicle control,[5] improving freeway merging efficiency,[6] vehicle speed control strategies,[7] multi-agent RL for platoon following,[8] origin-destination ride-hailing demand prediction,[9] accelerating model predictive path integral,[10] drill boom hole-seeking control,[11] etc.

Documentation and Usage

The GOPS package is available on GitHub at Intelligent-Driving-Laboratory/GOPS,[12] where users can access the source code and contribute to its development. Further details, including installation instructions, usage guidelines, and examples, are provided in the GOPS documentation.[13]

References

  1. ↑ Wang, Wenxuan; Zhang, Yuhang; Gao, Jiaxin; Jiang, Yuxuan; Yang, Yujie; Zheng, Zhilong; Zou, Wenjun; Li, Jie et al. (2023). "GOPS: A general optimal control problem solver for autonomous driving and industrial control applications". Communications in Transportation Research 3: 100096. doi:10.1016/j.commtr.2023.100096. ISSN 2772-4247. https://doi.org/10.1016/j.commtr.2023.100096. 
  2. ↑ "iDLab, Tsinghua (清华大学智能驾驶实验室)". http://www.idlab-tsinghua.com/thulab/labweb/index.html. 
  3. ↑ Zhang, Hao; Chen, Boli; Lei, Nuo; Li, Bingbing; Chen, Chaoyi; Wang, Zhi (2024). "Coupled velocity and energy management optimization of connected hybrid electric vehicles for maximum collective efficiency". Applied Energy 360: 122792. doi:10.1016/j.apenergy.2024.122792. ISSN 0306-2619. https://doi.org/10.1016/j.apenergy.2024.122792. 
  4. ↑ Lin, Hongyi; He, Yixu; Li, Shen; Liu, Yang (2024). "Insights into Travel Pattern Analysis and Demand Prediction: A Data-Driven Approach in Bike-Sharing Systems" (in en). Journal of Transportation Engineering, Part A: Systems 150 (2). doi:10.1061/JTEPBS.TEENG-8137. ISSN 2473-2907. https://ascelibrary.org/doi/10.1061/JTEPBS.TEENG-8137. 
  5. ↑ He, Yixu; Liu, Yang; Yang, Lan; Qu, Xiaobo (2024-01-17). "Exploring the design of reward functions in deep reinforcement learning-based vehicle velocity control algorithms" (in en). Transportation Letters: 1–15. doi:10.1080/19427867.2024.2305018. ISSN 1942-7867. https://www.tandfonline.com/doi/full/10.1080/19427867.2024.2305018. 
  6. ↑ Zhu, Jie; Wang, Liang; Tasic, Ivana; Qu, Xiaobo (2024). "Improving Freeway Merging Efficiency via Flow-Level Coordination of Connected and Autonomous Vehicles". IEEE Transactions on Intelligent Transportation Systems: 1–13. doi:10.1109/TITS.2023.3346832. ISSN 1524-9050. https://ieeexplore.ieee.org/document/10400985. 
  7. ↑ Ma, Changxi; Li, Yuanping; Meng, Wei (2023). "A Review of Vehicle Speed Control Strategies". Journal of Intelligent and Connected Vehicles 6 (4): 190–201. doi:10.26599/JICV.2023.9210010. ISSN 2399-9802. https://ieeexplore.ieee.org/document/10409227. 
  8. ↑ Lin, Hongyi; Lyu, Cheng; He, Yixu; Liu, Yang; Gao, Kun; Qu, Xiaobo (2024). "Enhancing State Representation in Multi-Agent Reinforcement Learning for Platoon-Following Models". IEEE Transactions on Vehicular Technology: 1–5. doi:10.1109/TVT.2024.3373533. ISSN 0018-9545. https://ieeexplore.ieee.org/document/10463154. 
  9. ↑ Lin, Hongyi; He, Yixu; Liu, Yang; Gao, Kun; Qu, Xiaobo (2024). "Deep Demand Prediction: An Enhanced Conformer Model With Cold-Start Adaptation for Origin–Destination Ride-Hailing Demand Prediction". IEEE Intelligent Transportation Systems Magazine: 2–15. doi:10.1109/MITS.2023.3309653. ISSN 1939-1390. https://ieeexplore.ieee.org/document/10248948. 
  10. ↑ Qu, Yue; Chu, Hongqing; Gao, Shuhua; Guan, Jun; Yan, Haoqi; Xiao, Liming; Li, Shengbo Eben; Duan, Jingliang (2023). "RL-Driven MPPI: Accelerating Online Control Laws Calculation With Offline Policy". IEEE Transactions on Intelligent Vehicles: 1–12. doi:10.1109/TIV.2023.3348134. ISSN 2379-8904. https://ieeexplore.ieee.org/document/10376303. 
  11. ↑ Yan, Haoqi; Xu, Haoyuan; Gao, Hongbo; Ma, Fei; Li, Shengbo Eben; Duan, Jingliang (2023-10-13). "Integrated Drill Boom Hole-Seeking Control via Reinforcement Learning". 2023 IEEE International Conference on Unmanned Systems (ICUS). IEEE. pp. 1247–1254. doi:10.1109/ICUS58632.2023.10318393. ISBN 979-8-3503-1630-8. https://ieeexplore.ieee.org/document/10318393. 
  12. ↑ "Intelligent-Driving-Laboratory/GOPS". March 7, 2024. https://github.com/Intelligent-Driving-Laboratory/GOPS. 
  13. ↑ "Welcome to GOPS's documentation! — GOPS 1.1.0 documentation". https://gops.readthedocs.io/en/latest/. 


Retrieved from "https://handwiki.org/wiki/index.php?title=Software:General_Optimal_control_Problem_Solver&oldid=3727705"