Collision probability reduction method for tracking control in automatic docking/berthing using reinforcement learning

1. Shouji K, Ohtsu K, Mizoguchi S (1992) An automatic berthing

study by optimal control techniques. IFAC Proc Vol 25:185

13

860

2. Shouji K, Ohtsu K (1992) A study on the optimization of ship

maneuvering by optimal control theory (1st report). J Soc Nav

Archit Jpn 1992:365

3. Ahmed YA, Hasegawa K (2015) Consistently trained artificial

neural network for automatic ship berthing control. TransNav Int

J Mar Navig Saf Sea Transp 9(3):417

4. Bitar G, Martinsen AB, Lekkas AM, Breivik M (2020) Trajectory

planning and control for automatic docking of ASVs with fullscale experiments this work is supported by the research council

of Norway through the project number 269116 as well as through

the centres of excellence funding scheme with project number

223254. IFAC-PapersOnLine 53(2), 14488, 21st IFAC World

Congress

5. Martinsen AB, Bitar G, Lekkas AM, Gros S (2020) Optimizationbased automatic docking and berthing of ASVs using exteroceptive sensors: theory and experiments. IEEE Access 8:204974

6. Sawada R, Hirata K, Kitagawa Y, Saito E, Ueno M, Tanizawa K,

Fukuto J (2021) Path following algorithm application to automatic

berthing control. J Mar Sci Technol (Jpn) 26(2):541

7. Bitar G, Eriksen BOH, Lekkas AM, Breivik M (2021) Threephase automatic crossing for a passenger ferry with field trials.

In: 2021 European control conference (ECC), pp 2271–2277

8. Shuai Y, Li G, Cheng X, Skulstad R, Xu J, Liu H, Zhang H (2019)

An efficient neural-network based approach to automatic ship

docking. Ocean Eng 191:106514

9. Li S, Liu J, Negenborn RR, Wu Q (2020) Automatic docking

for underactuated ships based on multi-objective nonlinear model

predictive control. IEEE Access 8:70044

10. Miyauchi Y, Sawada R, Akimoto Y, Umeda N, Maki A (2022)

Optimization on planning of trajectory and control of autonomous

berthing and unberthing for the realistic port geometry. Ocean Eng

245:110390

11. Rachman DM, Maki A, Miyauchi Y, Umeda N (2022) Warmstarted semionline trajectory planner for ship’s automatic docking

(berthing). Ocean Eng 252:111127

12. Bartels S, Helling S, Meurer T (2022) Inequality constrained

optimal control for rope-assisted asv docking maneuvers. IFACPapersOnLine 55(31), 44 (2022). 14th IFAC conference on control applications in marine systems, robotics, and vehicles CAMS

13. Ødven PK, Martinsen AB, Lekkas AM (2022) Static and dynamic

multi-obstacle avoidance and docking of ASVs using computational geometry and numerical optimal control. IFAC-PapersOnLine 55(31), 50 (2022). 14th IFAC conference on control applications in marine systems, robotics, and vehicles CAMS

14. Fossen TI (2000) A survey on nonlinear ship control: from theory

to practice. IFAC Proc Vol 33(21):1

15. Sørensen AJ (2011) A survey of dynamic positioning control systems. Annu Rev Control 35(1):123

16. Ding F, Wang Y, Wang Y (2011) Trajectory-tracking controller

design of underactuated surface vessels. In: OCEANS’11 MTS/

IEEE KONA, pp 1–5

17. Zheng H, Negenborn RR, Lodewijks G (2014) Trajectory tracking

of autonomous vessels using model predictive control. IFAC Proc

Vol 47(3):8812

18. Yang Y, Du J, Liu H, Guo C, Abraham A (2014) A trajectory

tracking robust controller of surface vessels with disturbance

uncertainties. IEEE Trans Control Syst Technol 22(4):1511

19. Wen G, Ge SS, Chen CLP, Tu F, Wang S (2019) Adaptive tracking

control of surface vessel using optimized backstepping technique.

IEEE Trans Cybern 49(9):3420

20. Jiang X, Huang L, Peng M, Li Z, Yang K (2022) Nonlinear model

predictive control using symbolic computation on autonomous

marine surface vehicle. J Mar Sci Technol 27(1):482

21. Rachman DM, Miyauchi Y, Umeda N, Maki A (2021) Feasibility

study on the use of evolution strategy: CMA-ES for ship automatic

13

Journal of Marine Science and Technology (2023) 28:844–861

22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. docking problem. In: Proceedings of 1st international conference

on the stability and safety of ships and ocean vehicles (2021),

STAB &S

Cheng Y, Zhang W (2018) Concise deep reinforcement learning

obstacle avoidance for underactuated unmanned marine vessels.

Neurocomputing 272:63

Martinsen AB, Lekkas AM (2018) Straight-path following for

underactuated marine vessels using deep reinforcement learning.

IFAC-PapersOnLine 51(29):329

Martinsen AB, Lekkas AM (2018) Curved path following with

deep reinforcement learning: results from three vessel models. In:

OCEANS 2018 MTS/IEEE Charleston, pp 1–8

Martinsen AB, Lekkas AM, Gros S, Glomsrud JA, Pedersen TA

(2020) Reinforcement learning-based tracking control of USVs in

varying operational conditions. Front Robot AI 7:32

Meyer E, Heiberg A, Rasheed A, San O (2020) Colreg-compliant

collision avoidance for unmanned surface vehicle using deep reinforcement learning. IEEE Access 8:165344

Meyer E, Robinson H, Rasheed A, San O (2020) Taming an autonomous surface vehicle for path following and collision avoidance

using deep reinforcement learning. IEEE Access 8:41466

Martinsen AB, Lekkas AM, Gros S (2022) Reinforcement learning-based NMPC for tracking control of ASVs: theory and experiments. Control Eng Pract 120:105024

Sutton RS, Barto AG (2018) Reinforcement learning: an introduction, 2nd edn. MIT Press, Cambridge

Japan Hamworthy & Co., Ltd. (2022) Steering systems: new

VecTwin system. https://​www.​japan​ham.​com/​en/​servi​ce/​new_​

vectw​in.​html. Accessed 12 July 2023

Hasegawa K, Kang D, Sano M, Nabeshima K (2006) Study on

the maneuverability of a large vessel installed with a mariner type

Super VecTwin rudder. J Mar Sci Technol 11:88

Rachman DM, Aoki Y, Miyauchi Y, Umeda N, Maki A (2023)

Experimental low-speed positioning system with VecTwin rudder

for automatic docking (berthing). J Mar Sci Technol 28:689–703

Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M

(2014) Deterministic policy gradient algorithms. In: Proceedings

of the 31st international conference on machine learning, Proceedings of machine learning research, vol 32, ed. by Xing EP, Jebara

T (eds) (PMLR, Bejing, China), Proceedings of machine learning

research, vol 32, pp 387–395

Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver

D, Wierstra D (2016) Continuous control with deep reinforcement

learning. In: Bengio Y, LeCun Y (eds) 4th international conference on learning representations, ICLR 2016, San Juan, Puerto

Rico, May 2–4, 2016, conference track proceedings

Fujimoto S, van Hoof H, Meger D (2018) Addressing function

approximation error in actor-critic methods. In: Proceedings of the

35th international conference on machine learning, Proceedings of

machine learning research, vol 80, Dy J, Krause A (eds) (PMLR),

Proceedings of machine learning research, vol 80, pp 1587–1596

Ogawa A, Kasai H (1978) On the mathematical model of manoeuvring motion of ships. Int Shipbuild Prog 25(292):306

Yoshimura Y, Nakao I, Ishibashi A (2009) Unified mathematical model for ocean and harbour manoeuvring. In: International

conference on marine simulation and ship maneuverability, pp

116–124

Kang D, Nagarajan V, Hasegawa K, Sano M (2008) Mathematical

model of single-propeller twin-rudder ship. J Mar Sci Technol

13(3):207

Kobayashi E (1988) A simulation study on ship manoeuvrability

at low speeds. Akishima Laboratory, Ocean Engineering Research

Journal of Marine Science and Technology (2023) 28:844–861 40. 41. 42. 43. 44. Section, Mitsubishi Heave Industries Ltd. Published in: Mitsubishi

Technical Bulletin No. 180

Fujiwara T, Ueno M, Nimura T (1998) Estimation of wind forces

and moments acting on ships. J Soc Nav Archit Jpn 1998(183):77

Maki A, Maruyama Y, Dostal L, Sakai M, Sawada R, Sasa K,

Umeda N (2022) Practical method for evaluating wind influence

on autonomous ship operations. J Mar Sci Technol 27:1302

Hino M (1971) Spectrum of gusty wind. In: Proceedings of the 3rd

international conference on wind effects on buildings and structures, Tokyo, Japan, vol 77

Maki A, Sakamoto N, Akimoto Y, Nishikawa H, Umeda N (2020)

Application of optimal control theory based on the evolution

strategy (CMA-ES) to automatic berthing. J Mar Sci Technol

25(1):221

Maki A, Akimoto Y, Naoya U (2021) Application of optimal control theory based on the evolution strategy (CMA-ES) to automatic berthing (part: 2). J Mar Sci Technol 26(3):835

861

45. Sakamoto N, Akimoto Y (2017) Modified box constraint handling

for the covariance matrix adaptation evolution strategy. In: Proceedings of the genetic and evolutionary computation conference

companion. Association for Computing Machinery, New York,

NY, USA, GECCO ’17, pp 183–184

46. Kose K, Fukudo J, Sugano K, Akagi S, Harada M (1986) On a

computer aided maneuvering system in harbours. J Soc Nav Archit

Jpn 1986(160):103

47. Miyauchi Y, Maki A, Umeda N, Rachman DM, Akimoto Y (2022)

System parameter exploration of ship maneuvering model for

automatic docking/berthing using CMA-ES. J Mar Sci Technol

27:1065

13

...