时序逆强化学习：基于逆强化学习的港口拥堵与泊位调度建模
原文标题：Temporal-IRL: Modeling Port Congestion and Berth Scheduling with Inverse Reinforcement Learning
作者：Guo Li, Zixiang Xu, Wei Zhang, Yikuan Hu, Xinyu Yang, Nikolay Aristov, Mingjie Tang, Elenna R Dugundji
原文摘要：Predicting port congestion is crucial for maintaining reliable global supply chains. Accurate forecasts enable improved shipment planning, reduced delays and costs, and optimize inventory and distribution strategies, thereby ensuring timely deliveries and enhancing supply chain resilience. To achieve accurate predictions, analyzing vessel behavior and their stay times at specific port terminals is essential, focusing particularly on berth scheduling under various conditions. Crucially, the model must capture and learn the underlying priorities and patterns of berth scheduling. Berth scheduling and planning are influenced by a range of factors, including incoming vessel size, waiting times, and the status of vessels within the port terminal. By observing historical Automatic Identification System (AIS) positions of vessels, we reconstruct berth schedules, which are subsequently utilized to determine the reward function via Inverse Reinforcement Learning (IRL). For this purpose, we modeled a specific terminal at the Port of New York/New Jersey and developed Temporal-IRL. This Temporal-IRL model learns berth scheduling to predict vessel sequencing at the terminal and estimate vessel port stay, encompassing both waiting and berthing times, to forecast port congestion. Utilizing data from Maher Terminal spanning January 2015 to September 2023, we trained and tested the model, achieving demonstrably excellent results.
中文翻译：预测港口拥堵对维护可靠的全球供应链至关重要。准确的预测能够改善货运规划、减少延误和成本，并优化库存和配送策略，从而确保及时交付并增强供应链韧性。为实现精确预测，分析船舶行为及其在特定港口终端的停留时间是必要的，特别关注各种条件下的泊位调度。关键的是，模型必须捕获并学习泊位调度的潜在优先级和模式。泊位调度和规划受多种因素影响，包括进港船舶大小、等待时间以及港口终端内船舶的状态。通过观察历史船舶自动识别系统（AIS）位置数据，我们重建了泊位调度，并随后利用逆强化学习（IRL）确定奖励函数。为此目的，我们对纽约/新泽西港的特定终端进行建模，并开发了时序逆强化学习（Temporal-IRL）。该Temporal-IRL模型学习泊位调度以预测终端的船舶排序，并估算船舶港口停留时间（包括等待时间和靠泊时间），从而预测港口拥堵。利用2015年1月至2023年9月期间来自马希尔终端的数据，我们训练和测试了该模型，取得了显著优异的结果。
智能问答：
Q: 这篇论文试图解决什么问题？
A: 这篇论文试图解决港口拥堵预测和泊位调度优化的问题。具体来说，论文关注以下几个核心问题：1）港口拥堵预测的准确性问题：准确预测港口拥堵对维护可靠的全球供应链至关重要，可以改善货运规划、减少延误和成本、优化库存和配送策略；2）泊位调度的智能化问题：需要分析船舶行为和停留时间，特别是在不同条件下的泊位调度模式，模型必须能够捕获和学习泊位调度的潜在优先级和模式；3）复杂因素的综合建模问题：泊位调度受到多种因素影响，包括进港船舶大小、等待时间、港口终端内船舶状态等，需要一个能够综合考虑这些因素的智能模型；4）历史数据的有效利用问题：如何从历史AIS（船舶自动识别系统）位置数据中提取有价值的信息，重建泊位调度并用于预测模型的训练。
阅读全文 →
