The arrival of a ship at a port is a complex process. It has been reported that ships in port can spend 5–10% of their time in unnecessary anchorage. Predicting the accurate time of ship arrivals could support port actors to optimally allocate their resources and reduce this time, to the benefit of port competitiveness. Currently, the rise of real-time automated ship identification data has given ports, shipping lines and other actors the possibility to predict but also to follow the consecutive steps a ship undertakes while entering a port. Much work has been done on the development of advanced estimated time of arrival (ETA) predictors, but little is known about the quality of ETA information that shipping lines provide to ports. We explore, quantitatively, the predictive power of the sequences of ETAs that ships transmit to the Port of Antwerp while arriving. We propose several forecast evaluation measures to assess the quality of a sequence of predictions and their ability to converge to the actual time of arrival. For the Port of Antwerp, we compute and interpret the measures on a dataset of port calls in the year 2019. Our aim is to compare ETA prediction quality across ships. Understanding variations in prediction quality gives port authorities a basis for discussion with individual shipowners, enabling them to improve their predictions and thus contribute to the competitive performance of the port. As an innovation, we cluster the different berths in our data set with the OPTICS algorithm to gain a better understanding of the effects of the river locks a ship must transit on her way to the allocated berth. Our findings indicate that, on average, shipping lines offer poor and optimistic arrival estimates, which they gradually adjust. The adjustments increase as the ship approaches the port and sometimes they overcompensate earlier predictions. In this, we find little evidence that shipping lines are adopting the advanced arrival prediction methods that have been developed in recent years. Our proposed prediction evaluation measures open the door for a uniform and quantitative approach to assessing the quality of prediction sequences of ships entering ports, to the benefit of a miscellany of port actors.