Point taken: Translating the physical rail domain to cyber space using point clouds from mobile laser scanning

There is an active interest to digitise the rail infrastructure, because it is expected that the coming years rail transport will be an important key factor to combat climate change. Compared to other means of transport, such as air or road transportation, rail transport is viewed as being more environment-friendly. With the envisioned increase of passenger and freight rail traffic in the near future, there will be an increased burden on the railway infrastructure. To ensure continued reliability, availability, maintainability and safety of the railway network in an efficient way, it is vital to have an accurate and up-to-date digital representation of the railway environment. Often such a digital representation is also referred to as an `as-is' model or representation because it reflects the current state. These as-is representations can be used for various tasks, such as planning work, monitoring the infrastructural health, automated inventory assessment, robotised construction, and eventually predictive maintenance.

Automated creation of these as-is models is challenging because railway environments pose a complex scene structure and span several thousands of kilometres. Scenes contain discrete objects, such as signals, catenary arches, and relay cabinets on the one hand. On the other hand there are continuous objects, such as overhead catenary wires and rail tracks. Adding to the complexity, is the fact that the number of distinct objects around the track is large. Especially for countries with a dense rail network and a long history of rail transportation such as the Netherlands. These factors, combined with the large intra-class variance, make the digitalisation of the rail environment a challenging task.

To accomplish the task of digitising the rail environment, point cloud data is considered a viable data format. Point clouds are captured using Light Detection and Ranging (LiDAR) technology. This is done through laser scanning, which operates by emitting a usually non-visible laser light pulse and measuring its time-of-flight. This time is directly proportional to the distance of the object. This distance, combined with the known angles of direction of the laser are used to calculate the 3D position of the reflection occurrence relative to the sensor. Point clouds are able to capture the geometric surface structure of a scene with high accuracy, under varying illumination conditions, and produces an immediate representation.

This thesis focuses on digitising the rail environment using point cloud data. It takes a top-down approach, first objects are detected within the point cloud data with a low level of detail. Thereafter this thesis focuses on dissecting these large objects into smaller semantic meaningful pieces, providing a higher level of detail. These two steps conclude the translation from the physical world to cyber space. To complete the cycle, going from cyber space to the physical world again, this thesis also focuses on how information from cyber space can be visualised in an intuitive, immersive way using a head mounted display.

The thesis starts by looking at how large objects can be detected at large scale from point cloud data. To do so, existing models which have been successful in the domain of autonomous driving vehicles are evaluated for this task. A key insight from this evaluation is that the absolute positional accuracy required for `as-is' models cannot be obtained from these existing models. Another insight is that models do not generalise well to new scenarios, and that to combat this issue, transfer learning is very suitable to train new models using fewer labels.

To further scrutinise these large objects and thereby increase the level of detail of the digitisation process, semantic segmentation can be used. Semantic segmentation assigns a class to each point within the point cloud. This decomposition can then for instance be used to retrieve CAD models from a CAD library to create an accurate digital representation. This thesis looks at the semantic segmentation of catenary arches which are captured at high resolution. Catenary arches are decomposed into fourteen distinct classes and labelled accordingly. An overall mIoU of 71% was accomplished for this task.

Object detection and semantic segmentation conclude the process of translating the physical world to cyberspace. Once a representation is available in cyberspace, a wide range of applications become available. For instance, planning maintenance, robotised construction, predictive maintenance, and computerised clearance inspections. To complete the cycle, and go from cyberspace to the physical world again, this thesis explores immersive visualisation of point cloud data. A proof-of-concept for this immersive visualisation is presented in this thesis. It is based on a head-mounted display and enables the user to interact with the point cloud using hand gestures and voice commands.

To conclude, this thesis has explored the digitisation of the rail environment using point cloud data. It provides valuable insights and pointers to move this relatively new research area to the next level. One promising research direction to highlight is the use of self-supervised learning (SSL). By using SSL, models can learn some `common sense' about the data without external labels. This alleviates the expensive, and tedious task of labelling large amounts of data.