The automated labeling of point clouds from highway scans

published: 24.06.2021

Various industries can benefit from using AI to classify laser scan point clouds into different object classes. One of them is the road and highway sector, e.g. in this Use Case the Bavarian State Construction Administration regularly carries out surveys of the Bavarian road networks. When surveying kilometres of highways, huge amounts of data are produced which need to be analyzed to extract relevant information. Once analyzed they can serve the process of planning, construction, and operation of road infrastructure. To reduce the manual processing efforts and automate the analysis of the scanned roads the Bavarian State Construction Administration contracted Pointly.

Up till now the classification of point clouds was time-consuming as the work was done mainly manually and huge amounts of data need to be labeled. Furthermore, traditional methods lack precision or granularity. Nowadays, modern techniques make it possible to accelerate and digitize this process to a new level of precision.

The goal of this infrastructure project was to fully automate point cloud labeling by using deep learning models.

For this task, high-resolution mobile mapping LiDAR scans of two German highways were provided by Cloud-Vermessung + Planung GmbH, which surveyed the road network for the Bavarian State Construction Administration. The used vehicle was a conventional MB V class. The measuring unit was the Trimble MX9 with built-in Riegl scanners. With this setup very high accuracies (+/- 1 cm) and very dense point, clouds were created. Also, the scanner captured buildings such as bridges around the highway and delivers point clouds including intensity values and other additional data that can be used for the deep learning task.

The classification of 3D point clouds from laser scans of highways is a manual and laborious activity. To make the process scalable and economically efficient it should be automated with the help of AI.

One of the biggest challenges was the clear definition of object classes to collect suitable training data for the AI. There are 3 requirement criteria for this definition:

1. Complete: Each data point must be assigned to a class
2. Explicit: Object classes must be clearly distinguishable
3. Representative:
   • Classes should be represented as equally as possible
   • No class should be seriously underrepresented

In this project, the most important classes were street, road marking, ground, vegetation, rails, signs and vehicles. However, it was a demanding task to exactly distinguish a road from other asphalt surfaces and determine the beginning and end of particular objects (e.g. components of bridge structures).

Firstly, the provided datasets were labeled with Pointly by in-house employees. Here it is beneficial that Pointly lets you create customized classes. Like this, all the classes needed for the customer could be classified. Several kilometres of sampled highway sections were completely classified for the model training.

Figure 1: a point cloud from the highway classified in Pointly

With the generated training data, our pretrained 3D neural network was trained and finetuned to detect the different classes in the LiDAR Scans. Eventually, the trained network is used to classify new point clouds automatically with high precision corresponding to the custom object classes. All provided point clouds were classified within a few hours as they were parallel processed in a cloud environment.

The trained neural network works with a strong and convincing precision and speeds up the classification process tremendously. The existing point clouds were automatically classified by the AI and delivered to the customer. Upcoming scans can be processed almost without any manual labour within hours. Also, the model can be enhanced anytime by feeding it additional training data.

In a follow-up project with the Bavarian State Construction Administration, the classified point clouds were used furthermore to automatically extract road outlines as 3D polylines and to create CAD models from these.

Soon the classification models will be made available for everyone on the Pointly platform so that they can be easily used on demand.

Figure 2: left side = manually classified by Pointly’s employees / right side = classified automatically by Pointly’s 3D neural network

Pointly – Point out what matters.