Digital documentation, computer vision and machine learning for masonry surveying and maintenance

Masonry structures constitute a significant proportion of the international traditional, and historic built environment. In Scotland alone, there are 500,000 pre-1919 traditional buildings mainly masonry-built, and more than two-thirds of the 300 properties currently in the care of Historic Environment Scotland
(HES) are masonry-built.

Traditional masonry structures are clearly a highly valued part of the wider built environment and their maintenance is societally fundamental. Climate change projections for the UK suggest that the built environment in general, and the masonry-built historic environment in particular, is being placed under increasingly significant strain, which raises fundamental challenges to the monitoring and maintenance of those structures. Indeed, the impact of climate change upon their performance and upkeep, requires not just continuous, but more frequent inspection and maintenance. These activities logically have corresponding cost increases both financially and in terms of expended CO₂ during interventions.

Inspection systems and more specifically provision of access constitute an important cost of reactive and proactive (planned) maintenance intervention. Worryingly, condition assessment has been widely reported to yield variable results due to subjective perceptions of inspectors, with the consequence
of defects remaining undetected or inaccurately characterised. Such fundamental errors can result in unnecessary, and often costly maintenance being conducted. Clearly, processes that yield a more accurate, complete and objective inspection of building fabric would contribute significantly to the enhancement of maintenance and repair activities. In addition to the aforementioned, the cost of
attaining and structuring survey data is often prohibitive and if not undertaken correctly can create significant problems with information processing, and exchanging data. Importantly, survey data acquisition is not risk-free and if not undertaken correctly can raise safety issues, particularly when working at heights.

The last decade has seen a rapid growth in the deployment of new technologies in the construction sector, from the use of remote sensing technologies for data capture to new powerful software and data-driven processes to obtain meaningful information. These novel operations are often captured under the umbrella terms of Building Information Modelling (BIM), and nowadays increasingly digital twinning (DT). In the context of building inspection and documentation, three-dimensional (3D) surveying technologies, like laser scanning (LS) and photogrammetry (PG), have seen great uptake under the leadership of organisations like HES. These solutions have the potential to transform inspection practice with time, cost and also safety benefits. However, while data acquisition and modelling have developed apace, the challenge remains on how to extract useful information from the data to support effective condition assessment and maintenance decision making.

This report summarises collaborative work conducted between HES, the University of Edinburgh and Heriot-Watt University since 2015. Whilst the team first assessed and contrasted the suitability of different 3D surveying techniques, its main effort has then focused on the processing of such
data. This report particularly presents data processing solutions for: (1) segmenting 3D survey data (i.e. point clouds) of masonry walls into the individual stone units for both ashlar and rubble masonry; and (2) detecting and classifying visible defects on the surface of those units.