Stella X. Yu : Papers / Google Scholar

AdaLN: A Vision Transformer for Multidomain Learning and Predisaster Building Information Extraction from Images
Yunhui Guo and Chaofeng Wang and Stella X. Yu and Frank McKenna and Kincho H. Law
Journal of Computing in Civil Engineering, 36(5):04022024, 2022

Satellite and street view images are widely used in various disciplines as a source of information for understanding the built environment. In natural hazard engineering, high-quality building inventory data sets are crucial for the simulation of hazard impacts and for supporting decision-making. Screening the building stocks to gather the information for simulation and to detect potential structural defects that are vulnerable to natural hazards is a time-consuming and labor-intensive task. This paper presents an automated method for extracting building information through the use of satellite and street view images. The method is built upon a novel transformer-based deep neural network we developed. Specifically, a multidomain learning approach is employed to develop a single compact model for multiple image-based deep learning information extraction tasks using multiple data sources (e.g., satellite and street view images). Our multidomain Vision Transformer is designed as a unified architecture that can be effectively deployed for multiple classification tasks. The effectiveness of the proposed approach is demonstrated in a case study in which we use pretrained models to collect regional-scale building information that is related to natural hazard risks.

natural hazard engineering, building information extraction, image segmentation, multi-domain vision transformer