This article presents an approach for the automatic semantic segmentation of floorplan images, predicting room boundaries (walls, doors, windows) and semantic labels of room types. A multi-task network was designed to represent and learn inherent dependencies by combining a Convolutional Neural Network to generate suitable features with a Graph Convolutional Network (GCN) to capture long-range dependencies. In particular, a Self-Constructing Graph module is applied to automatically induce an input graph for the GCN. Experiments on different datasets demonstrate the superiority and effectiveness of the multi-task network compared to state-of-the-art methods. The accurate results not only allow for subsequent vectorization of the existing floorplans but also for automatic inference of layout graphs including connectivity and adjacency relations. The latter could serve as basis to automatically sample layout graphs for architectural planning and design, predict missing links for unobserved parts for as-built building models and learn important latent topological and architectonic patterns.

doi:10.1016/j.autcon.2024.105649

@article{Knechtel2024FloorplanSCG,
    abstract = {This article presents an approach for the automatic semantic segmentation of floorplan images, predicting room boundaries (walls, doors, windows) and semantic labels of room types. A multi-task network was designed to represent and learn inherent dependencies by combining a Convolutional Neural Network to generate suitable features with a Graph Convolutional Network (GCN) to capture long-range dependencies. In particular, a Self-Constructing Graph module is applied to automatically induce an input graph for the GCN. Experiments on different datasets demonstrate the superiority and effectiveness of the multi-task network compared to state-of-the-art methods. The accurate results not only allow for subsequent vectorization of the existing floorplans but also for automatic inference of layout graphs including connectivity and adjacency relations. The latter could serve as basis to automatically sample layout graphs for architectural planning and design, predict missing links for unobserved parts for as-built building models and learn important latent topological and architectonic patterns.},
    author = {Knechtel, Julius and Rottmann, Peter and Haunert, Jan-Henrik and Dehbi, Youness},
    doi = {10.1016/j.autcon.2024.105649},
    journal = {Automation in Construction},
    pages = {105649},
    title = {Semantic Floorplan Segmentation Using Self-Constructing Graph Networks},
    url = {https://www.sciencedirect.com/science/article/pii/S0926580524003856},
    volume = {166},
    year = {2024}
}

Immersive virtual reality (IVR) allows viewing abstract concepts and entities in a three dimensional (3D) visuospatial environment. In this paper, we innovatively introduced IVR technology into the verification of the as-built state of electric line networks in buildings. On the one hand, using a reasoning-based estimation of electric networks as a starting point, we demonstrated the usability of IVR technology for verifying installed utilities in buildings. On the other hand, we established the communication between the Reasoner and the practitioner and also simulated the verification action of electric line networks in buildings in the real world. The principal findings of this work pave the way for a subsequent and systematic evaluation of the different reasoning strategies for estimating and generating the as-built state of building utilities.

doi:10.1007/978-3-031-43699-4_8

@inproceedings{knechtel2024immersiveVRElectricNetworks,
    abstract = {Immersive virtual reality (IVR) allows viewing abstract concepts and entities in a three dimensional (3D) visuospatial environment. In this paper, we innovatively introduced IVR technology into the verification of the as-built state of electric line networks in buildings. On the one hand, using a reasoning-based estimation of electric networks as a starting point, we demonstrated the usability of IVR technology for verifying installed utilities in buildings. On the other hand, we established the communication between the Reasoner and the practitioner and also simulated the verification action of electric line networks in buildings in the real world. The principal findings of this work pave the way for a subsequent and systematic evaluation of the different reasoning strategies for estimating and generating the as-built state of building utilities.},
    address = {Cham},
    author = {Knechtel, Julius and Li, Weilian and Orgeig, Yannick and Haunert, Jan-Henrik and Dehbi, Youness},
    booktitle = {Recent Advances in 3D Geoinformation Science (Proceedings of the 18th 3D GeoInfo Conference 2023, Munich)},
    doi = {10.1007/978-3-031-43699-4\_8},
    editor = {Kolbe, Thomas H. and Donaubauer, Andreas and Beil, Christof},
    isbn = {978-3-031-43699-4},
    pages = {129--143},
    publisher = {Springer Nature Switzerland},
    title = {{I}mmersive {V}irtual {R}eality to {V}erify the {A}s-built {S}tate of {E}lectric {L}ine {N}etworks in {B}uildings},
    year = {2024}
}

Knowledge on the utilities hidden in the wall, e.g., electric lines or water pipes, is indispensable for work safety and valuable for planning. Since most of the existing building stock originates from the pre-digital era, no models as understood for Building Information Modeling (BIM) exist. To generate these models often labor-intensive procedures are necessary; however, recent research has dealt with the efficient generation and verification of a building’s electric network. In this context, a reliable measurement method is a necessity. In this paper we test different measurement techniques, such as point-wise measurements with hand-held devices or area-based techniques utilizing thermal imaging. For this purpose, we designed and built a simulation environment that allows various parameters to be manipulated under controlled conditions. In this scenario the low-cost handheld devices show promising results, with a precision between 92% and 100% and a recall between 89% and 100%. The expensive thermal imaging camera is also able to detect electric lines and pipes if there is enough power on the line or if the temperature of the water in the pipe and the environment’s temperature are sufficiently different. Nevertheless, while point-wise measurements can directly yield results, the thermal camera requires post-processing in specific analysis software. The results reinforce the idea of using reasoning methods in both the do-it-yourself and commercial sector, to rapidly gather information about hidden installations in a building without prior technical knowledge. This paves the way for, e.g., exploring the possibilities of an implementation and presentation in augmented reality (AR).

doi:10.5194/isprs-annals-X-1-W1-2023-263-2023

@article{knechtel2023sensorSuitability,
    abstract = {Knowledge on the utilities hidden in the wall, e.g., electric lines or water pipes, is indispensable for work safety and valuable for planning. Since most of the existing building stock originates from the pre-digital era, no models as understood for Building Information Modeling (BIM) exist. To generate these models often labor-intensive procedures are necessary; however, recent research has dealt with the efficient generation and verification of a building’s electric network. In this context, a reliable measurement method is a necessity. In this paper we test different measurement techniques, such as point-wise measurements with hand-held devices or area-based techniques utilizing thermal imaging. For this purpose, we designed and built a simulation environment that allows various parameters to be manipulated under controlled conditions. In this scenario the low-cost handheld devices show promising results, with a precision between 92\% and 100\% and a recall between 89\% and 100\%. The expensive thermal imaging camera is also able to detect electric lines and pipes if there is enough power on the line or if the temperature of the water in the pipe and the environment’s temperature are sufficiently different. Nevertheless, while point-wise measurements can directly yield results, the thermal camera requires post-processing in specific analysis software. The results reinforce the idea of using reasoning methods in both the do-it-yourself and commercial sector, to rapidly gather information about hidden installations in a building without prior technical knowledge. This paves the way for, e.g., exploring the possibilities of an implementation and presentation in augmented reality (AR).},
    author = {Knechtel, Julius and Behmann, Jan and Haunert, Jan-Henrik and Dehbi, Youness},
    doi = {10.5194/isprs-annals-X-1-W1-2023-263-2023},
    journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
    pages = {263--270},
    title = {{S}uitability {A}ssessment of {D}ifferent {S}ensors to {D}etect {H}idden {I}nstallations for {A}s-built {BIM}},
    url = {https://isprs-annals.copernicus.org/articles/X-1-W1-2023/263/2023/},
    volume = {X-1/W1-2023},
    year = {2023}
}

Abstract Detecting and collecting public opinion via social media can provide near real-time information to decision-makers, which plays a vital role in urban disaster management and sustainable development. However, there has been little work focusing on identifying the perception and the sentiment polarity expressed by users during and after disasters, particularly regional flood events. In this article, we comprehensively analyze tweets data related to the “European floods in 2021” over time, topic, and sentiment, forming a complete workflow from data processing, topic modeling, sentiment analysis, and topic and sentiment prediction. The aim is to address the following research questions: (1) What are the public perception and main concerns during and after floods? (2) How does the public sentiment change during and after floods? Results indicate that there is a significant correlation between a flood's trend and the heat of corresponding tweets. The three topics that receive the most public concern are: (1) climate change and global warming; (2) praying for the victims: and (3) disaster situations and information. Negative sentiments are predominant during the floods and will continue for some time. We tested five different classifiers, of which TextCNN-attention turned out to deliver the best predictions in topic and sentiment prediction, and performed well for sparse flood tweets, it can be used to predict the topic and sentiment polarity of a single tweet in real-time during the flood events. Our findings can help disaster agencies to better understand the dynamics of social networks and develop stronger situational awareness towards a disaster, which can contribute to scientifically justified decision-making in urban risk management and also meet the challenges associated with the global sustainable development goal 11 (SDGs) on Sustainable Cities and Communities.

doi:https://doi.org/10.1111/tgis.13097

@article{li2023socialMediaDisaster,
    abstract = {Abstract Detecting and collecting public opinion via social media can provide near real-time information to decision-makers, which plays a vital role in urban disaster management and sustainable development. However, there has been little work focusing on identifying the perception and the sentiment polarity expressed by users during and after disasters, particularly regional flood events. In this article, we comprehensively analyze tweets data related to the “European floods in 2021” over time, topic, and sentiment, forming a complete workflow from data processing, topic modeling, sentiment analysis, and topic and sentiment prediction. The aim is to address the following research questions: (1) What are the public perception and main concerns during and after floods? (2) How does the public sentiment change during and after floods? Results indicate that there is a significant correlation between a flood's trend and the heat of corresponding tweets. The three topics that receive the most public concern are: (1) climate change and global warming; (2) praying for the victims: and (3) disaster situations and information. Negative sentiments are predominant during the floods and will continue for some time. We tested five different classifiers, of which TextCNN-attention turned out to deliver the best predictions in topic and sentiment prediction, and performed well for sparse flood tweets, it can be used to predict the topic and sentiment polarity of a single tweet in real-time during the flood events. Our findings can help disaster agencies to better understand the dynamics of social networks and develop stronger situational awareness towards a disaster, which can contribute to scientifically justified decision-making in urban risk management and also meet the challenges associated with the global sustainable development goal 11 (SDGs) on Sustainable Cities and Communities.},
    author = {Li, Weilian and Haunert, Jan-Henrik and Knechtel, Julius and Zhu, Jun and Zhu, Qing and Dehbi, Youness},
    doi = {https://doi.org/10.1111/tgis.13097},
    journal = {Transactions in GIS},
    number = {6},
    pages = {1766-1793},
    title = {Social {M}edia {I}nsights on {P}ublic {P}erception and {S}entiment {D}uring and {A}fter {D}isasters: {T}he {E}uropean {F}loods in 2021 as a {C}ase {S}tudy},
    volume = {27},
    year = {2023}
}

doi:10.5194/ica-abs-6-11-2023

@article{arzoumanidis2023SCGHistMaps,
    author = {Arzoumanidis, Lukas and Knechtel, Julius and Haunert, Jan-Henrik and Dehbi, Youness},
    doi = {10.5194/ica-abs-6-11-2023},
    journal = {Abstracts of the ICA},
    pages = {11},
    title = {{S}elf-{C}onstructing {G}raph {C}onvolutional {N}etworks for {S}emantic {S}egmentation of {H}istorical {M}aps},
    volume = {6},
    year = {2023}
}

This article addresses the augmentation of existing building models by a-priori not observable structures such as electric installations. The aim is to unambiguously determine an electric network in an incremental manner with a minimum number of local measurements, e.g. using wire detectors, by suggesting the next measurement. Different reasoning strategies, e.g. utilizing graph-theoretical algorithms, have been presented and tested based on a hypothesis which is generated using Mixed Integer Linear Programming (MILP) while incorporating standards regarding the installation of electric wiring and findings from previous measurements. The presented method has been successfully applied on simulated and real-world buildings, it saves up to 80% of the necessary measurements compared to an exhaustive verification of the whole existing electric network, and paves the way for efficiently extending existing models, e.g. GIS models, with information on hidden utilities. This opens up new opportunities to model further infrastructures, e.g. water pipes, in future research.

doi:10.1016/j.autcon.2022.104571

@article{dehbi2022incrementalConstraint-based,
    abstract = {This article addresses the augmentation of existing building models by a-priori not observable structures such as electric installations. The aim is to unambiguously
determine an electric network in an incremental manner with a minimum number of local measurements, e.g. using wire detectors, by suggesting the next measurement.
Different reasoning strategies, e.g. utilizing graph-theoretical algorithms, have been presented and tested based on a hypothesis which is generated using Mixed Integer
Linear Programming (MILP) while incorporating standards regarding the installation of electric wiring and findings from previous measurements. The presented
method has been successfully applied on simulated and real-world buildings, it saves up to 80% of the necessary measurements compared to an exhaustive verification
of the whole existing electric network, and paves the way for efficiently extending existing models, e.g. GIS models, with information on hidden utilities. This opens
up new opportunities to model further infrastructures, e.g. water pipes, in future research.},
    author = {Dehbi, Youness and Knechtel, Julius and Niedermann, Benjamin and Haunert, Jan-Henrik},
    doi = {10.1016/j.autcon.2022.104571},
    journal = {Automation in Construction},
    pages = {104571},
    title = {Incremental Constraint-based Reasoning for Estimating As-built Electric Line Routing in Buildings},
    volume = {143},
    year = {2022}
}

Terrestrial laser scanning has become more and more popular in recent years. The according planning of the standpoint network is a crucial issue influencing the overhead and the resulting point cloud. Fully static approaches are both cost and time extensive, whereas fully kinematic approaches cannot produce the same data quality. Stop-and-go scanning, which combines the strengths of both strategies, represents a good alternative solution. In the scanning process, the standpoint planning is by now mostly a manual process based on expert knowledge and relying on the surveyor's experience. This paper provides a method based on Mixed Integer Linear Programming (MILP) ensuring an optimal placement of scanner standpoints considering all scanner-related constraints (e.g. incidence angle), a full coverage of the scenery, a sufficient overlap for the subsequent registration and an optimal route planning solving a Traveling Salesperson Problem (TSP). This enables the fully automatic application of autonomous systems for providing a complete model while performing a stop-and-go laser scanning, e.g. with the Spot robot from Boston Dynamics. Our pre-computed solution, i.e. standpoints and trajectory, has been evaluated surveying a real-world environment and successfully compared with a precise LoD2 building model of the underlying scene. The performed ICP-based registration issued from our fully automatic pipeline turns out to be a very good and safe alternative of the otherwise laborious target-based registration.

doi:10.5194/isprs-annals-V-4-2022-129-2022

@article{knechtel2022OptimalPositionPath,
    abstract = {Terrestrial laser scanning has become more and more popular in recent years.
The according planning of the standpoint network is a crucial issue influencing the overhead and the resulting point cloud. Fully static approaches are both cost and time extensive, whereas fully kinematic approaches cannot produce the same data quality.
Stop-and-go scanning, which combines the strengths of both strategies, represents a good alternative solution. In the scanning process, the standpoint planning is by now mostly a manual process based on expert knowledge and relying on the surveyor's experience. This paper provides a method based on Mixed Integer Linear Programming (MILP) ensuring an optimal placement of scanner standpoints considering all scanner-related constraints (e.g. incidence angle),
a full coverage of the scenery, a sufficient overlap for the subsequent registration and an optimal route planning solving a Traveling Salesperson Problem (TSP). This enables the fully automatic application of autonomous systems for providing a complete model while performing a stop-and-go laser scanning, e.g. with the Spot robot from Boston Dynamics.
Our pre-computed solution, i.e. standpoints and trajectory, has been evaluated surveying a real-world environment and successfully compared with a precise LoD2 building model of the underlying scene. The performed ICP-based registration issued from our fully automatic pipeline turns out to be a very good and safe alternative of the otherwise laborious target-based registration.},
    author = {Knechtel, Julius and Klingbeil, Lasse and Haunert, Jan-Henrik and Dehbi, Youness},
    doi = {10.5194/isprs-annals-V-4-2022-129-2022},
    journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
    pages = {129--136},
    title = {Optimal position and path planning for stop-and-go laserscanning for the acquisition of 3D building models},
    url = {https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-4-2022/129/2022/},
    volume = {V-4-2022},
    year = {2022}
}