Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology

Qinjun Qiu; Zhong Xie; Die Zhang; Kai Ma; Liufeng Tao; Yongjian Tan; Zhipeng Zhang; Baode Jiang

doi:10.1007/s12583-022-1641-1

Volume 34 Issue 5

Oct 2023

Turn off MathJax

Article Contents

Journal of Earth Science > 2023 > 34(5): 1418-1432.

Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1

Citation:

Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1

Citation:

Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1

PDF( 3567 KB)

Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology

doi: 10.1007/s12583-022-1641-1

Qinjun Qiu^{1, 2, 3
,},
Zhong Xie^{1, 2, 3},
Die Zhang⁴,
Kai Ma^{5, 6},
Liufeng Tao^{1, 2, 3
,
,
,},
Yongjian Tan^{5, 6},
Zhipeng Zhang⁴,
Baode Jiang^{1, 2, 3}

1.
School of Computer Science, China University of Geosciences, Wuhan 430078, China
2.
Key Laboratory of Geological Survey and Evaluation of Ministry of Education, China University of Geosciences, Wuhan 430074, China
3.
National Engineering Research Center of Geographic Information System, Wuhan 430074, China
4.
School of Resource and Environmental Sciences, Wuhan University, Wuhan 430074, China
5.
College of Computer and Information Technology, China Three Gorges University, Yichang 443002, China
6.
Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, China Three Gorges University, Yichang 443002, China

More Information

Corresponding author: Liufeng Tao, taoliufeng@cug.edu.cn
Received Date: 08 Nov 2021
Accepted Date: 24 Feb 2022
Issue Publish Date: 30 Oct 2023

Abstract

Abstract

The occurrence of geological disasters can have a large impact on urban safety. Protecting people's safety is the most important concern when disasters occur. Safety improvement requires a large amount of comprehensive and representative risk analysis and a large collection of information related to geological hazards, including unstructured knowledge and experience. To address the relevant information and support safety risk analysis, a geological hazard knowledge graph is developed automatically based on computer vision and domain-geoscience ontology to identify geological hazards from input images while obeying safety rules and regulations, even when affected by changes. In the implementation of the knowledge graph, we design an ontology schema of geological disasters based on a top-down approach, and by organizing knowledge as a logical semantic expression, it can be shared using ontology technologies and therefore enable semantic interoperability. Computer vision approaches are then used to automatically detect a set of entities and attributes, using the data from input images, and object types and their attributes are identified so that they can be stored in Neo4j for reasoning and searching. Finally, a reasoning model for geological hazard identification was developed using the Neo4j database to create nodes, relationships, and their properties for modeling, and geological hazards in the images can be automatically identified by searching the Neo4j database. An application on geological hazard is presented. The results show the effectiveness of the proposed approach in terms of identifying possible potential hazards in geological hazards and assisting in formulating targeted preventive measures.
- geological hazard,
- computer vision,
- knowledge graph,
- city safety,
- ontology

Conflict of Interest
The authors declare that they have no conflict of interest.

FullText(HTML)

References(53)

References

Andrés, S., Arvor, D., Mougenot, I., et al., 2017. Ontology-Based Classification of Remote Sensing Images Using Spectral Rules. Computers & Geosciences, 102: 158–166. https://doi.org/10.1016/j.cageo.2017.02.018

Ashish, V., Noam, S., Niki, P., et al., 2017. Attention is All You Need. NIPS, 5998–6008

Bharambe, U., Durbha, S. S., 2018. Adaptive Pareto-Based Approach for Geo-Ontology Matching. Computers & Geosciences, 119: 92–108. https://doi.org/10.1016/j.cageo.2018.06.008

Bittner, T., Donnelly, M., Smith, B., 2009. A Spatio-Temporal Ontology for Geographic Information Integration. International Journal of Geographical Information Science, 23(6): 765–798. https://doi.org/10.1080/13658810701776767

Buccella, A., Cechich, A., Gendarmi, D., et al., 2011. Building a Global Normalized Ontology for Integrating Geographic Data Sources. Computers & Geosciences, 37(7): 893–916. https://doi.org/10.1016/j.cageo.2011.02.022

Carion, N., Massa, F., Synnaeve, G., et al., 2020. End-to-End Object Detection with Transformers. ECCV, 1: 213–229

Dai, Z. G., Cai, B. L., Lin, Y. G., et al., 2021. UP-DETR: Unsupervised Pre-Training for Object Detection with Transformers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20–25, 2021, Nashville. https://doi.org/10.1109/cvpr46437.2021.00165

Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al., 2020. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929. https://doi.org/10.48550/arXiv.2010.11929

Fan, C., Esparza, M., Dargin, J., et al., 2020. Spatial Biases in Crowdsourced Data: Social Media Content Attention Concentrates on Populous Areas in Disasters. Computers, Environment and Urban Systems, 83: 101514. https://doi.org/10.1016/j.compenvurbsys.2020.101514

Fan, R. Y., Wang, L. Z., Yan, J. N., et al., 2019. Deep Learning-Based Named Entity Recognition and Knowledge Graph Construction for Geological Hazards. ISPRS International Journal of Geo-Information, 9(1): 15. https://doi.org/10.3390/ijgi9010015

Fang, W. L., Ding, L. Y., Luo, H. B., et al., 2018a. Falls from Heights: A Computer Vision-Based Approach for Safety Harness Detection. Automation in Construction, 91: 53–61. https://doi.org/10.1016/j.autcon.2018.02.018

Fang, W. L., Ding, L. Y., Zhong, B. T., et al., 2018b. Automated Detection of Workers and Heavy Equipment on Construction Sites: A Convolutional Neural Network Approach. Advanced Engineering Informatics, 37: 139–149. https://doi.org/10.1016/j.aei.2018.05.003

Fang, W. L., Zhong, B. T., Zhao, N., et al., 2019. A Deep Learning-Based Approach for Mitigating Falls from Height with Computer Vision: Convolutional Neural Network. Advanced Engineering Informatics, 39: 170–177. https://doi.org/10.1016/j.aei.2018.12.005

Fang, W. L., Ma, L., Love, P. E. D., et al., 2020. Knowledge Graph for Identifying Hazards on Construction Sites: Integrating Computer Vision with Ontology. Automation in Construction, 119: 103310. https://doi.org/10.1016/j.autcon.2020.103310

Garcia, L. F., Abel, M., Perrin, M., et al., 2020. The GeoCore Ontology: A Core Ontology for General Use in Geology. Computers & Geosciences, 135: 104387. https://doi.org/10.1016/j.cageo.2019.104387

Girshick, R. B., 2015. Fast R-CNN. IEEE International Conference on Computer Vision (ICCV), December 7–13, Santiago. https://doi.org/10.1109/iccv.2015.169

Girshick, R., Donahue, J., Darrell, T., et al., 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23–28, 2014, Columbus. https://doi.org/10.1109/cvpr.2014.81

Goh, Y. M., Chua, D. K. H., 2010. Case-Based Reasoning Approach to Construction Safety Hazard Identification: Adaptation and Utilization. Journal of Construction Engineering and Management, 136(2): 170–178. https://doi.org/10.1061/(asce)co.1943-7862.0000116

Guarino, N., 1997. Understanding, Building and Using Ontologies. International Journal of Human-Computer Studies, 46(2/3): 293–310. https://doi.org/10.1006/ijhc.1996.0091

Guia, J., Soares, V. G., Bernardino, J., 2017. Graph Databases: Neo4j Analysis. 351–356. https://doi.org/10.5220/0006356003510356

Guo, B. H. W., Goh, Y. M., 2017. Ontology for Design of Active Fall Protection Systems. Automation in Construction, 82: 138–153. https://doi.org/10.1016/j.autcon.2017.02.009

He, K. M., Zhang, X. Y., Ren, S. Q., et al., 2015. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9): 1904–1916. https://doi.org/10.1109/tpami.2015.2389824

He, K. M., Gkioxari, G., Piotr, D., et al., 2017. Mask R-CNN. arXiv: 1703.06870. https://doi.org/10.48550/arXiv.1703.06870

Huang, Q. Y., Cervone, G., Zhang, G. M., 2017. A Cloud-Enabled Automatic Disaster Analysis System of Multi-Sourced Data Streams: an Example Synthesizing Social Media, Remote Sensing and Wikipedia Data. Computers, Environment and Urban Systems, 66: 23–37. https://doi.org/10.1016/j.compenvurbsys.2017.06.004

Janowicz, K., Raubal, M., Kuhn, W., 2011. The Semantics of Similarity in Geographic Information Retrieval. Journal of Spatial Information Science, 2: 29–57. https://doi.org/10.5311/josis.2011.2.3

Jasper, R. R., Uijlings, K. E. A., van de Sande, T. G., et al., 2013. Smeulders: Selective Search for Object Recognition. Int. J. Comput. Vis., 104(2): 154–171

Jiao, L. C., Zhang, F., Liu, F., et al., 2019. A Survey of Deep Learning-Based Object Detection. IEEE Access, 7: 128837–128868. https://doi.org/10.1109/access.2019.2939201

Johnpaul, C. I., Mathew, T., 2017. A Cypher Query Based NoSQL Data Mining on Protein Datasets Using Neo4j Graph Database. 2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS). January 6–7, 2017, Coimbatore, India. IEEE: 1–6. https://doi.org/10.1109/icaccs.2017.8014558

Kim, D., Liu, M. Y., Lee, S., et al., 2019. Remote Proximity Monitoring between Mobile Construction Resources Using Camera-Mounted UAVs. Automation in Construction, 99: 168–182. https://doi.org/10.1016/j.autcon.2018.12.014

Li, L., Liu, Y., Zhu, H. H., et al., 2017. A Bibliometric and Visual Analysis of Global Geo-Ontology Research. Computers & Geosciences, 99: 1–8. https://doi.org/10.1016/j.cageo.2016.10.006

Liu, L., Ouyang, W., Wang, X. G., et al., 2020. Deep Learning for Generic Object Detection: A Survey. International Journal of Computer Vision, 128(2): 261–318. https://doi.org/10.1007/s11263-019-01247-4

Liu, W., Anguelov, D., Erhan, D., et al., 2016. Single Shot Multibox Detector. arXiv: 1512.02325. https://arxiv.org/abs/1512.02325

Lu, P., Qin, Y. Y., Li, Z. B., et al., 2019a. Landslide Mapping from Multi-Sensor Data through Improved Change Detection-Based Markov Random Field. Remote Sensing of Environment, 231: 111235. https://doi.org/10.1016/j.rse.2019.111235

Lu, P., Bai, S. B., Tofani, V., et al., 2019b. Landslides Detection through Optimized Hot Spot Analysis on Persistent Scatterers and Distributed Scatterers. ISPRS Journal of Photogrammetry and Remote Sensing, 156: 147–159. https://doi.org/10.1016/j.isprsjprs.2019.08.004

Ma, X. G., 2017. Linked Geoscience Data in Practice: Where W3C Standards Meet Domain Knowledge, Data Visualization and OGC Standards. Earth Science Informatics, 10(4): 429–441. https://doi.org/10.1007/s12145-017-0304-8

Ma, X. G., Carranza, E. J. M., Wu, C. L., et al., 2012. Ontology-Aided Annotation, Visualization, and Generalization of Geological Time-Scale Information from Online Geological Map Services. Computers & Geosciences, 40: 107–119. https://doi.org/10.1016/j.cageo.2011.07.018

Ma, X. G., Ma, C., Wang, C. B., 2020. A New Structure for Representing and Tracking Version Information in a Deep Time Knowledge Graph. Computers & Geosciences, 145: 104620. https://doi.org/10.1016/j.cageo.2020.104620

Mao, S., Zhao, Y. M., Chen, J. H., et al., 2020. Development of Process Safety Knowledge Graph: A Case Study on Delayed Coking Process. Computers & Chemical Engineering, 143: 107094. https://doi.org/10.1016/j.compchemeng.2020.107094

Qiu, Q. J., Xie, Z., Wu, L., et al., 2019. Geoscience Keyphrase Extraction Algorithm Using Enhanced Word Embedding. Expert Systems with Applications, 125: 157–169. https://doi.org/10.1016/j.eswa.2019.02.001

Redmon, J., Farhadi, A., 2018. YOLOv3: An Incremental Improvement. arXiv: 1804.02767. https://arxiv.org/abs/1804.02767

Redmon, J., Farhadi, A., 2016. YOLO9000: Better, Faster, Stronger. arXiv: 1612.08242. https://arxiv.org/abs/1612.08242

Redmon, J., Divvala, S., Girshick, R., et al., 2015. You Only Look Once: Unified, Real-Time Object Detection. arXiv: 1506.02640. https://arxiv.org/abs/1506.02640

Ren, S. Q., He, K. M., Girshick, R., et al., 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149. https://doi.org/10.1109/tpami.2016.2577031

Steiner, T., Verborgh, R., Troncy, R., et al., 2012. Adding Realtime Coverage to the Google Knowledge Graph. In: 11th International Semantic Web Conference (ISWC 2012). Citeseer

UNDRR (United Nations Office for Disaster Risk Reduction), 2020. UNDRR Annual Report. https://www.undrr.org/about-undrr (2020)

Wang, C. B., Ma, X. G., Chen, J. G., 2018a. Ontology-Driven Data Integration and Visualization for Exploring Regional Geologic Time and Paleontological Information. Computers & Geosciences, 115: 12–19. https://doi.org/10.1016/j.cageo.2018.03.004

Wang, C. B., Ma, X. G., Chen, J. G., et al., 2018b. Information Extraction and Knowledge Graph Construction from Geoscience Literature. Computers & Geosciences, 112: 112–120. https://doi.org/10.1016/j.cageo.2017.12.007

Wang, J. J., He, Z. C., Weng, W. G., 2020. A Review of the Research into the Relations between Hazards in Multi-Hazard Risk Analysis. Natural Hazards, 104(3): 2003–2026. https://doi.org/10.1007/s11069-020-04259-3

Weber, E., Kané, H., 2020. Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion. arXiv: 2004.05525. https://arxiv.org/abs/2004.05525

Xiao, T. T., Liu, Y. C., Zhou, B. L., et al., 2018. Unified Perceptual Parsing for Scene Understanding. arXiv: 1807.10221. https://arxiv.org/abs/1807.10221

Zheng, M. H., Gao, P., Zhang, R. R., et al., 2020. End-to-End Object Detection with Adaptive Clustering Transformer. arXiv: 2011.09315. https://arxiv.org/abs/2011.09315

Zheng, X., Wang, B., Zhao, Y. M., et al., 2021. A Knowledge Graph Method for Hazardous Chemical Management: Ontology Design and Entity Identification. Neurocomputing, 430: 104–111. https://doi.org/10.1016/j.neucom.2020.10.095

Zhu, X. Z., Su, W. J., Lu, L. W., et al., 2020. Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv: 2010.04159. https://arxiv.org/abs/2010.04159

Relative Articles

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(13)

Get Citation

PDF

XML

Article Metrics

Article views(279) PDF downloads(70)

Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology

doi: 10.1007/s12583-022-1641-1

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Related

Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology

doi: 10.1007/s12583-022-1641-1

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Related

Export File

Citation

Format

Content