Advanced Search

Indexed by SCI、CA、РЖ、PA、CSA、ZR、etc .

Volume 34 Issue 5
Oct 2023
Turn off MathJax
Article Contents
Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1
Citation: Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1

Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology

doi: 10.1007/s12583-022-1641-1
More Information
  • Corresponding author: Liufeng Tao, taoliufeng@cug.edu.cn
  • Received Date: 08 Nov 2021
  • Accepted Date: 24 Feb 2022
  • Issue Publish Date: 30 Oct 2023
  • The occurrence of geological disasters can have a large impact on urban safety. Protecting people's safety is the most important concern when disasters occur. Safety improvement requires a large amount of comprehensive and representative risk analysis and a large collection of information related to geological hazards, including unstructured knowledge and experience. To address the relevant information and support safety risk analysis, a geological hazard knowledge graph is developed automatically based on computer vision and domain-geoscience ontology to identify geological hazards from input images while obeying safety rules and regulations, even when affected by changes. In the implementation of the knowledge graph, we design an ontology schema of geological disasters based on a top-down approach, and by organizing knowledge as a logical semantic expression, it can be shared using ontology technologies and therefore enable semantic interoperability. Computer vision approaches are then used to automatically detect a set of entities and attributes, using the data from input images, and object types and their attributes are identified so that they can be stored in Neo4j for reasoning and searching. Finally, a reasoning model for geological hazard identification was developed using the Neo4j database to create nodes, relationships, and their properties for modeling, and geological hazards in the images can be automatically identified by searching the Neo4j database. An application on geological hazard is presented. The results show the effectiveness of the proposed approach in terms of identifying possible potential hazards in geological hazards and assisting in formulating targeted preventive measures.

     

  • Conflict of Interest
    The authors declare that they have no conflict of interest.
  • loading
  • Andrés, S., Arvor, D., Mougenot, I., et al., 2017. Ontology-Based Classification of Remote Sensing Images Using Spectral Rules. Computers & Geosciences, 102: 158–166. https://doi.org/10.1016/j.cageo.2017.02.018
    Ashish, V., Noam, S., Niki, P., et al., 2017. Attention is All You Need. NIPS, 5998–6008
    Bharambe, U., Durbha, S. S., 2018. Adaptive Pareto-Based Approach for Geo-Ontology Matching. Computers & Geosciences, 119: 92–108. https://doi.org/10.1016/j.cageo.2018.06.008
    Bittner, T., Donnelly, M., Smith, B., 2009. A Spatio-Temporal Ontology for Geographic Information Integration. International Journal of Geographical Information Science, 23(6): 765–798. https://doi.org/10.1080/13658810701776767
    Buccella, A., Cechich, A., Gendarmi, D., et al., 2011. Building a Global Normalized Ontology for Integrating Geographic Data Sources. Computers & Geosciences, 37(7): 893–916. https://doi.org/10.1016/j.cageo.2011.02.022
    Carion, N., Massa, F., Synnaeve, G., et al., 2020. End-to-End Object Detection with Transformers. ECCV, 1: 213–229
    Dai, Z. G., Cai, B. L., Lin, Y. G., et al., 2021. UP-DETR: Unsupervised Pre-Training for Object Detection with Transformers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20–25, 2021, Nashville. https://doi.org/10.1109/cvpr46437.2021.00165
    Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al., 2020. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929. https://doi.org/10.48550/arXiv.2010.11929
    Fan, C., Esparza, M., Dargin, J., et al., 2020. Spatial Biases in Crowdsourced Data: Social Media Content Attention Concentrates on Populous Areas in Disasters. Computers, Environment and Urban Systems, 83: 101514. https://doi.org/10.1016/j.compenvurbsys.2020.101514
    Fan, R. Y., Wang, L. Z., Yan, J. N., et al., 2019. Deep Learning-Based Named Entity Recognition and Knowledge Graph Construction for Geological Hazards. ISPRS International Journal of Geo-Information, 9(1): 15. https://doi.org/10.3390/ijgi9010015
    Fang, W. L., Ding, L. Y., Luo, H. B., et al., 2018a. Falls from Heights: A Computer Vision-Based Approach for Safety Harness Detection. Automation in Construction, 91: 53–61. https://doi.org/10.1016/j.autcon.2018.02.018
    Fang, W. L., Ding, L. Y., Zhong, B. T., et al., 2018b. Automated Detection of Workers and Heavy Equipment on Construction Sites: A Convolutional Neural Network Approach. Advanced Engineering Informatics, 37: 139–149. https://doi.org/10.1016/j.aei.2018.05.003
    Fang, W. L., Zhong, B. T., Zhao, N., et al., 2019. A Deep Learning-Based Approach for Mitigating Falls from Height with Computer Vision: Convolutional Neural Network. Advanced Engineering Informatics, 39: 170–177. https://doi.org/10.1016/j.aei.2018.12.005
    Fang, W. L., Ma, L., Love, P. E. D., et al., 2020. Knowledge Graph for Identifying Hazards on Construction Sites: Integrating Computer Vision with Ontology. Automation in Construction, 119: 103310. https://doi.org/10.1016/j.autcon.2020.103310
    Garcia, L. F., Abel, M., Perrin, M., et al., 2020. The GeoCore Ontology: A Core Ontology for General Use in Geology. Computers & Geosciences, 135: 104387. https://doi.org/10.1016/j.cageo.2019.104387
    Girshick, R. B., 2015. Fast R-CNN. IEEE International Conference on Computer Vision (ICCV), December 7–13, Santiago. https://doi.org/10.1109/iccv.2015.169
    Girshick, R., Donahue, J., Darrell, T., et al., 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23–28, 2014, Columbus. https://doi.org/10.1109/cvpr.2014.81
    Goh, Y. M., Chua, D. K. H., 2010. Case-Based Reasoning Approach to Construction Safety Hazard Identification: Adaptation and Utilization. Journal of Construction Engineering and Management, 136(2): 170–178. https://doi.org/10.1061/(asce)co.1943-7862.0000116
    Guarino, N., 1997. Understanding, Building and Using Ontologies. International Journal of Human-Computer Studies, 46(2/3): 293–310. https://doi.org/10.1006/ijhc.1996.0091
    Guia, J., Soares, V. G., Bernardino, J., 2017. Graph Databases: Neo4j Analysis. 351–356. https://doi.org/10.5220/0006356003510356
    Guo, B. H. W., Goh, Y. M., 2017. Ontology for Design of Active Fall Protection Systems. Automation in Construction, 82: 138–153. https://doi.org/10.1016/j.autcon.2017.02.009
    He, K. M., Zhang, X. Y., Ren, S. Q., et al., 2015. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9): 1904–1916. https://doi.org/10.1109/tpami.2015.2389824
    He, K. M., Gkioxari, G., Piotr, D., et al., 2017. Mask R-CNN. arXiv: 1703.06870. https://doi.org/10.48550/arXiv.1703.06870
    Huang, Q. Y., Cervone, G., Zhang, G. M., 2017. A Cloud-Enabled Automatic Disaster Analysis System of Multi-Sourced Data Streams: an Example Synthesizing Social Media, Remote Sensing and Wikipedia Data. Computers, Environment and Urban Systems, 66: 23–37. https://doi.org/10.1016/j.compenvurbsys.2017.06.004
    Janowicz, K., Raubal, M., Kuhn, W., 2011. The Semantics of Similarity in Geographic Information Retrieval. Journal of Spatial Information Science, 2: 29–57. https://doi.org/10.5311/josis.2011.2.3
    Jasper, R. R., Uijlings, K. E. A., van de Sande, T. G., et al., 2013. Smeulders: Selective Search for Object Recognition. Int. J. Comput. Vis., 104(2): 154–171
    Jiao, L. C., Zhang, F., Liu, F., et al., 2019. A Survey of Deep Learning-Based Object Detection. IEEE Access, 7: 128837–128868. https://doi.org/10.1109/access.2019.2939201
    Johnpaul, C. I., Mathew, T., 2017. A Cypher Query Based NoSQL Data Mining on Protein Datasets Using Neo4j Graph Database. 2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS). January 6–7, 2017, Coimbatore, India. IEEE: 1–6. https://doi.org/10.1109/icaccs.2017.8014558
    Kim, D., Liu, M. Y., Lee, S., et al., 2019. Remote Proximity Monitoring between Mobile Construction Resources Using Camera-Mounted UAVs. Automation in Construction, 99: 168–182. https://doi.org/10.1016/j.autcon.2018.12.014
    Li, L., Liu, Y., Zhu, H. H., et al., 2017. A Bibliometric and Visual Analysis of Global Geo-Ontology Research. Computers & Geosciences, 99: 1–8. https://doi.org/10.1016/j.cageo.2016.10.006
    Liu, L., Ouyang, W., Wang, X. G., et al., 2020. Deep Learning for Generic Object Detection: A Survey. International Journal of Computer Vision, 128(2): 261–318. https://doi.org/10.1007/s11263-019-01247-4
    Liu, W., Anguelov, D., Erhan, D., et al., 2016. Single Shot Multibox Detector. arXiv: 1512.02325. https://arxiv.org/abs/1512.02325
    Lu, P., Qin, Y. Y., Li, Z. B., et al., 2019a. Landslide Mapping from Multi-Sensor Data through Improved Change Detection-Based Markov Random Field. Remote Sensing of Environment, 231: 111235. https://doi.org/10.1016/j.rse.2019.111235
    Lu, P., Bai, S. B., Tofani, V., et al., 2019b. Landslides Detection through Optimized Hot Spot Analysis on Persistent Scatterers and Distributed Scatterers. ISPRS Journal of Photogrammetry and Remote Sensing, 156: 147–159. https://doi.org/10.1016/j.isprsjprs.2019.08.004
    Ma, X. G., 2017. Linked Geoscience Data in Practice: Where W3C Standards Meet Domain Knowledge, Data Visualization and OGC Standards. Earth Science Informatics, 10(4): 429–441. https://doi.org/10.1007/s12145-017-0304-8
    Ma, X. G., Carranza, E. J. M., Wu, C. L., et al., 2012. Ontology-Aided Annotation, Visualization, and Generalization of Geological Time-Scale Information from Online Geological Map Services. Computers & Geosciences, 40: 107–119. https://doi.org/10.1016/j.cageo.2011.07.018
    Ma, X. G., Ma, C., Wang, C. B., 2020. A New Structure for Representing and Tracking Version Information in a Deep Time Knowledge Graph. Computers & Geosciences, 145: 104620. https://doi.org/10.1016/j.cageo.2020.104620
    Mao, S., Zhao, Y. M., Chen, J. H., et al., 2020. Development of Process Safety Knowledge Graph: A Case Study on Delayed Coking Process. Computers & Chemical Engineering, 143: 107094. https://doi.org/10.1016/j.compchemeng.2020.107094
    Qiu, Q. J., Xie, Z., Wu, L., et al., 2019. Geoscience Keyphrase Extraction Algorithm Using Enhanced Word Embedding. Expert Systems with Applications, 125: 157–169. https://doi.org/10.1016/j.eswa.2019.02.001
    Redmon, J., Farhadi, A., 2018. YOLOv3: An Incremental Improvement. arXiv: 1804.02767. https://arxiv.org/abs/1804.02767
    Redmon, J., Farhadi, A., 2016. YOLO9000: Better, Faster, Stronger. arXiv: 1612.08242. https://arxiv.org/abs/1612.08242
    Redmon, J., Divvala, S., Girshick, R., et al., 2015. You Only Look Once: Unified, Real-Time Object Detection. arXiv: 1506.02640. https://arxiv.org/abs/1506.02640
    Ren, S. Q., He, K. M., Girshick, R., et al., 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149. https://doi.org/10.1109/tpami.2016.2577031
    Steiner, T., Verborgh, R., Troncy, R., et al., 2012. Adding Realtime Coverage to the Google Knowledge Graph. In: 11th International Semantic Web Conference (ISWC 2012). Citeseer
    UNDRR (United Nations Office for Disaster Risk Reduction), 2020. UNDRR Annual Report. https://www.undrr.org/about-undrr (2020)
    Wang, C. B., Ma, X. G., Chen, J. G., 2018a. Ontology-Driven Data Integration and Visualization for Exploring Regional Geologic Time and Paleontological Information. Computers & Geosciences, 115: 12–19. https://doi.org/10.1016/j.cageo.2018.03.004
    Wang, C. B., Ma, X. G., Chen, J. G., et al., 2018b. Information Extraction and Knowledge Graph Construction from Geoscience Literature. Computers & Geosciences, 112: 112–120. https://doi.org/10.1016/j.cageo.2017.12.007
    Wang, J. J., He, Z. C., Weng, W. G., 2020. A Review of the Research into the Relations between Hazards in Multi-Hazard Risk Analysis. Natural Hazards, 104(3): 2003–2026. https://doi.org/10.1007/s11069-020-04259-3
    Weber, E., Kané, H., 2020. Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion. arXiv: 2004.05525. https://arxiv.org/abs/2004.05525
    Xiao, T. T., Liu, Y. C., Zhou, B. L., et al., 2018. Unified Perceptual Parsing for Scene Understanding. arXiv: 1807.10221. https://arxiv.org/abs/1807.10221
    Zheng, M. H., Gao, P., Zhang, R. R., et al., 2020. End-to-End Object Detection with Adaptive Clustering Transformer. arXiv: 2011.09315. https://arxiv.org/abs/2011.09315
    Zheng, X., Wang, B., Zhao, Y. M., et al., 2021. A Knowledge Graph Method for Hazardous Chemical Management: Ontology Design and Entity Identification. Neurocomputing, 430: 104–111. https://doi.org/10.1016/j.neucom.2020.10.095
    Zhu, X. Z., Su, W. J., Lu, L. W., et al., 2020. Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv: 2010.04159. https://arxiv.org/abs/2010.04159
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(13)

    Article Metrics

    Article views(244) PDF downloads(70) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return