Figure
1.
Workflow to automatically develop a knowledge graph with geological hazard images based on computer vision and ontology.
| Citation: | Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1
|
The occurrence of geological disasters can have a large impact on urban safety. Protecting people's safety is the most important concern when disasters occur. Safety improvement requires a large amount of comprehensive and representative risk analysis and a large collection of information related to geological hazards, including unstructured knowledge and experience. To address the relevant information and support safety risk analysis, a geological hazard knowledge graph is developed automatically based on computer vision and domain-geoscience ontology to identify geological hazards from input images while obeying safety rules and regulations, even when affected by changes. In the implementation of the knowledge graph, we design an ontology schema of geological disasters based on a top-down approach, and by organizing knowledge as a logical semantic expression, it can be shared using ontology technologies and therefore enable semantic interoperability. Computer vision approaches are then used to automatically detect a set of entities and attributes, using the data from input images, and object types and their attributes are identified so that they can be stored in Neo4j for reasoning and searching. Finally, a reasoning model for geological hazard identification was developed using the Neo4j database to create nodes, relationships, and their properties for modeling, and geological hazards in the images can be automatically identified by searching the Neo4j database. An application on geological hazard is presented. The results show the effectiveness of the proposed approach in terms of identifying possible potential hazards in geological hazards and assisting in formulating targeted preventive measures.
The total number of natural disasters worldwide has risen sharply in the first two decades of the 21st century, with 7 348 natural disasters recorded between 2000 and 2019, killing 1.23 million people, affecting a total of 4 billion people and causing economic losses of ＄2.97 trillion worldwide, as stated by the United Nations in a report released on 12 December (UNDRR et al., 2020). The deadliest disaster of the past decade was the Indian Ocean tsunami in 2004, which killed 226 408 people. Furthermore, there exists a large amount of video, image, text and other data on the Internet after geological disasters, and how to effectively extract information and discover knowledge from these data is an important question for data mining (Fang et al., 2020; Weber and Kané, 2020; Wang et al., 2014). Especially for image data, how to identify a target or object and extract and identify its spatial location relationship in the image are key scientific problems to address to make full use of image data (Fan C et al., 2020; Fan R Y et al., 2019; Wang et al., 2020; Lu et al., 2019a, b; Huang et al., 2017).
Usually, a large amount of geological hazard data (especially from images in the news) is generated at the time of and shortly following a geological hazard, and a large amount of the image data is currently underutilized. Image data as a visual display of targets (e.g., people, vehicles, houses, etc.) and scenes in images provide a good data source for our research. It is often necessary to manually identify and analyze geological hazards when they occur and after they occur, which can be a labor-intensive and time-consuming process. The use of computer vision methods to avoid the complexity of traditional manual disaster identification has been widely studied (Fang et al., 2019, 2018a, b).
Other fields have successfully implemented computer vision-related techniques for hazard identification tasks (such as assessing the safety of construction workers) (Fang et al., 2020; Liu et al., 2020; Mao et al., 2020), but it is not possible to effectively identify newly created hazards and predict risks that will arise in the future. Since secondary hazards may arise after the occurrence of geological hazards, existing methods suffer from the following drawbacks: (1) most existing methods use a single computer vision method to identify relatively homogeneous hazards, such as identifying the relationship between people and scenes directly in an image; (2) the currently used computer vision methods cannot effectively extract the relationships between objects and hazards in an image and conduct a deep analysis by combining the extracted semantic relationships. Therefore, a semantic gap exists between primary feature extraction and the advanced semantic information extraction of images, which affects the mining and utilization of advanced semantic information. In addition, due to lack of automatic construction, the implementation and subsequent improvement of these approaches require a lot of human resources.
As a new powerful tool of knowledge representation, knowledge graph was first proposed by Google in 2012 (Steiner et al., 2012), and it focuses on describing various entities and concepts in the real world (Ma et al., 2020). The knowledge graph can be regarded as a new semantic web, where the ontology is extended at the entity level. Knowledge atlas has been widely used in safety monitoring of various industries (Fang et al., 2020; Liu et al., 2020; Mao et al., 2020). These successful applications in other industries indicate that the knowledge graph technology is a potential solution to promote the process safety by making full use of knowledge and information in the production process of the geological hazard.
A semi-automatic knowledge graph development solution for process safety in the geological hazard by integrating computer vision and domain ontology is designed in this paper. The approach consists of three main steps: ontology modeling, knowledge extraction and knowledge inference. Geological hazard documents, experts' experience and related standard codes is used to develop a geological hazard taxonomy, which includes both the specialization and relation between entities. Knowledge extraction step focuses on automatically detecting a set of entities and attributes by using the data derived from input image. After identifying objects and their attributes, an intersection over union (IoU) is used to extract the spatial relationships between objects (i.e., within, away, or overlap) by using geometric and spatial features. After extracting knowledge using the above approach, a reasoning model for geological hazard process safety is constructed using the Neo4j database, which is used to build a set of extracted nodes, relationships and their related properties. A set of experiments, which consists of entity recognition, spatial relationship extraction and automated reasoning and query, is conducted to validate the effectiveness of the proposed methodology.
The main contributions of this research are summarized as follows:
(1) From an algorithmic perspective, a semi-automatic knowledge graph development solution for improve geological hazard safety is proposed in this research, which can qualitatively and comprehensively model various processes in the form of semantic networks. This development framework makes full use of the knowledge of safety in the geological hazard domain and provides the foundation for improving safety level.
(2) We construct a schema representing the process technologies and concepts in the domain of geological hazard, which is the foundation of the knowledge graph. The main challenge of developing this schema is how to properly define classes, relations, and properties to describe the process safety analysis process, while minimizing ambiguity and ensuring inference.
(3) From an application perspective, based on the constructed knowledge graphs, several useful applications, containing knowledge visualization, automatic information retrieval and extraction of disaster chain, are realized in the case.
Computer vision-based object detection can be divided into traditional object detection and deep learning-based object detection. Traditional object detection methods adopt manual extraction of feature information in three steps: (1) using a sliding window to select candidate regions at different locations in an image; (2) extracting feature information; and (3) recognizing feature information. Traditional object detection methods have shortcomings such as substantial time consumption, poor robustness, and limited computing power. With the continuous development and maturity of deep learning technology, it has quickly become the most popular research method in the field of object detection due to the advantages of accuracy, speed, and end-to-end implementation. According to the different implementation steps, deep learning-based object detection algorithms can be divided into two-stage object detection algorithms and single-stage object detection algorithms (Jiao et al., 2019). Two-stage object detection algorithms include a series of CNN models (Liu et al., 2020), and single-stage object detection algorithms include YOLO and SSD.
To utilize neural networks for object positioning and to take advantage of small-scale datasets to train large-scale network models, Girshick proposed Region-CNN (R-CNN) (Girshicks et al., 2014), which combines region proposal and a convolutional neural network. Compared with traditional methods, R-CNN has better performance, but it also has many shortcomings, including cumbersome steps, long model training time and slow image detection speed. He et al. (2015) proposed spatial pyramid pooling (SPP), which enables images of any size to be input into the network and has achieved considerable improvements in speed and detection accuracy. Fast R-CNN (Girshick et al., 2015) is an improved algorithm proposed by Grishick et al. (2014) that solves the problem of repeated calculations. However, the R-CNN algorithm is highly time-consuming when performing feature extraction in the candidate region. Compared with R-CNN and SPP, Fast R-CNN integrates multiple steps into a single model, and the training process is not step-by-step. Fast R-CNN improves both network performance and training speed. Faster R-CNN (Ren et al., 2017), which is based on Fast R-CNN, uses the region proposal network (RPN) to replace the selective search method (SS) (Jasper et al., 2013) for candidate frame recommendation, which improves the detection speed. Faster R-CNN realized end-to-end detection for the first time. Mask R-CNN (He et al., 2017) was proposed by He, who achieved object instance segmentation and improved Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. YOLO (Redmon et al., 2016) (you only look once) is an end-to-end object detection algorithm that uses a regression strategy. The algorithm abandons the generation of suggestion boxes, which only need to "look once" to obtain the regression location information and category information. Compared to Faster R-CNN, YOLO has only one convolutional network, a simpler structure and fast detection speed. To overcome the problem of low detection accuracy and low classification accuracy of YOLO, YOLO v2 (Redmon et al., 2017) introduced batch normalization, a high-resolution classifier and other methods. YOLO v3 (Redmon et al., 2018), which is based on YOLO v2, adjusts the network structure, utilizes multiscale features for object detection, and uses the logistic function to replace softmax for object classification. To address the high missed detection rate of YOLO, Liu et al. (2016) proposed the single-shot detector (SSD) algorithm, which combines the regression strategy and the anchor mechanism of Faster R-CNN to improve the detection accuracy.
Most computer vision-based object detection models are based on convolutional architectures. In natural language processing, the self-attention-based architecture Transformer (Ashish et al., 2017) has been widely used. Recently, Dosovitskiy et al. (2020) proposed Vision Transformer (VIT) (Dosovitskiy et al., 2020), which applies a standard Transformer directly to the image. The results show that the ViT model performs as well as convolutional networks in computer vision. DETR (Carion et al., 2020) is the first end-to-end object detector that successfully utilizes transformers for object detection. This method regards object detection as a set prediction problem and achieves good performance. Deformable DERT (Zhu et al., 2020), UP-DERT (Dai et al., 2021), and ACT (Zheng et al., 2020) were recently proposed as improvements to DERT and achieved good performance.
Ontology, originally a philosophical concept, describes explicit, formal expressions of conceptualization and their relationships among shared communities (Guarino, 1997). Explicit means that there must be a clear definition of concepts and their constraints. Formal refers to the standard formal expression, which makes the computer readable and processable. Shared means knowledge in ontology is not privately owned by individuals but accepted by groups in specialized fields. Therefore, it is convenient for people to realize semantic sharing and a common understanding of concepts (Buccella et al., 2011). This approach can represent the implied relationship between concepts through logical reasoning and effectively express general knowledge in specific fields. Ontology is playing an increasingly important role in information integration and text mining (Wang et al., 2018b; Janowicz et al., 2011; Bittner et al., 2009).
Many researchers have been dedicated to the study of ontology in recent decades, which has deepened our insight into ontology (Garcia et al., 2020; Bharambe and Durbha, 2018; Andrés et al., 2017; Li et al., 2017). For geological topics, early works include information extraction (Qiu et al., 2019; Wang et al., 2018b), data integration and visualization based on geo-ontology (Bharambe and Durbha, 2018; Wang et al., 2018a; Ma, 2017), ontology or knowledge graph construction (Garcia et al., 2020; Ma et al., 2020). Well-organized geological knowledge graphs have resulted from this effort (Ma et al., 2020). Wang et al. (2018b) extracted key information from the geoscience literature based on a hybrid corpus consisting of generic and geological terms. They used a conditional random fields (CRF) model to train Chinese word segmentation rules and realized the visualization of a knowledge graph. To address the limitations of existing deep-time knowledge graphs, Ma et al.)(2020) proposed a new structure that reveals the changes and updates of the International Stratigraphic Chart (ISC) version history in a single knowledge graph. The changes included concept change in the ISC version, attributes of concept change and changes in the relationships between concepts. In addition, Wang et al. (2018a) achieved data integration and visualization among geologic time scales (GTSs), general geology and paleontology with the help of ontology.
Knowledge and data of geological hazard are usually stored in geological reports and separate tables. In this paper, we try to establish a new knowledge graph to construct connections among a substantial volume of geological hazards, objects, and other types of related knowledge. One major challenge in constructing a knowledge graph for geological hazard management is how to build an appropriate ontology structure. A critical part is defining categories, relations, and attributes. Another major challenge is how to make better use of images in related geological documents, namely efficient entity recognition (e.g., object in image). In this paper, we propose an ontology framework for geological hazard management about related images, which provides the foundation and solution for improving process safety. We also carry out named entity recognition based on deep learning approach to identify entities in image so that the information could be better utilized in the domain of geological hazard.
Figure 1 depicts the workflow to automatically develop a knowledge graph with geological hazard images based on computer vision and ontology, which consist of three main steps: ontology modeling, knowledge extraction and knowledge inference.
Ontology modeling: Geological hazard reports and geological hazard standards/codes are applied to develop hazard taxonomy is constructed, which includes both the specialization and relations between entities.
Knowledge extraction: Computer vision methods are applied to automatically identify a set of entities and attributes using the input image. Based on this approach, object types and their attributes (e.g., geometric, coordinates in images) are identified so that the extracted objects could be stored in Neo4j for reasoning and searching. After recognizing objects and their attributes, an intersection over union (IoU) is applied to extract the spatial relationships between objects (e.g., within, away, or overlap) based on geometric and spatial features. Herein, geological hazards come from the ontology in the previous step, and the relationship also comes from the instance relationship defined in the ontology.
Knowledge inference: A reasoning model for geological hazard identification was developed using the Neo4j database to create nodes, relationships, and their properties for modelling. The Neo4j database stores and records all types of objects, their attributes, and the relationship of objects, which were obtained from step two. Thus, geological hazards in the images are automatically identified by querying the created Neo4j database.
A set of geological hazard documents/reports, professional literature and a large amount of expert experience are applied to develop a geological hazard taxonomy, which includes not only specialization but also relations from a list of entities.
The knowledge graph is formed by fusing the constructed ontology with the related data that belong to different concepts. Thus, with the constructed ontology as a guide, the knowledge graph represents a collection of several specific individuals based on the constructed ontology. In most domain-general knowledge graphs (e.g., DBpedia, YAGO and OpenKG), the main and core data are encyclopedia data and the aim are to develop knowledge graphs (Zheng et al., 2021; Mao et al., 2020). Thus, a bottom-up approach is used to construct schemas of these knowledge graphs to cover a large amount of multidomain data. However, for domain-specific knowledge graphs, schemas can be built in a top-down manner because domain-specific ontologies are highly specialized and require the support of a large number of experts (Wang et al., 2018a, b; Ma et al., 2012). The application of these two approaches requires ensuring that they do not present a conceptual conflict level, and the main difference between the approaches is at the data level. Therefore, to select which approach to use to design a geological hazard safety-oriented knowledge graph, it is first necessary to consider detailed requirements and collect relevant data. In this research, relevant databases and geological hazard reports and a set of controlled vocabularies are collected.
For processing geological hazards in the domain of geoscience, several core concepts (e.g., landslide, debris flow, collapse, ground collapse, ground settlement and ground fissure) that have been constructed by domain experts in the domain of geoscience (named GeoOnto) (Qiu et al., 2019) can directly describe the different types of geological hazards and their classification. Based on this framework, similar concepts may be developed in a top-down manner. However, for concepts (e.g., hazards, losses) related to process safety in geological reports or disaster news, the top-down approach is not necessarily complete, as it no longer ensures minimal ambiguity and guaranteed reasoning. In addition, it does not ensure that all available and needed information is covered. Considering the above analysis, in this research, a mixed approach is used to define the schema of geological hazard process safety. With this mixed approach, several controlled ontologies are first defined and constructed by a set of experts in a top-down manner; then, by collecting large amounts, other ontologies can be extracted in a bottom-up manner. The ontologies constructed via the above two methods require manual checking and modification.
Figure 2 shows an illustrative example of a geological graph and its ontology.
In this research, the main objective is to develop a geological hazard process safety knowledge graph in the domain of geoscience using images as the input. Computer vision methods are applied to automatically detect as many entities and their attributes as possible in the images. Thus, the types and relevant attributes (mainly the geometric coordinates for each object) of entities (the object is identified in the image, and the knowledge graph treats it as an entity) are recognized so that they can be stored and recorded in the Neo4j database for further reasoning and searching.
With all objects and relevant attributes in hand, intersection over union (IoU) is applied to calculate the spatial relationships (including within, away, and overlap) between the extracted entities by applying geometric and spatial features.
After extracting knowledge using the above approach, a reasoning model for geological hazard process safety is constructed using the Neo4j database to build a set of extracted nodes, relationships and their related properties. This database aims to store and record all types of extracted entities (in this paper, the extracted objects are treated as entities in the knowledge graph), relevant attributes, and the relationships of these entities. Therefore, geological hazards in the input images can be automatically recognized by searching the constructed Neo4j database.
Geological hazards include a wide range of types, such as landslide, debris flow, subsidence, uplift, volcanoes, etc. In this paper, we focus on the proposed semi-automatic knowledge graph construction method to realize the understanding of image information and disaster risk evaluation by constructing objects and their semantic relationships in images through computer vision and knowledge graph technology. Further knowledge graph construction for each geological hazard type is the next research step.
Geological hazard-related images are chosen for this research. These images can be obtained from various news websites, websites specializing in geological hazards, etc. As a result, we collected 60 000 geological hazard images, including 4 000 images of each type of geological hazards that contain debris_flow, landslide, earthquake, volcano, ground_fissure, ground_settlement, desert_and_desertificated, soil_erosion, special_soil, marine_dynamic_disaster, mine_and_underground _engineering_disaster, surface_collapse, river_and_lake_disasters, and salinization_and_swampization. In the process of image collection, we focus on the geological hazard images including tasks, 15 geological hazard types, houses, traffic, environment, etc. The collected images are input to the Mask R-CNN model as a training set, where we use default values for the parameters in the model.
A variety of natural disasters, such as earthquakes, typhoons, droughts, floods and storm surges, occur globally every year, posing a great threat to people's lives and property. Risk management after a geological disaster is crucial to the safety of life and property. Therefore, effective risk management of geological hazards is an important task. Since a large amount of knowledge and information about geological disasters has not been fully utilized and mined, it is challenging to manage geological disaster risks via information technology. Here, we suggest the use of knowledge mapping to overcome unidentified information gaps in a large number of geological disaster images to improve geological disaster risk management.
A large amount of various data, such as geological hazard news or other ubiquitous textual resources, is created after the occurrence of geological hazards. Textual data have been heavily researched, whereas extraction and knowledge mining of image information formed after the occurrence of geological hazards is relatively rare. There is often important information in geological disaster images. For example, mudslides may lead to flooding, and we can use the spatial relationship between the target (people or houses, etc.) and the geological hazard (flooding, etc.) in the image to provide early warning about the safety of the target and provide supplementary reference on hazards caused by the reoccurrence of the disaster.
In this research, a computer vision approach and ontology are combined to develop a knowledge graph that can be used to automatically detect hazards to address the semantic gap mentioned above. Our goal is to verify whether our semantic vision-based knowledge graph can recognize hazards by means of complex rules. Hence, a knowledge graph framework that combines a computer vision approach and ontology is constructed for identifying hazards. Expert knowledge can be described by ontologies, and the relationships between entities can be presented formally by ontologies. Thus, the relationships between these entities can form computer-understandable, computable, representative and inferred security knowledge. Moreover, computer vision techniques and ontology knowledge can be used to automate the extraction of information from images to form knowledge.
The initial step in constructing the knowledge graph model to identify hazards is to build an ontology of geological hazards. In this paper, the ontology is constructed with the graph database language, which differs from traditional RDF mapping models. In our semantic ontological model, the contents are classified into seven classes: thing, part, attribute, time, space, event and attribute value. For a geological hazard that has occurred, temporal information, spatial information and entity or entity attribute information constitute a complete series of activities that are also inherent properties of the geological hazard itself (Ma et al., 2020; Guo and Goh, 2017; Goh and Chua, 2010). Thus, a basic geological hazard event contains semantic information, which consists of the following:
(1) Entity. Entities represent objectively existing units of information. In this paper, entities can be categorized into five types: (1) people; (2) hazard; (3) house; (4) transportation; and (5) environment. Figure 3 shows an example of taxonomy entities.
(2) Activity. Changes caused by hazards, such as their properties, states and relationships, consist of static and dynamic subclasses. For example, "multiple people are observing the fissures that have occurred", where "observing" represents an activity.
(3) Location. Location information is an inherent property of geological hazard occurrence, including specific location information of different entities in the image.
(4) Time. Time information is an inherent attribute of the occurrence of geological hazards, including the time of occurrence and the duration of geological hazards.
(5) Attribute. This represents entity-specific descriptive information, such as height, color, thickness, speed, and distance information.
An illustrative example of the entities in the constructed ontology model is presented in Fig. 4. As shown in Fig. 4, the entity relationships are mainly spatial relationships in this research. Thus, a relationship exists for all the connections between entities in the image, such as between people and people, between people and houses, and between people and ground cracks. This model can be used to address the following questions:
(1) Is the current environment hostile? Are there people in the environment?
(2) Are there people present within the environment with geological hazards?
(3) What is the spatial relationship between people and houses in the current geological hazard environment? Is it safe to move away from this environment?
A GHIKG should be developed to explore geological hazard identification using a knowledge graph by applying an image as the input based on topological analysis. A knowledge graph is composed of a set of knowledge entities, their relevant relationships and their attributes (Fang et al., 2020; Liu et al., 2020).
To develop the GHIKG, recognizing a set of knowledge entities and their relationships (e.g., relationships between entities and relationships between entities and attributes) and then mapping them into a graph network consists of three primary steps.
The object of this research is to construct a methodology that combines computer vision and ontology to recognize and warn people who people may be in danger. For example, after a ground fracture occurs, there may be some people who are watching, and whether these people are moving or stationary, they should be considered unsafe in the model we have constructed and require a warning.
Our proposed approach focuses solely on object entity and attribute extraction from images using a computer vision approach, and it is applied to explore the construction of a geological hazard knowledge graph. Based on this object, the following tasks aim to extract contextual information from a geological hazard image:
(1) Entity Recognition: As described above, entities can be categorized into five types: (1) people; (2) hazard; (3) house; (4) transportation; and (5) environment. In this paper, two tasks are required to recognize and extract: object and scene recognition. Object detection is applied to recognize people, houses and transportation. Scene recognition aims to identify the scene of the input image. Scene recognition methods a signature component of computer vision that enable us to define a context based on recognized objects. To solve this problem, the Mask R-CNN approach (He et al., 2017) is used for object and scene recognition.
Mask R-CNN has achieved great success in object detection. Therefore, the model is adopted in our research for entity (i.e., people, houses and transportation) and scene detection (i.e., hazards and the environment). The specific details of Mask R-CNN are introduced in Fang et al. (2018). Mask R-CNN consists of two main stages to identify objects and scenes: (1) the ResNet network is used to develop feature maps based on input images and then to obtain a list of candidate object bounding boxes with the region proposal network; and (2) the RoiAlign stage focuses on persevering and extracting all spatial locations for each object based on the candidate box from the above stage and performing classification, regressing bounding boxes, and generating masks.
To accurately understand and identify a set of scenes (i.e., the current scene), a unified perceptual parsing method (named UPP) that integrates a feature pyramid network is applied to break the input images into concepts. The UPP method, which was proposed by Xiao et al. (2018), can be used to effectively discover and infer a set of rich visual information and knowledge from input images. In terms of performance, the UPP approach has achieved higher accuracy than that of other machine learning approaches, especially deep learning approaches (e.g., SegNet, DilatedNet and fast-RCNN) applied for segmentation. The details of Mask R-CNN are introduced in Xiao et al. (2018).
(2) Attribute Extraction: In our research, distance and spatial features are used to recognize geological hazards since we consider only the relevant information from an image without other auxiliary data. Two types of attributes must be extracted: (1) the coordinate information about each object based on the input images and (2) the distance information between objects extracted by Mask R-CNN. Herein, a transformation matrix integrated with a hybrid semantic computer vision model is used to calculate the distance between objects.
Based on the output of Step 1, we can obtain the types of all objects and their attributes, and three spatial relationships between these objects are used in this research. These spatial relationships can be classified as within, overlap and away, as illustrated in Fig. 5. Since images alone are used as input, the spatial relationship between two objects is calculated in terms of the spatial distance.
Herein, the spatial relationship between two extracted objects (e.g., object A and object B in Fig. 5) is computed using the IoU of the contents of bounding box A and bounding box B. The detailed equation is as follows:
|
IoU(A,B)=area(A)area(B)min{area(A),area(B)}{1,within[0,1],overlap0,away |
(1) |
For the first two cases, we represent the space between two targets by calculating the IoU. For the last case, we describe the spatial relationship using the transfer transformation calculation method.
After obtaining the spatial relationships between knowledge entities from the input images, the third step is to map the spatial relationships and the knowledge entities into a graph database to represent the knowledge as a network graph (i.e., GHIKG). With the graph structure, a list of semantic queries by nodes, links and properties can be represented and searched. The Neo4j graph database management system is used to store and visualize these nodes and relevant relationships because this graph database has the ability to represent data in a robust and scalable manner (Guia et al., 2017; Johnpaul and Mathew, 2017). The objective of step 3 is to automatically recognize hazards via data storing and automated reasoning and querying.
Data storing. After the procedure of step 2, we obtain the knowledge entities and their relevant spatial relationships. The final outputs after these procedures are recorded in.csv file format and then input into the Neo4j database for visualization. This database handles the data and then offers a set of outputs with the network graph.
Automated reasoning and query. On the basis of the stored data, geological hazards can be identified and obtained by searching a set of unsafe behavior rules included in the model. Unsafe behavior consists mainly of a set of rules that we define. For example, in the event of a ground crack, if there is a crowd observing or walking on top of the crack, we can identify such unsafe behavior by calculating how much the border of the crowd overlaps with the border of the ground crack, and if the overlap occurs, the crowd is potentially in danger.
In this subsection, a set of primary experiments are conducted to validate the performance of the proposed methodology for developing a knowledge graph. First, we construct a list of unsafe rules that includes geological hazard ontologies. Second, the geological hazard identification results consist of entity recognition from the input images, spatial relationship extraction based on the IoU and results of automated reasoning and query.
To share and reuse information in the field of geological hazards, it is necessary to build and develop a formal and standardized ontology. In this paper, a geological hazard ontology is created by Protégé, which is an open-source toolkit often used to build ontologies based on knowledge. This platform provides a large number of knowledge model architectures and toolkits to create, visualize, and manipulate various forms of ontology. The ontology is constructed with reference to the Standard of Classification for Geological Hazard released by the China Association of Geological Disaster Prevention and Control Engineering, which defines different kinds of geological hazard terminology, categories and grades. Domain experts also participated in the construction of the ontology and added reliability and professionalism. Each class is connected by is-a and part-of relations. Both relations are key components in constructing the conceptual hierarchy and hyponymy relations of this ontology. For the relation part-of, a parent class has a series of subclasses, but the subclass has only one parent. In addition, there are many instance triples related to hazard events, such as < earthquake, happen-time, 2021-03-24 > , < earthquake, happen-place, Xinjiang > , and < earthquake, magnitude, 5.4 > . With the help of a well-organized and well-designed geological hazard ontology, it is easier to realize semantic extraction, querying, recommendation and reasoning about geological disaster information.
The top class is geological hazards, which describe the most abstract and generalized entity in this ontology. All geological hazards are described and classified according to the genesis, scale and other characteristics of geological disasters in the lower-level class. Geological hazards include 15 classes: collapse, debris_flow, landslide, earthquake, volcano, ground_fissure, ground_settlement, desert_and_desertificated, soil_erosion, special_soil, marine_dynamic_disaster, mine_and_underground_engineering_disaster, surface_collapse, river_and_lake_disasters, and salinization_and_swampization. These 15 classes are specific types of geological hazards, and their subclasses are detailed descriptions of the corresponding geological hazard categories (see Fig. 6).
To illustrate and validate the effectiveness of the proposed knowledge graph semantic model, unsafe conditions must be identified from input images alone, and these unsafe conditions may lead to a number of risks, as shown in Table 1. A series of online news images about geological hazards was collected for corpus construction and experiments; in total, we collected 4 000 news images about geological hazards, some of which are presented in Fig. 7. Some representative unsafe geological hazard conditions are shown in Fig. 7, and the geological hazard entities, possible types of activities, calculated attributes and spatial relationships are described. For example, in image number one, people are in a mudslide disaster scene with cart activities. Mudslide disasters often produce secondary disasters or re-emerging disasters; in this case, there is a certain level of safety risk in the scene, and the people should be warned.
Hence, based on the constructed ontology, we can combine computer vision techniques to identify possible risks related to geological hazards to reduce the number of accidents. We identify and predict only simple security risks because more complex image scenarios are not considered.
Initially, we use computer vision and ontology methods to detect targets and their attribute values in image scenes, geological hazard types, etc. As explained earlier, we use spatial relations in the image scene to represent direct connections between targets by calculating the crossover or overlap between targets using the IoU method, as illustrated in Figs. 8 and 9. In Figs. 8a and 8c, a group of people use a raft to slide to a safe area in the case of flooding, and the object recognition methodology can accurately identify the target in the scene. In Figs. 9a and 9c, there is a ground collapse in the city and a group of people are watching or passing through the area.
The experimental performance of this study depends mainly on two aspects: entity recognition and the identification of relevant attributes. Hence, the precision and recall are selected as performance metrics to evaluate the object detection performance. Additionally, pixel accuracy and mean IoU are used to assess the scene recognition performance. Our approach achieved a pixel accuracy and mean IoU of 89.9 and 48.88 on the collected dataset, respectively.
The performance of attribute detection relies mainly on the extraction of coordinates and the calculation of distances to geohazard images. Previous studies have demonstrated that the transformation matrix can be used for distance calculation of objects (Kim et al., 2019). Based on these performance metrics, the semantic computer vision approach we developed achieves acceptable accuracy in identifying unsafe behaviors.
Since we use only image information as the data source, based on the image information in this paper, we use only spatial relationships to represent connections among extracted targets to construct the knowledge graph of the geological hazard scenario. In the extraction of attribute information based on images, the core content is dependent on the extraction of coordinate information from the graph and the calculation of the distance to the target in the image. Related studies have demonstrated that the object or target distance calculation can be performed via the transformation matrix method. Based on the described performance metrics, the computer vision-based semantic computation method designed in this paper achieves user-acceptable levels and standards for performing unsafe behavior in geological hazard scenes.
Figure 10 shows an example of extracting spatial relationships by applying the IoU with the relevant distance. As shown in Fig. 10, first, the target and scene types are identified, including pedestrians, ground collapse, cyclists, and bicycles. On the basis of the identification, the coordinate information and IoU information of the target are computed and analyzed. Finally, the spatial relationships between different targets and between targets and scenes are constructed to form a knowledge graph. Figure 11 also shows the detection results after a ground collapse in the city. The results show that there is an overlapping relationship between pedestrians and ground collapse, which requires special attention and precaution.
After obtaining the objects and the spatial relationships between objects, we combine entities (in this research, objects are regarded as entities) and spatial relationships as the input for the Neo4j database; then, a geological hazard knowledge graph can be automatically developed and visualized. Based on the constructed knowledge graph, we can determine or read the location relationship between the target and the geological hazard in the image scene by inputting a query to determine whether the current behavior of the target is safe (see Figs. 12–13).
Figure 12 shows an example of an unsafe scene in which a group of people stand on a crack in the ground for observation, which can be recognized by applying the rule "MATCH(x: people)-[r: overlap]-(y: hazard) RETURN x, r, y". Based on the rules of the query, we can determine that there is an overlapping relationship between the target A in the image and the geological hazard scene; that is, the target is exposed to safety hazards and risks.
The constructed knowledge graph can provide service and support before, during and after disasters. Before a disaster occurs, the knowledge may generate early warnings based on knowledge in the existing graph; when a disaster occurs, pictures can be monitored, and trapped people can be found; after a disaster occurs, early warnings can be provided for secondary disasters. People are at risk in disaster scenarios and are not always aware of what behaviors are dangerous. The developed method can be used to raise awareness of risk prevention.
When a geological disaster incident occurs, emergency and rescue personnel need to obtain the most logical solution. Because geological disaster rescue is complex, emergency and rescue personnel have difficulty obtaining reasonable solutions or techniques without fully understanding the process. The constructed geological disaster safety knowledge graph and the information retrieval function provided enables the use of other data sources, such as text data and other ontology data, to enrich the formation of the knowledge graph combined with graphical and textual tables and to complement the knowledge map and finally automatically provide solution recommendations. Moreover, information about the causes and losses caused by geological disasters can be provided via information retrieval, and the knowledge in the knowledge graph can be used to evaluate the losses caused by disasters and provide decision and service support for different departments.
To improve the effectiveness and efficiency of safety behavior inspection and reduce the occurrence of unsafe behavior in geological hazard scenes, we propose an algorithm based on a combination of computer solutions and ontologies to automatically monitor and identify risks from geological hazard images when they occur or when they are not yet complete. Our proposed method provides an effective discovery mechanism for geological hazard scene management and can be used to automate the identification of targets and record and analyze the relationships between targets and geological hazards (mainly unsafe behaviors) and thus provide decision recommendations for relevant authorities to reduce or minimize the possibility of injuries. Furthermore, the constructed behavioral safety rules provide a means of management for the relevant departments and can highlight the potential consequences of unsafe behavior for the targets (especially people) on site and monitor and advocate for greater compliance with basic safety rules.
Despite the novelty of the approach proposed, we require to acknowledge that it has several limitations. Firstly, the current data source of our method is just ordinary image information, and then identify the spatial relationship between targets in the image. For geological disasters, the identification and damage assessment caused by the disaster can be performed with the help of remote sensing images when the disaster occurs. By fusing remote sensing images with common images, the accuracy of the model and recognition can be further improved. In the future, we will focus on creating a database with a significant number of images in order to further validate and improve the reliability of our proposed approach.
Secondly, our study extracts coordinates and distances of objects from 2D images, and then obtains spatial relationships according to the obtained information (i.e., coordinates, distances). Mistakes may occur when using the transformation matrix to calculate the distances of objects in a single image. Therefore, we suggest that future studies need to use stereo cameras to collect data and calculate depth information to improve the accuracy of calculating spatial relationships.
Thirdly, our study only considered attributes (i.e., distances between entities) in the ontology model to determine whether hazards with complex rules can be identified. An accurate definition of a hazard should include a combination of more semantic information to determine it, such as time, location, activity, hazard scale, hazard level, etc. For example, a person is not allowed to approach the area where a large landslide occurs. In this case, we should obtain the scale or magnitude of the geohazard. We suggest that our approach can be extended by considering other semantic information, based on the built ontology model.
Based on the overall good results, we believe that the proposed approach has various important implications for different tasks involving information retrieval and knowledge inference. Future research may be conducted in several directions to generalize this study. For example, we intend to combine more features (such as color features, shape features, and texture features) to complete and enrich the knowledge graph in the future.
In this study, we proposed a geological hazard identification knowledge graph (GHIKG) approach based on computer vision and a domain-geoscience ontology to automatically recognize geological hazards from a set of images. Three main steps are combined to develop the proposed model: ontology modeling, knowledge extraction and knowledge inference. Ontology modeling is used to construct the domain-geoscience ontology as a guide for developing a knowledge graph. Knowledge extraction focuses on extracting a list of entities and their attributes from input images based on computer vision algorithms. The last step is knowledge inference, in which a reasoning model for geological hazard process safety is constructed using the Neo4j database to build a set of extracted nodes, relationships and related properties. We also show that our proposed approach can detect geological hazards in various contexts from images. We reveal that our semantic model can accurately recognise geological hazards from images with complex rules. We also suggest that our proposed semantic model can be used by site management to automatically identify potential geological hazards and therefore put in place strategies to mitigate potential injuries and accidents.
ACKNOWLEDGMENTS: Additional support was provided by the IUGS Deep-time Digital Earth (DDE) Big Science Program. This study was financially supported by the National Key R & D Program of China (No. 2022YFF0711601), the Natural Science Foundation of Hubei Province of China (No. 2022CFB640), the Opening Fund of Hubei Key Laboratory of Intelligent Vision-Based Monitoring for Hydroelectric Engineering (No. 2022SDSJ04), the Opening Fund of Key Laboratory of Geological Survey and Evaluation of Ministry of Education (No. GLAB 2023ZR01) and the Fundamental Research Funds for the Central Universities. The final publication is available at Springer via https://doi.org/10.1007/s12583-022-1641-1.| Andrés, S., Arvor, D., Mougenot, I., et al., 2017. Ontology-Based Classification of Remote Sensing Images Using Spectral Rules. Computers & Geosciences, 102: 158–166. https://doi.org/10.1016/j.cageo.2017.02.018 |
| Ashish, V., Noam, S., Niki, P., et al., 2017. Attention is All You Need. NIPS, 5998–6008 |
| Bharambe, U., Durbha, S. S., 2018. Adaptive Pareto-Based Approach for Geo-Ontology Matching. Computers & Geosciences, 119: 92–108. https://doi.org/10.1016/j.cageo.2018.06.008 |
| Bittner, T., Donnelly, M., Smith, B., 2009. A Spatio-Temporal Ontology for Geographic Information Integration. International Journal of Geographical Information Science, 23(6): 765–798. https://doi.org/10.1080/13658810701776767 |
| Buccella, A., Cechich, A., Gendarmi, D., et al., 2011. Building a Global Normalized Ontology for Integrating Geographic Data Sources. Computers & Geosciences, 37(7): 893–916. https://doi.org/10.1016/j.cageo.2011.02.022 |
| Carion, N., Massa, F., Synnaeve, G., et al., 2020. End-to-End Object Detection with Transformers. ECCV, 1: 213–229 |
|
Dai, Z. G., Cai, B. L., Lin, Y. G., et al., 2021. UP-DETR: Unsupervised Pre-Training for Object Detection with Transformers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20–25, 2021, Nashville. |
|
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al., 2020. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929. |
| Fan, C., Esparza, M., Dargin, J., et al., 2020. Spatial Biases in Crowdsourced Data: Social Media Content Attention Concentrates on Populous Areas in Disasters. Computers, Environment and Urban Systems, 83: 101514. https://doi.org/10.1016/j.compenvurbsys.2020.101514 |
| Fan, R. Y., Wang, L. Z., Yan, J. N., et al., 2019. Deep Learning-Based Named Entity Recognition and Knowledge Graph Construction for Geological Hazards. ISPRS International Journal of Geo-Information, 9(1): 15. https://doi.org/10.3390/ijgi9010015 |
| Fang, W. L., Ding, L. Y., Luo, H. B., et al., 2018a. Falls from Heights: A Computer Vision-Based Approach for Safety Harness Detection. Automation in Construction, 91: 53–61. https://doi.org/10.1016/j.autcon.2018.02.018 |
| Fang, W. L., Ding, L. Y., Zhong, B. T., et al., 2018b. Automated Detection of Workers and Heavy Equipment on Construction Sites: A Convolutional Neural Network Approach. Advanced Engineering Informatics, 37: 139–149. https://doi.org/10.1016/j.aei.2018.05.003 |
| Fang, W. L., Zhong, B. T., Zhao, N., et al., 2019. A Deep Learning-Based Approach for Mitigating Falls from Height with Computer Vision: Convolutional Neural Network. Advanced Engineering Informatics, 39: 170–177. https://doi.org/10.1016/j.aei.2018.12.005 |
| Fang, W. L., Ma, L., Love, P. E. D., et al., 2020. Knowledge Graph for Identifying Hazards on Construction Sites: Integrating Computer Vision with Ontology. Automation in Construction, 119: 103310. https://doi.org/10.1016/j.autcon.2020.103310 |
| Garcia, L. F., Abel, M., Perrin, M., et al., 2020. The GeoCore Ontology: A Core Ontology for General Use in Geology. Computers & Geosciences, 135: 104387. https://doi.org/10.1016/j.cageo.2019.104387 |
|
Girshick, R. B., 2015. Fast R-CNN. IEEE International Conference on Computer Vision (ICCV), December 7–13, Santiago. |
|
Girshick, R., Donahue, J., Darrell, T., et al., 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition. June 23–28, 2014, Columbus. |
| Goh, Y. M., Chua, D. K. H., 2010. Case-Based Reasoning Approach to Construction Safety Hazard Identification: Adaptation and Utilization. Journal of Construction Engineering and Management, 136(2): 170–178. https://doi.org/10.1061/(asce)co.1943-7862.0000116 |
| Guarino, N., 1997. Understanding, Building and Using Ontologies. International Journal of Human-Computer Studies, 46(2/3): 293–310. https://doi.org/10.1006/ijhc.1996.0091 |
|
Guia, J., Soares, V. G., Bernardino, J., 2017. Graph Databases: Neo4j Analysis. 351–356. |
| Guo, B. H. W., Goh, Y. M., 2017. Ontology for Design of Active Fall Protection Systems. Automation in Construction, 82: 138–153. https://doi.org/10.1016/j.autcon.2017.02.009 |
| He, K. M., Zhang, X. Y., Ren, S. Q., et al., 2015. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9): 1904–1916. https://doi.org/10.1109/tpami.2015.2389824 |
|
He, K. M., Gkioxari, G., Piotr, D., et al., 2017. Mask R-CNN. arXiv: 1703.06870. |
| Huang, Q. Y., Cervone, G., Zhang, G. M., 2017. A Cloud-Enabled Automatic Disaster Analysis System of Multi-Sourced Data Streams: an Example Synthesizing Social Media, Remote Sensing and Wikipedia Data. Computers, Environment and Urban Systems, 66: 23–37. https://doi.org/10.1016/j.compenvurbsys.2017.06.004 |
| Janowicz, K., Raubal, M., Kuhn, W., 2011. The Semantics of Similarity in Geographic Information Retrieval. Journal of Spatial Information Science, 2: 29–57. https://doi.org/10.5311/josis.2011.2.3 |
| Jasper, R. R., Uijlings, K. E. A., van de Sande, T. G., et al., 2013. Smeulders: Selective Search for Object Recognition. Int. J. Comput. Vis., 104(2): 154–171 |
| Jiao, L. C., Zhang, F., Liu, F., et al., 2019. A Survey of Deep Learning-Based Object Detection. IEEE Access, 7: 128837–128868. https://doi.org/10.1109/access.2019.2939201 |
|
Johnpaul, C. I., Mathew, T., 2017. A Cypher Query Based NoSQL Data Mining on Protein Datasets Using Neo4j Graph Database. 2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS). January 6–7, 2017, Coimbatore, India. IEEE: 1–6. |
| Kim, D., Liu, M. Y., Lee, S., et al., 2019. Remote Proximity Monitoring between Mobile Construction Resources Using Camera-Mounted UAVs. Automation in Construction, 99: 168–182. https://doi.org/10.1016/j.autcon.2018.12.014 |
| Li, L., Liu, Y., Zhu, H. H., et al., 2017. A Bibliometric and Visual Analysis of Global Geo-Ontology Research. Computers & Geosciences, 99: 1–8. https://doi.org/10.1016/j.cageo.2016.10.006 |
| Liu, L., Ouyang, W., Wang, X. G., et al., 2020. Deep Learning for Generic Object Detection: A Survey. International Journal of Computer Vision, 128(2): 261–318. https://doi.org/10.1007/s11263-019-01247-4 |
|
Liu, W., Anguelov, D., Erhan, D., et al., 2016. Single Shot Multibox Detector. arXiv: 1512.02325. |
| Lu, P., Qin, Y. Y., Li, Z. B., et al., 2019a. Landslide Mapping from Multi-Sensor Data through Improved Change Detection-Based Markov Random Field. Remote Sensing of Environment, 231: 111235. https://doi.org/10.1016/j.rse.2019.111235 |
| Lu, P., Bai, S. B., Tofani, V., et al., 2019b. Landslides Detection through Optimized Hot Spot Analysis on Persistent Scatterers and Distributed Scatterers. ISPRS Journal of Photogrammetry and Remote Sensing, 156: 147–159. https://doi.org/10.1016/j.isprsjprs.2019.08.004 |
| Ma, X. G., 2017. Linked Geoscience Data in Practice: Where W3C Standards Meet Domain Knowledge, Data Visualization and OGC Standards. Earth Science Informatics, 10(4): 429–441. https://doi.org/10.1007/s12145-017-0304-8 |
| Ma, X. G., Carranza, E. J. M., Wu, C. L., et al., 2012. Ontology-Aided Annotation, Visualization, and Generalization of Geological Time-Scale Information from Online Geological Map Services. Computers & Geosciences, 40: 107–119. https://doi.org/10.1016/j.cageo.2011.07.018 |
| Ma, X. G., Ma, C., Wang, C. B., 2020. A New Structure for Representing and Tracking Version Information in a Deep Time Knowledge Graph. Computers & Geosciences, 145: 104620. https://doi.org/10.1016/j.cageo.2020.104620 |
| Mao, S., Zhao, Y. M., Chen, J. H., et al., 2020. Development of Process Safety Knowledge Graph: A Case Study on Delayed Coking Process. Computers & Chemical Engineering, 143: 107094. https://doi.org/10.1016/j.compchemeng.2020.107094 |
| Qiu, Q. J., Xie, Z., Wu, L., et al., 2019. Geoscience Keyphrase Extraction Algorithm Using Enhanced Word Embedding. Expert Systems with Applications, 125: 157–169. https://doi.org/10.1016/j.eswa.2019.02.001 |
|
Redmon, J., Farhadi, A., 2018. YOLOv3: An Incremental Improvement. arXiv: 1804.02767. |
|
Redmon, J., Farhadi, A., 2016. YOLO9000: Better, Faster, Stronger. arXiv: 1612.08242. |
|
Redmon, J., Divvala, S., Girshick, R., et al., 2015. You Only Look Once: Unified, Real-Time Object Detection. arXiv: 1506.02640. |
| Ren, S. Q., He, K. M., Girshick, R., et al., 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149. https://doi.org/10.1109/tpami.2016.2577031 |
| Steiner, T., Verborgh, R., Troncy, R., et al., 2012. Adding Realtime Coverage to the Google Knowledge Graph. In: 11th International Semantic Web Conference (ISWC 2012). Citeseer |
|
UNDRR (United Nations Office for Disaster Risk Reduction), 2020. UNDRR Annual Report. |
| Wang, C. B., Ma, X. G., Chen, J. G., 2018a. Ontology-Driven Data Integration and Visualization for Exploring Regional Geologic Time and Paleontological Information. Computers & Geosciences, 115: 12–19. https://doi.org/10.1016/j.cageo.2018.03.004 |
| Wang, C. B., Ma, X. G., Chen, J. G., et al., 2018b. Information Extraction and Knowledge Graph Construction from Geoscience Literature. Computers & Geosciences, 112: 112–120. https://doi.org/10.1016/j.cageo.2017.12.007 |
| Wang, J. J., He, Z. C., Weng, W. G., 2020. A Review of the Research into the Relations between Hazards in Multi-Hazard Risk Analysis. Natural Hazards, 104(3): 2003–2026. https://doi.org/10.1007/s11069-020-04259-3 |
|
Weber, E., Kané, H., 2020. Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion. arXiv: 2004.05525. |
|
Xiao, T. T., Liu, Y. C., Zhou, B. L., et al., 2018. Unified Perceptual Parsing for Scene Understanding. arXiv: 1807.10221. |
|
Zheng, M. H., Gao, P., Zhang, R. R., et al., 2020. End-to-End Object Detection with Adaptive Clustering Transformer. arXiv: 2011.09315. |
| Zheng, X., Wang, B., Zhao, Y. M., et al., 2021. A Knowledge Graph Method for Hazardous Chemical Management: Ontology Design and Entity Identification. Neurocomputing, 430: 104–111. https://doi.org/10.1016/j.neucom.2020.10.095 |
|
Zhu, X. Z., Su, W. J., Lu, L. W., et al., 2020. Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv: 2010.04159. |
| [1] | Song Du, Diquan Li, Zhan Yang, Qiaohui Che, Yinglin Fan, Degao Zhang, Zhiqing Xie. Detection of High-Salinity water in Deep Geological Sequestration Using the Wide-Field Electromagnetic Method[J]. Journal of Earth Science. doi: 10.1007/s12583-025-0224-3 |
| [2] | Yiying Deng, Sicun Song, Junxuan Fan, Mao Luo, Le Yao, Shaochun Dong, Yukun Shi, Linna Zhang, Yue Wang, Haipeng Xu, Huiqing Xu, Yingying Zhao, Zhaohui Pan, Zhangshuai Hou, Xiaoming Li, Boheng Shen, Xinran Chen, Shuhan Zhang, Xuejin Wu, Lida Xing, Qingqing Liang, Enze Wang. Paleontology Knowledge Graph for Data-Driven Discovery[J]. Journal of Earth Science, 2024, 35(3): 1024-1034. doi: 10.1007/s12583-023-1943-9 |
| [3] | Shaochun Dong, Yukun Shi, Yizao Ran, Haijun Wu, Yiying Deng, Junxuan Fan, Xinyu Dai. Biological Classification System Knowledge Graph and Semi-automatic Construction of Its Invertebrate Fossil Branches[J]. Journal of Earth Science, 2024, 35(6): 2119-2128. doi: 10.1007/s12583-023-1941-y |
| [4] | Xiangyun Hu, Bo Han, Tao Yu. How to Effectively Prevent and Mitigate Geomagnetic Hazard?[J]. Journal of Earth Science, 2023, 34(4): 1304-1305. doi: 10.1007/s12583-023-1883-6 |
| [5] | Jia Wang, Wen Zhang, Donghui Chen, Han Yin, Junqi Chen. Multi-scale structural geological model and quantification of stability evaluation for a high-steep fractured rock slope[J]. Journal of Earth Science. doi: 10.1007/s12583-023-1953-9 |
| [6] | Qinjun Qiu, Miao Tian, Zhong Xie, Yongjian Tan, Kai Ma, Qingfang Wang, Shengyong Pan, Liufeng Tao. Extracting Named Entity Using Entity Labeling in Geological Text Using Deep Learning Approach[J]. Journal of Earth Science, 2023, 34(5): 1406-1417. doi: 10.1007/s12583-022-1789-8 |
| [7] | Qinjun Qiu, Zhen Huang, Dexin Xu, Kai Ma, Liufeng Tao, Run Wang, Jianguo Chen, Zhong Xie, Yongsheng Pan. Integrating NLP and Ontology Matching into a Unified System for Automated Information Extraction from Geological Hazard Reports[J]. Journal of Earth Science, 2023, 34(5): 1433-1446. doi: 10.1007/s12583-022-1716-z |
| [8] | Xiumian Hu, Yiwei Xu, Xiaogang Ma, Yunqiang Zhu, Chao Ma, Chao Li, Hairong Lü, Xinbing Wang, Chenghu Zhou, Chengshan Wang. Knowledge System, Ontology, and Knowledge Graph of the Deep-Time Digital Earth (DDE): Progress and Perspective[J]. Journal of Earth Science, 2023, 34(5): 1323-1327. doi: 10.1007/s12583-023-1930-1 |
| [9] | Jinglun Xi, Jin Wu, Mingbo Wu. Design and Construction of Lightweight Domain Ontology of Tectonic Geomorphology[J]. Journal of Earth Science, 2023, 34(5): 1350-1357. doi: 10.1007/s12583-022-1779-x |
| [10] | Linna Zhang, Zhangshuai Hou, Boheng Shen, Qing Chen, Shaochun Dong, Junxuan Fan. Paleobiogeographic Knowledge Graph: An Ongoing Work with Fundamental Support for Future Research[J]. Journal of Earth Science, 2023, 34(5): 1339-1349. doi: 10.1007/s12583-023-1845-z |
| [11] | Xiaobo Zhang, Hao Li, Qiang Liu, Zhenhua Li, Claire E. Reymond, Min Zhang, Yuangeng Huang, Hongfei Chen, Zhong-Qiang Chen. A New Machine-Learning Extracting Approach to Construct a Knowledge Base: A Case Study on Global Stromatolites over Geological Time[J]. Journal of Earth Science, 2023, 34(5): 1358-1373. doi: 10.1007/s12583-022-1801-3 |
| [12] | Yiwei Xu, Xiumian Hu, Zhong Han. Carbonate Ontology and Its Application for Integrating Microfacies Data[J]. Journal of Earth Science, 2023, 34(5): 1328-1338. doi: 10.1007/s12583-023-1808-4 |
| [13] | Kai Ma, Miao Tian, Yongjian Tan, Qinjun Qiu, Zhong Xie, Rong Huang. Ontology-Based BERT Model for Automated Information Extraction from Geological Hazard Reports[J]. Journal of Earth Science, 2023, 34(5): 1390-1405. doi: 10.1007/s12583-022-1724-z |
| [14] | Qinjun Qiu, Bin Wang, Kai Ma, Hairong Lü, Liufeng Tao, Zhong Xie. A Practical Approach to Constructing a Geological Knowledge Graph: A Case Study of Mineral Exploration Data[J]. Journal of Earth Science, 2023, 34(5): 1374-1389. doi: 10.1007/s12583-023-1809-3 |
| [15] | Renguang Zuo, Carranza Emmanuel John M.. A Fractal Measure of Spatial Association between Landslides and Conditioning Factors[J]. Journal of Earth Science, 2017, 28(4): 588-594. doi: 10.1007/s12583-017-0772-2 |
| [16] | Xiaogang Ma, Peter Fox, Eric Rozell, Patrick West, Stephan Zednik. Ontology Dynamics in a Data Life Cycle: Challenges and Recommendations from a Geoscience Perspective[J]. Journal of Earth Science, 2014, 25(2): 407-412. doi: 10.1007/s12583-014-0408-8 |
| [17] | Hao Li, Changsong Lin, Yanmei Zhang, Rongxi Zhang, Zhifeng Rui, Lili Huang. Stratigraphic Architecture and Computer Modelling of Carbonate Platform Margin, Late Ordovician Lianglitage Formation, Central Tarim Basin[J]. Journal of Earth Science, 2012, 23(4): 627-638. doi: 10.1007/s12583-012-0279-9 |
| [18] | Junqi Liu, Xuebin Huang, Chonglong Wu, Wenming Cheng, Wei Xiang, He Xia. From the Area to the Point-Study on the Key Technology of 3D Geological Hazard Modeling in Three Gorges Reservoir Area[J]. Journal of Earth Science, 2012, 23(2): 199-206. doi: 10.1007/s12583-012-0246-5 |
| [19] | Kunlong Yin, Guirong Zhang, Liangfeng Zhu. Overview Landslide Hazard Assessment of China[J]. Journal of Earth Science, 2004, 15(3): 306-311. |
| [20] | Xianzhong Li, Zousheng Yang, Huizhu Yang, Zunguo Hu, Minglei Shi. Engineering Geological Mapping and Land-Capability Analysis in Tangshan City[J]. Journal of Earth Science, 2002, 12(3): 274-277. |
| 1. | Liufeng Tao, Qirui Wu, Miao Tian, et al. Construction and Application of Feature Recommendation Model for Remote Sensing Interpretation of Rock Strata Based on Knowledge Graph. Remote Sensing, 2025, 17(6): 973. doi:10.3390/rs17060973 | |
| 2. | Xin Yang, Li Sun, Mei-Ling Liu, et al. Knowledge graph construction with BERT-BiLSTM-IDCNN-CRF and graph algorithms for metallogenic pattern discovery: A case study of pegmatite-type lithium deposits in China. Ore Geology Reviews, 2025, 179: 106514. doi:10.1016/j.oregeorev.2025.106514 | |
| 3. | Qiuli Lv, Lijie Gao, Longyu Shi, et al. Design of a sustainable development path in Chenzhou based on a knowledge graph. Chinese Journal of Population, Resources and Environment, 2024, 22(4): 455. doi:10.1016/j.cjpre.2024.11.009 | |
| 4. | Donghui Shi, Zhigang Li, Jozef Zurada, et al. Ontology-based text convolution neural network (TextCNN) for prediction of construction accidents. Knowledge and Information Systems, 2024, 66(4): 2651. doi:10.1007/s10115-023-02036-9 | |
| 5. | Wenjia Li, Liang Wu, Xinde Xu, et al. Deep Learning and Network Analysis: Classifying and Visualizing Geologic Hazard Reports. Journal of Earth Science, 2024, 35(4): 1289. doi:10.1007/s12583-021-1589-6 | |
| 6. | Yalin Zou, Yi Huang, Yifan Wang, et al. The Construction of Urban Rainstorm Disaster Event Knowledge Graph Considering Evolutionary Processes. Water, 2024, 16(7): 942. doi:10.3390/w16070942 | |
| 7. | Qirui Wu, Zhong Xie, Miao Tian, et al. Integrating Knowledge Graph and Machine Learning Methods for Landslide Susceptibility Assessment. Remote Sensing, 2024, 16(13): 2399. doi:10.3390/rs16132399 |
Figures(13)
Copyright © 2013-2020 Journal of Earth Science 鄂ICP备15021562号-2
Tel: +86-27-67885075 Fax: +86-27-67885075 E-mail: xbb@cug.edu.cn
Address: Editorial Office of Journal, China University of Geosciences, Yujiashan, Wuhan, Hubei 430074, P. R. China
Supported by:
Beijing Renhe Information Technology Co. Ltd
E-mail:
info@rhhz.net
Qinjun Qiu, Zhong Xie, Die Zhang, Kai Ma, Liufeng Tao, Yongjian Tan, Zhipeng Zhang, Baode Jiang. Knowledge Graph for Identifying Geological Disasters by Integrating Computer Vision with Ontology. Journal of Earth Science, 2023, 34(5): 1418-1432. doi: 10.1007/s12583-022-1641-1
DownLoad:
DownLoad:
