Driven by China’s “AI+” strategy and policies supporting digital and intelligent transformation in the energy sector, the oil and gas industry is accelerating its transition from experience-led operations to data-driven intelligence. The growing use of AI in well-log interpretation, reservoir prediction, and drilling incident early warning has made high-quality datasets essential to overcoming barriers to intelligent development. Three major challenges remain: inconsistent alignment between data and business, with heterogeneous geological, geophysical, and production data dispersed across dozens of systems without common semantics or sharing mechanisms; unclear boundaries for collaboration between large models and specialized small models, as general-purpose large models lack deep petroleum-domain understanding while small models remain isolated with limited data reuse; and inefficient annotation, as traditional manual labeling cannot meet the needs of small-model training and large-model fine-tuning.
Addressing the full exploration and development lifecycle, this study establishes three categories of high-quality datasets: business support, small-model training, and large-model fine-tuning. The classification clarifies their application boundaries, data standards, and implementation paths. Centered on Five-Dimensional Business Ontology Modeling and integrated with the open OSDU architecture, the approach decomposes oil and gas workflows into more than 40,000 minimum Business Nodes and constructs an ontology spanning the object, business, work, discipline, and business-process domains. Business data, standards, knowledge cases, and business rules are integrated into a comprehensive business knowledge graph that provides a common semantic backbone. Data governance, feature engineering, and knowledge extraction then support standardized construction of all three dataset categories, addressing data silos, inconsistent standards, and inefficient annotation. The resulting foundation supports production management, intelligent modeling, and decision optimization, advancing governance from data cleansing to semantic, object-oriented, and knowledge-based data management.
Each category follows a distinct technical path. Business-support datasets use end-to-end governance—multi-source ingestion, cleansing and repair, standardization, and automated quality inspection—to ensure completeness and business consistency. Small-model training datasets use an enterprise feature store to align granularity across time, depth, and stratigraphy, with knowledge graphs supporting automated feature labeling. Large-model fine-tuning datasets use 15 types of question-and-answer templates, including definitions, procedures, and causality, together with extensive petroleum-domain semantic lexicons to generate structured Q&A pairs.
Keywords: high-quality datasets; Five-Dimensional Business Ontology Modeling; minimum Business Nodes; knowledge graphs; artificial intelligence; data governance; OSDU; semantic lexicons; Q&A pairs.
Under policy guidance, the oil and gas industry, a central pillar of energy security, is accelerating its shift from experience-led operations to data-driven intelligence. Applications combining AI with exploration and development, production, and operations are expanding, making intelligent capabilities a major driver of high-quality industry development.
Data is a core production input for this transformation, and its quality directly affects model training and business value. As AI adoption deepens, both specialized small models for automated well-log interpretation and reservoir prediction and industry large models covering multiple business domains require increasingly accurate, complete, consistent, and business-relevant data. High-quality datasets underpin precise training, connect business and technology workflows, enable cross-department collaboration, and support closed-loop intelligent decision-making.
Two major challenges nevertheless persist. The first is consistency between data and business: long value chains and complex operations disperse data across sources without common semantics or governance, creating silos and disconnecting data from business workflows. The second is unclear positioning of large and small models: general-purpose large models lack deep domain understanding, specialized small models remain isolated, and model systems lack a closed feedback loop with real-world data. Building datasets tailored to different applications and establishing standardized construction and management processes are therefore essential to overcoming these barriers.
This study establishes a systematic classification framework for high-quality datasets addressing the core needs of intelligent development in oil and gas, defining application domains, data requirements, and technical standards. By examining construction methods and key technologies for three dataset categories, it develops an end-to-end solution centered on a five-dimensional business ontology, supported by knowledge graphs and safeguarded by data governance, addressing inconsistent business alignment, mismatches between models and data, and nonstandard knowledge representation.
The ultimate objective is a deployable, extensible framework that standardizes business data management, enables efficient reuse of modeling data, and structures knowledge data. It supports stable operation of everyday business systems, accurate small-model training, and domain adaptation of large models, moving intelligent development from technical exploration toward deeper business application and providing a foundation for intelligent transformation across the value chain.
The study proposes an integrated framework comprising one core and three supporting branches. The five-dimensional oil and gas business ontology is the central foundation for a unified business knowledge graph and management framework supporting standardized construction of all three dataset categories (Figure 1).
Figure 1. Overall framework for building high-quality oil and gas datasets
The foundational layer combines five-dimensional business modeling with OSDU open architecture standards. Its ontology covers the object, business, work, discipline, and business-process domains, decomposing exploration, development, and production into more than 40,000 minimum Business Nodes. Each node specifies input data, processing workflows, outputs, and relationships. A comprehensive business knowledge graph integrates business data, standards, knowledge cases, business rules, and component algorithms into a common semantic backbone and data management platform, providing standardized data and relationship support for all three categories.
The dataset layer builds on this foundation to address different applications. Business-support datasets prioritize completeness and business consistency for everyday systems. Small-model training datasets emphasize standardized, reusable features for specialized modeling tasks. Large-model fine-tuning datasets focus on structured industry knowledge and semantic accuracy.
The application-support layer provides foundational intelligent services through data service interfaces, connecting datasets with business systems, small models, and large models to establish closed-loop management across data, models, and business operations.
1. High-Quality Business Datasets for Everyday Business Applications
(1) Application Scenarios
These datasets underpin enterprise information systems and serve routine production management, reserves evaluation, equipment operations and maintenance, and compliance auditing. They support ERP, MES, PCS, reserves management, and other business platforms. Their core value is stable operation and efficient coordination, providing reliable, accurate data for daily decisions such as single-well production reporting, equipment maintenance planning, and dynamic reserves evaluation.
(2) Core Data Requirements
Comprehensive coverage: Include the full business lifecycle and resources such as master data (well identifiers, blocks, and equipment codes), transactional data (production, pressure, and operational records), reference data (lithology codes and fluid types), standards, knowledge cases, and business rules.
Consistent management: Strictly separate business descriptions from physical storage, linking the two through metadata to preserve flexibility and extensibility.
Complete metadata: Record provenance, collection time, data type, accuracy requirements, business meaning, and relationships. Metadata must support lifecycle traceability for business objects and precise identification of business relationships.
Quality compliance: Apply comprehensive governance to meet six core dimensions—accuracy, completeness, consistency, uniqueness, timeliness, and traceability—with no erroneous records, conflicting formats, or semantic ambiguity.
(3) Implementation Path
Standardized ingestion: Establish common multi-source ingestion protocols based on OSDU open architecture and OiO business modeling standards. Support structured databases, unstructured documents, time-series curves, images, and video, using format conversion and semantic mapping to standardize data from different systems.
End-to-end governance: Establish an ingestion–cleansing–standardization–inspection–archiving workflow with automated tools. Combine rule engines and machine learning to handle missing, anomalous, and duplicate values. Standardize terminology, coding conventions, units, and formats. Build automated inspection rules covering all six quality dimensions, using the Drools rule engine to scan data assets periodically and produce issue lists and remediation recommendations.
Metadata framework: Establish business, technical, and management metadata, supported by tools for automatic collection, manual entry, relationship mapping, and version management. Use metadata for lifecycle traceability and business relationship identification.
Storage and services: Combine a data lake with a data warehouse, storing structured data in the warehouse and unstructured and time-series data in the lake. Unified interfaces provide query, retrieval, and statistical services with secure, controlled, and efficient access.
2. High-Quality Datasets for Big-Data Modeling with Small Models
(1) Application Scenarios
These datasets bridge raw business data and specialized AI small models. Task-specific datasets support automated well-log interpretation, reservoir prediction, drilling incident early warning, production forecasting, and injection–production optimization. Standardized, relevant features reduce lengthy preparation, inconsistent features, and high labeling costs, improving training efficiency and prediction accuracy.
(2) Core Data Requirements
Unified graph foundation: Source data through the comprehensive oil and gas business knowledge graph to preserve business relevance and semantic consistency and avoid weak generalization caused by data silos.
Standardized granularity: Align data from different types and disciplines to common units across time (days, months, years), depth (meters, decimeters, stratigraphic horizons), events (incident and operation types), and grids (horizontal and vertical grids) to enable integration.
Features and labels: Extract meaningful features such as porosity, permeability, and shale content through feature engineering, and automatically label them using the knowledge graph and historical cases. Produce structured feature-and-label data compatible with machine learning inputs.
Reusability: Manage features centrally for reuse across models, avoiding repeated engineering and inconsistent results while maintaining common calculation definitions.
(3) Implementation Path
Feature extraction and standardization: Identify core modeling features through the knowledge graph and develop specialized tools supporting moving averages, Fourier transforms, quantile statistics, and interval aggregation. Convert raw business data into standardized features, using nearest-neighbor resampling, linear interpolation, and kriging to align multi-source granularity.
Automated feature labeling: Use graph relationships and business rules for automatic labeling. For complex cases, combine automated labeling with expert review: business metadata associations and historical event matching generate initial labels, which domain experts validate and correct to reduce manual effort.
Feature store: Establish an enterprise Feature Hub for registration, storage, management, and reuse. Assign each feature a unique ID and record its name, description, calculation logic, source, applicable scenarios, and version. Support searches by keyword, business scenario, and data source for rapid feature access.
Dynamic dataset construction: A graph-driven feature retrieval engine allows users to select features and granularity for a modeling task. The system links and aggregates relevant features into training datasets, supporting filtering, sorting, aggregate calculations, and export to formats compatible with TensorFlow and PyTorch.
Dataset quality evaluation: Assess both individual features and their combinations. Feature evaluation covers accuracy, stability, and leakage detection. Correlation matrices, SHAP analysis, and cross-validation assess relationships with target variables and redundancy among features to establish modeling suitability.
3. High-Quality Industry Knowledge Datasets for Fine-Tuning Oil and Gas Large Language Models
(1) Application Scenarios
These datasets structure petroleum-domain knowledge for domain-adaptive fine-tuning, improving understanding and reasoning in specialized contexts. Applications include professional Q&A, workflow consultation, standards interpretation, automated research report generation, and natural-language-driven chart generation. Examples include consultation on drilling incident response, reserves evaluation report outlines, and intelligent data visualization from natural-language requests, improving efficiency and reducing the effort needed to acquire domain knowledge.
(2) Core Data Requirements
Comprehensive knowledge coverage: Use the comprehensive business knowledge graph to include business data, standards, knowledge cases, business rules, and component algorithms, describing the full exploration and development lifecycle.
Standardized structure: Organize data through a standard ontology specifying Business Nodes, entity types, and relationship types to ensure consistent, logical knowledge representation and efficient model parsing.
Rich Q&A pairs: Include extensive structured or semi-structured Q&A covering 15 question types, including definitions, causality, procedures, and comparisons, to provide diverse fine-tuning inputs.
Complete semantic lexicons: Supply petroleum-specific terminology, synonyms, abbreviations, and industry jargon to map natural language precisely to data resources associated with Business Nodes.
(3) Implementation Path
Comprehensive knowledge graph construction: Use the five-dimensional business ontology to model geological bodies, wells, equipment, processes, standards, problems, and indicators, with relationships such as “belongs to,” “located in,” “causes,” and “resolves.” Combine manual construction with NLP-based extraction from technical documents, standards, and historical cases, incorporating expert-reviewed entities and relationships to ensure accuracy and completeness.
Standardized Q&A generation: Design 15 graph-based template types, including definitions (“What is a low-permeability sandstone reservoir?”), procedures (“What is the procedure for responding to stuck pipe during drilling?”), and causality (“What are the main causes of lost circulation?”). Generate pairs in batches using natural language generation (NLG), supplementing them with Q&A extracted from papers and operating manuals and high-quality manually authored pairs.
Industry semantic lexicon: Compile more than 30,000 core terms with definitions, synonyms, abbreviations, and application contexts. Map terms to knowledge graph entities so that models can locate relevant business resources when interpreting natural language.
Dataset optimization: Standardize generated Q&A and lexicons, remove semantic conflicts and ambiguous wording, and classify content by business domain, including geology, engineering, safety, and management. Store data using standard JSON Schema definitions for direct loading through fine-tuning interfaces, ensuring usability and compatibility.
1. Five-Dimensional Business Ontology Modeling
Based on business coordinate theory and centered on the Minimum Business Unit (MBU), this approach describes oil and gas business logic across the object, business, work, discipline, and business-process domains. The ontology precisely represents spatial, temporal, and logical relationships between entities, providing a common semantic backbone for all three dataset categories and closely connecting data with business operations.
Figure 2. Five-Dimensional Business Ontology Modeling
2. Multi-Source Heterogeneous Data Integration
Combine traditional ETL tools with modern data lake architectures to establish common description structures and ingestion protocols. Format conversion, semantic mapping, and unified interfaces integrate structured, unstructured, time-series, and volumetric data, breaking down silos.
3. Business Contextualization of Data
Raw data exists in many forms and requires business contextualization during and after ingestion. This includes automatically dividing unstructured documents into business-relevant units, extracting business data, automatically generating charts and tables, and processing raw measurements—for example, converting measured depth to true vertical depth. Manage and store all resulting outputs by the minimum Business Nodes of the five-dimensional ontology.
4. Automated Data Quality Assessment
Combine rule engines and machine learning in a dynamic inspection system. Rules validate business logic such as “tubing pressure > casing pressure,” while isolation forests and the three-sigma rule identify anomalies. Quantified quality scorecards and remediation recommendations support controlled data quality.
5. Knowledge Graph and NLP Integration
Store business knowledge graphs in graph databases such as Neo4j for complex relationship queries and reasoning. Integrate NLP tools such as spaCy and BERT-NER for document knowledge extraction, Q&A generation, and terminology standardization, providing high-quality knowledge inputs for fine-tuning.
6. Distributed Storage and Efficient Retrieval
Combine time-series, vector, and graph databases. Time-series databases store high-frequency sensor data, vector databases store text features for similarity search, and graph databases support knowledge graph queries, enabling efficient storage and rapid retrieval.
Systematic analysis and development based on these methods have produced standardized services, resources, and software tools supporting all three dataset categories, as described below.
1. Development Results
A five-dimensional business description model covers exploration, development, production, storage, and transportation, with supporting maintenance tools. It includes more than 40,000 minimum Business Nodes, 16,000 standard datasets, and 150,000 standard data items, covering core exploration, development, and production domains. Enterprises can extend the model with new nodes and data items. Tools support visual editing, version management, and conflict detection.
2. Customer Value
The model supported the data lake platform of a major oil company, resolving inconsistent standards across disciplines and systems. It automated model construction, migration, business-component configuration, and back-end query and search configuration, supporting multidimensional queries, retrieval, and business encyclopedias. Cross-department data collaboration efficiency improved by more than 60%, and query response times fell from minutes to seconds.
1. Development Results
An ingestion–cleansing–standardization–inspection–archiving–monitoring framework was established, with quality inspection tools spanning six major categories and 58 subcategories. It supports structured data such as database tables and CSV files, unstructured data such as technical documents and core images, and time-series data such as sensor readings and production curves. More than 2,000 industry-specific inspection rules support customization, batch import, and automated execution, alongside quality reporting, issue tracing, and remediation tracking.
2. Customer Value
The framework has been deployed at three major oil companies. One company used it to cleanse and ingest 140 million structured exploration records and 2.6 million unstructured text and image documents, establishing unique, consistent, complete, and trustworthy management of historical data assets. A unified exploration data asset management system centered on the data lake reduced the quality inspection cycle by more than 50%.
1. Development Results
A graph-based Q&A repository covers more than 40,000 Business Nodes and 15 question types, including definitions, procedures, causality, and comparisons, with more than 500,000 high-quality pairs. Management tools support batch generation, human review, classification, archiving, version iteration, rapid retrieval, semantic similarity matching, and automatic updates.
2. Customer Value
The repository supplied core fine-tuning data for an oil company, increasing professional Q&A accuracy from 30% to more than 80%. It supports questions over actual oil and gas data, comparative analysis, and automated diagnostic analysis. The average time for staff to obtain professional knowledge fell from 30 minutes to under five minutes.
1. Development Results
Domestic and international standards, including SY/T, GB, and API, were consolidated into a repository of more than 3,000 core business rules covering exploration and development, production safety, reserves calculation, and equipment maintenance. Intelligent tools automatically extract rules from technical documents, standards, and expert knowledge into standardized expressions. They integrate with governance tools and business systems through more than 100 standard API interfaces.
2. Customer Value
An oil company used these resources to automate generation and validation of drilling, completion, and reserves reports, reducing preparation time from five days to one day, substantially improving validation accuracy, and eliminating 80% of repetitive manual work. In production compliance checks, the rule engine automatically identified violations, delivering a threefold improvement in hazard detection efficiency and reducing safety risks.
1. Development Results
The graph contains more than 100,000 entities and 500,000 relationships, covering over 20 entity types, including geological bodies, wells, equipment, traps, and structures, and 15 relationship types, such as “belongs to,” “located in,” “causes,” “resolves,” and “references.” Its management platform supports visual editing, entity and relationship queries, path analysis, inference, and dynamic updates, with OSDU service architecture compatibility.
2. Customer Value
A unified knowledge hub for an oil company connected multi-source data through business relationships, improving linked-data query efficiency by 70%. It enabled automatic aggregation of small-model training data and structured representation of large-model knowledge, reducing small-model data preparation cycles by 60% and improving large-model domain semantic understanding by 50%. Graph-based identification of related data and resources also substantially improved cross-department collaboration.
1. Development Results
JuraML, a platform for building small-model training datasets, integrates ingestion, feature extraction, granularity alignment, automated labeling, dataset construction, and quality evaluation. It includes more than 20 specialized feature extraction algorithms, supports alignment across time, depth, and stratigraphy, and exports datasets compatible with TensorFlow and PyTorch, supporting rapid model construction, reuse, and deployment.
2. Customer Value
Deployments at a shale gas field and an offshore oilfield extracted eight categories of features for applications including reservoir prediction, automated wireline and mud-log interpretation, and drilling incident early warning, supporting small-model training for 15 intelligent scenarios. Reservoir identification accuracy increased from 30% to 70%, while well-log interpretation time fell from eight hours to 40 minutes, significantly improving practical deployment outcomes.
High-quality datasets underpin digital and intelligent transformation in oil and gas. This study systematically classifies datasets for three major applications—business-system support, AI small-model training, and industry large-model fine-tuning—and proposes tailored construction methods, management mechanisms, and quality assurance strategies.
The research indicates that data governance must advance beyond cleansing toward semantic, object-oriented, and knowledge-based management. Guided by Five-Dimensional Business Ontology Modeling, the integration of OSDU open architecture, feature stores, knowledge graphs, and AI generation technologies can establish a standardized data ecosystem that is continuously updated and accessible to intelligent applications, providing a fundamental path to intelligent transformation across the oil and gas value chain.
This approach is essential to realizing the value of data and enabling AI to achieve its full potential in the complex, high-risk, and capital-intensive oil and gas sector.
References
[1] National Energy Administration of the People’s Republic of China. SY/T 6025-2012: Specification for Oil and Gas Reserves Data Item Names [S]. Beijing: Petroleum Industry Press, 2012.
[2] Patidar S. Challenges and Solutions While Adopting OSDU in Oil & Gas [J]. International Journal of Financial Management Research, 2023, 11(3): 45–56.
[3] Wang Jian, Li Yang, Zhang Lei. Knowledge-Graph-Based Intelligent Decision Support for Energy Consumption Anomalies in Oilfield Gathering, Transportation, and Water Injection Systems [J]. Journal of China University of Petroleum (Edition of Natural Science), 2025, 49(2): 121–130.
[4] Dutta S, Roy A, Sen S. AI/ML Method for Seismic Well Tie Support on the OSDU Platform: Predicting Missing Wireline and Checkshot Data [J]. SPE Journal, 2025, 30(2): 897–912.
[5] Institute of Electrical and Electronics Engineers. IEEE 3821:2024 Standard for Digital Intelligent Comprehensive Energy Service Station Data Management System [S]. New York: IEEE Standards Association, 2024.
[6] CNPC Logging Technology Services Co., Ltd. Construction and Application of High-Quality Geophysical Well-Logging Datasets [R]. Xi’an, 2025.
[7] Max180123. Chinese-PetroEngineering-Data-Distill-From-R1 [DB/OL]. Hugging Face Datasets, 2025-08-27.
[8] Chen Hongzhi, Lin Xiufeng. Application of Large Language Models in Exploration and Development Data Governance [J]. Computer Technology and Development, 2024, 34(10): 134–139.
[9] Wang Lina, Zhang Wei, Li Qiang. Modular AI Training and High-Quality Dataset Adaptation Technologies for Oil and Gas [J]. Oil Drilling & Production Technology, 2025, 47(4): 512–520.
Quick Links
About Us