BACKGROUND OF THE INVENTION

THIS invention relates to a method of, and system for, predicting strata- related risk in an underground environment.

In an underground mining environment, the possible occurrence of strata failure (e.g. as a result of cavities) when expanding into unmined areas provides a significant health risk to people working underground and can also have a detrimental impact on the operations of the mine.

Currently, strata conditions in future mining areas are forecasted by technical personnel, using a range of tools and methodologies. These include geomechanical assessments, numerical based models, and performing historical back analysis on previous mining areas. Areas that have the potential for poor strata conditions to occur can be identified ahead of mining, such that they can be communicated and assessed in a risk-based approach, allowing controls to be implemented where required to ensure that the resulting risk is as low as reasonably achievable. These forecasting methods are however all performed manually by a technical person, which means that it is time-consuming to perform, and it is subject to human error. In a high- risk environment such as underground mines, such an error could lead to a loss in human lives and also have a detrimental impact on mining productivity.

In certain underground mining environments, such as in longwall mining operations, strata control can also be complex, with numerous factors affecting the behaviour of an active mining face. Some factors are readily quantifiable and manageable, while others are more difficult. There is currently no ‘one size fits all’ approach to understanding and quantify risk due to poor strata conditions on a longwall face.

The Inventors wish to address at least some of the issues mentioned above.

SUMMARY OF THE INVENTION

In accordance with a first aspect of the invention there is provided a method of predicting strata-related risk in an underground mining environment, wherein the method includes:

(a) obtaining geological data which relates to a particular unmined area in an underground mine; and

(b) utilising a risk prediction module, which includes a machine learning model, in order to predict a likelihood or risk of a strata-related event occurring in the particular unmined area, by using the geological data as input to the machine learning model, wherein the machine learning model has been trained using data related to already-mined areas.

A “module”, in the context of the specification, includes an identifiable portion of code, computational or executable instructions, or a computational object to achieve a particular function, operation, processing, or procedure. A module may be implemented in software, hardware or a combination of software and hardware. Furthermore, modules need not necessarily be consolidated into one device.

The method may include obtaining operational data which relates to the particular unmined area in the underground mine. Step (b) may include using the geological data and the operational data as inputs to the machine learning model, in order to predict the likelihood or risk of the strata-related event occurring in the particular unmined area.

The strata-related event may be a cavity event. The method may more specifically be for predicting a cavity event in a particular mining panel which is at least partially unmined. The predictions may be applicable to near real time during operations or as part of operational planning and scheduling, further in the future.

The underground mine may be an underground longwall mine and the mining panel may be a longwall mining panel. The method may however be applicable to any form of underground operations with similar strata event risks.

The method may include dividing, by using a processor, the mining panel into a grid which consists of a plurality of cells. Step (b) may then include using the geological data and the operational data as inputs to the machine learning model, in order to predict the likelihood or risk of the strata- related event occurring in each cell.

The geological data may include spatial geological data which indicates the location of one or more geological faults/structures. The method may include determining, by using a processor, the relative positions of these one or more geological faults/structures in relation to the cells. The data on these relative positions may form part of the geological data used as input to the machine learning model.

The operational data may include any one or more of the following: data related to a cut height; data related to a coal roof beam thickness; data related to a cut horizon; equipment position data; mining progress data; data on leg pressure measurements of roof supports located in the underground mining environment; data from equipment sensors; and data related to strata-related production delays. The method may include training the machine learning model using geotechnical data and/or geological data related to already-mined areas.

Step (b) may include: predicting a likelihood of a strata-related event occurring in the unmined area, by using the geological data and the operational data which relate to the particular unmined area as inputs to the machine learning model, in order to predict the likelihood; and/or predicting a consequence should a strata-related event occur in the unmined area, wherein the consequence includes an indication of an estimated operational downtime as a result of the event occurring.

The method may include comparing, by using a processor, recommended/obtained operational data which relates to the particular unmined area in the underground mine with operational data obtained from already-mined areas, in order to predict a likelihood or risk of a strata-related event occurring in the particular unmined area. Recommended/optimised operational data may be obtained from the machine learning model. Planned operational data may be obtained from direct user input.

In accordance with a second aspect of the invention there is provided a prediction system for predicting strata-related risk in an underground mining environment, wherein the system includes a risk prediction module which is configured to receive/obtain geological data which relates to a particular unmined area in an underground mine and utilise the geological data as input to a machine learning model which is configured to predict a likelihood or risk of a strata-related event occurring in the particular unmined area, by using the geological data, wherein the machine learning model has been trained using data related to already-mined areas.

The risk prediction module may be configured to: receive/obtain operational data which relates to the particular unmined area in the underground mine, and utilise the geological data and the operational data as inputs to the machine learning model, in order to predict the likelihood or risk of the strata-related event occurring in the particular unmined area.

The risk prediction module may be configured to compare recommended/obtained operational data which relates to the particular unmined area in the underground mine with operational data obtained from already-mined areas, in order to predict a likelihood or risk of a strata-related event occurring in the particular unmined area. The recommended operational data may be obtained from the machine learning model. Planned operational data may be obtained from direct user input.

The strata-related event may be a cavity event.

The system may be for predicting a cavity event in a particular mining panel which is at least partially unmined. The predictions may be applicable to near real time during operations or as part of operational planning and scheduling, further in the future.

The underground mine may be an underground longwall mine and the mining panel may be a longwall mining panel. The method may however be applicable to any form of underground operations with similar strata event risks.

The risk prediction module may be configured to: divide the mining panel into a grid which consists of a plurality of cells; and use the geological data and the operational data as inputs to the machine learning model, in order to predict the likelihood or risk of the strata-related event occurring in each cell.

The geological data may include spatial geological data which indicates the location of one or more geological faults/structures. The risk prediction module may be configured to determine the relative positions of these one or more geological faults/structures in relation to the cells, and wherein data on these relative positions form part of the geological data used as input to the machine learning model.

The operational data may include any one or more of the following: data related to a cut height; data related to a coal roof beam thickness; data related to a cut horizon; equipment position data; mining progress data; data on leg pressure measurements of roof supports located in the underground mining environment; data from equipment sensors; and data related to strata-related production delays.

The system may include sensors for obtaining at least some of the operational data.

The system may include a user interface on which the predicted likelihood or risk is displayed (e.g. a graphical user interface displayed on a display screen.

The risk prediction module may be configured to train the machine learning model using geotechnical data and/or geological data obtained in relation to already-mined areas.

The risk prediction module may be configured to: predict a likelihood of a strata-related event occurring in the unmined area, by using the geological data and the operational data which relate to the particular unmined area as inputs to the machine learning model, in order to predict the likelihood; and/or. predict a consequence should a strata-related event occur in the unmined area, wherein the consequence includes an indication of an estimated operational downtime as a result of the event occurring.

In accordance with a third aspect of the invention there is provided a method of predicting a certain event/target variable, wherein the method includes:

(a) obtaining geological data which relates to a particular area; and

(b) utilising a prediction module, which includes a machine learning model, in order to predict (i) a likelihood of the event occurring or (ii) the target variable, in the particular area, by using the geological data as input to the machine learning model, wherein the machine learning model has been trained using historical data related to other areas.

In step (a), the particular area may be an area yet to be explored or exploited, while the historical data may relate to areas already explored or exploited.

The event may be a strata event or strata-related event (e.g. a cavity in the ground).

Every one of the features described in respect of the first aspect of the invention also applies to this third aspect of the invention. In other words, the methodology may have application outside of an underground mining environment.

Step (b) may therefore include: predicting a likelihood of an event occurring in an area yet to be explored or exploited, by using geological data and the operational data which relate to the particular area yet to be explored or exploited, as inputs to the machine learning model, in order to predict the likelihood; and/or. predicting a consequence should an event occur in the area yet to be explored or exploited, wherein the consequence includes an indication of an estimated operational downtime as a result of the event occurring. In accordance with a fourth aspect of the invention there is provided a prediction system for predicting a certain event/target variable, wherein the system includes a prediction module which is configured to receive/obtain geological data which relates to a particular area and utilise the geological data as input to a machine learning model which is configured to predict (i) a likelihood of the event occurring or (ii) the target variable, in the particular area, by using the geological data, wherein the machine learning model has been trained using historical data related to other areas.

In particular area may be an area yet to be explored or exploited, while the historical data may relate to areas already explored or exploited.

The event may be a strata event or strata-related event (e.g. a cavity in the ground).

Every one of the features described in respect of the second aspect of the invention also applies to this fourth aspect of the invention. In other words, the system may have application outside of an underground mining environment.

The risk prediction module may therefore be configured to: predict a likelihood of an event occurring in an area yet to be explored or exploited, by using geological data and the operational data which relate to the particular area yet to be explored or exploited, as inputs to the machine learning model, in order to predict the likelihood; and/or. predicting a consequence should an event occur in the area yet to be explored or exploited, wherein the consequence includes an indication of an estimated operational downtime as a result of the event occurring.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings will now be described, by way of example, with reference to the accompanying drawings. In the drawings: Figure 1a shows a schematic layout of mining panels in an underground longwall mine, which have been divided by a system, in accordance with the invention, into blocks to form a grid, and whereby two of the panels relate to already mined areas where blocks are marked as either (i) “clear” to indicate that no strata event has occurred in that block or (ii) with an “X” to indicate that a strata event has occurred in that block, while a third panel relates to strata event predictions (predicted by the system) in an unmined area, where darker areas indicate a higher risk for a strata event, compared to the lighter areas;

Figure 1 b shows an enlarged view of an example of a portion of an already mined mining panel shown in Figure 1 a which contains 75 blocks, where the blocks have been marked by the system as either (i) “clear” to indicate that no strata event has occurred in that block or (ii) with an “X” to indicate that a strata event has occurred in that block;

Figure 2 shows a schematic layout of where known geological faults/structures have, through the use of the system in accordance with the invention, been superimposed/aggregated over the locations of three mining panels which also illustrates predicted strata events and historical strata events on a block level basis;

Figure 3 shows a schematic layout of where known geological faults/structures have, through the use of the system in accordance with the invention, been superimposed/aggregated over the locations of a portion of a mining panel 100;

Figure 4 shows a schematic layout of the mining panels illustrated in Figure 1 , where spatial and temporal data have been integrated to indicate a cut height at the various locations of the individual blocks in which the panels are divided/sectioned;

Figure 5 shows a high-level flow diagram which illustrates a data flow of the system in accordance with the invention; Figure 6a shows a comparison between model predictions and actual performance from real mining activities, and how true positive and true negative scores are calculated, in order to measure the model ‘accuracy’ or performance;

Figure 6b shows a simplified visual layout of the predictions and actual performance shown in Figure 6a, where the areas/locations where the actual results differ from the predicted results are clearly indicated;

Figure 7 shows a flow diagram of a model training pipeline/process flow;

Figure 8 shows a virtual illustration of an example of training dataset monitoring;

Figure 9 shows a virtual and interactive representation of the mining panels, divided into blocks (similar to Figure 1 a);

Figure 10 shows a virtual illustration which illustrates panels with associated strata event likelihood predictions on a block level, versus the actual results from mined areas;

Figure 11 shows an example where operational data and geological data are combined;

Figure 12 shows an example of a few operational parameters which can be used by the system in accordance with the invention, whereby the operational parameters are divided into (i) actual operating parameter values (mined areas) and (ii) recommended operating parameter values, and whereby a contribution to a model’s risk predictions is also illustrated;

Figure 13 shows a visual comparison between model predictions of the system and actual mining results - in this case referring to consequence predictions and actual consequence results in the form of operational downtime;

Figure 14 shows a feature provided by the system, in accordance with the invention, where users are able to compare and analyse specific areas or blocks, for multiple panels or the same panel via a user interface; Figure 15 shows a table which sets out operational data which is used by the system in accordance with the invention;

Figure 16 shows a table which sets out a collection of general geological and operational variables used in predictive models (machinelearning models) which are incorporated/used in the present invention, whereby these model ‘features’ are used to train the models and explain the predictions;

Figure 17 shows a table which summarises data relevant to a machinelearning model which is implemented by the system in accordance with the invention;

Figure 18 shows a planned data capture screen where future operational parameters can be specified;

Figure 19 shows a validation feedback screen where users can specify already mined areas on longwall panels that need to be excluded from model training; and

Figure 20 shows a confidence measure screen where a confidence measure for model predictions is illustrated.

DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention relates to a system for, and method of, predicting strata-related risk in an underground mining environment in respect of certain unmined areas. The strata-related risk relates to (i) the likelihood of a particular strata event taking place in an unmined area and (ii) the consequence/severity (e.g. in operational downtime) should a particular strata event (also referred to as a “target variable”) occur at specific areas within an underground mine.

The invention will now specifically be described in relation to an underground longwall mine (e.g. an underground longwall coal mine). It should however be appreciated that the invention can also be implemented in other underground environments, such as other types of underground mines. It should also be noted that the invention can also be applied in other applications/environments where the system is configured to define a target variable associated with certain geological data. The present invention can therefore also have application outside the underground mining environment.

Underground longwall mines are typically found in coal mining where a longwall panel (typically 3km-4km long and about 200-400m wide) is mined in single slices (e.g. about 0.6m-1 m thick per slice). Reference is in this regard made to Figure 1 a where reference numerals 100.1 -100.3 indicate three longwall panels (collectively hereinafter referred to as the “panels 100”).

The system 10 in accordance with the invention includes one or more processors and suitable software (i.e. computer-readable instructions) to set out, divide and analyse these panels 100 in a manner as set out below. More specifically, the panels 100 of certain unmined areas (i.e. panels 100 still to be mined) are divided as set out below.

The 4 corner coordinates (see reference numerals 12.1 -12.4) of each mining panel 100 form the basis for calculating the shape, width and length of the mining panels 100. Coded functions of the system 10 calculate the length and the width of the panel using the panel corner coordinates and mine design parameters (e.g. the intended length and width of the panel and the number of roof support units that will be used (depending on the panel width)).

The panels 100 are divided up by the system 10 into cells . Each cell represents an average shear or drum width (typically 1 m along a length of the panel) and a roof support width (typically 2.05m along a width of the panel). Each cell therefore represents a theoretical roof support unit and shearer position expected in normal operations. This then provides a panel grid made up of about 1 m by about 2m cells/shields (when viewed from above). It should however be appreciated that the specific dimensions of these cells can vary and are therefore not fixed. Based on the 4 corner coordinates per panel 100 and the 1 m x 2m grid, x and y coordinates are assigned by the system 10 to each corner of the grid’s cells/shields. These coordinates are then used by the system 10 to match the nearest spatial data (discussed later) to each particular cell.

The 1 m x 2m cells/shields in the panel grid effectively represent theoretical mining progress, and are used together with actual mining progress (e.g. in meters) and actual equipment positions, which are captured and calculated from production and delay data capture systems and equipment sensors.

The calculated cell level panel grid forms the foundation of the integration of spatial geological data with temporal production data, which will be described further below.

The system 10 is configured to match the “nearest” coordinates (e.g. using a “nearest neighbour” method) in the panel grid with relevant spatial geological data points (discussed further below), in order to allocate geological variables to positions on the panel 100. In a similar manner, the system 10 can also be used to allocate coordinates of geological faults or structures to the coordinates of the panel grid.

Reference is in this regard made to Figure 2 which shows an example of where the system 10 has superimposed/aggregated the location of known geological faults/structures (generally indicated by reference numeral 200) over the location of the panels 100.

Considering the size of mining panels 100 and the spacing of observed spatial geological inputs (i.e. data on geological faults or structures 200), the cells are grouped by the system 10 into larger blocks 14 for analysis and visualisation. Block sizes used thus far are 10m by 10m and 20m by 20m. However, various other sizes can also be used. These are typically configured based on the meters along the panel length and the number of roof support units required per block. The system 10 is configured to aggregate actual and recommended operational data (discussed further below) to the block sizes from cell level data.

Reference is in this regard specifically made to Figure 3 which illustrates where known geological faults or structures 200 have been superimposed/aggregated over the location of a portion of a mining panel 100.

Figure 4 represents an example of the integration of operating parameters and geological data. In this instance, it is a view of actual and recommended/planned cut height values across the longwall panels, for the mined and unmined areas. The integration of spatial and temporal data allows the system 10 to create these views for any of the operating parameters.

Geological and Geotechnical Data Inputs:

Using generally available geological modelling tools (e.g. Geovia Minex ^TM or MAPTEK VULCAN), borehole data/records is converted into three- dimensional, spatial geological data. Borehole data/records is a well-known term used in the industry and is typically produced from a geologist's or surveyor's observations of the rock core extracted from the ground and typically include locality and lithological descriptions with depth and thickness. Geophysical logs may also be noted from on-site measurements.

Well-known geostatistical methods are employed to calculate relationships between boreholes to construct a block, grid or mesh representation of the relevant composite intervals and variables (hereinafter referred to the “geological data”) for export, at a scale relevant to the solution (e.g. 5m x 5m x 0.2m blocks) (see Figure 16, reference numeral 60). In other words, data from the borehole logs are used to create exportable data in a block, grid or mesh format at a scale relevant for the solution. This is done using the geological modelling software. The system 10 matches this supplied data to the nearest coordinates in the dataset in order to apply the values to the cells (and aggregated blocks 14).

It is up to the relevant technical subject matter experts to identify and specify the most appropriate general geological data/inputs to calculate from the borehole data.

The geological data is represented in an x, y, z format (in terms of geographic location). The system 10 typically performs an integration of this spatial geological data with temporal operating data, that forms a critical input to a machine learning model and a user interface application, which are described in more detail further below.

In Figure 16, rows 10 to 56 (see reference numeral 60) refers to the geological attributes/data (e.g. typically defined by subject matter experts) and used in the model training and predictions.

Operational Data

Data is sourced from various systems and equipment that measure aspects related to mining production, to measure and calculate mining progress, operating parameters, and cavity events.

The process effectively creates a snapshot of the past operations at a roof support and shear pass level, and provides this data as a temporal dataset that is joined with the geospatial dataset to form the foundation of the input to the machine-learning model and the user interface. Reference is in this regard made to Figure 5 which illustrates a high-level data flow depiction.

The integration of geological and operating data allows for the calculation of critical operating parameters, such as extraction height, also referred to as the cut horizon, and the coal left behind in the seam, also called the coal roof beam. This integration also allows for the calculation of the recommended operating parameters in the unmined areas. Equipment Position

Data (hereinafter referred to as “equipment position data”) from an automated horizon control system is used to calculate the relative positions during mining of the roof support units and the shearer by integrating shearer odometry and a horizon control profile, based on relative time stamps. The shearer odometry data indicates the position of the shearer along the face of the mining panel, while the profile data identifies the start and end of profiles or shears.

Mining Progress

The number of shears or metres retreat data from a production and delay accounting system of a mine, is used to calculate how mining progressed in terms of the length of the mining panel. This is measured in metres and can be captured manually or retrieved from underground sensor data.

The mining progress data is integrated with the equipment position data to enrich the equipment position data along the length of the mining panel. This provides a virtual representation of the mining progress throughout the panels 100 and supports the functions of integrating the operating parameters and the geospatial data. This is done in preparation for the integration of the temporal operating data with the spatial geological data.

Consequence

All production delays are captured in a Production and Delay Accounting system. The delay classification is usually manually captured and can be measured in minutes or hours. Data from the production stoppages related to strata events are used to calculate one aspect of the severity or consequence. This data from the delay accounting system is aligned with the identified strata events (see Cavity Events below) in order to assign a production stoppage duration to a cavity event.

The combination of cavity events and the consequence (duration) assigned to the events are used as the target variables for training the consequence machine-learning model(s). Cut Heights

Cut heights for the relative equipment positions can be calculated by integrating the shearer drum height sensor data from the automated horizon control system and data from the historian that explains what state or method the shearer arms (aka Booms) were in. The leading and lagging drums need to be identified and the sensor data aligned to the same roof support unit to match the top and bottom cut heights, so that the total cut height can be calculated.

The shearer arm or Boom states will indicate where cut heights may not be accurate, and need to be assumed or calculated from previous shears.

Cut heights are very important operating data points that may impact the behaviour of the underground mining conditions.

Cavity Events

Knowing where the roof support units were located in the panel 100, at what point in time, allows operating parameters or other related data points to be assigned to relative areas of the mining panel 100.

One of these related data points is the maximum leg pressure of a roof support unit while in a relative position. These leg pressure measurements for each roof support unit are then used to derive the existence or occurrence of strata risk events or cavity events. If the pressure is below a configured value (bar) for a predefined number of consecutive roof support units and consecutive shears, that grouping is identified as a strata risk event or cavity. The grouping is used, instead of a single pressure reading for a single roof support unit, to negotiate bad data or equipment malfunction.

The cavity events are the target variables used by the machine learning model for training of the cavity likelihood predictions. Cut Floor

For the purposes of understand the various operating factors that can affect the risk of strata events or cavities occurring, the position in seam is captured to calculate the cut floor and assign these values to the relevant equipment position data. To better understand the conditions and mining activities, it is important to understand whether the mining equipment are mining below a particular seam floor, on top of the seam floor or above the seam floor. This cut floor is captured in the system 10 as a thickness or distance from the seam floor.

Integration with Geological data

Some operating parameters require integration with the geological data. By knowing the cut height and the position relative to the seam floor at each roof support unit and shear position, the cut horizon can be calculated, meaning the actual height above the coal seam floor that is being mined.

When also integrating the geological data related to the seam thickness, the coal roof beam thickness can be calculated.

- Cut Horizon = Cut Height - Floor Cut

Coal Roof Beam = Seam thickness - Cut Horizon

The coal roof beam has proven to be a critical consideration in the prediction of cavity likelihood.

All operational data explained above is used in the training of the machine learning model(s), along with the relevant geological data also discussed above.

Planned or Recommended Operating Parameters

Users have the ability to enter planned operational data for unmined regions. The planned data in combination with the geological data may form the basis for the initial risk predictions. The model can provide recommended operating parameters for the unmined regions that support the specific site’s agreed operating strategy.

Operating Scenario Analysis

Using a minimum coal roof beam value along with the calculated panel blocks (discussed earlier) and the supplied coal seam thickness (geological input), the model calculates the recommended operating parameters at a 100m length level, for the full width of the panel. These 100m zones are configurable and can be changed according to the business requirements and processes.

The model aims to maximise coal extraction and minimize stone floor (see Figure 17) with the recommended operating parameters.

Within the 100m zones, the minimum coal seam thickness is established from the geological data points assigned to the panel blocks. Using this minimum coal seam value along with the site’s agreed minimum coal roof beam value, the model calculates the highest ‘recommended’ cut height value that results in the minimum ‘recommended’ cut floor at the 100m zone level, thus providing the ‘recommended’ cut height and ‘recommended’ cut floor / stone floor operating parameters. Having the coal seam thickness and referring to the formula in the previous section, this is sufficient to calculate the ‘recommended’ cut horizon.

‘Recommended’ Cut Horizon = Seam Thickness - ‘minimum’ Coal Roof Beam

Machine Learning

The table illustrated in Figure 17 includes a summary of important geological data which can be taken into account by the system 10 as inputs to the machine-learning model. The data under “Input Model Dataset” can collectively also be referred to as the geospatial dataset. Intended use

During coal mining, cavity formations above a mining area are formed and may affect production. The present system includes a risk prediction module which incorporates a machine-learning model. The model combines existing geological and mining/operational data to predict strata risks, likelihood and consequence, along an unmined area.

These risk estimations and the risk predictors provide insights for potential upcoming strata hazards in the mine. The model predictions can then be used during operations, for planning purposes, risk assessment and planning regarding mitigating actions.

Model Type

The machine-learning model can be/include a classification type model which estimates the likelihood of a strata event, or cavity, in underground mining.

Techniques which can be implemented by the machine-learning model include:

• An oversampling methodology for imbalanced data: SMOTE

• A Binary Variable classification - Likelihood estimation: XGBoost

• Likelihood estimation predictors, such as Shapley Values

In one example, a Boost algorithm was used as the machine learning algorithm in order to generate likelihood scores for an unmined region.

Model Validation

When new actual mining data (from a newly mined region) is received (more specifically operating parameter data, which is typically a combination of automatically sourced data (e.g. from sensors) and/or manually sourced data (e.g. from surveys)), the machine-learning model is scored using the actual events and consequences, along with operating parameters (e.g. cut heights, stone floor and coal roof beam). The estimated risk scores (calculated before the mining took place in a particular area) are compared versus the actual events (model performance validation). A model performance validation output is then fed into an application database for front-end visualisation. In other words, the results or data from mining activities are compared to the model predictions to calculate model performance. The data used for, and generated from, the validation is included in the application database in order to visualize it (see Figures 6a&b).

Model Refresh / Re-train

The machine-learning model is retrained with the inclusion of the new actuals (i.e. the actual mining data received as discussed above) in the model training process/dataset. Users have the ability to exclude data from mined areas of a panel to preserve data quality for model training to ensure optimum model performance, using the user interface. A model variable importance output is fed into the application database for front-end visualisation. The variable importance that may change compared to a previous execution are also provided for user analysis and supporting documentation (e.g. via a graphical user interface).

Model Scoring

The refreshed/re-trained model is scored for an unmined region based on the plan data provided or recommended data for a particular operating scenario. The scoring outputs are fed into the application database for frontend visualisation. It should also be noted that 2 or 3 additional scenarios with different operational parameters may possibly be included as pre-calculated scenarios for users to assess alternative options, as part of the scenario analysis functionality.

Risk/Likelihood/Consequence drivers (predictors)

A Shapley values technique is applied to rank the contribution of each predictor to the estimated risk / likelihood. These predictors are illustrated in some of the rows of the table shown in Figure 16. Each of those data points are used as predictors in the model training and scoring. Shapley Values output is fed into the application database for front-end visualisation.

Relevant Factors

Important factors which the machine-learning model takes into account are: a) Geological attributes which are mapped/matched at geo-location level using coordinates (x,y,z) as matching keys (also referred to as the “geological data”); b) Actual operating parameters provided by equipment sensors, surveys and engineering reports for model training (also referred to as “operational data”); c) Planned/Recommended operating parameters for an unmined region model scoring. d) Relevant geological interval attributes which are estimated above the actual & recommended operating cut horizon for model training & scoring purposes respectively.

The model can be regularly re-trained with the inclusion of new actual operational data to enable the machine-learning algorithm to learn from the newly mined geological conditions and the interactions with the operating parameters. The predictors' strength of the most recent model is estimated by the refreshed model variable importance.

Evaluation Factors for Evaluating Model Performance

The following factors can typically be taken into account, when evaluating model performance: a) Predictor categories include: Geo-attributes, Operating Parameters, Geo-interval above Cut Horizon attributes b) Target variable: Strata Event Likelihood and Consequence score estimations.

Evaluation facto rs/metrics: True Positive & True Negative Rates. When new actual operational data is received, the model is scored with the actual operating parameters and a customized (based on the business/domain requirements) confusion matrix is generated providing the True Positive & True Negative rate metrics for the likelihood model. The customized confusion matrix applies a level of domain-specific criteria to calculate the likelihood model performance. These criteria are the ‘business rules’ applied when calculating model performance. These are guidelines or agreements on how to measure model performance, due to the ‘non-specific’ nature of the data and predictions. This can refer to the location of a predicted high risk block relative to an actual cavity.

When new actual operational data is received, the model is scored with the newly mined data using the actual operating parameters, and the estimated risk score is compared versus the actual event at block level. Each block is characterized as True Positive, True Negative, False Positive or False Negative case taking into consideration business/domain rules.

When new actual operational data is received, the model is scored with the actual operating parameters and a customized (based on the business/domain requirements) accuracy matrix is generated providing the Strict and Loose Accuracy metrics for the Consequence model. Strict Accuracy refers to matching predictions vs actuals, and Loose accuracy refers to the prediction vs actual differing by one Consequence category level. The customized matrix applies a level of domain-specific criteria to calculate the Consequence model performance. These criteria are the ‘business rules’ applied when calculating model performance. These are guidelines or agreements on how to measure model performance, due to the ‘non-specific’ nature of the data and predictions. This can refer to the location of a predicted high risk block relative to an actual cavity.

A model performance / validation output is fed into the risk prediction module and aggregated True Positive and True Negative Rates are calculated along with an extensive confusion matrix for a pre-defined validation region. Reference is in this regard made to Figures 6a, 6b and 13 which visualises the actual cavities/results from actual mining with the model predictions when mining took place through that area. The model predictions may change for each retrain depending on the input data and how much of the same data has been used for model training before. In Figures 6a&b:

Reference sign 400 refers to actual results from mining (where “x” refers to a cavity);

Reference sign 402 refers to likelihood predictions;

Reference sign 404 refers to bordered blocks which indicate differences between likelihood predictions and actual results; and

Reference sign 406 refers to actual results from mining which match likelihood predictions;

Reference sign 410 refers to consequence predictions if a cavity event were to occur; and

Reference sign 411 refers to actual cavity events where the shading (or colour) of the ‘X’ indicates the actual consequence severity of the cavity. If the shading/colour match the predictions matches the actual consequence from the event, otherwise the prediction does not match the actual.

Approaches to Uncertainty and Variability:

The True Positive / True Negative / Strict Accuracy / Loose Accuracy rates are adapted using block level True Positive / True Negative / False Positive I False Negative and accuracy rules, based on the model risk estimation versus actual events and special geological conditions (i.e. the geological faults, e.g. normal faults and reverse faults): a) Model Likelihood Risk Estimation: Rare, Unlikely, Possible, Likely, Almost Certain b) Model Consequence Risk Estimation: Insignificant, Minor, Moderate, High, Major c) Actual Event: Strata Event, Non-Event, Strata Event Surround d) Geo-conditions: Fault Structures

Prediction Confidence Measure:

The geological and operational data per cell is assessed and compared to the rest of the available data to determine a relative confidence measure for model predictions. The confidence measure is based on how much similar data is available for model training in already mined areas. The ‘similarity’ measure is based on specific geological characteristics that can be used to divide the mining lease into different geological domains. A 10m x 10m block is assigned a confidence measure of Low, Medium or High depending on how much of the already mined area consists of similar characteristics. The confidence measure is also aggregated to a 100m area along the length of the panel. The majority confidence category, ranked based on a count of each category in the 100m area, is assigned to the 100m area.

Reference is in this regard made to Figure 20. In this regard,

Reference sign 418 refers to the confidence category assigned to the selected block (as described above);

Reference sign 420 refers to the aggregated confidence category calculated from the confidence categories per block that make up the associated 100m area (as described above); and

Reference sign 422 is a selected unmined block. Selecting the block conveys information about the confidence category (Low, Medium, High) assigned to the block;

Decision threshold

Subject matter experts can review the model performance and decide if a new model selection is required or whether data should be excluded from model training.

There are cases where ‘lower than expected’ model performance may require intervention through validation feedback functionality: a) Transitional or new geological condition regions: the algorithm needs to receive new actual data in the training dataset to learn and decompose the new strata events mechanisms. b) Operating or equipment malfunctions may cause strata events which are ‘out-of-scope’ for the model predictions and can be excluded from model training using the user interface. c) Where special mitigation actions were applied, and are not captured in the data, should be ignored and can be excluded from model training using the user interface.

Training Datasets

Reference is in this regard made to Figure 7 which illustrates an example of a model training pipeline/process flow.

The training dataset represents all the already mined areas or panels. The model is re-trained I refreshed with the periodic inclusion of new ‘actual data’ from newly mined areas. Thus, the training dataset is increased, and the data profile is changing with the inclusion of new geological profiles / conditions as well as strata event mechanisms.

Data Exclusion

There are strata events caused by equipment or sensor failures or other factors outside the scope of this problem that should be excluded from the model training process. The regions in which this takes places are provided by users (e.g. technical personnel) and are applied to the model through validation feedback such that data relating to these types of failures are effectively excluded from the model training. Reference is in this regard made to Figure 19, which shows an example of a validation feedback screen which allows users to specify already mined areas on longwall panels that need to be excluded from model training for whatever reason (reference numeral 412 shows the rows excluded from future model training, while reference numeral 414 refers to a specific reason which must be provided/selected for excluding the rows from the model training). Excluding irrelevant data is intended to improve predictive model performance over time.

Model Training Dataset monitoring

Since the training dataset is increased gradually and the data profile changes, reports are generated to monitor how the dataset changes with the inclusion of the new ‘actual data’. Within the Extract, Transform, Load (ETL) step phase of the model training, important variables are visualized (heatmap) in the different mining panels 100 that provide the data distributions. In this regard, reference is specifically made to Figure 8 which shows an example of training dataset monitoring.

User Interface

The geological (spatial) and production (temporal) data that result from the data engineering, along with the prediction-related data and insights resulting from the machine learning processes, are modelled and stored for consumption/utilisation.

A graphical user interface is configured to consume/utilise this data and communicate the following information to the users: a) A virtual and interactive representation of the mining panels, divided into blocks (as explained in previous sections) - using open-source mapping libraries (see Figure 9); b) Model predictions - predictions for the likelihood and consequence of cavities/strata events, respectively, and the predictors (reasons) that support the predictions; c) Model validation and SME input - visually comparing model predictions with actual mining and strata events, providing users the ability to exclude events outside the scope of the solution; d) Past mining operations - aggregated operational metrics through mined areas; e) Planned / Recommended operational scenarios - aggregated operational metric recommendations for unmined areas; f) Operational Plan Data - aggregated operational data provided for unmined areas; g) Past cavities - areas that were affected by strata events; and h) Geological structures - faults that may impact cavity likelihood and consequence predictions.

Reference is in this regard made to Figure 10 which illustrates panels 100 showing risk predictions versus the actual results from mined areas (see the units/shells marked with “X” which indicate that a strata event has occurred in a particular cell/shield). In this drawing, reference numeral 71 indicates the results from already-mined areas, while reference numeral 73 refers to predictions for unmined areas. More specifically, reference numeral 70 indicates low risk (i.e. rare), 72 indicates fairly low risk (i.e, unlikely), 74 indicates a medium risk (i.e. likely), 76 indicates high risk (i.e. likely) and 78 indicates extremely high risk (i.e. almost certain).

Figure 11 illustrates an example where the geological fault data is being displayed for analysis.

Figure 12 shows an example of a few operational parameters which can be used by the system 10. The operational parameters are divided into (i) actual operating parameter values (mined areas) 450 and (ii) recommended / planned operating parameter values (prior to mining) 452, and whereby a contribution to a model’s risk predictions is also illustrated 454.

The functionality and visualisation available in the graphical user interface application supports the following: a) Back analyses of mined areas and conditions. b) Analyses of predictions, operational plans / recommendations, and upcoming conditions. c) Mine planning and risk assessment processes associated with underground mining. d) Planned operational data capture that supports more accurate base predictions in the absence of operational recommendations (see Figure 18). The plan data capture feature allows users to specify future operational parameters so that the model can use this data for ‘base’ or initial predictions. This replaces a ‘hardcoded’ assumption based on a minimum coal roof beam that would be used to calculate ‘recommended’ cut height and floor stone through areas of the panels. e) Model validation and user validation feedback to compare predictions with actuals and allow exclusions of areas that should not be used for model training.

From the above it will be clear that the present invention provides an effective way of reducing the risk of unwanted strata events occurring during underground mining operations. As a result, the system also helps to reduce the likelihood of production downtime and loss of human life.