IMT Faridabad Volume III, Special Issue I April 2017 Page 41
A Hybridized Network for Grade Approximation using Neural
and Fuzzy Logic Method
Lalit Negi*, Preeti Ranga∗∗ , Anju Gupta∗∗∗
Abstract
Keywords: Grade approximation,
artificial neural networks,
Genetic Algorithm, Fuzzy
Logic, Coactive neuro-
fuzzy inference system
(CANFIS).
Author correspondence:
The grade approximation is a very essential and money/time
consuming stage has become the major challenge tackled by the
geologists and mining engineers due to the structural difficulties in
mineral ore deposits. The explanation to this problem is given by
several artificial intelligence methods like Artificial Neural
Networks (ANN) and Fuzzy Logic (FL). In this paper “Coactive
Neuro-Fuzzy Inference System” (CANFIS) method is used, which is
a combination of both ANN and FL. In this network, Genetic
algorithm has been employed to standardize the parameters of the
network.
Lalit Negi, M.Tech Student, Department of Electronics Engineering
YMCA University of Science and Technology, Faridabad, India Email: [email protected]
1. Introduction
A standout amongst the most imperative parameters which can majorly affect mining plausibility and its future
administration is review estimation exactness. In this manner, there is an extraordinary affectability on the
techniques which are utilized for hold assessment, since these strategies can have a noteworthy part in the mining
future arranging. Moreover, it is connected as an instrument to recognize the outskirts for financial as well as
non-monetary stores [1]. The estimation is used amid the mining essential stages and it might be reused up to the
finish of the mine exercises. In this manner, the exactness of the estimation techniques has a continuous impact
on the mining venture. A few techniques and techniques have as of now been used with a specific end goal to
expand the exactness of the review or tonnage estimation, for example, geostatistics[2], fake neural systems
(ANN) [3][4][5][6] and fuzzy logic (FL) [7][8][9][10][11][12][13]. Clearly, geostatistics is a standout amongst
the most predominant systems for review estimation. A large portion of the regular geostatistical strategies, for
example, Krigning [1] [2] [14] are straight estimators that limit the difference. Now and again in which the
review appropriation and spatial examples connections are complicated, the ''kriging strategy'' is not generally
ready to give the best answer. In any case, to conquer these issues, some geostatistical recreations have been
proposed, yet each of them has its own particular issues [15]. What's more, two-point based geostatistical
strategies have a low precision which cause to make some compels and confinements [16] [15]. Accordingly, it is
attempted to utilize the nonlinear estimators like ANN to defeat the complex spatial relationship. Amid the
current years, ANN has been utilized more than different techniques [3] [17] [18] [19] [20] [21] [5] [22] [23] [6].
IMT Faridabad Volume III, Special Issue I April 2017 Page 42
2. Methodology
Both ANN and FL are finely evident for a large portion of scientists and we allude the peruser to the accessible
references about ANN and FL reffered to in the last area. Along these lines, from now on the technique will be
illustrated.
*M.Tech Student, Department of Electronics Engineering, YMCA University of Science and Technology, Faridabad, India
** M.Tech Student, Department of Electronics Engineering, YMCA University of Science and Technology, Faridabad, India
***Associate Professor, Department of Electrical Engineering, YMCA University of Science and Technology, Faridabad,
India
Essentially, FL and ANN are the without model and nonlinear estimators that their point is generally
accomplishing a steady and solid model which can legitimize the commotion and instabilities in the mind
boggling information [24]. As per prior dialogs, clearly a few issues, for example, deciding the shape and the
area of participation capacities (MFs) for each fluffy variable are included with FL. The FL productivity
fundamentally relies on upon the estimation of commence and the subsequent parts. Plus, the issues like number
of shrouded layers, number of neurons in each concealed layer, learning rate and force coefficient are
additionally required with ANN demonstrating. Be that as it may, a standout amongst the most vital capacities of
FL is to show the subjective parts of human by utilizing the basic guidelines. Conversely, the ANN likewise have
a few focal points, for example, its capacity of learning and high computational power. Subsequently, it is
conceivable to join the benefits of ANN and FL to improve an apparatus. In any case, Asadi and Tahmasebi
(2011) introduced a thorough review in which a worldwide philosophy for ANN is illustrated. Besides, an
affectability examination on various ANN parameters can be found in their review.
Jang, 1992, 1993 joined both FL and ANN to create a capable handling apparatus, named adaptive neuro-
fuzzy inference system (ANFIS). ANFIS utilizes an ANN learning calculation to set fluffy govern with the
suitable MFs from info and yield information. Really, this strategy is a suitable answer for capacity estimate in
which a half breed learning calculation connected for the shape and the area of MFs [25].
Fig. 1. A multi-layer back propogation
Fig. 2. Fuzzy inference system
IMT Faridabad Volume III, Special Issue I April 2017 Page 43
One of the principle systems used in this review is CANFIS which has a place with a more broad class of
ANFIS. Truth be told, the inclination of CANFIS to alternate ANFIS structures is a result of its capacity to create
multi-output(s) by nonlinear fluffy principles in which both ANN and FIS assume a noteworthy part to achieve a
superior estimation[20] . To be all the more clear, by mix these two strategies, ANN will characterize the FL's
guidelines, on the grounds that the vast majority of the mining and geography conditions (and particularly review
estimation) are blended with uncontrolled ambiguities which make the tenets be hard to characterize.
Subsequently it is viewed as a smart thought to utilize the capacity of ANN to prepare the FL. At the end of the
day, by utilizing this new crossover framework, one can utilize both abilities of FL's capability and ANN's
evaluation angles.
2.1. Genetic Algorithm
The GA was initially presented by Holland (1975). It is an all inclusive technique for settling the
assortment of obliged and unconstrained improvement issues (Holland, 1975). GA can likewise be utilized to
take care of a differences of enhancement issues that are not appropriate for standard improvement calculations,
incorporating issues in which the target capacity is intermittent, non-differentiable, stochastic, or very nonlinear.
A few scientists likewise proposed that worldwide inquiry methods including the GA may keep ANN from
falling into a nearby least. At first, the factors ought to be spoken to by a parallel string which encodes the
parameters of the CANFIS and each chromo-a few (people) comprises of a few qualities which speak to the
system's parameters [15].
In this review, the roulette wheel technique is utilized to decide the following chromosomes with arbitrarily
chose length. Also, it is conceivable to allow to the pervious chromosome to collaborate later on era to wind up
noticeably a more grounded chromo-a few. It ought to likewise be noticed that in this review, we utilized the
rulette-wheel, two-point strategy and limit technique for the hereditary administrators of choice, hybrid and
change, respectively. A while later, the estimations of wellness capacity (which is fundamentally identified with
the contrast between the yield CANFIS–GA and the genuine review) are sorted and afterward the best and the
most noticeably bad chromosomes are distinguished (note that exclusive as well as can be expected hybrid or
change by rating). Hence, the GA with the specified parameters is utilized to get the ideal Structure of CANFIS
(e.g., Number of information neurons, enrollment capacity, learning and energy rate).
IMT Faridabad Volume III, Special Issue I April 2017 Page 44
Fig. 3. The flowchart for grade approximation and the applied methods with their designing details.
3.1 Geological setting
At first, researched Sungun copper porphyry widely and they detailed that this save is a Skarn-sort
mineralization at the contact between the Cretaceous limestone and the granodioritic stock. As per the accessible
components and adjustment pointers, gathered that this store is fundamentally the same as displayed porphyry
Store characters and he con-cluded that this store is one of the porphyry stores.
IMT Faridabad Volume III, Special Issue I April 2017 Page 45
Fig. 4. Simplified geologic map of the Sun gun area
3.2 Information planning
Toward the start of the displaying, the information is standardized which lessens the commotions lastly
prompts a superior expectation. For this point, diverse standardization techniques ought to be tried to enhance the
system preparing. Each of the factors are standardized by applying the accompanying three techniques to locate
the best and exact one.
These connected strategies are as per the following: ''The first information'', ''the standardized information
which is in the scope of [11] by utilizing the most extreme and least dataset'', ''the standardized information by
utilizing the mean and standard deviation of dataset'', lastly ''the standardized information in the scope of [0 1] by
utilizing the accompanying condition'':
The connected information incorporates 156 exploratory boreholes. For review estimation, the directions
are utilized as info factors, and grade property is utilized as yield variable for the individual dataset [18]. In this
paper, the accessible information for both ANN and ANFIS is separated into three and two subsets for CANFIS–
GA organize. For CANFIS– GA, the preparation and testing sets are arbitrarily chosen in the inexact proportion
of 85% and 15% from the database, individually. In any case, in the standard techniques (ANN, ANFIS) which
depend on experimentation, a few parameters ought to be changed amid the demonstrating. Subsequently, the
dataset should be validated to control the ANN and ANFIS execution. The primary subset is the preparation set
by which the system finds an input–output spatial relationship by redundant investigation of the preparation set
(70% of entire information). The second subset is the approval set (_15% of entire information). The distinction
between these two sorts of information appropriations is because of applying the GA to CANFIS. While applying
the GA, there is no compelling reason to return and right the model, in light of the fact that the preparation stage
will be controlled by the GA and the best parameters are put away at every cycle [21]. Likewise, we pick the sets
at similarly divided focuses all through the first information. Table 1 speaks to the rundown insights of the
datasets in which the copper dataset uncovers less change.
IMT Faridabad Volume III, Special Issue I April 2017 Page 46
Fig. 5. Profiles showing the distribution of Cu grades along cross-sections D–D and C–C. Numbers on surface indicate
drill holes
3.3 Modeling
All the specified properties, for example, information determination, ANN's parameters, and so on were
chosen and considered for the displaying. It is ideal to clarify some powerful parameters in both CANFIS and
GA before the displaying.
There are two fluffy MFs: Bell-molded and the Gaussian-formed bends. The ringer formed bend is
somewhat more adaptable, in light of the fact that it has three free parameters to change contrasted with the
Gaussian-molded MF which has just two parameters. Therefore, the ringer formed has been chosen as the
utilized participation work. There are likewise two learning refreshes: on-line realizing, which refreshes the
system in the wake of displaying every model, and cluster realizing, which refreshes it after the introduction of
the whole preparing set. We utilized the last refresh in this review. Besides, we chose the energy and axon as the
learning calculation and the exchange work, separately. Mention that axon exchange capacity is the personality
outline is ordinarily utilized just as a capacity unit, and as it is normal, its yield is equivalent to the info.
Moreover, the force ought to be included as a structure of two fundamental parameters. At the end of the day,
while seeking with the energy part, there are two parameters to be chosen: the progression measure and the force.
One arrangement is to set an incentive for the learning rates by experimentation strategy. As another
arrangement, one can utilize the GA. In fact, we utilized GA for finding the best esteem/number of preparing
components, step size, and energy rate. At long last, there are two variations from accessible CANFIS systems to
IMT Faridabad Volume III, Special Issue I April 2017 Page 47
pick: the Tsukamoto fluffy model which is straightforward and runs quick, while the TSK fluffy model
(otherwise called the Sugeno fluffy model) is by and large more prominent [15].
Since the chromosomes are made out of qualities, the variable ought to be exhibited to GA encoded by
chromosomes. As clarified, there are three parameters which ought to be upgraded; subsequently, every
chromosome (or a hopeful of arrangement) is made out of three qualities (learning rate, energy of the net-work
and the quantity of MFs for each info) which demonstrate the estimations of those parameters. We likewise
utilized haploid chromosomes to present them to the GA. At the initial step, the quantity of people will be created
haphazardly. At that point, wellness capacity is assessed for these people. Next, encoded chromosomes are
sought to boost the wellness work in which the nature of the arrangement is sorted in the wellness esteem and can
be spoken to by the normal expectation precision of the preparation information. In this review, we utilized MSE
(Mean Square Error) as the wellness work which measures normal of the square of blunder. The measure of
blunder is translated by the contrast between the reactions of CANFIS–GA and the genuine (real) grades
In each effectively keep running of the calculation, the wellness capacity is computed by people. In this
segment, it is conceivable that the figured wellness capacity of people may not be suitable for new era
contrasting with the underlying criteria; hence, for this situation the people are chosen by their wellness capacity
to create the posterity which that is without a doubt the consequence of parent recombination. At that point, by
utilizing the transformation administrator (which is a likelihood sort), all the posterity will be changed. Next, the
guardians will be supplanted by the new posterity keeping in mind the end goal to deliver another era [24]. This
basic cycle is the fundamental stride in the GA since rehashing this method and as indicated by gathered criteria,
the best response for streamlining issue would be inferred. Clearly, keeping in mind the end goal to discover the
settings that deliver the most minimal blunder, this type of cycle based enhancement requires the system to be
prepared in a few circumstances.
Essentially, the GA depends on the three parameters: hybrid, transformation and populace. The distinctive
estimations of these parameters are imperative and the GA may be touchy to the diverse qualities. At that point,
the populace size was set from 15 to 60 and the hybrid and change rates may differ to counteract starting up any
conceivable issue and blunder. The scope of the hybrid rate was set in the vicinity of 0.1 and 0.9 while the
transformation rate ranges from 0.01 to 0.2 and their outcomes were contrasted all together with locate the ideal
estimations of various parameters. Subsequently, a sum of 45 examinations were performed to show the mix of
the distinctive level of specified parameters. Since no errors were seen between the different parameters values,
the center level of every parameter was picked as the last GA parameters. There-fore, the GA was begun with 50
haphazardly created chromosomes, and their parameters were hybrid rate, change rate and populace measure
with the estimations of 0.6, 0.12 and 50, individually.
4. Results and discussion
4.1. Review estimation by ANN
As clarified before, the info information incorporates the coordination and the yield is review esteem. In
non-ideal techniques, the experimentation strategy is the best way to decide the ideal structure. The outcomes for
testing dataset are appeared in Figs. 8 and 9. The acquired structure has the most reduced MSE (Mean Square
Error), RMS (Root Mean Squared), AARE (Average Absolute Relative Error), ARE (Average Relative Error)
and the greatest R (correlation coefficient). Clearly the best and sought qualities for those parameters are zero
aside from the R esteem which is equivalent to 1. In is additionally noticed that for more than single layer
structure, the ANN was tried all through this phase for multi-layers, however the outcomes were not fulfilling.
4.2. Review estimation by ANFIS
The preparation of an ANFIS organize which includes review estimation, directs the info preparing vectors
to target vectors with a base aggregate entirety of squared blunder. Every one of the information sources and
yields are introduced to organize in a few emphases. As the system takes in, the blunder.
The preparation depended on 60 ages which the cross breed learning calculation was utilized as a part of its
engineering. Toward the finish of 35 preparing ages, the mean square mistake for approval vectors was
Met to at least 0.0925. Consequently, 35 ages were chosen for review estimation prepare. The prepared Takagi–
Sugeno sort fluffy deduction framework was utilized for review estimation of accessible dataset. At that point,
IMT Faridabad Volume III, Special Issue I April 2017 Page 48
four chime wrote fluffy MFs per each information were chosen to depict the contribution to yield factors. This is
interpreted in 16 leads for everyone in regards to the two contributions with four fluffy sets. The aftereffects of
definite estimator demonstrate for testing dataset.
4.3. Review estimation: CANFIS–GA
As indicated by the before discourse, the wellness capacity is ""MSE"" and it implies that in all the system
assessments, that is considered generally speaking to achieve the best outcome. In this demonstrating, a few
systems with various hybrid and transformation rates were examined to locate the best likelihood. As it is
appeared, among 50 eras, the coveted system with the least mistake acquired in the twentieth era. This system is
the consequences of applying the GA to the best got parameter that can be utilized to pick up the most reduced
mistake for review estimation and the best system along these lines. Then again, by testing a few qualities for the
given parameters, the GA makes the system to merge to the best structure and reaction. These parameters are
portrayed as takes after: a change rate of 0.08, the system with four hubs in the MFs layer, a learning rate of 0.65
and an energy of 0.7. The ideal got system was exhibited in the twentieth era that it was situated in 24th position
of the 52 systems. The MSE of this ideal system in the preparation stage was equivalent to 0.003.
Keeping in mind the end goal to assess the execution of the current methodology, 15% of the entire dataset were
chosen as the test dataset. The execution of CANFIS–GA picked up by the test dataset. The expectations were
then contrasted and the real review values. Fig. 8 demonstrates the correlation of 24 review forecast tests versus
the real review values for CANFIS–GA. The level pivot speaks to the genuine review and the vertical hub
demonstrates the acquired outcomes from applying a few procedures in this paper.
Fig. 8. Comparison of predicted results with actual grade values for Sungun copper porphyry deposit based on the ANN,
ANFIS and CANFIS–GA.
Fig. 9. Comparison of AARE, RMSE, ARE, R, MSE for ANN, ANFIS and CANFIS–GA
IMT Faridabad Volume III, Special Issue I April 2017 Page 49
As indicated by above figures, the forecast capacity of CANFIS– GA model is reported. Subsequently, this
system is thought to be utilized to foresee the review values. By applying this strategy, a conspicuous change on
the review estimation is gotten. The most exceedingly bad outcomes were acquired from the neural system
method. The consequences of ANFIS are superior to those of ANN because of the way that this procedure is a
mix of ANN and FL techniques and is better in getting the spatial connections between the factors. The
CANFIS–GA system is relied upon to give a noteworthy change when the new information originates from
blended or complex dispersions. Since a large portion of the mining and land exercises can be considered to
tumble down in some fluffy conditions, the CANFIS–GA could be a fantastic decision contrasted with alternate
techniques from the execution perspective. The proposed philosophy not just has the characteristic preferences of
the neural-fluffy strategies, additionally the parameters are improved by the GA under the new approach. Also,
this coordinated approach could enhance the expectations, and notwithstanding topography and mining issues,
this versatile method can be utilized as a part of various fields.
5. Conclusion and future works
This paper presents a coordinated CANFIS and GA (CANFIS– GA) to anticipate the mineral review from the
boreholes in Sungun porphyry copper store in Iran. It is additionally a synchronous look for ideal choice to alter
the system parameters. Due to having a ton of parameters in various strategies for manmade brainpower (e.g.,
ANN, FL and GA), these techniques require a huge time and push to locate the ideal structure, Then again,
utilizing this new proposed strategy for review estimation in this paper, the issues with FL, for example, meaning
of fluffy if-then standards and number of MFs could be settled. Moreover, by applying the learning procedure, it
is conceivable to create an arrangement of fluffy if-then guidelines to inexact a coveted review, as appeared in
this paper. The ANFIS is the aftereffect of ANN and FL blend. Joining these two astute methodologies, a great
thinking would be accomplished in both quality and amount. As such, both fluffy thinking and system estimation
could be accessible at the same time. Since the greater part of the related review estimation degrees are
exceptionally fluffy and the connection between factors is additionally entangled sometimes, this new technique
can be stretched out to be connected in the majority of mining and land star balms. By utilizing the GA, it is
additionally conceivable to decrease contributions to request to get the best outcomes. In the present review the
information factors were three dimensional and there was no compelling reason to decrease the factors. In cases
with heaps of factors, in any case, this technique could be exceptionally useful in information diminishments. In
this review, the CANFIS–GA execution was explored widely, though the relationship coefficient was equivalent
to 0.9327 which demonstrates that this technique has an amazing execution for review estimation. This technique
was contrasted with the other current strategies, which the outcomes showed that the coordinated neural-fluffy
and GA (GA–ANFIS) gives minimal blunder on the testing dataset. What's more, applying the GA–CANFIS
requires substantially less exertion and furthermore that is not exceptionally tedious to tackle the issues. In
addition, this strategy does not have the basic issues existing in the ANN and the FL, as both the expectation
exactness and the time prerequisite for the arrangements are enhanced by the proposed technique.
Acknowledgement
I would like to take this opportunity to express my profound gratitude and deep regard to my project guides
Ms. Neetu Gupta and Dr. Anju Gupta for their exemplary guidance, valuable feedback and constant
encouragement throughout the duration of project. Their valuable suggestions were of immense help in completing
this project.
I would also like to give my sincere gratitude to all my friends and colleagues especially Preeti Ranga who
filled in the survey, without whom this the research work would be incomplete.
IMT Faridabad Volume III, Special Issue I April 2017 Page 50
References
[1]Journel, A.G., Huijbregts, C., 1978. Mining Geostatistics. Academic Press, London 600 pp.
[2]Hornik, K., Stinchcombe, M., White, H., 1989. Multilayer feedforward networks are universal approximators. Neural
Network 2 (5), 359–366. doi:10.1016/0893- 6080(89)90020-8.
[3]Wu, X., Zhou, Y., 1993. Reserve estimation using neural network techniques. Computers & Geosciences 19 (4), 567–575.
[4]Koike, K., Matsuda, S., Suzuki, T., Ohmi, M., 2002. Neural network-based estimation of principal metal contents in the
Hokuroku district, Northern Japan, for exploring Kuroko-type deposits. Natural Resources Research 11 (2), 135–156 [5]Koike, K., Matsuda, S., 2003. Characterizing content distributions of impurities in a limestone mine using a feed forward
neural network. Natural Resources Research 12 (3), 209–223.
[6]Samanta, B., Bandopadhyay, S., Ganguli, R., 2004. Data segmentation and genetic algorithms for sparse data division in Nome placer gold grade estimation using neural network and geostatistics. Mining Exploration Geology 11 (1–4), 69–76. [7]Bardossy, Gy., Szabo, I.R., Varga, G., 2003. A new method of resource estimation for bauxite and other solid mineral
deposits. Journal of Hungarian Geomatematics 1, 14–26.
[8]Bardossy, Gy., Fodor, J., 2005. Assessment of the completeness of mineral exploration by the application of fuzzy
arithmetic and prior information. Acta Polytechnica Hungaricae 2 (1), 217–224.
[9]Galatakis, M., Theodoridis, K., Kouridou, O., 2002. Lignite quality estimation using ANN and adaptive neuro-fuzzy inference systems (ANFIS). APCOM, 425–431. [10]Luo, X., Dimitrakopoulos, R., 2003. Data-driven fuzzy analysis in quantitative mineral resource assessment. Computers & Geosciences 29, 3–13. [11]Pham, T.D., 1997. Grade estimation using fuzzy-set algorithms. Mathematical Geology 29 (2), 291–305. [12]Tutmez, B., 2005. Reserve estimation using fuzzy set theory. Unpublished Ph.D dissertation, Hacettepe University,
Ankara, pp. 168. [13]Tutmez, B., Tercan, E.A., Kaymak, U., 2007. Fuzzy modeling for reserve estimation based on spatial variability.
Mathematical Geology 39 (1), 87–111. [14]Rendu, J.M., 1979. Kriging, logarithmic Kriging, and conditional expectation: comparison of theory with actual results, Proc. 16th APCOM Symposium. Tucson, Arizona, pp. 199–212. [15]Strebelle, S., 2002. Conditional simulation of complex geological structures using multiple-point geostatistics. Mathematical Geology 34 (1), 1–22. [16]Kapageridis, I.K., Denby, B., Hunter, G., 1999. Integration of a neural ore grade estimation tool in a 3D resource
modeling package, neural networks. IJCNN ‘99. International Joint Conference on Neural Network 6, 3908–3912.
[17]Singer, D.A., Kouda, R., 1996. Application of a feed forward neural network in the search for Kuroko deposits in the Hokuroku district, Japan. Mathematical Geology 28 (8), 1017–1023. [18]Yama, B.R., Lineberry, G.T., 1999. Artificial neural network application for a predictive task in mining. Mining
Engineering 51 (2), 59–64.
[19]Denby, B., Burnett, C., 1993. A neural network based tool for grade estimation, 24th International Symposium on the
Application of Computer and Operation Research in the Mineral Industries (APCOM), Montreal, Quebec.
[20]Clarici, E., Owen, D.B., Durucan, S., Ravencroft, P.J., 1993. Recoverable reserve estimation using a neural network. In:
Elbrond, J., Tang, X. (Eds.), 24th International Symposium on the Application of Computer and Operation Research in the
Mineral Industries (APCOM), pp. 145–152.
[21]Ke, J., 2002. Neural network modeling of placer ore grade spatial variability. Unpublished Ph.D Dissertation, University of Alaska Fairbanks, pp. 251.
[22]Koike, K., Matsuda, S., Gu, B., 2001. Evaluation of interpolation accuracy of neural kriging with application to
temperature-distribution analysis. Mathematical Geology 33 (4), 421–448.
[23]Porwal, A., Carranza, E.J.M., Hale, M., 2004. A hybrid neuro-fuzzy model for mineral potential mapping. Mathematical Geology 36 (7), 803–826.
[24]Lacassie, J.P., Solar, J.R., Roser, B., Herve´, F., 2006. Visualization of volcanic rock geochemical data and classification
with artificial neural networks. Mathema-tical Geology 38 (6), 697–710.
[25]Weller, A.F., Corcoran, J., Harris, A.J., Ware, J.A., 2005. The semi-automated classification of sedimentary organic
matter in palynological preparations. Computers & Geosciences 31 (10), 1213–