Share:


Cockpit crew safety performance prediction based on the integrated machine learning multi-class classification models and Markov chain

    Naimeh Borjalilu Affiliation
    ; Fariborz Jolai   Affiliation
    ; Mahdieh Tavakoli Affiliation

Abstract

The main tool of cockpit crew performance evaluation is the recorded flight data used for flight operations safety improvement since all certified airlines require implementation of a safety and quality management system. The safety performance of a flight has been a challenging issue in the aviation industry and plays an important role to acquire competitive benefits. In this study, an integrated multi-class classification machine learning models and Markov chain were developed for cockpit crew performance evaluation during their flights. At the outset, the main features related to a flight are identified based on the literature review, flight operations expert’s statements, and the case study dataset (as numerical example). Afterwards, the flights’ performance is evaluated as a target column based on four multi-class classification models (Decision Tree, Support Vector Machine, Neural Network, and Random Forest). The results showed that the random forest classifier has the greatest value in all evaluation metrics (i.e., accuracy = 0.90, precision = 0.91, recall = 0.97, and F1-score = 0.93). Therefore, this model can be used by the airline companies to predict flight crew performance before the flight in order to prevent or decrease flight safety risks.

Keyword : cockpit crew performance, flight evaluation prediction, multi-class classification, Markov chain, safety management system

How to Cite
Borjalilu, N., Jolai, F., & Tavakoli, M. (2023). Cockpit crew safety performance prediction based on the integrated machine learning multi-class classification models and Markov chain . Aviation, 27(3), 152–161. https://doi.org/10.3846/aviation.2023.19739
Published in Issue
Oct 18, 2023
Abstract Views
398
PDF Downloads
322
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

References

Aljedani, N., Alotaibi, R., & Taileb, M. (2021). HMATC: Hierarchical multi-label Arabic text classification model using machine learning. Egyptian Informatics Journal, 22(3), 225–237. https://doi.org/10.1016/j.eij.2020.08.004

Ashiku, L., Al-Amin, M., Madria, S., & Dagli, C. (2021). Machine learning models and big data tools for evaluating kidney acceptance. Procedia Computer Science, 185(June), 177–184. https://doi.org/10.1016/j.procs.2021.05.019

Castillo-Botón, C., Casillas-Pérez, D., Casanova-Mateo, C., Ghimire, S., Cerro-Prada, E., Gutierrez, P. A., Deo, R. C., & Salcedo-Sanz, S. (2022). Machine learning regression and classification methods for fog events prediction. Atmospheric Research, 272, 106157. https://doi.org/10.1016/j.atmosres.2022.106157

Corker, K. M., & Pisanich, G. M. (1995). Analysis and modeling of flight crew performance in automated air traffic management systems. IFAC Proceedings Volumes, 28(15), 547–552. https://doi.org/10.1016/S1474-6670(17)45289-X

Delgado, F., Trincado, R., & Pagnoncelli, B. K. (2019). A multistage stochastic programming model for the network air cargo allocation under capacity uncertainty. Transportation Research Part E: Logistics and Transportation Review, 131(November), 292–307. https://doi.org/10.1016/j.tre.2019.09.011

EASA. (2019). AMC1 ORO.AOC.130 – Annex III. https://www.easa.europa.eu/

European Aviation Safety Agency (EASA). (2016). Developing Standardised Fdm-Based Indicators Focus (2 December, pp. 1–55). https://www.easa.europa.eu/sites/default/files/dfu/EAFDM__standardised_FDM-based_indicators_v2_Ed2017.pdf

Filippone, A. (2008). Comprehensive analysis of transport aircraft flight performance. Progress in Aerospace Sciences, 44(3), 192–236. https://doi.org/10.1016/j.paerosci.2007.10.005

Fodeh, S. J., & Tiwari, A. (2018). Exploiting MEDLINE for gene molecular function prediction via NMF based multi-label classification. Journal of Biomedical Informatics, 86(August 2017), 160–166. https://doi.org/10.1016/j.jbi.2018.08.009

Gharaibeh, A., Shaamala, A., Obeidat, R., & Al-Kofahi, S. (2020). Improving land-use change modeling by integrating ANN with Cellular Automata-Markov Chain model. Heliyon, 6(9), e05092. https://doi.org/10.1016/j.heliyon.2020.e05092

Güven, İ., & Şimşir, F. (2020). Demand forecasting with color parameter in retail apparel industry using artificial neural networks (ANN) and support vector machines (SVM) methods. Computers and Industrial Engineering, 147. https://doi.org/10.1016/j.cie.2020.106678

Hon, K. K., Ng, C. W., & Chan, P. W. (2020). Machine learning based multi-index prediction of aviation turbulence over the Asia-Pacific. Machine Learning with Applications, 2, 100008. https://doi.org/10.1016/j.mlwa.2020.100008

International Civil Aviation Organization. (2013). Operation of Aircraft – Fatigue. Excerpts of Fatigue Management Realted Provisions from: Annex 6 to the Convention of International Civil Aviation, February. https://www.icao.int/safety/fatiguemanagement/FRMS%20Tools/Amendment%2037%20for%20FRMS%20SARPS%20%28en%29.pdf

Kulkarni, V. G. (2011). Brownian motion. In Introduction to modeling and analysis of stochastic systems (pp. 247–280). Springer. https://doi.org/10.1007/978-1-4419-1772-0_7

Lan, C. E., Wu, K., & Yu, J. (2012). Flight characteristics analysis based on QAR data of a jet transport during landing at a high-altitude airport. Chinese Journal of Aeronautics, 25(1), 13–24. https://doi.org/10.1016/S1000-9361(11)60357-9

Li, H., Wang, W., Fan, L., Li, Q., & Chen, X. (2020). A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL, entropy weighting and later defuzzification VIKOR. Applied Soft Computing, 91, 106207. https://doi.org/10.1016/j.asoc.2020.106207

Lyu, Y., & Liem, R. P. (2020). Flight performance analysis with data-driven mission parameterization: Mapping flight operational data to aircraft performance analysis. Transportation Engineering, 2(September), 100035. https://doi.org/10.1016/j.treng.2020.100035

Martini, G., Scotti, D., & Volta, N. (2013). Including local air pollution in airport efficiency assessment: A hyperbolic-stochastic approach. Transportation Research Part D: Transport and Environment, 24(2007), 27–36. https://doi.org/10.1016/j.trd.2013.05.002

Mokhtarimousavi, S., & Mehrabi, A. (2022). Flight delay causality: Machine learning technique in conjunction with random parameter statistical analysis. International Journal of Transportation Science and Technology, 12(1), 230–244. https://doi.org/10.1016/j.ijtst.2022.01.007

Moral-García, S., Mantas, C. J., Castellano, J. G., & Abellán, J. (2020). Non-parametric predictive inference for solving multi-label classification. Applied Soft Computing Journal, 88. https://doi.org/10.1016/j.asoc.2019.106011

Moshkov, M. (2021). On the depth of decision trees over infinite 1-homogeneous binary information systems. Array, 10(March), 100060. https://doi.org/10.1016/j.array.2021.100060

Nguyen, T. P., & Lin, Y. K. (2021). Reliability assessment of a stochastic air transport network with late arrivals. Computers and Industrial Engineering, 151(January). https://doi.org/10.1016/j.cie.2020.106956

Okwuashi, O., & Ndehedehe, C. E. (2021). Integrating machine learning with Markov chain and cellular automata models for modelling urban land use change. Remote Sensing Applications: Society and Environment, 21(January). https://doi.org/10.1016/j.rsase.2020.100461

Onan, A., Korukoğlu, S., & Bulut, H. (2016a). Ensemble of keyword extraction methods and classifiers in text classification. Expert Systems with Applications, 57, 232–247. https://doi.org/10.1016/j.eswa.2016.03.045

Onan, A., Bal, V., & Yanar Bayam, B. (2016b). The use of data mining for strategic management: A case study on mining association rules in student information system. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje, 18(1), 41–70. https://doi.org/10.15516/cje.v18i1.1471

Onan, A. (2019). Two-stage topic extraction model for bibliometric data analysis based on word embeddings and clustering. IEEE Access, 7, 145614–145633. https://doi.org/10.1109/ACCESS.2019.2945911

Onan, A., & Korukoğlu, S. (2017). A feature selection model based on genetic rank aggregation for text sentiment classification. Journal of Information Science, 43(1), 25–38. https://doi.org/10.1177/0165551515613226

Onan, A. (2021). Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks. Concurrency and Computation: Practice and Experience, 33(23), e5909. https://doi.org/10.1002/cpe.5909

Onan, A. (2015). A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Systems with Applications, 42(20), 6844–6852. https://doi.org/10.1016/j.eswa.2015.05.006

Onan, A. (2020). Mining opinions from instructor evaluation reviews: A deep learning approach. Computer Applications in Engineering Education, 28(1), 117–138. https://doi.org/10.1002/cae.22179

Onan, A. (2016). Classifier and feature set ensembles for web page classification. Journal of Information Science, 42(2), 150–165. https://doi.org/10.1177/0165551515591724

Onan, A. (2018a). An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1), 28–47. https://doi.org/10.1177/0165551516677911

Onan, A. (2018b). Biomedical text categorization based on ensemble pruning and optimized topic modelling. Computational and Mathematical Methods in Medicine, 2018. https://doi.org/10.1155/2018/2497471

Onan, A. (2019a). Consensus clustering-based undersampling approach to imbalanced learning. Scientific Programming, 2019. https://doi.org/10.1155/2019/5901087

Onan, A. (2019b). Topic-enriched word embeddings for sarcasm identification. In R. Silhavy, Software Engineering Methods in Intelligent Algorithms: Proceedings of 8th Computer Science On-line Conference 2019 (Vol. 984, pp. 293–304). Springer International Publishing. https://doi.org/10.1007/978-3-030-19807-7_29

Onan, A., & Toçoğlu, M. A. (2021). A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification. IEEE Access, 9, 7701–7722. https://doi.org/10.1109/ACCESS.2021.3049734

Oreschko, B., Kunze, T., Schultz, M., Fricke, H., Kumar, V., & Sherry, L. (2012). Turnaround prediction with stochastic process times and airport specific delay pattern airport delays. In The 5th International Conference on Research in Air Transportation, 9. ResearchGate.

Papadopoulos, C. T., Li, J., & O’Kelly, M. E. J. (2019). A classification and review of timed Markov models of manufacturing systems. Computers and Industrial Engineering, 128(November 2018), 219–244. https://doi.org/10.1016/j.cie.2018.12.019

Qian, W., Xiong, C., & Wang, Y. (2021). A ranking-based feature selection for multi-label classification with fuzzy relative discernibility. Applied Soft Computing, 102. https://doi.org/10.1016/j.asoc.2020.106995

Rey, M., Aloise, D., Soumis, F., & Pieugueu, R. (2021). A data-driven model for safety risk identification from flight data analysis. Transportation Engineering, 5, 100087. https://doi.org/10.1016/j.treng.2021.100087

Ross, S. M. (2014). Introduction to probability models. Academic Press. https://doi.org/10.1016/B978-0-12-407948-9.00001-3

Samaee, S., & Kobravi, H. R. (2020). Predicting the occurrence of wrist tremor based on electromyography using a hidden Markov model and entropy based learning algorithm. Biomedical Signal Processing and Control, 57(March). https://doi.org/10.1016/j.bspc.2019.101739

Shone, R., Glazebrook, K., & Zografos, K. G. (2021). Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty. European Journal of Operational Research, 292(1), 1–26. https://doi.org/10.1016/j.ejor.2020.10.039

Toçoğlu, M. A., & Onan, A. (2020, July). Sentiment analysis on students’ evaluation of higher educational institutions. In International Conference on Intelligent and Fuzzy Systems (pp. 1693–1700). INFUS 2020: Intelligent and Fuzzy Techniques: Smart and Innovative Solutions. Springer. https://doi.org/10.1007/978-3-030-51156-2_197

Utami, N. A., Maharani, W., & Atastina, I. (2021). Personality classification of Facebook users according to big five personality using SVM (Support Vector Machine) method. Procedia Computer Science, 179(2020), 177–184. https://doi.org/10.1016/j.procs.2020.12.023

Van Giffen, B., Herhausen, D., & Fahse, T. (2022). Overcoming the pitfalls and perils of algorithms: A classification of machine learning biases and mitigation methods. Journal of Business Research, 144, 93–106. https://doi.org/10.1016/j.jbusres.2022.01.076

Wang, L., Wu, C., & Sun, R. (2014). An analysis of flight Quick Access Recorder (QAR) data and its applications in preventing landing incidents. Reliability Engineering and System Safety, 127, 86–96. https://doi.org/10.1016/j.ress.2014.03.013

Yaakoubi, Y., Soumis, F., & Lacoste-Julien, S. (2020). Machine learning in airline crew pairing to construct initial clusters for dynamic constraint aggregation. EURO Journal on Transportation and Logistics, 9(4), 100020. https://doi.org/10.1016/j.ejtl.2020.100020

Yamaguchi, S., Nakashima, H., Moriwaki, Y., Terada, T., & Shimizu, K. (2022). Prediction of protein mononucleotide binding sites using AlphaFold2 and machine learning. Computational Biology and Chemistry, 107744. https://doi.org/10.1016/j.compbiolchem.2022.107744

Yan, S., & Tang, C.-H. (2007). A heuristic approach for airport gate assignments for stochastic flight delays. European Journal of Operational Research, 180(2), 547–567. https://doi.org/10.1016/j.ejor.2006.05.002

Yang, C., Yin, T., Zhao, W., Huang, D., & Fu, S. (2014). Human factors quantification via boundary identification of flight performance margin. Chinese Journal of Aeronautics, 27(4), 977–985. https://doi.org/10.1016/j.cja.2014.03.016

Zhou, Y., Liu, Y., Wang, D., Liu, X., & Wang, Y. (2021). A review on global solar radiation prediction with machine learning models in a comprehensive perspective. Energy Conversion and Management, 235(13), 113960. https://doi.org/10.1016/j.enconman.2021.113960