Agricultural Engineering

Improving the accuracy of soil heavy metal concentration estimation based on spectral reflectance by minimizing the effect of moisture with the external parameter orthogonalization algorithm

Document Type : Research Paper

Authors

Soheila Alioghli 1
Mahmoud Shabanpour 2
Hosein Ali Bahrami 3

1 Ph.D. Student, Department of Soil Science, Faculty of Agricultural Sciences, University of Guilan, Rasht, Gilan, Iran

2 Associate Professor, Department of Soil Science, Faculty of Agricultural Sciences, University of Guilan, Rasht, Gilan, Iran

3 Professor,Department of Soil Sciences, Faculty of Agriculture, Tarbiat Modares University, Tehran, Iran

https://doi.org/10.22055/agen.2024.47640.1738
Abstract
Introduction: Rapid global industrial development has led to a significant increase in the risk of environmental contamination with heavy metals. Soil contamination with heavy metals is a challenge due to the risks associated with human health and the environment, as well as soil-related food security. It is considered universal. Soil moisture, as one of the influencing parameters on soil spectral reflectance and its high spatio-temporal variability, is considered the most important confounding factor in using Visible and Near Infrared Reflectance Spectroscopy (VNIRS) techniques to estimate soil heavy metals. . In this study, the ability of the external parameter orthogonal (EPO) algorithm to reduce the effect of moisture on soil spectral reflectance was evaluated to improve the performance of machine learning methods for heavy metal estimation.

Materials and Methods: In the present study, a modeling approach based on spectral information obtained from VNIRS technique is used to investigate the effect of soil moisture on the estimation of heavy metal concentration (nickel and lead). Soil samples were obtained from areas suspected of heavy metal contamination. For this purpose, 129 soil samples were collected from fields contaminated with heavy metals in Tehran, Gilan and East Azerbaijan provinces. Nickel and lead concentrations in air-dried and sieved soil samples were measured in the laboratory using the ISO 11466 method. Then the soil samples were coded and transferred to the dark room for spectroscopy. Spectral reflectance of soil samples at 7 moisture levels (air-dried, 6, 12, 18, 24, 30 and 36%) using a FieldSpec-3 spectrometer and a contact probe in the spectral range of 350-2500 nm in a room Darkness was measured. After applying the necessary pre-processing, the soft and de-noised spectra related to the soil samples were randomly separated into two sets of data sets for modeling and validation. EPO using a set of calibration samples was developed. To estimate heavy metals, machine learning algorithms including PLSR and SVR were used.

Results and Discussion: The results indicate that with the increase in soil moisture, the spectral reflectance in the entire range of 2450-400 nm decreases non-linearly. This means that the amount of reduction in different wavelengths is not the same. The greatest reduction occurs in the range of absorption peaks located in the range of wavelengths of 1600-1400 nm and 1850-2000 nm. VNIR spectroscopy has a high ability to estimate nickel and lead heavy metals in dry soil. The presence of moisture in the soil, even at the level of 6%, leads to a significant decrease in the ability of this technology to accurately estimate heavy metals in the soil. At humidity greater than 24%, both machine learning models evaluated for both heavy metals are in the medium class.

Conclusion: In a general conclusion, it can be stated that the use of EPO algorithm significantly improves the ability of machine learning methods in the estimation of heavy metals in soil in the presence of moisture in soil samples. In general, SVR algorithm has the best performance compared to PLSR methods for modeling soil heavy metals. On the other hand, VNIR reflectance spectrum information provides more capability in estimating nickel than lead.

In this study, the ability of the EPO method was evaluated in order to minimize the effect of soil moisture disturbing parameters on the ability of VNIR spectroscopy to improve the accuracy of soil heavy metal concentration modeling including nickel and lead based on PLSR and SVR machine learning methods. The results indicate that with the increase in soil moisture, the spectral reflectance in the entire range of 2450-400 nm decreases non-linearly. This means that the amount of reduction in different wavelengths is not the same. The greatest decrease occurs in the range of absorption peaks located in the wavelength range of 1600-1400 nm and 1850-2000 nm. VNIR spectroscopy has a high ability to estimate nickel and lead heavy metals in dry soil. The presence of moisture in the soil, even at the level of 6%, leads to a significant decrease in the ability of this technology to accurately estimate heavy metals in the soil. At humidity greater than 24%, both machine learning models are evaluated for both heavy metals in the medium class. Using the EPO algorithm significantly improves the ability of machine learning methods to estimate heavy metals in soil in the presence of moisture in soil samples. In general, SVR algorithm has the best performance compared to PLSR methods for modeling soil heavy metals. On the other hand, the information of VNIR reflectance spectrum offers more capability in estimating nickel than lead.

Keywords

Soil pollution
Heavy metals
VNIR Spectroscopy
Soil moisture
EPO
Machine learning
Subjects
Reference
1. Abdu, N., Abdulkadir, A., Agbenin, J. O., and Buerkert, A. 2011. Vertical distribution of heavy metals in wastewater irrigated vegetable garden soils of three West African cities. Nutrient Cycling in Agroecosystems, 89, 387 397. https://doi.org/10.1007/s10705 010 9403 3
2. Abdu, N., Abdullahi, A. A., and Abdulkadir, A. 2017. Heavy metals and soil microbes. Environmental Chemistry Letters, 15(1), 65 84. https://doi.org/10.1007/s10311 016 0587 x 3. Abdu, N., Agbenin, J. O., and Buerkert, A. 2011. Geochemical assessment, distribution, and dynamics of trace elements in urban agricultural soils under long‐term wastewater irrigation in Kano, northern Nigeria. Journal of Plant Nutrition and Soil Science, 174(3), 447 458. https://doi.org/10.1002/jpln.201000333
4. Ackerson, J. P., Demattê, J. A. M., and Morgan, C. L. S. 2015. Predicting clay content on field moist intact tropical soils using a dried, ground VisNIR library with external parameter orthogonalization. Geoderma, 259, 196 204. https://doi.org/10.1016/j.geoderma.2015.06.002
5. Angelopoulou, T., Dimitrakos, A., Terzopoulou, E., Zalidis, G., Theocharis, J., Stafilov, T., and Zouboulis, A. 2017. Reflectance spectroscopy (Vis NIR) for assessing soil heavy metals concentrations determined by two different analytical protocols, based on ISO 11466 and ISO 14869 1. Water, Air, and Soil Pollution, 228, 1 12. https://doi.org/10.1007/s11270 017 3609 9
6. Briffa, J., Sinagra, E., and Blundell, R. 2020. Heavy metal pollution in the environment and their toxicological effects on humans. Heliyon, 6(9). DOI: 10.1016/j.heliyon.2020.e04691
7. Cao, Y., Bao, N., Liu, S., Zhao, W., and Li, S. 2020. Reducing moisture effects on soil organic carbon content prediction in visible and near infrared spectra with an external parameter othogonalization algorithm. Canadian Journal of Soil Science. 100(3), 1 10. http://dx.doi.org/10.1139/cjss 2020 0009
8. Castaldi, F., Palombo, A., Pascucci, S., Pignatti, S., Santini, F., and Casa, R. 2015. Reducing the influence of soil moisture on the estimation of clay from hyperspectral data: a case study using simulated PRISMA data. Remote Sensing, 7(11), 15561 15582. https://doi.org/10.3390/rs71115561
9. Chakraborty, S., Li, B., Weindorf, D.C., and Morgan, C.L.S. 2019. External parameter orthogonalisation of Eastern European VisNIR DRS soil spectra. Geoderma, 337, 65 75. https://doi.org/10.1016/j.geoderma.2018.09.015
10. Chang, W., Laird, D. A., Mausbach, M. J., and Hurburgh, C. R. 2001. Near‐infrared reflectance spectroscopy principal components regression analyses of soil properties. Soil Science Society of America Journal, 65(2), 480 490. https://doi.org/10.2136/sssaj2001.652480x
11. Cheng, H., Shen, R., Chen, Y., Wan, Q., Shi, T., Wang, J., Wan, Y., Hong, Y., and Li, X. 2019. Estimating heavy metal concentrations in suburban soils with reflectance spectroscopy. Geoderma, 336, 59 67. https://doi.org/10.1016/j.geoderma.2018.08.010
12. de Santana, F.B., de Giuseppe, L.O., de Souza, A.M., and Poppi, R.J. 2019. Removing the moisture effect in soil organic matter determination using NIR spectroscopy and PLSR with external parameter orthogonalization. Microchemical Journal, 145, 1094 1101. https://doi.org/10.1016/j.microc.2018.12.027
13. Ge, Y., Morgan, C.L.S., and Ackerson, J.P. 2014. VisNIR spectra of dried ground soils predict properties of soils scanned moist and intact. Geoderma, 221 222, 61 69. https://dx.doi.org/10.1016/j.geoderma.2014.01.011
14. Geladi, P. and Kowalski, B.R. 1986. Partial least--squares regression: a tutorial. Analytical chemistry, 185, 1--17. https://doi.org/10.1016/0003--2670(86)80028--9
15. Ji, W., Viscarra Rossel, R.A., and Shi, Z. 2015. Accounting for the effects of water and the environment on proximally sensed vis––NIR soil spectra and their calibrations. European Journal of Soil Science, 66, 555––565. https://doi.org/10.1111/ejss.12239
16. Jiang, Q., Chen, Y., Guo, L., Fei, T., and Qi, K. 2016. Estimating soil organic carbon of cropland soil at different levels of soil mois--ture using VIS--NIR spectroscopy. Remote Sensing, 8(9), 755. https://doi.org/10.3390/rs8090755
17. Knadel, M., Castaldi, F., Barbetti, R., Ben--Dor, E., Gholizadeh, A., and Lorenzetti, R. 2023. Mathematical techniques to remove moisture effects from visible––near--infrared––shortwave--infrared soil spectra. Applied Spectroscopy Reviews, 58(9), 629--662. https://doi.org/10.1080/05704928.2022.2128365
18. Liu, Z., Lu, Y., Peng, Y., Zhao, L., Wang, G., and Hu, Y. 2019. Estimation of soil heavy metal content using hyperspectral data. Remote Sensing, 11(12), 1464. https://doi.org/10.3390/rs11121464
19. Liu, Y., Pan, X., Wang, C., Li, Y., and Shi, R., 2015. Predicting soil salinity with Vis––NIR spectra after removing the effects of soil moisture using External Parameter Orthogonalization. PLOS ONE, 10(10), 1--13. https://doi.org/10.1371/journal.pone.0140688.
20. Lobell, D.B., and Asner, G.P. 2002. Moisture effects on soil reflectance. Soil Science. Society Of America Journal. J. 66, 722--727. https://doi.org/10.2136/sssaj2002.7220
21. Mao, Y., Liu, J., Cao, W., Ding, R., Fu, Y., and Zhao, Z. 2021. Research on the quantitative inversion model of heavy metals in soda saline land based on visible--near--infrared spectroscopy. Infrared Physics and Technology, 112, 103602. https://doi.org/10.1016/j.infrared.2020.103602
22. Masindi, V., and Muedi, K. L. 2018. Environmental contamination by heavy metals. Heavy Metals, 10(4), 115––133.http://dx.doi.org/10.5772/intechopen.71185
23. Minasny, B., Mcbratney, A.B., Bellon--Maurel, V., Roger, J.M., Gobrecht, A., Ferrand, L., and Joall, S. 2011. Removing the effect of soil moisture from NIR diffuse reflectance spectra for the prediction of soil organic carbón. Geoderma, 167--168, 118--124. https://dx.doi.org/10.1016/j.geoderma.2011.09.008
24. Mirzaei, S., Boloorani, A. D., Bahrami, H. A., Alavipanah, S. K., Mousivand, A., and Mouazen, A. M. 2022. Minimising the effect of moisture on soil property prediction accuracy using external parameter orthogonalization. Soil and Tillage Research, 215, 105225. https://doi.org/10.1016/j.still.2021.105225
25. Mouazen, A.M., and Al--Asadib, R.A. 2018. Influence of soil moisture content on assessment of bulk density with combined frequency domain reflectometry and visible and near infrared spectroscopy under semi field conditions. Soil and Tillage Research, 176, 95––103. https://doi.org/10.1016/j.still.2017.11.002
26. Nawar, S., Abdul Munnaf, M., and Mouazen, A.M. 2020. Machine learning based on--line prediction of soil organic carbon after removal of soil moisture effect. Remote Sensing, 12(8), 1308. https://doi.org/10.3390/rs12081308
27. Nyarko, F., Tack, F. M. G., and Mouazen, A. M. 2022. Potential of visible and near infrared spectroscopy coupled with machine learning for predicting soil metal concentrations at the regional scale. Science of The Total Environment, 841, 156582. https://doi.org/10.1016/j.scitotenv.2022.156582
28. Ogen, Y., Faigenbaum--Golovin, S., Granot, A., Shkolnisky, Y., Goldshleger, N., and Ben--Dor, E. 2019. Removing moisture effect on soil reflectance properties: A case study of clay content prediction. Pedosphere, 29(4), 421––431. https://doi.org/10.1016/S1002--0160(19)60811--8
29. Pyo, J., Hong, S. M., Kwon, Y. S., Kim, M. S., and Cho, K. H. 2020. Estimation of heavy metals using deep neural network with visible and infrared spectroscopy of soil. Science of the Total Environment, 741, 140162. https://doi.org/10.1016/j.scitotenv.2020.140162
30. Rodrigues, A. A. Z., De Queiroz, M. E. L. R., De Oliveira, A. F., Neves, A. A., Heleno, F. F., Zambolim, L., Freitas, J. F., and Morais, E. H. C. 2017. Pesticide residue removal in classic domestic processing of tomato and its effects on product quality. Journal of Environmental Science and Health, Part B, 52(12), 850––857. https://doi.org/10.1080/03601234.2017.1359049
31. Roger, J.M., Chauchard, F., and Maurel, V. 2003. EPO--PLS external parameter orthogonalisation of PLS application to tempera--ture--independent measurement of sugar content of intact fruits. Chemometrics and Intelligent Laboratory Systems, 66 (2), 191--204. https://doi.org/10.1016/S0169--7439(03)00051--0
32. Schwartz, G., Eshel, G., and Ben Dor, E. 2011. Reflectance spectroscopy as a tool for monitoring contaminated soils. Soil Contamination, 6790.
33. Seidel, M., Vohland, M., Greenberg, I., Ludwig, B., Ortner, M., Thiele--Bruhn, S., and Hutengs, C. 2022. Soil moisture effects on predictive VNIR and MIR modeling of soil organic carbon and clay content. Geoderma, 427, 116103. https://doi.org/10.1016/j.geoderma.2022.116103
34. Shi, T., Chen, Y., Liu, Y., and Wu, G. 2014. Visible and near--infrared reflectance spectroscopy——An alternative for monitoring soil contamination by heavy metals. Journal of Hazardous Materials, 265, 166––176. https://doi.org/10.1016/j.jhazmat.2013.11.059
35. Wang, Y.P., Lee, C.K., Dai, Y.H., and Shen, Y. 2020. Effect of wetting on the determination of soil organic matter content using visible and near--infrared spectrometer. Geoderma, 376, 114528. https://doi.org/10.1016/j.geoderma.2020.114528
36. Wijewardane, N.K., Ge, Y., and Morgan, C.L.S. 2016b. Prediction of soil organic and inorganic carbon at different moisture contents with dry ground VNIR: a comparative. European Journal of Soil Science, 67, 605––615. https://doi.org/10.1016/j.geoderma.2010.04.008.
37. Wijewardane, N.K., Hetrick, S., Ackerson, J., Morgan, C.L.S., and Ge, Y. 2020. VisNIR integrated multi--sensing penetrometer for in situ high--resolution vertical soil sensing. Soil and Tillage Research, 199, 104604, https://doi.org/10.1016/j.still.2020.104604.
38. Xu, J., Liu, Y., Yan, C., and Yuan, J. 2024. Estimation of Soil Organic Matter Based on Spectral Indices Combined with Water Removal Algorithm. Remote Sensing, 16(12), 2065. https://doi.org/10.3390/rs16122065
39. Zwolak, A., Sarzyńska, M., Szpyrka, E., and Stawarczyk, K. 2019. Sources of soil pollution by heavy metals and their accumulation in vegetables: A review. Water, Air, and Soil Pollution, 230, 1––9. https://doi.org/10.1007/s11270--019--4221--y
Agricultural Engineering
Volume 47, Issue 3
Autumn 2024
Pages 429-444

Home

Submit Manuscript

Editorial Staff

Editorial Board

Publication Ethics

Guide for Authors

Article Processing Charges

Reviewers

contact us