基于规则集和多层感知机的Argo温度数据质量控制方法*

doi:10.11978/2023172

热带海洋学报 ›› 2024, Vol. 43 ›› Issue (5): 190-202.doi: 10.11978/2023172CSTR: 32234.14.2023172

• 海洋调查与观测 • 上一篇

基于规则集和多层感知机的Argo温度数据质量控制方法^*

齐焕东¹^,²(), 朱程², 李序春², 景昕蒂², 宋德瑞²^,³()

1.信息学院(上海海洋大学), 上海 201306
2.国家海洋环境监测中心, 辽宁大连 116023
3.地理科学学院(辽宁师范大学), 辽宁大连 116029

收稿日期:2023-11-21 修回日期:2024-01-08 出版日期:2024-09-10 发布日期:2024-10-10
作者简介:
齐焕东(2000—), 男, 河南省济源市人, 硕士研究生, 从事海洋数据分析相关研究。email: m220901584@st.shou.edu.cn
*感谢中国Argo实时资料中心(http://www.argo.org.cn/)提供数据支撑。
基金资助:
国家重点研发计划项目(2021YFF0704000); 国家重点研发计划项目(2022YFC3106100)

Rule set and multilayer perceptron based quality control method for Argo temperature data^*

QI Huandong¹^,²(), ZHU Cheng², LI Xuchun², JING Xindi², SONG Derui²^,³()

1. College of Information Technology (Shanghai Ocean University), Shanghai 201306, China
2. National Marine Environmental Monitoring Center, Dalian 116023, China
3. School of Geographical Sciences (Liaoning Normal University), Dalian 116029, China

Received:2023-11-21 Revised:2024-01-08 Online:2024-09-10 Published:2024-10-10
Supported by:
National Key Research and Development Program of China(2021YFF0704000); National Key Research and Development Program of China(2022YFC3106100)

摘要/Abstract

摘要：

海洋温度数据在全球海洋观测和气候研究中发挥着关键作用, 质量控制对于确保这些数据的可靠性十分关键, 然而, 目前在大数据集上的异常数据召回率尚不理想。文章基于Argo温度数据, 提出一种基于规则集和多层感知机(rule set and multilayer perceptron, RS-MLP)的质量控制方法。首先对13种机器学习模型进行对比分析, 从中筛选出最优机器学习模型, 然后设计了由6种基于规则的质量控制检查模块组成的规则集, 最后集成规则集和最优机器学习模型构建出RS-MLP方法, 并以南海区域的Argo数据为例评估方法性能。研究结果表明: RS-MLP在351746条温度数据的测试集中真阳性率(true positive rate, TPR)、真阴性率(true negative rate, TNR)和接受者操作特性(receiver operating characteristic, ROC)曲线下面积(area under the curve, AUC)依次能达到93%、96%和95%, 并在不同深度层次上的异常数据召回率比较稳定, 具有优秀的质量控制性能。

关键词: Argo, 温度, 机器学习, 质量控制

Abstract:

The ocean temperature data plays a crucial role in global ocean observation and climate research. Quality control is essential to ensure the reliability of these data. However, the current recall rate of anomalous data in large datasets is unsatisfactory. This paper proposes a quality control method based on a rule set and multilayer perceptron (RS-MLP), using Argo temperature data. Initially, thirteen machine learning models are compared and analyzed to select the optimal model. Subsequently, a rule set consisting of six rule-based quality control check modules is designed. Finally, the RS-MLP method is constructed by integrating the rule set with the optimal machine learning model, and its performance is evaluated using Argo data from the South China Sea region. The results show that the RS-MLP achieves good performance with true positive rate (TPR), true negative rate (TNR), and area under the receiver operating characteristic (ROC) curve (AUC) reaching 94%, 96%, and 95% respectively in a test set of 351746 temperature data points. The recall rate of anomalous data at different depth levels is stable, demonstrating excellent quality control performance.

Key words: Argo, temperature, machine learning, quality control

齐焕东, 朱程, 李序春, 景昕蒂, 宋德瑞. 基于规则集和多层感知机的Argo温度数据质量控制方法^*[J]. 热带海洋学报, 2024, 43(5): 190-202.

QI Huandong, ZHU Cheng, LI Xuchun, JING Xindi, SONG Derui. Rule set and multilayer perceptron based quality control method for Argo temperature data^*[J]. Journal of Tropical Oceanography, 2024, 43(5): 190-202.

图/表 14

图1

图2

图3

图4

图5

表1

图6

表2

图7

表3

图8

表4

图9

图10

参考文献 35

[1]	蒋华, 武尧, 王鑫, 等, 2019. 改进K均值聚类的海洋数据异常检测算法研究[J]. 计算机科学, 46(7): 211-216. doi: 10.11896/j.issn.1002-137X.2019.07.032
	JIANG HUA, WU YAO, WANG XIN, et al, 2019. Study on ocean data anomaly detection algorithm based on improved K-means clustering[J]. Computer Science, 46(7): 211-216 (in Chinese with English abstract). doi: 10.11896/j.issn.1002-137X.2019.07.032
[2]	刘玉龙, 王国松, 侯敏, 等, 2021. 基于深度学习的海温观测数据质量控制应用研究[J]. 海洋通报, 40(3): 283-291.
	LIU YULONG, WANG GUOSONG, HOU MIN, et al, 2021. Quality control of sea temperature observation data using deep learning neural networks[J]. Marine Science Bulletin, 40(3): 283-291 (in Chinese with English abstract).
[3]	刘增宏, 李兆钦, 卢少磊, 等, 2021. 全球海洋Argo温盐度剖面散点数据集[J]. 全球变化数据学报(中英文), 5(3): 312-321, 451-460.
	LIU ZENGHONG, LI ZHAOQIN, LU SHAOLEI, et al, 2021. Scattered dataset of global ocean temperature and salinity profiles from the international Argo program[J]. Journal of Global Change Data & Discovery, 5(3): 312-321, 451-460 (in Chinese with English abstract).
[4]	卢少磊, 孙朝辉, 刘增宏, 等, 2016. COPEX和HM2000与APEX型剖面浮标比测试验及资料质量评价[J]. 海洋技术学报, 35(1): 84-92.
	LU SHAOLEI, SUN CHAOHUI, LIU ZENGHONG, et al, 2016. Comparative testing and data quality evaluation for COPEX, HM2000 and APEX profiling buoys[J]. Journal of Ocean Technology, 35(1): 84-92 (in Chinese with English abstract).
[5]	沈锐, 王德亮, 刘增宏, 等, 2019. HM2000型剖面浮标的主要特征及其应用[J]. 数字海洋与水下攻防, 2(2): 20-27.
	SHEN RUI, WANG DELIANG, LIU ZENGHONG, et al, 2019. Main characteristics and applications of HM2000 profile float[J]. Digital Ocean & Underwater Warfare, 2(2): 20-27 (in Chinese with English abstract).
[6]	石洪波, 陈雨文, 陈鑫, 2019. SMOTE过采样及其改进算法研究综述[J]. 智能系统学报, 14(6): 1073-1083.
	SHI HONGBO, CHEN YUWEN, CHEN XIN, 2019. Summary of research on SMOTE oversampling and its improved algorithms[J]. CAAI Transactions on Intelligent Systems, 14(6): 1073-1083 (in Chinese with English abstract).
[7]	谭哲韬, 张斌, 吴晓芬, 等, 2022. 海洋观测数据质量控制技术研究现状及展望[J]. 中国科学: 地球科学, 52(3): 418-437.
	TAN ZHETAO, ZHANG BIN, WU XIAOFEN, et al, 2022. Quality control for ocean observations: from present to future[J]. Science China Earth Sciences, 65(2): 215-233 (in Chinese with English abstract).
[8]	王东晓, 邱春华, 舒业强, 等, 2022. 南海环流多尺度动力过程演变特征与机制研究进展[J]. 海洋科学进展, 40(4): 605-623.
	WANG DONGXIAO, QIU CHUNHUA, SHU YEQIANG, et al, 2022. Progress in the dynamic process and mechanisms of multi-scale currents in the South China Sea[J]. Advances in Marine Science, 40(4): 605-623 (in Chinese with English abstract).
[9]	许自舟, 宋德瑞, 赵辉, 等, 2009. 海洋环境监测数据质量计算机控制方法研究[J]. 海洋环境科学, 28(3): 320-323.
	XU ZIZHOU, SONG DERUI, ZHAO HUI, et al, 2009. Technique study on data quality control in sea environmental monitoring by computer[J]. Marine Environmental Science, 28(3): 320-323 (in Chinese with English abstract).
[10]	杨剑锋, 乔佩蕊, 李永梅, 等, 2019. 机器学习分类问题及算法研究综述[J]. 统计与决策, 35(6): 36-40.
	YANG JIANFENG, QIAO PEIRUI, LI YONGMEI, et al, 2019. A review of machine-learning classification and algorithms[J]. Statistics & Decision, 35(6): 36-40 (in Chinese with English abstract).
[11]	张桐, 2018. 基于Argo数据的海洋温度预测方法研究[D]. 长春: 吉林大学: 1-2.
	ZHANG TONG, 2018. Research of ocean temperature prediction based on Argo data[D]. Changchun: Jilin University: 1-2 (in Chinese with English abstract).
[12]	张雪薇, 韩震, 2022. Argo温度数据的ConvGRU模型预测分析[J]. 海洋环境科学, 41(4): 628-635.
	ZHANG XUEWEI, HAN ZHEN, 2022. Prediction and analysis of Argo temperature data by ConvGRU model[J]. Marine Environmental Science, 41(4): 628-635 (in Chinese with English abstract).
[13]	CASTELÃO G P, 2021. A machine learning approach to quality control oceanographic data[J]. Computers & Geosciences, 155: 104803.
[14]	CUMMINGS J A, 2011. Ocean data quality control[M]// SCHILLER A, BRASSINGTON G B. Operational oceanography in the 21st Century. Dordrecht: Springer: 91-121.
[15]	DOBBIN K K, SIMON R M, 2011. Optimally splitting cases for training and testing high dimensional classifiers[J]. BMC Medical Genomics, 4(1): 31.
[16]	FAWCETT T, 2006. An introduction to ROC analysis[J]. Pattern Recognition Letters, 27(8): 861-874.
[17]	FOSIĆ I, ŽAGAR D, GRGIĆ K, et al, 2023. Anomaly detection in NetFlow network traffic using supervised machine learning algorithms[J]. Journal of Industrial Information Integration, 33: 100466.
[18]	GOOD S A, MARTIN M J, RAYNER N A, 2013. EN4: Quality controlled ocean temperature and salinity profiles and monthly objective analyses with uncertainty estimates[J]. Journal of Geophysical Research: Oceans, 118(12): 6704-6716.
[19]	GULLI A, PAL S, 2017. Deep learning with Keras[M]. Birmingham: Packt Publishing: 1-2.
[20]	HAN HUI, WANG WENYUAN, MAO BINGHUAN, 2015. Borderline-SMOTE: A new over-sampling method in imbalanced data sets learning[C]// International conference on intelligent computing. Hefei, China: Springer: 878-887.
[21]	INGLEBY B, HUDDLESTON M, 2007. Quality control of ocean temperature and salinity profiles-Historical and real-time data[J]. Journal of Marine Systems, 65(1-4): 158-175.
[22]	Intergovernmental Oceanographic Commission, 2010. GTSPP real-time quality control manual. Revised edition 2010[Z]. Paris: United Nations Educational, Scientific and Cultural Organization.
[23]	KAMIKAWAJI Y, MATSUYAMA H, FUKUI K I, et al, 2016. Decision tree-based feature function design in conditional random field applied to error detection of ocean observation data[C]// 2016 IEEE symposium series on computational intelligence (SSCI). Athens, Greece: IEEE: 1-8.
[24]	LIU ZHENGHONG, XING XIAOGANG, CHEN ZHAOHUI, et al, 2023. Twenty years of ocean observations with China Argo[J]. Acta Oceanologica Sinica, 42(2): 1-16.
[25]	MIERUCH S, DEMIREL S, SIMONCELLI S, et al, 2021. SalaciaML: A deep learning approach for supporting ocean data quality control[J]. Frontiers in Marine Science, 8: 611742.
[26]	NGUYEN Q H, LY H B, HO L S, et al, 2021. Influence of data splitting on performance of machine learning models in prediction of shear strength of soil[J]. Mathematical Problems in Engineering, 2021: 4832864.
[27]	ONO S, MATSUYAMA H, FUKUI K I, et al, 2015. Error detection of oceanic observation data using sequential labeling[C]// 2015 IEEE international conference on data science and advanced analytics (DSAA). Paris, France: IEEE: 1-8.
[28]	PAWLOWICZ R, 2013. Key physical variables in the ocean: temperature, salinity, and density[J]. Nature Education Knowledge, 4(4): 13.
[29]	PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al, 2011. Scikit-learn: Machine learning in Python[J]. The Journal of Machine Learning Research, 12: 2825-2830.
[30]	RASCHKA S, MIRJALILI V, 2019. Python machine learning:Machine learning and deep learning with Python, scikit-learn, and TensorFlow 2[M]. 3rd ed. Birmingham: Packt Publishing.
[31]	TAN ZHETAO, CHENG LIJING, GOURETSKI V, et al, 2023. A new automatic quality control system for ocean profile observations and impact on ocean warming estimate[J]. Deep Sea Research Part Ⅰ: Oceanographic Research Papers, 194: 103961.
[32]	UDAYA BHASKAR T V S, RAO E P R, SHESU R V, et al, 2012. A note on three way quality control of Argo temperature and salinity profiles - A semi-automated approach at INCOIS[J]. International Journal of Earth Sciences and Engineering, 5(6): 1510-1514.
[33]	UDAYA BHASKAR T V S, SHESU R V, BOYER T P, et al, 2017. Quality control of oceanographic in situ data from Argo floats using climatological convex hulls[J]. MethodsX, 4: 469-479.
[34]	YU FANGJIE, ZHANG XUAN, CHEN XIN, et al, 2020. Altimetry-derived ocean thermal structure reconstruction for the Bay of Bengal cyclone season[J]. Ocean Dynamics, 70(11): 1449-1459.
[35]	ZHANG QI, QIAN CHENYAN, DONG CHANGMING, 2020. A machine learning approach to quality-control Argo temperature data[J]. Atmospheric and Oceanic Science Letters, 16(4): 100292.

模型名称	TPR	TNR
LOF	0.94	0.05
IF	0.85	0.08
SVM	0.99	0.33
DT	0.79	0.64
KNN	0.96	0.60
RF	0.99	0.48
MLP	0.94	0.90
SGD	0.86	0.88
GNB	0.90	0.72
XGB	0.98	0.61
LGB	0.98	0.66
CatBoost	0.97	0.69
NGB	0.92	0.68

模型名称	TPR	TNR	AUC
LOF	0.94	0.09	0.52
IF	0.95	0.06	0.51
SVM	0.99	0.28	0.63
DT	0.99	0.59	0.79
KNN	0.98	0.64	0.81
RF	0.99	0.55	0.77
MLP	0.92	0.89	0.91
SGD	0.72	0.81	0.76
GNB	0.86	0.60	0.73
XGB	0.90	0.56	0.72
LGB	0.98	0.74	0.86
CatBoost	0.96	0.81	0.89
NGB	0.96	0.74	0.85

模型名称	TPR	TNR
规则集	0.99	0.11
MLP	0.94	0.90
MFP	0.77	0.92
GCH	0.80	0.70
RS-MLP	0.93	0.96

模型名称	TPR	TNR	AUC
规则集	0.99	0.12	0.55
MLP	0.92	0.89	0.91
MFP	0.69	0.60	0.64
GCH	0.64	0.65	0.65
RS-MLP	0.92	0.95	0.94

基于规则集和多层感知机的Argo温度数据质量控制方法^*

Rule set and multilayer perceptron based quality control method for Argo temperature data^*

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 35

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	孙泽铭, 韩树宗, 王明杰, 苏翰祥. 不同路径台风对中国近海海温的影响特征统计分析[J]. 热带海洋学报, 2024, 43(5): 17-31.
[2]	林桂焕, 严幼芳, 刘颖. 印度尼西亚—澳大利亚海盆上层海洋层结变化及其影响因素[J]. 热带海洋学报, 2024, 43(4): 57-67.
[3]	张现清, 李彩, 周雯, 刘聪, 许占堂, 曹文熙, 杨跃忠. 基于体散射函数及吸收系数的南海水体漫射衰减系数研究^*[J]. 热带海洋学报, 2023, 42(3): 86-95.
[4]	唐超礼, 陶鑫华, 魏圆圆, 戴聪明, 魏合理. 东亚及西太平洋地表温度时空模态分析及预测研究^*[J]. 热带海洋学报, 2022, 41(6): 183-192.
[5]	陈克海, 解学通, 张金兰, 郑艳. 一种温度相关的HY-2A散射计地球物理模型函数[J]. 热带海洋学报, 2022, 41(2): 90-102.
[6]	李杨, 黄鹏起, 鲁远征, 屈玲, 郭双喜, 岑显荣, 周生启, 张佳政, 丘学林. 基于精细温度观测的南海东北部陆坡-深海盆底层湍流混合^*[J]. 热带海洋学报, 2022, 41(1): 62-74.
[7]	汪浩, 王静, 郑佳喻. 南印度洋热带气旋快速增强事件气候特征及其年际变率[J]. 热带海洋学报, 2022, 41(1): 94-105.
[8]	郑承志, 左丽明, 马旺, 朱琴, 王火火, 吕颂辉, 陈亨, 黄凯旋. 不同温度下抑食金球藻、中肋骨条藻和海洋卡盾藻间的相互作用[J]. 热带海洋学报, 2021, 40(3): 124-131.
[9]	张培军, 周水华, 梁昌霞. 基于卫星遥感海温数据的南海SST预报误差订正[J]. 热带海洋学报, 2020, 39(6): 57-65.
[10]	刘磊, 许兰芳, 管红香, 孙治雷, 王利波, 茅晟懿, 刘丽华, 吴能友. 冲绳海槽中部8.2ka以来GDGTs组成及温度重建*[J]. 热带海洋学报, 2020, 39(6): 77-92.
[11]	严瑾, 林明晴, 曹文浩, 林岳光, 邹立. 温度对网纹纹藤壶幼虫发育和附着的生物学效应[J]. 热带海洋学报, 2020, 39(5): 55-61.
[12]	刘叶取, 唐世林. 夏季琼东上升流的高频变化特征研究^*[J]. 热带海洋学报, 2020, 39(4): 1-12.
[13]	薛文璟, 余锦华, 陈林. 2016与1998年春季北大西洋海表温度异常的差异及成因[J]. 热带海洋学报, 2020, 39(3): 19-30.
[14]	陈武阳, 李骏旻, 何庆友, 唐世林, 施平. 南海岛礁周边海域表面叶绿素浓度的时空特征[J]. 热带海洋学报, 2019, 38(6): 21-28.
[15]	方珂,余锦华. 北半球热带中太平洋与印度洋海表温度梯度对夏季西北太平洋热带气旋生成频数变化的影响[J]. 热带海洋学报, 2019, 38(5): 42-51.

基于规则集和多层感知机的Argo温度数据质量控制方法*

Rule set and multilayer perceptron based quality control method for Argo temperature data*

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 35

相关文章 15

Metrics

本文评价

推荐阅读 0

基于规则集和多层感知机的Argo温度数据质量控制方法^*

Rule set and multilayer perceptron based quality control method for Argo temperature data^*