基于CNN-NLSTM的脑电信号注意力状态分类方法

沈振乾,李文强,任甜甜,王瑶,赵慧娟

PDF(13369 KB)
PDF(13369 KB)
中文信息学报 ›› 2024, Vol. 38 ›› Issue (4) : 38-49.
语言认知计算模型

基于CNN-NLSTM的脑电信号注意力状态分类方法

  • 沈振乾1,李文强1,任甜甜2,王瑶2,赵慧娟2
作者信息 +

EEG-based Attention States Classification via CNN-NLSTM Model

  • SHEN Zhenqian1, LI Wenqiang1, REN Tiantian2, WANG Yao2, ZHAO Huijuan2
Author information +
History +

摘要

通过脑电信号进行注意力状态检测,对扩大脑-机接口技术的应用范围具有重要意义。为了提高注意力状态的分类准确率,该文提出一种基于CNN-NLSTM的脑电信号分类模型。首先采用Welch方法获得脑电信号的功率谱密度特征并将其表示为二维灰度图像。然后使用卷积神经网络从灰度图像中学习表征注意力状态的特征,并将相关特征输入到嵌套长短时记忆神经网络依次获得所有时间步骤的注意力特征。最后将两个网络依次连接来构建深度学习框架进行注意力状态分类。实验结果表明,该文所提出的模型通过进行多次5-折交叉验证评估后得到89.26%的平均分类准确率和90.40%的最大分类准确率,与其他模型相比具有更好的分类效果和稳定性。

Abstract

Electroencephalogram (EEG)-based attention states detection is of great significance for expanding the application the brain-computer interface. In this paper, a classification approach is presented to improve the accuracy of EEG-based attention states classification via the Convolutional Neural Network and Nested Long Short-term Memory (CNN-NLSTM) model. First, the power spectral density of the EEG signals is obtained by the Welch method and represented as a two-dimensional grayscale image. Then, the CNN is used to learn features that represent attention states from grayscale images, and resulted features are input into the NLSTM neural network to sequentially obtain attention characteristics for all time steps. Finally, the two networks are connected to build a deep learning framework for attentional states classification. The experimental results show that the proposed model evaluated by multiple 5-fold cross-validation outperforms other models by an average accuracy of 89.26% and a maximum accuracy of 90.40%.

关键词

注意力状态 / 脑电信号 / 卷积神经网络 / 嵌套长短时记忆神经网络 / 功率谱密度

Key words

attention state / electroencephalogram signal / convolutional neural network / nested long short-term memory network / power spectral density

引用本文

EndNote

Ris (Procite)

Bibtex

导出引用
沈振乾,李文强,任甜甜,王瑶,赵慧娟. 基于CNN-NLSTM的脑电信号注意力状态分类方法. 中文信息学报. 2024, 38(4): 38-49
SHEN Zhenqian, LI Wenqiang, REN Tiantian, WANG Yao, ZHAO Huijuan. EEG-based Attention States Classification via CNN-NLSTM Model. Journal of Chinese Information Processing. 2024, 38(4): 38-49

参考文献

[1] MOHAMMADPOUR M, MOZAFFARI S. Classification of EEG-based attention for brain computer interface [C]//Proceedings of the 3rd Iranian Conference on Intelligent Systems and Signal Processing Shahrood: IEEE, 2017: 34-37.
[2] DELVIGNE V, WANNOUS H, DUTOIT T, et al. PhyDAA: Physiological dataset assessing attention[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(5): 2612-2623.
[3] PAPAKOSTAS M, TSIAKAS K, GIANNAKOPOULOS T, et al. Towards predicting task performance from EEG signals [C]//Proceedings of the IEEE International Conference on Big Data. Boston: IEEE, 2017: 4423-4425.
[4] HWANG H J, KIM S, CHOI S, et al. EEG-based brain-computer interfaces: A thorough literature survey[J]. International Journal of Human-computer Interaction, 2013, 29(12): 814-826.
[5] MOGHADDARI M, LIGHVAN M Z, DANISHVAR S. Diagnose ADHD disorder in children using convolutional neural network based on continuous mental task EEG[J]. Computer Methods and Programs in Biomedicine, 2020, 197: 105738.
[6] CANTISANI G, TRGOAT G, ESSID S, et al. MAD-EEG: An EEG dataset for decoding auditory attention to a target instrument in polyphonic music[C]//Proceedings of the Workshop on Speech, Music and Mind. ISCA, 2019: 51-55.
[7] WOLPAW J R, BIRBAUMER N, MCFARLAND D J, et al. Brain-computer interfaces for communication and control[J]. Clinical Neurophysiology, 2002, 113(6): 767-791.
[8] WALTER S, QUIGLEY C, ANDERSEN S K, et al. Effects of overt and covert attention on the steady-state visual evoked potential[J]. Neuroscience Letters: An International Multidisciplinary Journal Devoted to the Rapid publication of Basic research in the Brain Sciences, 2012, 519(1): 37-41.
[9] ARNS M, CONNERS C K, KRAEMER H C. A decade of EEG theta/beta ratio research in ADHD: A meta-analysis[J]. Journal of Attention Disorders, 2013, 17(5): 374-383.
[10] ALBARES M, CRIAUD M, WARDAK C, et al. Attention to baseline: Does orienting visuospatial attention really facilitate target detection? [J]. Journal of Neurophysiology, 2011, 106(2): 809-816.
[11] BAJAJ N, CARRIN J R, BELLOTTI F, et al. Automatic and tunable algorithm for EEG artifact removal using wavelet decomposition with applications in predictive modeling during auditory tasks[J]. Biomedical Signal Processing and Control, 2020, 55: 101624.
[12] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[13] YAO Z, WANG Z, LIU W, et al. Speech emotion recognition using fusion of three multi-task learning-based classifiers: HSF-DNN, MS-CNN and LLD-RNN[J]. Speech Communication, 2020, 120: 11-19.
[14] 王星,李超,陈吉.基于膨胀卷积神经网络模型的中文分词方法[J].中文信息学报,2019,33(09): 24-30.
[15] CAI S, SU E, SONG Y, et al. Low latency auditory attention detection with common spatial pattern analysis of EEG signals[C]//Proceedings of the Interspeech. ISCA, 2020: 2772-2776.
[16] MONTILLA-TROCHEZ D, SALAS R, BERTIN A, et al. Convolutional neural network for cognitive task prediction from EEG's auditory steady state responses [C]//Proceedings of the Congress on Robotics and Neuroscience. Proceedings of the 5th Congress on Robotics and Neuroscience. Valparaíso: CRoNe,2019: 44-50.
[17] TSIOURIS K M, PEZOULAS V C, ZERVAKIS M, et al. A long short-term memory deep learning network for the prediction of epileptic seizures using EEG signals[J]. Computers in Biology and Medicine, 2018, 99: 24-37.
[18] HOCHREITER S, SCHMIDHUBER J. Long short-term memory [J]. Neural Computation,1997, 9(8): 1735-1780.
[19] 张亚伟,吴良庆,王晶晶,等.基于多层LSTM融合的多模态情绪识别[J].中文信息学报,2022,36(05): 145-152.
[20] KURUVILA I, MUNCKE J, FISCHER E, et al. Extracting the locus of attention at a cocktail party from single-trial EEG using a joint CNN-LSTM model [EB/OL]. https: //ui.adsabs.harvard.edu/abs/2021arXiv210203957K/abstract[2022-08-10]
[21] BAJAJ N, CARRIN J R, BELLOTTI F. PhyAAt: Physiology of auditory attention to speech dataset [EB/OL]. https: //arxiv.org/abs/2005.11577[2022-08-14]
[22] ACCOU B, JALILPOUR MONESI M, MONTOYA J, et al. Modeling the relationship between acoustic stimulus and EEG with a dilated convolutional neural network[C]//Proceedings of the 28th European Signal Processing Conference. Amsterdam: IEEE,2021: 1175-1179.
[23] ACHARYA U R, FUJITA H, OH S L, et al. Application of deep convolutional neural network for automated detection of myocardial infarction using ECG signals[J]. Information Sciences, 2017, 415/416: 190-198.
[24] 汤鹏杰,王瀚漓,许恺晟.LSTM逐层多目标优化及多层概率融合的图像描述[J].自动化学报,2018,44(07): 1237-1249.
[25] WANG S, WANG X, WANG S, et al. Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting[J]. International Journal of Electrical Power & Energy Systems, 2019, 109: 470-479.
[26] IERACITANO C, MAMMONE N, BRAMANTI A, et al. A convolutional neural network approach for classification of dementia stages based on 2D-spectral representation of EEG recordings[J]. Neurocomputing, 2019, 323: 96-107.
[27] FRIES P, REYNOLDS J H, RORIE A E, et al. Modulation of oscillatory neuronal synchronization by selective visual attention [J]. Science, 2001, 291 (5508): 1560-1563.

基金

国家自然科学基金(61701342);天津市科技计划项目(22KPXMRC00060)
PDF(13369 KB)

507

Accesses

0

Citation

Detail
段落导航
相关文章

  • 版权所有 © 《中文信息学报》编辑部 网站统计

    地址：北京市海淀区中关村南四街4号　邮编：100190　电话：010-62562916　E-mail：cips@iscas.ac.cn

    本系统由北京玛格泰克科技发展有限公司设计开发

/

〈 〉