Keywords
How to Cite
Abstract
Annotatsiya. Maqolada matnlarni tasniflash modellarining sifatini oshirish maqsadida o'qitish ma'lumotlarini ko'paytirishning diqqat mexanizmi ballariga asoslangan tanlab so'z almashtirish algoritmi ko'rib chiqiladi. An'anaviy usullarda almashtiriladigan so'zlar tasodifiy tanlanadi, bu esa sinf belgisini bashorat qilishga ta'siri kam bo'lgan tokenlarning o'zgartirilishiga va past qiymatli sintetik namunalar hosil bo'lishiga olib keladi. Ko'rib chiqilayotgan yondashuvda har bir tokenning muhimlik darajasi ikki yo'nalishli rekurrent tarmoq asosidagi diqqat modeli orqali baholanadi, chegaraviy qiymatdan yuqori ballga ega so'zlargina NER va POS teglari mos keluvchi lug'at birliklari bilan almashtiriladi. Algoritm bosqichlari formal ifodalar va psevdokod ko'rinishida bayon etilgan, hisoblash murakkabligi tahlil qilingan. TREC va IMDb benchmark to'plamlarida RNN, CNN va BERT modellarida o'tkazilgan eksperimentlar tanlab almashtirish strategiyasi tasodifiy almashtirishga asoslangan usullardan ustunligini, eng katta samara esa ma'lumot tanqisligi sharoitida kuzatilishini ko'rsatdi.
References
1. Yu Y.J., Yoon S.J., Jun S.Y., Kim J.W. TABAS: Text augmentation based on attention score for text classification model // ICT Express. – 2022. – Vol. 8, No. 4. – P. 549–554. DOI: 10.1016/j.icte.2021.11.002.
2. Wei J., Zou K. EDA: Easy data augmentation techniques for boosting performance on text classification tasks // Proc. of EMNLP-IJCNLP 2019. – Hong Kong, 2019. – P. 6382–6388.
3. Xie Q., Dai Z., Hovy E., Luong M.T., Le Q.V. Unsupervised data augmentation for consistency training // arXiv:1904.12848. – 2019.
4. Kobayashi S. Contextual augmentation: Data augmentation by words with paradigmatic relations // Proc. of NAACL-HLT 2018. – New Orleans, 2018. – Vol. 2. – P. 452–457.
5. Kumar V., Choudhary A., Cho E. Data augmentation using pre-trained transformer models // arXiv:2003.02245. – 2020.
6. Sennrich R., Haddow B., Birch A. Improving neural machine translation models with monolingual data // Proc. of the 54th Annual Meeting of the ACL. – Berlin, 2016. – Vol. 1. – P. 86–96.
7. Edunov S., Ott M., Auli M., Grangier D. Understanding back-translation at scale // Proc. of EMNLP 2018. – Brussels, 2018. – P. 489–500.
8. Anaby-Tavor A., Carmeli B., Goldbraich E. et al. Do not have enough data? Deep learning to the rescue // Proc. of the 34th AAAI Conference on Artificial Intelligence. – New York, 2020. – P. 7383–7390.
9. Bahdanau D., Cho K., Bengio Y. Neural machine translation by jointly learning to align and translate // arXiv:1409.0473. – 2014.
10. Luong M.T., Pham H., Manning C.D. Effective approaches to attention-based neural machine translation // Proc. of EMNLP 2015. – Lisbon, 2015. – P. 1412–1421.
11. Vaswani A., Shazeer N., Parmar N. et al. Attention is all you need // Advances in Neural Information Processing Systems 30. – Long Beach, 2017. – P. 5998–6008.
12. Cho K., van Merriënboer B., Gulcehre C. et al. Learning phrase representations using RNN encoder–decoder for statistical machine translation // Proc. of EMNLP 2014. – Doha, 2014. – P. 1724–1734.
13. Kim Y. Convolutional neural networks for sentence classification // Proc. of EMNLP 2014. – Doha, 2014. – P. 1746–1751.
14. Devlin J., Chang M.W., Lee K., Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding // Proc. of NAACL-HLT 2019. – Minneapolis, 2019. – P. 4171–4186.
15. Shorten C., Khoshgoftaar T.M. A survey on image data augmentation for deep learning // Journal of Big Data. – 2019. – Vol. 6, Article 60.