Evaluating ChatGPT's Translation of Harry Potter: A Qualitative Study of Translation Techniques, Accuracy, and Acceptability
Abstract
This research investigates the translation techniques employed by ChatGPT in translating J.K. Rowling's Harry Potter and the Sorcerer's Stone, focusing on the accuracy and acceptability of the translations produced. Utilizing a qualitative approach, the study analyzes data from the original English text, the Indonesian translation generated by ChatGPT, and feedback from five evaluators based on predetermined criteria. The data analysis process involved comparing the source and target texts to categorize translation strategies and calculating average scores from raters. The findings reveal that ChatGPT effectively utilized various translation strategies, with Translation by Paraphrase Using a Related Word being the most used (62%). Other strategies included Translation by a More Neutral or Less Expressive Word (24%) and Translation by a More General Word (8%). The overall accuracy score was relatively high; however, challenges arose in maintaining acceptability, particularly regarding expressive tones and cultural nuances. This study contributes significantly to understanding AI's role in literary translation, showcasing both its potential and limitations in handling complex literary elements. By highlighting these aspects, it advances the discourse on AI's evolving capabilities in creative fields.
Downloads
References
Apriliya, R., Simamora, A., Budiarti, V., & Syafitri, Y. D. T. (2020). Translation Based on Ogden and Richard’s Semantic Triangle Theory: How the Idea Is Connected to Real-World Objects. Journal of English Language Teaching and Literature (JELITA), 1(1), 20–26.
Badruzaman, A,& Halim A. (2021). Strategies Used in Translating the Harry Potter and the Sorcerer’s Stone Novel into Indonesian. Bulletin of Science Education 1(1), 60–67. https://dx.doi.org/10.512
Bahdanau, D. Cho, K.H. & Bengi, Y.. (2015). Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 1–15. https://doi.org/10.48550/arXiv.1409.0473
Baker, Mona. 2018. 134 Development In Other Words:A Coursebook on Translation. Routledge
Biswa, P., Neogi, S.G., Daniel, A. & Mitra, A. (2024).Evaluating Generative Artificial Intelligence on Multilingual Sentiment Analysis.Journal of Electrical Systems 20(3), 3502–3513. https://doi.org/10.52783/jes.4986
Castilho, S . Mallon, C. Q. Meister, R. &Yue, S. (2023). Do online Machine Translation Systems Care for Context? What About a GPT Model?. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 393–417, Tampere, Finland. European Association for Machine Translation.
Chen, X. (2024). Research on the Application of Artificial Intelligence in Translation Courses. International Journal of Education and Humanities, 12(1), 41-44. https://doi.org/10.54097/3c1b8w36
Darma, A.& Pratama, Y.(2015)Aspek Legibilitas Dalam Penerjemahan. RETORIKA: Jurnal Ilmu Bahasa 1(1): 137–54. https://doi.org/10.22225/jr.1.1.20.137-154
Di, J., & Nida, E. A. (2006). On translation: An expanded edition.
Hang, T. (2018). Translating proper names in a literary text: a case of Harry Potter novel in Vietnam. VNU journal of foreign studies, 34(2). https://doi.org/10.25073/2525-2445/vnufs.4245
Harjanti, D. (2020). Indonesian Translation Acceptability of Directive Utterances in Harry Potter Novel and the Application in Translation Teaching. KnE Social Sciences, 4(4). https://doi.org/10.18502/kss.v4i4.6467
Hasyim, M., Saleh, F. Yusuf, R. , & Abbas, A. (2021). Artificial Intelligence: Machine Translation Accuracy in Translating French-Indonesian Culinary Texts. (IJACSA) International Journal of Advanced Computer Science and Applications, 12(3), 186-191. http://dx.doi.org/10.2139/ssrn.3816594
Khanna, P.; Dhillon, G.; Buddhavarapu, V.; Verma, R.; Kashyap, R.; Grewal, H. (2024). Artificial Intelligence in Multilingual Interpretation and Radiology Assessment for Clinical Language Evaluation (AI-MIRACLE). J. Pers. Med., 14, 923. https://doi.org/10.3390/jpm14090923
Kocmi,T., & Federmann, c, (2023). Large Language Models Are State-of-the-Art Evaluators of Translation Quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pp 193–203, Tampere, Finland. European Association for Machine Translation.
Liu, Y., Gu, J., Goyal, N., Li, X.,Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020). Multilingual Denoising Pre-training for Neural Machine Translation. Transactions of the Association for Computational Linguistics, 8, 726–742. https://doi.org/10.1162/tacl_a_00343
Lopez, A.(2008). Statistical machine translation. ACM Comput. Surv. 40(3), 1-49 https://doi.org/10.1145/1380584.1380586
Nababan, M., Nuraeni, A., & Sumardiono, S. (2012). Pengembangan Model Penilaian Kualitas Terjemahan. Kajian Linguistik dan Sastra 24(1), 39–57.
Newmark, P. (2009). The linguistic and communicative stages in translation theory. In The Routledge companion to translation studies (pp. 34-49). Routledge.
Ruhmadi, A., & Al Farisi, M. (2023). Analisis Kesalahan Morfologi Penerjemahan Arab–Indonesia pada ChatGPT. Aphorisme: Journal of Arabic Language, Literature, and Education, 4(1), 55-75. https://doi.org/10.37680/aphorisme.v4i1.3148
Sudarmaji, S.,i,Santoso, I., &Mulyati, R.E.S.(2023) Analisis Kesalahan Hasil Terjemahan Mesin Penerjemah Teks Bahasa Jerman Ke Dalam Bahasa Indonesia. Diglosia: Jurnal Kajian Bahasa, Sastra, dan Pengajarannya 6(2), 483–500. https://doi.org/10.30872/diglosia.v6i2.668
Suwondo, A. S., & Hamsia, W. (2023). Process of Indonesian Transnational Students to be Polyglots and Their Language Varieties in Japan. Jelita, 4(2), 68–84.
Tan, F.,&Wang, H..(2024)A Semantic Context-Aware Automatic Quality Scoring Method for Machine Translation Based on Pretraining Language Model. IEEE Access, 12, 72023-72033, https://doi.org/10.1109/ACCESS.2024.3402360.
Wang, L., Gaikar Vilas, B., Sun, H., & Erokhin, V. (2023). The Impacts and Challenges of Artificial Intelligence Translation Tool on Translation Professionals. SHS Web of Conferences, 163, 2021-. https://doi.org/10.1051/shsconf/202316302021
Zhao J. (2021). Optimization of Machine Online Translation System Based on Deep Convolution Neural Network Algorithm. Comput Intell Neurosci. 2021 Sep 29;2021:7388825. https://doi.org/10.1155/2021/7388825.
