Linguistics and Large Language Models

Zeynab Mohammadebrahimi Jahromi; Arezoo Haghbin; Motahareh Ramezani Khouzestani

doi:10.63053/ijset.95

Authors

Zeynab Mohammadebrahimi Jahromi 1. Department of linguistics, Faculty of literature and Humanities, Shahid Beheshti University, Tehran, Iran.
Arezoo Haghbin 2. Department of linguistics, Faculty of literature and Humanities, Shahid Beheshti University, Tehran, Iran.
Motahareh Ramezani Khouzestani 3. Faculty of Computer Engineering, Natural Language Processing Lab, Shahid Beheshti University, Tehran, Iran.

DOI:

https://doi.org/10.63053/ijset.95

Keywords:

Large Language Models, Computational Linguistics, challenges, solutions

Abstract

Given the development and progress of artificial intelligence in large language models, this article attempts to first introduce large language models and the importance of linguistics on these language models. After that, in separate sections, we will examine the important and fundamental issues of large language models in relation to linguistics. Examining the challenges and issues that these models have and the influence of linguistics on large language models will be the main goal of our work. Some of the solutions that exist for these challenges are presented and we try to provide solutions for other challenges that do not yet have a solution. Proposed solutions to the challenges of large language models can be grouped into three areas: interdisciplinary collaboration, which helps reduce bias and improve interpretability; user-centric design, which aligns models with real-world needs through direct user involvement; and evolutionary trial-and-error approaches, where models are continuously refined with updated data and feedback. Together, these strategies foster the development of fairer, more interpretable, and context-sensitive LLMs.

References

• Aberer, K. (2001). P-Grid: A self-organizing access structure for P2P information systems. In Cooperative Information Systems,pages 179–194. Springer.

• Ashley-Rollman, M. (2010). personal communication.

• Ashley-Rollman, M., Lee, P., Goldstein, S. C., Pillai, P., and Campbell, J. (2009). A language for large ensembles of independently executing nodes. In International Conference on Logic Programming, pages 265–280. Springer.

• Atul, A. (2009). Compact Implementation of Distributed Inference Algorithms for Network. Master’s thesis, University of California, Berkeley.

• Aaron Craig, Alex Potanin, Lindsay Groves, and Jonathan Aldrich. Capabilities: Effects for Free. In Formal Methods and Software Engineering, 2018. ISBN 978-3-030-02450-5.3.7

• Andrej Bauer and Matija Pretnar. Programming with Algebraic Effects and Handlers. Journal of Logical and Algebraic Methods in Programming, 84(1):108 – 123, 2015. ISSN2352-2208. doi: http://dx.doi.org/10.1016/j.jlamp.2014.02.001. URL http://www.sciencedirect.com/science/article/pii/S2352220814000194. 3.8.

• Brunskill, E., Kollar, T., and Roy, N. (2007). Topological mapping using spectral clustering and classification. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007, pages 3491–3496.

• Butler, Z., Corke, P., Peterson, R., and Rus, D. (2004). Networked cows: Virtual fences for controlling cows. In WAMES 2004,volume i. Citeseer.

• Jack B. Dennis and Earl C. Van Horn. Programming Semantics forn Multiprogrammed Computations. Communications of the ACM, 9(3):143–155, 1966. 2.6

• Ramachandran, A., Feamster, N., and Vempala, S. (2007). Filtering spam with behavioral blacklisting. In CCS ’07: Proceedings of the 14th ACM conference on Computer and communications security, pages 342–351, New York, NY, USA. ACM.

• Dominique Devriese, Frank Piessens, and Lars Birkedal. Reasoning about Object Capabilities with Logical Relations and Effect Parametricity. In European Symposium on Security and Privacy, 2016. 1, 2.9, 3.8

• Christos Dimoulas, Scott Moore, Aslan Askarov, and Stephen Chong. Declarative Policies for Capability Control. In Computer Security Foundations Symposium, 2014. 2.9, 3.8, 5.1

• Darya Melicher, Yangqingwei Shi, Alex Potanin, and Jonathan Aldrich. A CapabilityBased Module System for Authority Control. In European Conference on Object-Oriented Programming, 2017. 2.6.2

• Darya Melicher, Yangqingwei Shi, Alex Potanin, and Jonathan Aldrich. A CapabilityBased Module System for Authority Control. Technical Report CMU-ISR-17-106, Carnegie Mellon University, 2017. URL http://reports-archive.adm.cs. cmu.edu/anon/isr2017/abstracts/17-106.html. 2.6.2

• Adrian Mettler, David Wagner, and Tyler Close. Joe-E: A Security-Oriented Subset of Java. In Network and Distributed System Security Symposium, 2010. 2.9, 3.8, 5.1

• Heather Miller, Philipp Haller, and Martin Odersky. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. In European Conference on ObjectOriented Programming, 2014. 2.5.1

• Gordon Plotkin and John Power. Algebraic Operations and Generic Effects. Applied Categorical Structures, 11(1):69–94, 2003. ISSN 1572-9095. doi: 10.1023/A:1023064908962. URL https://doi.org/10.1023/A:1023064908962. 3.8

• Gordon Plotkin and Matija Pretnar. Handlers of Algebraic Effects. In Programming Languages and Systems, 2009. ISBN 978-3-642-00590-9. 3.8

• Vineet Rajani, Deepak Garg, and Tamara Rezk. On Access Control, Capabilities, Their Equivalence, and Confused Deputy Attacks. In 2016 IEEE 29th Computer Security Foundations Symposium (CSF), pages 150–163, June 2016. doi: 10.1109/CSF.2016.18. 2.9

• Jonathan A. Rees. A Security Kernel Based on the Lambda-Calculus. Technical report, Massachusetts Institute of Technology, 1996. 2.9

• John M. Rushby. Design and Verification of Secure Systems. In Symposium on Operating Systems Principles, 1981. ISBN 0-89791-062-1. 1

• David Wagner and Dean Tribble. A Security Analysis of the Combex DarpaBrowser Architecture. http://combex.com/papers/darpa-review/security-review. pdf, March 2002. 2.9, 5.1 Esther Wang and Jonathan Aldrich. Capability Safe Reflection for the Wyvern Language. In Workshop on Meta-Programming Techniques and Reflection, 2016. 2.4

• Robert N. M. Watson. Exploiting Concurrency Vulnerabilities in System Call Wrappers.In USENIX Workshop on Offensive Technologies, 2007. 1

• Yizhou Zhang and Andrew C. Myers. Abstraction-safe Effect Handlers via Tunneling. Proceedings of the ACM on Programming Languages, 3(POPL):5:1–5:29, 2019. ISSN 2475- 1421. doi: 10.1145/3290318. URL http://doi.acm.org/10.1145/3290318.3.8

• Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, and Fei Sha. Generate-and-retrieve: Use your predictions to improve retrieval for semantic parsing. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung-Hoon Na, editors, Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 4946–4951. International Committee on Computational Linguistics, 2022. URL https://aclanthology.org/2022.coling-1.438. 107

• Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. Repocoder: Repository-level code completion through iterative retrieval and generation. CoRR, abs/2303.12570, 2023. doi: 10.48550/arXiv.2303.12570. URL https://doi.org/10.48550/arXiv.2303.12570. 107

• Ruohong Zhang, Luyu Gao, Chen Zheng, Zhen Fan, Guokun Lai, Zheng Zhang, Fangzhou Ai, Yiming Yang, and Hongxia Yang. A self-enhancement approach for domain-specific chatbot training via knowledge mining and digest. CoRR, abs/2311.10614, 2023. doi: 10.48550/ARXIV.2311.10614. URL https://doi.org/10.48550/arXiv.2311.

• Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards a unified multi-dimensional evaluator for text generation. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2023–2038. Association for Computational Linguistics, 2022. URL https://aclanthology.org/2022. emnlp-main.131. 103

• Zexuan Zhong, Tao Lei, and Danqi Chen. Training language models with memory augmentation. CoRR, abs/2205.12674, 2022. doi: 10.48550/arXiv.2205.12674. URL https://doi.org/10.48550/arXiv.2205.12674. 6

• Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: less is more for alignment. CoRR, abs/2305.11206, 2023. doi: 10.48550/ARXIV.2305.11206. URL https://doi.org/10.48550/arXiv.2305.