Hadi Babalou

Hi! I am Hadi Babalou, a senior bachelor student in Computer Engineering at the University of Tehran. Through my coursework, research experiences, and personal projects, I have developed a strong interest in the fields of Artificial Intelligence and Machine Learning.

Recently, my curiosity has deepened in the area of Trustworthy AI, particularly in topics such as robustness, interpretability, and fairness of machine learning models, with a specific focus on large language models, medical applications, and astrophysical applications. My goal is to expand my knowledge in these areas and contribute to advancing the field, working under the guidance of leading experts in the community.

selected publications

Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT

Amirhossein Abaskohi, Sara Baruni, Mostafa Masoudi, and 9 more authors

Accepted in LREC-COLING 2024, Apr 2024

Abs Bib PDF Code

This paper explores the efficacy of large language models (LLMs) for Persian. While ChatGPT and consequent LLMs have shown remarkable performance in English, their efficiency for more low-resource languages remains an open question. We present the first comprehensive benchmarking study of LLMs across diverse Persian language tasks. Our primary focus is on GPT-3.5-turbo, but we also include GPT-4 and OpenChat-3.5 to provide a more holistic evaluation. Our assessment encompasses a diverse set of tasks categorized into classic, reasoning, and knowledge-based domains. To enable a thorough comparison, we evaluate LLMs against existing task-specific fine-tuned models. Given the limited availability of Persian datasets for reasoning tasks, we introduce two new benchmarks: one based on elementary school math questions and another derived from the entrance exams for 7th and 10th grades. Our findings reveal that while LLMs, especially GPT-4, excel in tasks requiring reasoning abilities and a broad understanding of general knowledge, they often lag behind smaller pre-trained models fine-tuned specifically for particular tasks. Additionally, we observe improved performance when test sets are translated to English before inputting them into GPT-3.5. These results highlight the significant potential for enhancing LLM performance in the Persian language. This is particularly noteworthy due to the unique attributes of Persian, including its distinct alphabet and writing styles.
@article{abaskohi2024benchmarkinglargelanguagemodels, title = {Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT}, author = {Abaskohi, Amirhossein and Baruni, Sara and Masoudi, Mostafa and Babalou, Mohammad Hadi and Abbasi, Nesa and Edalat, Ali and Kamahi, Sepehr and Sani, Samin Mahdizadeh and Naghavian, Nikoo and Namazifard, Danial and Sadeghi, Pouya and Yaghoobzadeh, Yadollah}, year = {2024}, month = apr, eprint = {2404.02403}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2404.02403}, journal = {Accepted in LREC-COLING 2024}, }