Large Language Models

Large language models, or LLMs, generate text by learning patterns in extensive training material. For Iran, an important question is how well their capabilities transfer to Persian: a language whose writing conventions and cultural references cannot be assessed adequately through English tests alone. Persian-language capability is also distinct from Iranian ownership or development; a multilingual model can be evaluated in Persian regardless of where it was built. s1s2

A study published at LREC-COLING in 2024 compared general-purpose models with systems trained for particular Persian tasks. Its results resisted a simple ranking of large versus small: general-purpose models showed strengths in reasoning and general knowledge, while smaller models fine-tuned for particular tasks often remained competitive or performed better. Translation into English improved results in some experiments. These findings describe the tested models and procedures, rather than establishing a permanent hierarchy. s1

[s1] [s2]

Persian evaluation increasingly asks whether models understand material that cannot simply be transferred from an English-language setting. The 2025 paper Advancing Persian LLM Evaluation introduced PeKA and PK-BETS, addressing subjects including history, literature, and cultural knowledge. It also examined different answer formats, recognizing that selecting an option and composing an explanation place different demands on a system. s2

That distinction matters for readers encountering an impressive demonstration. A model may recognize a familiar fact while struggling to explain a literary reference or interpret a context-dependent question. The authors also identified possible contamination of existing benchmarks as a limitation: material already encountered during training can make evaluation less informative about performance on genuinely unfamiliar questions. s2

A separate FarsEval-PKBETS preprint describes questions covering Persian language, social knowledge, professional domains, ethics, and other subjects. Its authors report substantial shortcomings among the models they tested. The study provides evidence about those experiments; it does not establish that every later model has the same weaknesses. Together, these projects show why locally grounded questions are an essential complement to translated international benchmarks. s3

[s2] [s3]

The practical implication of this research is that Persian capability should be examined at the level of the intended task. An organization classifying messages has a different evaluation problem from one producing explanations about Iranian literature. The earlier comparison with fine-tuned systems makes model size alone an insufficient selection criterion, while the culturally focused benchmarks make general multilingual fluency an insufficient substitute for local knowledge. This is an editorial inference from the studies, rather than a measured statement about company purchasing decisions. s1s2

For international readers, the most useful picture is therefore an evolving research field with increasingly specific evaluation tools. Published benchmarks help expose weaknesses and organize comparisons. They do not, by themselves, establish national adoption, commercial revenue, or reliable performance in an organization's own documents and workflows. Those outcomes require separate evidence. s1s2s3

[s1] [s2] [s3]