CIOs Grapple with Subpar Global genAI Models
In the realm of generative AI, CIOs face a pressing challenge as they navigate the intricacies of deploying non-English language models. The surge in generative AI experiments within enterprises has led to a scenario where CIOs procure a multitude of large language models tailored for diverse geographies and languages. However, a stark reality emerges as these CIOs discover that non-English models significantly underperform compared to their English counterparts, even when sourced from the same vendor.
The primary reason underlying this disparity is the acute shortage of data available for training non-English models. Akhil Seth, the head of AI business development at UST, pointedly notes that the implementation of Large Language Models (LLMs) in languages other than English inherently suffers from reduced accuracy and relevance due to the vast differences in training sample sizes. This data scarcity manifests as diminished comprehensiveness, accuracy, and an increased occurrence of hallucinations within the models.
The Size Difference Can Be Extreme
The inadequacy of training datasets for non-English models is further emphasized by Vasi Philomin, VP and general manager for generative AI at Amazon Web Services (AWS). Philomin estimates that the training datasets for non-English models are typically “10 to 100 times smaller” than their English counterparts. While the exact data availability for training in a specific language may vary, Hans Florian from IBM suggests a practical indicator: the number of Wikipedia pages in that language often correlates with the available data volume.
Moreover, Mary Osborne, a senior product manager at SAS, highlights the significance of parallel data in enabling multilingual language models to excel. For instance, regions like Quebec, with bilingual government data in English and French, provide ample resources for training models proficient in both languages. However, the challenge intensifies when incorporating languages with limited data availability, such as Cree or Micmac, resulting in subpar performance compared to English and French models.
Quick Fixes Bring Limited Success
Amidst these challenges, potential solutions like automated translation and synthetic data augmentation emerge as quick fixes for enhancing non-English genAI models. However, as Seth from UST cautions, these approaches may introduce inconsistencies and biases that can compromise model accuracy and credibility. Flavio Villanustre, the global chief information security officer at LexisNexis Risk Solutions, raises concerns about the escalating risks of perpetuating biases and inaccuracies within generative AI models if not carefully managed.
In light of these complexities, CIOs are urged to adopt practical and sometimes costly strategies to address data disparities in non-English models. It is imperative to prioritize transparency in data provenance during model procurement and consider sourcing from regional genAI firms specializing in native languages. However, as Rowan Curren, a senior analyst at Forrester, notes, enterprises often lean towards established hyperscalers for model deployments due to familiarity and trust, posing challenges for diversifying model sources.
Looking Ahead
As the landscape of generative AI continues to evolve, the future holds promise for mitigating disparities in model quality among different languages. One notable trend involves the shift towards leveraging private data sources to enhance data quality and expand training datasets for non-English models. This transition, coupled with the anticipated rise in unstructured data from diverse sources, presents opportunities to address language-specific challenges and elevate the overall performance of genAI models.
In conclusion, as CIOs navigate the complexities of deploying non-English genAI models, a concerted effort towards enhancing data transparency, exploring diverse sourcing strategies, and prioritizing data quality remains paramount. By embracing a forward-thinking approach and demanding accountability from model vendors, CIOs can steer towards a future where language barriers no longer impede the efficacy and reliability of generative AI technologies.