CIOs Navigate Challenges with Non-English GenAI Models
As the realm of generative AI expands within enterprises, Chief Information Officers (CIOs) are facing a significant hurdle when it comes to non-English language models. Despite procuring these models from various vendors, CIOs are realizing that the performance of non-English models lags far behind their English counterparts. This discrepancy stems from the scarcity of training data available for non-English languages, leading to reduced accuracy and relevance in their outputs.
Akhil Seth, an AI business development leader at UST, highlights the inherent issue, emphasizing that the disparity in training sample size between English and non-English models inevitably impacts their performance. The consequence of insufficient data manifests in reduced comprehensiveness, accuracy, and an increased likelihood of generating inaccurate responses or “hallucinations.”
For global companies relying on diverse languages to cater to customers and employees worldwide, addressing this data imbalance is crucial to avoid subpar outcomes. However, major model makers such as OpenAI, Microsoft, and Google remain opaque about the volume and quality of training data used for their models, making it challenging for enterprises to ascertain the efficacy of non-English models through conventional testing methods primarily focused on English models.
The disparity in dataset sizes for non-English models, estimated to be significantly smaller—ranging from 10 to 100 times less than English models—poses a substantial hurdle. Vasi Philomin from Amazon Web Services (AWS) suggests leveraging available data sources, such as the number of Wikipedia pages in a language, to gauge training data availability. Industries, topics, and use cases also influence the availability and quality of training data, underscoring the complexity of the issue.
One practical approach to mitigate the challenges posed by non-English genAI models involves investing in extensive testing and fine-tuning specific to each language. However, quick fixes like automated translation or synthetic data supplementation come with their own set of drawbacks, including potential biases and inaccuracies that can impact the credibility of the AI-generated outputs.
Moving forward, CIOs must demand greater transparency from model vendors regarding training data for non-English models. By scrutinizing the quality and sourcing of training data, enterprises can make more informed decisions and potentially drive model makers to enhance the accuracy of non-English models. Additionally, exploring regional genAI firms native to specific languages or considering marketplace solutions like AWS’s Bedrock could offer alternatives to address the data disparity issue.
While the road ahead may involve challenges and increased costs to refine non-English models adequately, the imperative for CIOs is clear: demand accountability, transparency, and rigorous testing to ensure the efficacy of genAI models across languages. By pushing for better data practices and exploring diverse sourcing options, CIOs can pave the way for more accurate and reliable AI solutions tailored to a global audience.