Ƶ

Training an Arabic LLM that reflects local values

Training an Arabic LLM that reflects local values

Training an Arabic LLM that reflects local values
The Arab world did not play a key role in the PC, internet and mobile eras. In the AI era, it will be different. (Shutterstock)
Short Url

Advances in the large language models that underpin generative AI are changing everything, from medicine and education to entertainment.

Our relationship with technology is becoming more intimate as machines change from passive tools into active assistants that amplify our innate human abilities.

This new era poses both a challenge and an opportunity for the Middle East.

The challenge is that leaders in this new field, like OpenAI’s ChatGPT and Google’s Gemini, come from Silicon Valley, or from China, where my team at 01.AI has built models that rival the Americans. In Europe, too, startups such as France’s Mistral have entered the race.

The opportunity is for the Middle East to join this league and make sure its voice is heard.

Inspired by my latest trip to Riyadh, I decided to test how the current crop of AI models would handle a simple request. I imagined myself as a young Saudi getting ready to host a dinner party and asked ChatGPT to prepare a menu.

The food it recommended sounded delicious — stuffed grape leaves, tabouleh salad, mandi and stuffed dates. But the beverages were a problem.

Aside from drinks such as mint lemonade and jallab, a mixture of dates, grape molasses and rose water, ChatGPT also offered this: “For alcoholic beverages, you could offer a selection of international wines, beers, or non-alcoholic mocktails.”

To its credit, when I repeated the question, it offered only non-alcoholic drinks.

If a model recommends breaking both the law and cultural norms, imagine how it might answer other more sensitive questions about politics or religion? Indeed, researchers have even shown that some models have exhibited an anti-Muslim bias.

My modest test underlines the urgent need to develop an Arabic large language model that reflects local values.

The first step to building this is creating enough high-quality Arabic digitized data to properly train a new generation of models.

Although there are 400 million Arabic speakers, only an estimated 2 percent of online content is in Arabic. Meta’s open source LLM model Llama is overwhelmingly trained on English data, with Arabic comprising less than 0.1 percent of the data.

The lack of data naturally skews the results. To fix this dearth of data, either a visionary entrepreneur or a government-backed organization should collect, digitize and convert the many Arabic books into training data for Arabic models.

Once the data is gathered, it can be fed into the breakthrough pre-training process, which reads trillions of words and creates its own virtual concept space or model of the world. This concept space has been shown to be mostly in English and Chinese.

Adding a sizable number of texts in Arabic, which has enormous cultural output and significance, will make the concept space more knowledgeable about Arabic and more balanced in its concepts and views.

After such pre-training, the model needs to be fine-tuned by data and labels from the Arab world, which will align with the values of the region. Those are different from American models, which are aligned to US values, and Chinese models, which reflect Chinese values.

The collection of alignment data, the coordination of human labeling and the alignment process will need to be done in-region by AI experts.

A new Arabic-enhanced large language model could encourage entrepreneurs and developers to build new applications tailored to the needs of their nations.

Kai-fu Lee

Finally, safety modules will need to be added to ensure legal compliance and to avoid harm. These will also need to be developed locally.

The above steps will create localized, sovereign models that will reflect the traditions of the Middle East. Privately developed or government-backed, it could be the foundation for a new wave of Arabic AI innovation.

A new Arabic-enhanced large language model could encourage entrepreneurs and developers to build new applications tailored to the needs of their nations.

Imagine an AI tool that could find, summarize, organize and write insightful content, an AI teacher that makes learning fun and customized, an AI doctor that is more knowledgeable than any human, an AI engineer that can write software and applications, and an AI assistant that knows its owner better than the owner themselves.

The Arab world did not play a leading role in the PC, internet and mobile eras. In the AI era, it will be different.

This transformation is by no means an easy feat. It will require an unprecedented investment of money, energy and human capital.

Middle Eastern leaders like Saudi Crown Prince Mohammed bin Salman and others have shown that they have the vision, determination and resources to lead their countries into the future.

Standing on my hotel balcony in Jeddah recently, overlooking the King Abdullah University of Science and Technology, I saw part of that vision coming to fruition.

Universities such as KAUST and the Mohamed bin Zayed University of Artificial Intelligence in the UAE are striking examples of the resources that have already been poured into this transformation.

These world-class academic institutions can attract and retain the best top tier global talent.  It is especially important to bring in the world’s best computer engineers to help fulfill this vision of the future AI.

Our team at 01.AI has shown what a group of talented and motivated computer scientists can achieve in just one year. With the right commitment of resources and drawing upon the best talent, countries like Ƶ can easily catch up with their global peers.

The Middle East can also lead the world in the use of renewables to run power-hungry generative AI models.

As it seeks to diversify its economy, Ƶ is actively promoting the use of alternative energy sources such as solar, which could power server farms and reduce their carbon footprint — a growing concern as AI becomes more widespread.

It may take time for countries to figure out their strategy for building a sovereign AI. But it is critical for the Arab world to quickly catalyze the creation of culturally appropriate LLMs and build a rich ecosystem to allow AI-powered Arabic apps to blossom.

A recent encounter with a female sales assistant at a computer store in Riyadh served as an apt reminder of what is at stake. Dressed in jeans and sporting a tattoo, she was a reminder of the transformative changes that the country is undergoing.

Where are you from, I asked. “I’m Saudi,” she said. “One day I want to be Ƶ’s Elon Musk.” I hope on my next visit she will pitch me a homegrown AI app.

Kai-Fu Lee is a computer scientist, CEO of 01.AI, chairman of Sinovation Ventures, former president of Google China, and author of “AI 2041” and “AI Superpowers”
 

Disclaimer: Views expressed by writers in this section are their own and do not necessarily reflect Arab News' point of view

Baha Craftsmen Forum engages youth, children

Baha Craftsmen Forum engages youth, children
Updated 2 min 57 sec ago

Baha Craftsmen Forum engages youth, children

Baha Craftsmen Forum engages youth, children

Riyadh: The Baha Craftsmen Forum, organized by the Technical and Vocational Training Corp., teaches youth and children the basics of the traditional handicrafts for which the region is renowned.

Forum supervisor Nouf Al-Harbi said the forum offers 10 specialized training courses, including crochet doll-making, palm weaving, and stone and marble carving, the Saudi Press Agency reported on Monday.

The courses are led by expert instructors who showcase works blending traditional craftsmanship with modern design, striking a balance between preserving cultural heritage and meeting the evolving demands of today’s market.

As part of the Baha Summer Festival, the forum’s second week began on Saturday, drawing a remarkable turnout of visitors.

The month-long forum, held in partnership with governmental and private entities, offers various training programs, dialogue sessions and live interactive activities celebrating cultural heritage.

The forum aims to showcase the region’s traditional crafts and cultural heritage, while fostering interaction between artisans and the public.


US officials express anger over Israel’s Syria strikes

US officials express anger over Israel’s Syria strikes
Updated 7 min 57 sec ago

US officials express anger over Israel’s Syria strikes

US officials express anger over Israel’s Syria strikes
  • PM Netanyahu ‘like a madman. He bombs everything all the time’
  • White House officials also describe growing consternation over Gaza war

LONDON: White House officials have expressed frustration over Israel’s bombing of Syria, The Times reported.

Israel carried out a series of attacks on government targets in the Syrian Arab Republic last week, including a strike on a tank convoy and the shelling of the Defense Ministry in Damascus.

US diplomats warned Israel to cease its intervention, which it claimed to be conducting in support of Syria’s Druze minority.

Clashes between local Bedouin and Druze forces had broken out in Syria’s southern province of Sweida, with the country’s government sending troops to quell the violence.

One White House official told Axios that Israeli Prime Minister Benjamin Netanyahu “acted like a madman. He bombs everything all the time. This could undermine what (US President Donald) Trump is trying to do.”

Trump lifted sanctions on Syria earlier this year after meeting President Ahmad Al-Sharaa, who has pledged to unite his country and bring an end to more than a decade of violence.

The US brokered a ceasefire last week that appeared to stop the clashes in Sweida, where more than 1,000 people were killed over seven days, according to the Syrian Observatory for Human Rights.

The violence was reportedly sparked by a series of kidnappings targeting members of various faiths, clans and tribal groups in the province.

Before launching strikes, Israel claimed that Syrian government forces were involved in targeting the Druze.

Israel has its own community of Druze, numbering about 130,000, and some Syrian members of the faith traveled to meet family members there to escape the violence in Sweida.

After the overthrow of Bashar Assad’s regime last year, Israel sent forces into Sweida to establish a buffer zone. The province borders Syria’s Golan Heights, which Israel has occupied since 1967.

Another US official told Axios: “Netanyahu is sometimes like a child who just won’t behave.”

US Secretary of State Marco Rubio on Sunday urged Al-Sharaa to halt the violence in his country, which he described as “horrifying and dangerous.”

The “rape and slaughter of innocent people, which has and is still occurring, must end,” Rubio said on X, adding that Syrian authorities “must hold accountable and bring to justice anyone guilty of atrocities including those in their own ranks.”

White House officials also described growing consternation over Israel’s war on Gaza, especially after the shelling of the Palestinian enclave’s only Catholic church last week. The attack killed three Palestinians.

A senior American official told Axios after the church strike: “The feeling is that every day there is something new … what the f***?”

Mike Huckabee, US ambassador to Israel, also delivered surprise public criticism in the wake of an arson attack on a Byzantine-era church in the occupied West Bank over the weekend.

“To commit an act of sacrilege by desecrating a place that is supposed to be a place of worship, it is an act of terror, and it is a crime,” he said. “There should be consequences.”

He also demanded “accountability” from Israel after a Palestinian American was killed in the West Bank last week.


Eight Pakistan paramilitary troops, four militants killed in northwest

Eight Pakistan paramilitary troops, four militants killed in northwest
Updated 8 min 6 sec ago

Eight Pakistan paramilitary troops, four militants killed in northwest

Eight Pakistan paramilitary troops, four militants killed in northwest
  • The clash, which lasted for several hours, took place in the Orakzai district, near the Afghan border
  • Pakistan has witnessed a sharp rise in violence in its regions bordering Afghanistan in recent years

PESHAWAR: At least eight Pakistani paramilitary troops and four militants were killed in a clash in the country’s northwest, senior local officials said Monday, in the latest violence to hit the restive region.

“Armed terrorists attacked a Frontier Corps (FC) convoy with heavy weapons... The fighting continued for several hours,” a senior local security official told AFP.

“Eight FC personnel were killed, and 11 were injured,” he said.

Three injured soldiers are in critical condition, while the clash also killed four militants, he added.

A senior administration official in the area confirmed the details to AFP.

The clash, which lasted for several hours, took place in Daburi, Orakzai district, near the Afghan border.

Pakistan has witnessed a sharp rise in violence in its regions bordering Afghanistan since the Taliban returned to power in Kabul in 2021.

Islamabad accuses its western neighbor of allowing its soil to be used for attacks against Pakistan — a claim the Taliban denies.

More than 320 people, mostly security personnel, have been killed in attacks since the start of the year by armed groups fighting in both Khyber Pakhtunkhwa and Balochistan, according to an AFP tally.


Ƶ hands over 129 housing units to Tunisian families

Ƶ hands over 129 housing units to Tunisian families
Updated 9 min 37 sec ago

Ƶ hands over 129 housing units to Tunisian families

Ƶ hands over 129 housing units to Tunisian families
  • The project was funded by the Saudi Development Fund in Sousse governorate, south of Tunis

RIYADH: The Kingdom has handed over 129 housing units to families in Tunisia, the Saudi Press Agency reported on Monday.

The nation’s ambassador to Tunisia, Abdulaziz bin Ali Al-Saqr, and Tunisia’s Minister of Equipment and Housing Salah Zouari were present for the handover ceremony.

The project was funded by the Saudi Development Fund in Sousse governorate, south of Tunis.

Zouari thanked the Kingdom for its continued support and solidarity with Tunisia, noting that the project benefits low-income families. 

Al-Saqr congratulated the beneficiaries and reaffirmed Ƶ’s commitment to supporting Tunisia’s growth and development across various sectors.

The units are a part of the first phase of a social housing program across several Tunisian governorates.

The full project aims to provide 4,715 units through concessional financing from the Saudi Development Fund, totaling $150 million.


Madinah presents SDG progress at UN in New York

Madinah presents SDG progress at UN in New York
Updated 18 min 28 sec ago

Madinah presents SDG progress at UN in New York

Madinah presents SDG progress at UN in New York

RIYADH: The Madinah Region Development Authority participated in the main session of the High-Level Political Forum on Sustainable Development at the UN headquarters in New York, the Saudi Press Agency reported on Monday.

In the session, titled “Transformation from the Ground Up: Acting at Local Level,” the authority highlighted Madinah’s pioneering role and the Kingdom’s leadership in localizing the Sustainable Development Goals.

Abdulrahman Ibrahim, the authority’s chief data and innovation officer, said Madinah had become a model for international best practices.

Key initiatives include the 2023 Madinah City Voluntary Local Review — the first in the Kingdom and second in the Arab region; the 2024 State of Sustainable Development Report for Madinah Region — the first Voluntary Subnational Review in the Arab region; and the launch of the Atlas of Sustainable Development in Madinah City Neighborhoods, a global first for precise, neighborhood-level SDG assessment.

The session also reviewed successful local and regional approaches to accelerating sustainable SDG solutions, discussing key enablers and opportunities to expand local action.

The authority’s participation, part of the Saudi delegation led by Minister of Economy and Planning Faisal Alibrahim, underscores the Kingdom’s commitment to sustainable development aligned with Vision 2030.

The UN forum, running through to July 23, also featured dialogue sessions on global frameworks for resilient urban development and strategies to create value for cities through data-driven trends.