Yearly, the international locations competing within the Worldwide Mathematical Olympiad (IMO) arrive with a booklet of their greatest, most unique issues. These booklets get shared amongst delegations, then quietly disappear. Nobody had ever collected them systematically, cleaned them, and made them obtainable, not for AI researchers testing the bounds of mathematical reasoning, and never for the scholars around the globe coaching for these competitions largely on their very own.

Researchers at MIT’s Laptop Science and Synthetic Intelligence Laboratory (CSAIL), King Abdullah College of Science and Know-how (KAUST), and the corporate HUMAIN have now accomplished precisely that.

MathNet is the biggest high-quality dataset of proof-based math issues ever created. Comprising greater than 30,000 expert-authored issues and options spanning 47 international locations, 17 languages, and 143 competitions, it’s 5 instances bigger than the next-biggest dataset of its type. The work can be offered on the Worldwide Convention on Studying Representations (ICLR) in Brazil later this month.

What makes MathNet completely different just isn’t solely its dimension, however its breadth. Earlier Olympiad-level datasets draw virtually solely from competitions in america and China. MathNet spans dozens of nations throughout six continents, covers 17 languages, consists of each text- and image-based issues and options, and spans 4 a long time of competitors arithmetic. The aim is to seize the total vary of mathematical views and problem-solving traditions that exist throughout the worldwide math neighborhood, not simply probably the most seen ones.

“Each nation brings a booklet of its most novel and most inventive issues,” says Shaden Alshammari, an MIT PhD pupil and lead creator on the paper. “They share the booklets with one another, however nobody had made the trouble to gather them, clear them, and add them on-line.”

Constructing MathNet required monitoring down 1,595 PDF volumes totaling greater than 25,000 pages, spanning digital paperwork and decades-old scans in additional than a dozen languages. A good portion of that archive got here from an unlikely supply: Navid Safaei, a longtime IMO neighborhood determine and co-author who had been amassing and scanning these booklets by hand since 2006. His private archive fashioned a lot of the spine of the dataset.

The sourcing issues as a lot as the dimensions. The place most present math datasets pull issues from neighborhood boards like Artwork of Downside Fixing (AoPS), MathNet attracts solely from official nationwide competitors booklets. The options in these booklets are expert-written and peer-reviewed, they usually usually run to a number of pages, with authors strolling by way of a number of approaches to the identical downside. That depth offers AI fashions a far richer sign for studying mathematical reasoning than the shorter, casual options typical of community-sourced datasets. It additionally means the dataset is genuinely helpful for college kids: Anybody making ready for the IMO or a nationwide competitors now has entry to a centralized, searchable assortment of high-quality issues and labored options from traditions around the globe.

“I keep in mind so many college students for whom it was a person effort. Nobody of their nation was coaching them for this type of competitors,” says Alshammari, who competed within the IMO as a pupil herself. “We hope this offers them a centralized place with high-quality issues and options to be taught from.”

The group has deep roots within the IMO neighborhood. Sultan Albarakati, a co-author, at the moment serves on the IMO board, and the researchers are working to share the dataset with the IMO basis straight. To validate the dataset, they assembled a grading group of greater than 30 human evaluators from international locations together with Armenia, Russia, Ukraine, Vietnam, and Poland, who coordinated collectively to confirm 1000’s of options.

“The MathNet database has the potential to be a superb useful resource for each college students and leaders looking for new issues to work on or on the lookout for the answer to a troublesome query,” says Tanish Patil, deputy chief of Switzerland’s IMO. “While different archives of Olympiad issues do exist (notably, the Contest Collections boards on AoPS), these sources lack standardized formatting system, verified options, and vital downside metadata that subjects and concept require. It’s going to even be fascinating to see how this dataset is used to enhance the efficiency of reasoning fashions, and if we’ll quickly have the ability to reliably reply an vital problem when creating novel Olympiad questions: figuring out if an issue is actually unique.”

MathNet additionally features as a rigorous benchmark for AI efficiency, and the outcomes reveal a extra sophisticated image than current headlines about AI math prowess may recommend. Frontier fashions have made extraordinary progress: Some have reportedly achieved gold-medal efficiency on the IMO, and on customary benchmarks they now resolve issues that might stump most people. However MathNet reveals that progress is uneven. Even GPT-5, the top-performing mannequin examined, averaged round 69.3 % on MathNet’s predominant benchmark of 6,400 issues, failing practically one-in-three Olympiad-level issues. And when issues embody figures, efficiency drops considerably throughout the board, exposing visible reasoning as a constant weak level for even probably the most succesful fashions.

A number of open-source fashions scored 0 % on Mongolian-language issues, highlighting one other dimension the place present AI programs fall quick regardless of their general power.

“GPT fashions are equally good in English and different languages,” Alshammari says. “However most of the open-source fashions fail utterly at less-common languages, akin to Mongolian.”

The range of MathNet can be designed to handle a deeper limitation in how AI fashions be taught arithmetic. When coaching information skews towards English and Chinese language issues, fashions take in a slender slice of mathematical tradition. A Romanian combinatorics downside or a Brazilian quantity concept downside might strategy the identical underlying idea from a totally completely different angle. Publicity to that vary, the researchers argue, makes each people and AI programs higher mathematical thinkers.

Past problem-solving, MathNet introduces a retrieval benchmark that asks whether or not fashions can acknowledge when two issues share the identical underlying mathematical construction, a functionality that issues each for AI growth and for the mathematics neighborhood itself. Close to-duplicate issues have appeared in actual IMO exams through the years as a result of discovering mathematical equivalences throughout completely different notations, languages, and codecs is genuinely onerous, even for knowledgeable human committees. Testing eight state-of-the-art embedding fashions, the researchers discovered that even the strongest recognized the proper match solely about 5 % of the time on the primary attempt, with fashions continuously rating structurally unrelated issues as extra related than equal ones.

The dataset additionally features a retrieval-augmented technology benchmark, testing whether or not giving a mannequin a structurally associated downside earlier than asking it to unravel a brand new one improves efficiency. It does, however solely when the retrieved downside is genuinely related. DeepSeek-V3.2-Speciale gained as much as 12 proportion factors with well-matched retrieval, whereas irrelevant retrieval degraded efficiency in roughly 22 % of instances.

Alshammari wrote the paper with Safaei, HUMAIN AI engineer Abrar Zainal, KAUST Academy Director Sultan Albarakati, and MIT CSAIL colleagues: grasp’s pupil Kevin Wen SB ’25; Microsoft Principal Engineering Supervisor Mark Hamilton SM ’22, PhD ‘25; and professors William Freeman and Antonio Torralba. Their work was funded, partially, by the Schwarzman Faculty of Computing Fellowship and the Nationwide Science Basis.

MathNet is publicly obtainable at mathnet.csail.mit.edu.