This paper describes a new test called FrontierMath for evaluating how well AI can solve advanced math problems. FrontierMath is different from other math tests because it uses brand new, really hard math problems that AI hasn't seen before, making it a more accurate measure of AI's abilities. The problems in FrontierMath cover many areas of math, like algebra, geometry, and calculus, and were created by over 60 mathematicians from top universities. The paper tested popular AI programs like GPT-4 and Claude on FrontierMath and found that they were only able to solve less than 2% of the problems. Even famous mathematicians, including winners of the Fields Medal (like a Nobel Prize for math), agree that these problems are very challenging. The auth ...