Researchers introduce the OpenDS task and OpenDS-World, a benchmark of 200 tasks for testing how well agents carry out multi-step deep search on the live Web. Among 12 frontier models evaluated, the top performer reached 39.0% accuracy, while humans scored 21.1% across 1,000 attempts.

— via arXiv Software Engineering, Xiaomin Yu, Hongliang Chen, Zhe Huang, Shilin Yan, Shao-Rong Su, Xuqiao Ma, Feiyang Wang, Jiayu Chen, Jiwei Fang, Yu Yan, Yini Huang, Chen Chen, Ruoxi Cheng, Yuzhi He, Yuhui Zhang, Xiaoxing Hu, Chen Liu, Jianbo Lin, Yijiang Li, Xiaobin Hu, Menglong Li, Zhuoran Lu, Weishi Wang, Zhongqi Yue, Chengwei Qin