Quesma Releases OTelBench: Independent Benchmark Reveals Frontier LLMs Struggle with Real-World SRE Tasks

New benchmark shows top LLMs achieve only 29% pass rate on OpenTelemetry instrumentation, exposing the gap between ...
Semiconductor Engineering

Benchmark For AI-Aided Chip Design That Evaluates LLMs Across 3 Critical Tasks (UCSD, Columbia)

Researchers at UCSD and Columbia University published “ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design.” Abstract “While Large Language Models (LLMs) show ...

Developers say AI coding tools work—and that’s precisely what worries them

Software developers have spent the past two years watching AI coding tools evolve from advanced autocomplete into something that can, in some cases, build entire applications from a text prompt. Tools ...

When AI can write code: Students rethink learning, exams, and careers

AI's coding capabilities prompt students to reevaluate the value of traditional computer science education and future career paths.