Drishtant Ghosh (Drix10)
drix10blogs
👁️8,960
GitHubLinkedIn
AI and Robotics Applications••6 min read•1196 words

🤖 Benchmarking AI Models

👁️0reads (human + AI)🤖0AI ingestions
⚡Direct Technical Summary

Benchmark numbers are marketing. All of them. Including the ones I like. Astra scored 62.7% and 99.9% on the same benchmark. Same weights, same games. The only thing that changed w

🤖 Benchmarking AI Models

Benchmark numbers are marketing. All of them. Including the ones I like. Astra scored 62.7% and 99.9% on the same benchmark. Same weights, same games. The only thing that changed was the harness. Guess which one went in the headline.

Key Points:

  • Benchmark numbers: are often misleading and should be taken with a grain of salt.

  • Astra scored 62.7%: and 99.9% on the same benchmark, highlighting the importance of harness selection.

  • The harness: can significantly impact model performance, and its selection should be carefully considered.

🔗 Resources:

🤖 Robot Family Live Stage Show

Galbot turned its robot family into a live stage show at WRC. The whole thing played out like a skit, but each robot got to show what it can do: the new ET1 and a G1 played hosts, S1 handled the heavy lifting, while another G1 worked the store counter. A much more fun way to

Key Points:

  • Galbot's robot family: was showcased in a live stage show at WRC.

  • The robots: demonstrated their capabilities in a fun and engaging way.

  • The show highlighted: the versatility and capabilities of Galbot's robots.

🔗 Resources:

🤖 Model Performance Gap

Right direction, and Astra just showed the gap it doesn't close. Arc-AGI-3, same weights same games: 62.7% on the standard harness, 99.9% on the provider adapter. Private test sets stop people gaming the questions. They don't stop the harness from doing the work. And on

Key Points:

  • Astra's performance gap: was highlighted by its inability to close the gap with Arc-AGI-3.

  • The standard harness: and provider adapter had significantly different performance results.

  • Private test sets: are not effective in stopping people from gaming the questions.

🔗 Resources:

🤖 Model Saturation

Everyone's reading "Arc-AGI-3 is saturated" as a model result. It isn't. Same weights, same games, two harnesses: standard: 62.7%, cost $26,098 provider adapter: 99.9%, cost $18,817 The run that scored 37 points higher cost $7,281 less. That's not the model getting smarter.

Key Points:

  • Arc-AGI-3's performance was: misinterpreted as model saturation.

  • The same weights: and games resulted in different performance outcomes with different harnesses.

  • The cost of: the run that scored 37 points higher was significantly lower.

🔗 Resources:

🤖 Home Robot Companion Wars

BREAKING: The Home Robot Companion Wars are coming. Choose your companion! They're all cute! 🇫🇷 - @pollenrobotics Microduck 🇨🇳 - @mondorobotics Beni 🇯🇵 - @aopico Mirumi 🇯🇵 - @LOVOT_official Lovot 🇨🇳 - @UnitreeRobotics Qmini Which one is your favorite one?

Key Points:

  • The Home Robot: Companion Wars are an upcoming event.

  • Several robot companions: are being showcased, each with its unique features.

  • The choice of companion: is a personal preference.

🔗 Resources:

🤖 Human Judgment in AI

TBH I think now geometry and perception annotations have become highly automated; what truly remains dependent on humans are detail-level semantics, intent, edge-case judgments, quality and success evaluation, and expert knowledge.

Key Points:

  • Geometry and perception annotations: have become highly automated.

  • Human judgment: is still required for detail-level semantics, intent, and edge-case judgments.

  • Expert knowledge: is also essential for quality and success evaluation.

🔗 Resources:

🤖 Annot Model Enrichment

Yup exactly, would add tactile in the mix. But the simple takeaway, is if you have diverse rich dataset you could build your own annot model that can enrich any existing dataset!

Key Points:

  • Annot models: can be used to enrich existing datasets.

  • A diverse and: rich dataset is required to build an effective annot model.

  • Tactile input: can also be added to the annot model.

🔗 Resources:

🤖 AI-powered Design and Manufacturing

Astra helped me: - designed this telescope star tracker PCB - created the order for the PCB+assembly on @JLCPCB and checked component stock / optimized for cost along the way - used OBS to record screen - rendered and edited this video Then I paid ~$200😅 I’d prefer if Astra

Key Points:

  • Astra was used: to design and manufacture a telescope star tracker PCB.

  • The design process: involved creating an order for PCB+assembly and optimizing component stock.

  • Astra was also: used to record and edit a video.

🔗 Resources:

🤖 AI-powered Robotics

GPT-6 Astra making CAD models and running robotics in seconds is crazy to watch. But serious respect to everyone who built stuff, ran wiring, and wrestled with CAD before any of this AI help existed. That physical intuition you have is still irreplaceable.

Key Points:

  • Astra was used: to create CAD models and run robotics in seconds.

  • The process: is impressive, but physical intuition is still essential.

  • Respect: is given to those who built and wired robots before AI assistance.

🔗 Resources:

🤖 Robotics Learning Course

ETH Zurich (Swiss Federal Institute of Technology in Zurich) has made the entire Robotics Learning course for 2026 publicly available. Not a simplified online version. It's the full course itself: slides, lecture videos, programming assignments, GitHub repository—all included.

Key Points:

  • The Robotics Learning: course for 2026 is publicly available.

  • The course includes slides: , lecture videos, programming assignments, and a GitHub repository.

  • The course: is not a simplified online version, but the full course itself.

🔗 Resources:

📂Source / Implementation:AI and Robotics Applications / resources-236.md
GitHub Repository↗

Related AI and Robotics Applications Breakdowns

🤖 Robotics - Industry Updates
5m readRead ↗
🤖 Robotics - Balloon-Powered Robots
7m readRead ↗
🤖 Technical Writing - Physical AI and Robotics
7m readRead ↗
🤖 Robotics - Humanoid Robotics
6m readRead ↗
Drishtant Ghosh (Drix10)
Drishtant Ghosh (Drix10)•Author & Engineer

Technical founder and engineer working across AI systems, developer infrastructure, and cybersecurity.

Portfolio·GitHub·LinkedIn·X·Email
← Back to all breakdowns
GitHub: AI and Robotics Applications/resources-236.md ↗https://blogs.drix10.com/articles/ai-and-robotics-applications/benchmarking-ai-models-236