China’s GLM 5.2: The Most Powerful Open-Source Model Yet — But Does It Deliver in Real Life?

+

China just dropped the first open source SOTA model: GLM 5.2. It crushes the benchmarks, but struggled in my real world testing.

Today, I ran GLM through a three-round test: building a pitch deck, teaching me about data centers, and designing a website.

GLM showed some powerful capabilities. But it also made surprising mistakes…

Round #1: Making a Startup Pitch Deck

For our first round, I had GLM 5.2 make a pitch deck for Uber.

I turned on Wispr Flow and rambled on what I wanted the deck to cover: team, traction, and vision. I gave GLM some style pointers, like putting high-contrast text on a black background.

GLM produced a beautiful deck!

I’ve seen the original UberCab deck. It didn’t look nearly as slick. If only Travis had had GLM!

This round gets an A+. Every founder should make their deck with GLM.

Round #2: GLM Teaches Me About Data Centers

I was excited to use GLM’s Knowledge/Teaching Material mode to learn about AI data centers. But it made me switch models 4 times due to high usage, landing on the weaker GLM-5-Turbo.

Rotten UX!

Eventually, GLM produced some solid info. I learned that five 1 gigawatt data centers consume as much electricity as all of New York City.

Power really is the bottleneck!

The report was great, but getting it was a pain. I’m giving this round a C.

Round #3: Designing a Website

GLM 5.2 claims to be strong in coding and design. Can it make a better design for this blog?

I gave GLM a vague prompt for a DOS-like interface, but cleaner and fresher. It switched to GLM-5-Turbo again, ran slowly, then outputted a blank page.

I told GLM to redo it. This time, I got a very nice homepage.

But I shouldn’t have to ask twice. Lovable nails this on the first try.

I’m giving this round a B-.

Wrap-Up

Overall, GLM earned a B- in my testing.

It excels on pitch decks, but struggles to make a basic webpage. Overall, it did not feel nearly as powerful as Claude or Grok.

Still, GLM 5.2 makes a lot of sense for startups.

Download it, post-train it on your data, and run it dirt cheap. The outputs will be good enough for most products.

If token bills are killing you, give GLM 5.2 a try!

China’s GLM 5.2: The Most Powerful Open-Source Model Yet — But Does It Deliver in Real Life?

Share this:

Leave a comment Cancel reply