I tested Gemini Ultra vs. ChatGPT and Claude

The results might surprise you...

šŸšØ Quick Reminder: My ChatGPT Accelerator course is now available. If you want to:

  • Transform from a ChatGPT beginner to advanced user fast

  • Stay ahead in the fast evolving world of tech and AI

  • Learn how to use ChatGPT for writing, brainstorming and learning

Get the course here (offer ends this Sunday).

Hey Warrior,

Gemini Ultra, the new LLM by Google, was released a few days ago and everyoneā€™s talking about how amazing it is.

So I decided to put it to the testā€¦

Specifically, I wanted to test its writing abilities.

In a previous post I had done a similar test between ChatGPT and Claude and spoiler alert: Claude came out ahead in almost all categories.

This time I did a three-way head-to-head, pitting Gemini Ultra against ChatGPT and Claude by using 5 identical prompts (youā€™ll find all my prompts linked below as well).

Letā€™s jump in!

Read time: 6 minutes

Guidde is an AI-powered screen recording tool (think Loom on steroids).

You simply record your screen and then AI will automatically turn that recording into a FULL step-by-step tutorial!

I can then share this how to guide with anyone, which makes it really easy and fast to demo a workflow or show a process.

*sponsored content

1) Newsletter Subject Lines

I started with newsletter subject lines.

This is a prompt I use for every newsletter I write. When Iā€™m done with the draft, I always generate subject line ideas using AI. This gives me inspiration and lets me piece together my favorite parts.

(See the full prompt I used here)

And here are the outputs of each model:

ChatGPT (GPT-4)

Claude 2

Gemini Ultra

Winner: Gemini

Why? ChatGPT's subject lines were over the top in my opinion. Claude's weren't concise enough. Gemini's were clear and compact. Therefore Iā€™ll give this first round to Gemini.

2) Twitter Hooks

Twitter hooks are tricky, since AI is notoriously bad at this.

ChatGPT has always been slightly worse than Claude at this. Letā€™s see if Gemini is any betterā€¦

(Full prompt here).

ChatGPT (GPT-4)

Claude 2

Gemini Ultra

Winner: It's a draw

Why? I can't determine a clear winner here. None of them are great and stood out to me. Iā€™ll still be writing Twitter hooks myself, and just using AI as inspiration.

3) Regular Tweets

AI is usually not great with regular tweets either. The best way to get a half-decent output is to include a specific template in the prompt.

(Hereā€™s the simple prompt I used in this case.)

Hereā€™s what the models came up with:

ChatGPT (GPT-4)

Claude 2

Gemini Ultra

Winner: Claude

Why? I have to give it to Claude here. ChatGPT was way too verbose (for a tweet). Gemini was good, but lost points for including the hashtags in the end. Claude did the best.

4/ Casual email

Default ChatGPT tends to sound too formal. Claude does a lot better in that aspect.

With this prompt I specifically instruct the AI to write in a casual tone. Lets see how well Gemini does here.

(Full prompt)

ChatGPT (GPT-4)

Claude 2

Gemini Ultra

Winner: Gemini

Why? ChatGPT and Claude's output both seem too long for a casual email pitch. Gemini was concise and straight to the point. I am not surprised that Gemini is good at emails, it probably has a lot of emails in its training data.

5/ Blog post

Finally, lets try out one of my favorite prompts: turning my newsletter drafts into (almost) done posts.

(Full prompt)

ChatGPT (GPT-4)

Claude 2

Gemini Ultra

Winner: Claude

Why? I like Claude's neutral and down-to-earth tone the most here. ChatGPT was a bit too much again. Gemini was ok, but the last sentence was too cringe. Ultimately I didnā€™t like any of them, and as you can tell from the intro of this email, I stuck with my original draft.

Wrapping up

Overall, ChatGPT struggled the most with these writing tasks. It's stronger at reasoning and brainstorming than writing. Nothing has really changed here since my comparison with Claude a few months ago.

Gemini Ultra impressed me, especially with its conciseness natural tone. I would say itā€™s almost on par with Claude now. I'll definitely be using it more and adding it to my workflow.

Of course, the release of GPT 4.5 may change everything soon. The AI landscape evolves quickly!

Thanks for reading!

If you enjoyed this email, consider forwarding this newsletter to a friend or colleague.

What did you think of today's email?

Your feedback helps me create better emails for you!

Login or Subscribe to participate in polls.

See you in the next one!

Reply

or to participate.