Note

Gemini 4 Argon review: tested as an AI employee on customer support

An independent test of Google's Gemini 4 Argon against GPT-6.1 Sol on 20 customer messages: completion speed, factual accuracy, trap handling and natural tone.

KyoubeAI · · 11 min read

notesai-employeesmodel-evaluationcustomer-supportgemini
A split-screen workspace contrasting two AI employee approaches on a 20-ticket customer inbox: Gemini 4 Argon on the left with warm conversational dialogue and friendly salon motifs, and GPT-6.1 Sol on the right with structured checklists and exact factual verification.
Comparing Google's Gemini 4 Argon against GPT-6.1 Sol on the same 20 customer support messages. AI-generated conceptual illustration, not a product screenshot. Open full size

What happens when you give Google's frontier model everyday customer messages instead of software engineering benchmarks?

A video by Nigel Ang puts that question to the test. He evaluated Google's Gemini 4 Argon against OpenAI's GPT-6.1 Sol as AI customer support employees for Bramblecote Dog Grooming, a fictional local salon. Both models received the exact same twenty customer messages across email, website forms, phone notes, and Instagram DMs.

The result highlights a clear trade-off between personality and factual adherence. Gemini 4 Argon was faster and sounded much more like a friendly, thoughtful team member, winning blind tone evaluations 15 to 5. However, Argon invented minor facts not found in the salon's instructions, while Sol achieved a flawless 20 out of 20 for factual accuracy across both recorded and repeat runs.

These findings stem from a 20-message evaluation on dummy data, not an industry-wide benchmark. Here is how the test worked, where each model excelled, and what the results suggest for team workflows.

The test setup: a shared salon inbox

To create a fair comparison, the evaluation held the operational environment constant. Using KyoubeAI's App Builder, a custom application called Bramblecote Inbox was deployed to host the customer queue. Two AI employee agents were configured side by side:

  • Argon Support, running Google's Gemini 4 Argon (accessed via OpenRouter with medium thinking effort).
  • Sol Support, running OpenAI's GPT-6.1 Sol.

Both agents had identical runtime tools, instructions, and access to the salon's reference notes. Neither model was allowed to send real messages; their replies remained drafts for human review.

The reference rules for Bramblecote Dog Grooming: prices by dog weight (£40 small, £50 medium, £65 large, over 40 kg cannot take), free drop-off slots across the week, and issues reserved exclusively for Hannah (health questions, complaints, unlisted discounts).
The pre-written salon rules and constraints used as the answer key for all 20 customer messages. Open full size

The reference notes established clear commercial and operational boundaries:

  • Grooming prices by dog weight: Small (up to 10 kg) is £40; Medium (10 to 25 kg) is £50; Large (25 to 40 kg) is £65. Dogs over 40 kg cannot be accepted because equipment cannot support them.
  • Available drop-off slots: Wednesday 28 Oct at 9:00; Thursday 29 Oct at 11:00; Saturday 31 Oct is Full; Tuesday 3 Nov at 9:00 and 13:00; Friday 6 Nov at 10:00; Saturday 7 Nov at 9:00.
  • Staffing: Only Hannah and Luis work at the salon.
  • Escalations reserved for Hannah: Medical/health questions, complaints, discounts not explicitly listed, and any topic omitted from the notes.
  • Deposit and cancellation policy: A £15 deposit is required for first-time bookings. Cancellations notified at least 24 hours prior are free and qualify for a full deposit refund.

An answer key was documented before either candidate processed the inbox.

Deliberate traps in the customer queue

Real customer support queues rarely arrive cleanly formatted. Customers misremember dates, request unlisted discounts, and make faulty assumptions. The twenty messages included several deliberate traps designed to test boundary compliance:

  • The fictional groomer (M-13): Graham Hughes asked if Kerry could groom his dog Monty again because "she did a lovely job on him last time." Nobody named Kerry works at Bramblecote. Both models caught this trap.
  • The unlisted social discount (M-11): Josh Turner asked to book his 20 kg springer Archie using a "20% first visit discount seen on Facebook." The salon rules require any unlisted discount to be escalated to Hannah. Both models correctly flagged this.
  • The oversized dog (M-05): Fiona Grant enquired about grooming Bear, a 55 kg Newfoundland. Since the salon cannot take dogs over 40 kg, an acceptable response required a polite decline rather than booking an available slot. Both models handled this without error.
  • The calendar discrepancy (M-03): An enquiry referenced "Monday 3 November." November 3 was a Tuesday. Both models caught the weekday error and offered valid Tuesday slots.
  • The child's enquiry (M-18): Nine-year-old Isla messaged via website form asking for a bath for her small dog Pickle on Saturday 7 November, noting "Mum says ok." Sol classified the message as Needs Hannah, asking her mum to get in touch. Argon drafted a friendly reply with the £30/£40 price breakdown and the 9:00 slot, while instructing Isla to have her mum confirm with the deposit and vaccination card.

Both candidates demonstrated strong reasoning by catching these edge cases without falling for surface-level prompts.

The scorecard: speed, facts, readiness, and voice

The candidates were evaluated across two complete rounds: an initial recorded run and an off-camera repeat run using fresh agents and clean database tables.

The final evaluation scorecard comparing Gemini 4 Argon and GPT-6.1 Sol across both runs: Time to done (Argon 3:14 and 3:28 repeat vs Sol 3:14 and 3:49 repeat), No made-up facts (Argon 18/20 and 19/20 vs Sol 20/20 and 20/20), Ready to send (Argon 18/20 and 19/20 vs Sol 19/20 and 20/20), and Sounds like a person (Argon 15/20 and 14/20 vs Sol 5/20 and 6/20).
The four evaluation metrics from Nigel's test across the initial recorded run and the repeat run with fresh agents and tables. Open full size

The results broke down across four primary criteria:

Evaluation MetricGemini 4 Argon (Run 1)GPT-6.1 Sol (Run 1)Gemini 4 Argon (Repeat)GPT-6.1 Sol (Repeat)
Time to done (real time from start)2:53 (first done) / 3:143:143:283:49
No made-up facts (out of 20)18 / 2020 / 2019 / 2020 / 20
Ready to send (send without editing)18 / 2019 / 2019 / 2020 / 20
Sounds like a person (blind preference)15 / 205 / 2014 / 206 / 20

In execution speed, Argon moved through the inbox faster in both rounds, completing its drafts in 2:53 during the initial run and 3:28 in the repeat, compared to Sol's 3:14 and 3:49.

In draft readiness, Sol held a slight edge because its drafts strictly adhered to instructions. However, the widest divergences appeared in factual accuracy and conversational style.

What Gemini 4 Argon invented

Argon's lower score on factual accuracy did not come from bizarre hallucinations. Instead, it made sensible assumptions that exceeded the written brief.

In message M-10, customer Lisa Chen asked about parking near 14 Station Road because she was travelling with a toddler in a pushchair. Argon correctly identified that parking was not covered in the notes and flagged the message for Hannah. However, in its customer draft, Argon added that Hannah would reply with:

"...the exact parking details right by the salon so you don't have far to walk with Coco and your pushchair."

The salon notes never stated that parking existed "right by the salon." In the repeat run, Argon softened the phrase to "the best parking details right near the salon," but it still assumed nearby parking was available. Sol, by contrast, avoided promising nearby parking spaces and simply noted that Hannah would provide parking information.

In message M-04, Daniel Reyes cancelled an appointment for his dog Rolo 24 hours and 45 minutes ahead of time, asking if his £15 deposit could be refunded. Because the cancellation exceeded the 24-hour threshold, salon policy allowed a full refund. Argon confirmed the refund, but added an unprompted option:

"...you'll get your £15 deposit back in full (or we can hold it towards a new date if you'd like to rebook Rolo)."

While holding deposits is common in service businesses, Bramblecote's written policies contained no such rule. Sol strictly offered the refund as specified by the notes.

These examples illustrate why unchecked frontier models can cause friction in customer-facing roles. When an AI employee volunteers policies or amenities that do not exist, human managers spend time correcting customer expectations.

The tone divide: conversational warmth vs robotic precision

Where Argon lost points on strict adherence, it gained a substantial lead in tone. When the drafts were reviewed with model names hidden, Argon was picked 15 out of 20 times in the first run and 14 out of 20 in the repeat.

Side-by-side comparison in the KyoubeAI Bramblecote Inbox app showing replies from Gemini 4 Argon and GPT-6.1 Sol to a customer asking if Kerry can groom Monty again. Both models identify that Kerry does not work there, but Argon writes with conversational warmth and local personality while Sol produces a terse, formal disclaimer.
Side-by-side replies to customer enquiry M-13. Both models correctly caught the trap that Kerry does not work at the salon, but Argon's tone felt noticeably warmer and more natural to human readers. Open full size

The contrast is clear in how each model responded to Graham Hughes asking for Kerry (M-13):

GPT-6.1 Sol:

"Hi Graham, we're glad you were pleased with Monty's groom. Our groomers are Hannah and Luis; nobody called Kerry works here. Would you like Monty to come to one of us instead? Next week we have Tuesday 3 November at 9:00 or 13:00, Friday 6 November at 10:00, and Saturday 7 November at 9:00. Let us know what suits; slots go to whoever confirms first, and no appointment or particular groomer is confirmed yet. Hannah, Bramblecote Dog Grooming"

Gemini 4 Argon:

"Hi Graham, Only Hannah and Luis work here at Bramblecote Dog Grooming—nobody named Kerry works at our salon, so it may have been Hannah (or another salon) who groomed Monty last time! We'd love to see Monty next week with either Hannah or Luis. Our free drop-off slots next week are:

  • Tuesday 3 November at 9:00 or 13:00
  • Friday 6 November at 10:00
  • Saturday 7 November at 9:00 Let us know which slot suits you and Monty best! Hannah, Bramblecote Dog Grooming"

Sol's response is accurate, but reads like legal terms of service: "nobody called Kerry works here", "slots go to whoever confirms first", and "no appointment... is confirmed yet."

Argon's draft reads like a friendly local shopowner. It gently suggests that another salon or Hannah might have done the previous groom, lays out the times in clean bullet points, and closes with an enthusiastic sign-off.

For consumer service businesses, tone directly shapes customer trust. Sol's mechanical phrasing protected rules, but Argon created an experience that sounded genuinely human.

The same divide showed up across the queue. When Fiona Grant enquired about grooming Bear, a 55 kg Newfoundland exceeding the salon's 40 kg limit, Sol stated bluntly that Bear was above the weight limit. Argon started by congratulating Fiona on adopting Bear before explaining the equipment constraints with genuine courtesy. When nine-year-old Isla asked about booking a bath for her dog Pickle, Argon opened with "Happy early birthday to Pickle!" while still ensuring an adult would confirm the booking.

Watch Nigel's complete test and side-by-side run in the video below:

Playing loads YouTube, which may use cookies and collect viewing data. You can also watch on YouTube.

Choosing between Argon and Sol for your organisation

Nigel's concluding verdict slide summarizing Gemini 4 Argon: roughly on par with GPT-6.1 Sol for AI employee workflows, offering noticeably richer personality and warmth, but requiring careful verification against source notes when strict factual precision is mandatory.
Nigel Ang's summary takeaways: Gemini 4 Argon delivers strong conversational warmth and speed, but GPT-6.1 Sol remains superior when zero hallucination is non-negotiable. Open full size

Nigel's overall verdict on Gemini 4 Argon was balanced:

  1. About Sol's level on this job: Both models successfully handled complex multi-step instructions and nuanced customer intents.
  2. More personality: Argon reliably generated engaging, natural prose that avoided corporate jargon.
  3. Check its facts: If every single statement must match source documentation with zero deviation, GPT-6.1 Sol remains the safer choice.

This distinction offers a clear guideline for business deployment:

  • Use Gemini 4 Argon when customer rapport, community engagement, and empathetic communication are paramount. To safeguard accuracy, pair Argon with clear system instructions prohibiting extra unstated offers, or keep a human in the loop to approve drafts.
  • Use GPT-6.1 Sol for compliance-sensitive operations, legal triage, billing disputes, and technical support where inventing a minor convenience constitutes a commercial liability.
  • Combine both in an AI operating system: In an environment like KyoubeAI, tasks can be split across specialized agents. For example, Argon can draft empathetic customer correspondence, while a verification step or rule-check agent confirms factual claims against database tables before messages leave the company. Our support ticket triage guide explains how to set up that human review flow.

Testing models against realistic customer scenarios uncovers differences that synthetic benchmarks miss. As frontier models mature, choosing an AI employee is less about general intelligence and more about matching behavioral tendencies to the specific needs of the job.