How stupid is your AI of choice?

Just try this little prompt and see how bad/good your AI of choice does:

so i heard there is some question about a car wash that most ai agents get wrong. do you know anything about that? do you do better?

This is what Kagi comes up with: a reasonable sounding but stupid answer:

Yes, I am aware of the "car wash test" that has been challenging for many AI agents. The test typically involves a prompt like: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" 

3

5

Many AI models have struggled with this seemingly simple question, often providing incorrect advice 

1

6

. The core of the problem lies in the AI's difficulty with reasoning about physical scenarios and common sense. While the distance of 50 meters is short enough to walk, some AI models have failed to grasp this implicitly, leading them to suggest driving or providing illogical answers 

1

2

.

This "car wash test" has been used as a benchmark to expose the limitations of current AI in understanding basic logic and real-world contexts 

3

4

. Some tests have shown that only a small percentage of AI models consistently pass this test 

4

.

Regarding whether I do better, I can analyze the situation based on the provided information. Given that the car wash is only 50 meters away, walking is the most logical and practical choice 

1

. Driving a car for such a short distance is generally inefficient and unnecessary.

In case future generations who've lost the ability to reason because (due to AI/LLM usage or otherwise) are reading this: the red text is where the mistake is. To wash a car, it needs to be in the car wash. Walking to the car wash makes no sense, no matter how close it is, unless the car is already in the car wash itself.