I'm impressed* by Amazon Rufus (*when it works)

Earlier this week #MuchCurious dropped in with a suggestion - Alchemy. 

Over the millennia the word itself is full to the brim with mysticism, mystery, magic, and disproportionate rewards. The last one is something every human secretly strives for. To be sure, The Alchemist made Paulo Coelho the legend in pop culture, with quotes from that book such as - “when you want something, all the universe conspires in helping you to achieve it”. Yes, it is that which gave Shah Rukh Khan one of the defining dialogues of his carrier (if you are a bollywood buff, you got the translation already; if you are not, it doesn’t matter really.)

Now, this Alchemy is by Rory Sutherland. The legend from Ogilvy who has dissected human behavior and decision making in making the global brands what they are. The snippet read - "Imagine a world where a $100,000 watch is sold not because of its intricate mechanics, but because of the story of its craftsmanship. Where a bottle of perfume is prized not for its fragrance, but for the mystique of its packaging. And where a pair of sneakers is coveted not for its performance, but for the status symbol it represents.

Take, for instance, the story of the De Beers diamond cartel, which created an illusion of scarcity to make diamonds a symbol of luxury and romance. Or the tale of the Red Bull energy drink, which became a global phenomenon not because of its taste, but because of its association with extreme sports and a "can-do" attitude. These are just a few examples of how Alchemy can transform the way we think about products, services, and experiences."

Enough said, I clicked on the link that took me to Amazon.

But there was a problem.

(Sit tight. This is where the real story begins). 

It appears that Sutherland has written two books with the exact same title. The subtexts are different, the cover designs are different, and the pricing difference is significant. It was virtually impossible to determine which one to choose simply based on the book cover. (No, not buying both, thank you. Because, Tsundoku!)

So I checked with little Rufus within the Amazon mobile app (Amazon’s love for dogs has always bemused me). What we have here as Rufus is a serious attempt at Agentic AI (That which we call a GenAI, by any other name would smell as chatty!). This attempt is aimed at potentially influencing purchase decisions for future transactions on the world’s largest e-commerce platform, with the primary objective of this Agentic model being product discovery.

Here is the exact chat with Rufus (beta) on mobile.

Not only that the Agent got the query right, it did provided inferences (yes!) based on data such as number of pages of the books, publication dates for each, and content reference from Amazon Preview of the books to help make the right decision. I revised and wrote the prompt a couple of times, but none of these decision points are explicitely asked for in the prompt. Given that Rufus is still in beta and a mobile-only interface for amazon.in, it was impressive. This was no short of checking for advise with someone who owns both the books.

Rufus (beta) is not perfect. It is buggy, forgetful, over confident and “thin”. It has earned its own redditor/haters. While shopping for a spray paint can, for instance, I asked Rufus what is the surface area that this 245gm paint can would typically cover? It mined through the customer reviews and came back with the answer “Enough for two coats on a complete 3-seater sofa” or “both rims of a motorcycle with three coats each”. Interesting. This provided some perspective, but then I had a follow up question - How many cans would I need to cover 8 feet by 10 feet area for two coats? That’s where the math was overwhelming for the model, and it fumbled (by suggesting “at least two cans”).

Can it be trained further and improved? Certainly. Will we be seeing such optimization in future releases? Less likely. You see the design decision for this LLM model is leaning towards information retrieval rather than towards math accuracy. As the Amazon release notes suggest (below), Rufus is a “Retrieval Augmented Generation (RAG) system with responses enhanced by retrieving additional information such as product information from Amazon search results.”

At the moment Rufus (beta) is like that small puppy who doesn’t know anything in life yet, but it does that one trick well. And when it does it you are overjoyed as the owner. In this chat transaction, for example, it did drag in “The alchemist”, a cough remedy by Alchem pharma, and Psychology of money by Housel. When I “scolded”, it revealed the search option “pathways” that the model "saw" as product possibilities.

This is when, instead of continuing the same prompt, I re-wrote the prompt by making it more precise. It is likely that the missives until then were taken into onsideration while generating the response for this final prompt (the impressive answers). This contextual continuity for a beta model is promising for an involved decision conversation with an engaging customer.

Rufus is scaled to engage with millions of customers concurrently (the reason I called it “thin”) on a day like Amazon Prime Day with less than 1 second latency to the first response, and provide product discovery and product feature details. This is very different from solving math or writing poems like a general purposes LLM like Amazon invested Claude.ai or ChatGPT. Keeping that perspective in sight is helpful.

Try here for Rufus technical architecture and release notes.

What do you think about an LLM or Agentic AI use-case like Rufus? Rufus is already out of Beta and on Amazon main website in many markets including the US. Have you tried it? How was your experience?