The Autonomous Shopping Cart

PR project to prove out human tracking in crowds, navigation, shopping conceptualization. Demoed at Whole Foods, Microsoft Tech Week, featured on Stephen Fry’s Gadgetman.

This project came out of some of the initial Kinect work that started with the Board of Awesomeness. While the BoA’s research goals were focused on trying to find the limits of Kinect’s capabilities (lighting, vibration, motion, etc), the Cart came out of using the Kinect with a Trossen Robotics Eddie platform to actually do something useful with the Kinect. I wasn’t even the first to do something like this - I remember a Paparazzi bot (I did a check and couldn’t immeidately find the actual details) that would wander around tech conferences and use a DLSR camera to take pictures of people by both navigating around people and detecting faces and taking photos. I assume the DLSR was there to provide higher resolution photos than the Kinect was capable of, but I never managed to chat with the project creator.

The original goal was to simply have something which was able to follow a person - simple navigation. But what? Luggage was the first idea, but the structure would have required a good bit of additional work to make it stable and honestly, there weren’t a lot of additional things to explore beyond navigation. At the time, it wouldn’t really have been possible to have an electronic boarding pass or really keep track of your gate to tell you if your flight had changed. But there were some external requests for how to garner attention for and with new clients - Whole Foods being on that list. So a quick brainstorming session and a trip down to the Whole Foods headquarters for a test cart and the project was started. Fortunately, I’d already been testing the Eddie system to tune the onboard controller and so most of the work to get the base functionality going involved shapelock and some general hardware for attaching the drive module under the cart.

And away it went - barcode lookups for nutrition information was pretty simple. Pricing was on the list, but unfortunately there wasn’t the infrastructure today of being able to pull live prices. The ability to do shopping list tracking with vision or RFID tags didn’t take a huge amount of effort. “Spagetti” became a running gag as I had mis-spelled it and the voice synth was happy to proclaim to everyone my oversight. And of course I never rememberd to fix it unless it was happening and there were other, more important things to focus on.

And so that’s how I found myself updating code in the Pasta aisle in Whole Foods and doing tech support from a closet in the Microsoft Home of the Future demo setup. But on the big plus side, it allowed me to meet Stephen Fry in person and nervously watch as it tried to keep up with his long, fast stride through a Waitrose while a woman informed me that she didn’t let the Nazis stop her from her shopping, and a camera crew filming in the store wasn’t going to either. And no, it never crossed my mind that I was going to stand between her and the Bakery section!

If there’s any interest, happy to do a follow on with more specific details.