I thought everybody would have heard of this story by now, but apparently not. So this week, I’m going to tell it here. The lesson is this: for as much as it seems like technology has progressed leaps and bounds even in the last year, we still have a very long way to go. And both as industry participants, and as consumers, we should not be complacent about our progress. But that’s not the only lesson here for retailers. Not by a long shot.

Here’s the story. About a month ago, a woman reported that her six year old daughter had managed to order a dollhouse and four pounds of cookies by interacting with Alexa via the woman’s recently acquired Amazon Dot. The woman said her kids were playing with Alexa by asking and telling knock knock jokes, and then the next thing she knew she had an email order confirmation for a $170 dollhouse and a giant tin of Danish butter cookies. The mother learned a lesson quickly – that when you have young kids, you need to make sure you set a PIN on Alexa’s ordering functionality.

That would be enough education for one day when it comes to artificial-intelligence powered devices, but the story didn’t end there. While the random dollhouse order had originated in Dallas, a San Diego news station ran the story on air, and when it did, prompted Amazon Echos and Dots in viewers’ houses to also order the dollhouse. Which, when you think about it, is both awesome and terrifying.

I have an Amazon Echo. I bought it to see what all this voice interactivity is about. I do not have voice ordering active, in part because I’m just not comfortable with that concept. Buying is visual to me, and I don’t even remotely like the idea of “Alexa, order me a pizza ” even when I know I have the default order for pizza set up just how I like it. Voice ordering is the ultimate in convenience over price – it’s so convenient you don’t even care about making sure it’s the right item or what price it is. That’s not for me. Though, retailers should be terrified if that’s where consumers are headed.

But what fascinates me the most about Amazon’s voice ordering system is how my own kids have taken to it. The mother in the story above reports that after asking if Alexa would play dollhouse with her, and Alexa confirmed the order, her daughter told Alexa “I love you! “. Her child is much younger than my two (which are 15 and 12), but both of them seem to have a certain amount of affection for “her ” – and they call it “her ” – already.

To my teen kids, we don’t own an Echo, we own an Alexa. And they are constantly asking her to tell them the weather, or a joke, or to play a song. My son (the 15-year-old) has the Alexa app on his phone, and gets the emails about what’s new with Alexa. He’s asked for us to download a to-do-list app that Alexa supports so that he can add things he wants to our grocery list via voice, instead of having to type it into the shared list we currently use.

Our Echo sits in our living room. And there have been times when the TV has run an ad from Amazon, where you can tell they are sending some kind of cancelling signal along with the ad, because the Echo will light up at the mention of Alexa, and then turn itself off after a second or two. And there have been times when we have been talking, or watching something, where Alexa thought it had been activated, and suddenly blurted out, “I’m sorry. I didn’t understand the question. ” One of us, my son usually, will tell Alexa to stop or shut up at that point.

I don’t think I realized how much this AI tool has wormed its way into our life until I caught an episode of Star Trek: Voyager. BBC America has been running them a lot lately. If you’re not familiar with this corner of the Star Trek franchise, then all you need to know is that one of the main characters is actually an AI – a holographic doctor, to be used in cases of a medical emergency. The ship has no other doctor, so “he ” is forced to be on all the time, both teaching people medical skills, and handling the usual batch of red-shirts that just got chewed up by the latest planet. In the episode I caught, the doctor has a new assistant, and she’s appalled at how the crew treat the doctor, “like he’s a piece of furniture or something “. The writers assumed that even though people might be faced with someone who looks real, sounds real, even feels real to the touch, they would still know deep down that it’s not a real person. And thus treat it more like a piece of furniture than a person.

After hearing about the girl who loves Alexa, and watching my own kids readily adapt to calling our Echo “her ” – and that’s just a disembodied voice – I have a feeling the human tendency is going to be to relate to anything that is moderately human as human, even when they know it’s not. I am certain that we will have more fun stories of AI-driven, voice-activated devices doing crazy things. But I’m also certain that for the retailers struggling to compete against Amazon, they need to think a lot more about what kind of personality and trust people are projecting on to services like Alexa. Increasingly, they may find they are not competing against some wacky bald guy and an internet behemoth. They are competing against Alexa, my kids’ favorite disembodied voice, who will always tell them jokes and find them the best songs to listen to. And, I guess, order the occasional toy if you’re not careful.