For Your Consideration : Emotion Sensing Machines, DIY Drugs, and Spider Tanks

Incredibly interesting piece on teaching computers to sense human emotion. I kept thinking about how you could combine this with thermal imaging and voice stress analysis to get a pretty reliable remote lie detector. When would this be admissible in court? Could emotion sensing systems be used on archival video for historians? To review congressional testimony? To analyze CEOs during quarterly meetings to see if they are being truthful or enthusiastic about their prospects?

Affectiva is the most visible among a host of competing boutique startups: Emotient, Realeyes, Sension. After Kaliouby and I sat down, she told me, “I think that, ten years down the line, we won’t remember what it was like when we couldn’t just frown at our device, and our device would say, ‘Oh, you didn’t like that, did you?’ ” She took out an iPad containing a version of Affdex, her company’s signature software, which was simplified to track just four emotional “classifiers”: happy, confused, surprised, and disgusted. The software scans for a face; if there are multiple faces, it isolates each one. It then identifies the face’s main regions—mouth, nose, eyes, eyebrows—and it ascribes points to each, rendering the features in simple geometries. When I looked at myself in the live feed on her iPad, my face was covered in green dots. “We call them deformable and non-deformable points,” she said. “Your lip corners will move all over the place—you can smile, you can smirk—so these points are not very helpful in stabilizing the face. Whereas these points, like this at the tip of your nose, don’t go anywhere.” Serving as anchors, the non-deformable points help judge how far other points move.

Affdex also scans for the shifting texture of skin—the distribution of wrinkles around an eye, or the furrow of a brow—and combines that information with the deformable points to build detailed models of the face as it reacts. The algorithm identifies an emotional expression by comparing it with countless others that it has previously analyzed. “If you smile, for example, it recognizes that you are smiling in real time,” Kaliouby told me. I smiled, and a green bar at the bottom of the screen shot up, indicating the program’s increasing confidence that it had identified the correct expression. “Try looking confused,” she said, and I did. The bar for confusion spiked. “There you go,” she said.

Many companies are moving to take advantage of this shift. “We put together a patent application for a system that could dynamically price advertising depending on how people responded to it,” Kaliouby told me one afternoon. I found more than a hundred other patents for emotion-sensing technology, many of them tied to advertising. Represented: A.O.L., Hitachi, eBay, I.B.M., Yahoo!, and Motorola. Sony had filed several; its researchers anticipated games that build emotional maps of players, combining data from sensors and from social media to create “almost dangerous kinds of interactivity.” There were patents for emotion-sensing vending machines, and for A.T.M.s that would understand if users were “in a relaxed mood,” and receptive to advertising. Anheuser-Busch had designed a responsive beer bottle, because sports fans at games “wishing to use their beverage containers to express emotion are limited to, for example, raising a bottle to express solidarity with a team.”

Not long ago, Verizon drafted plans for a media console packed with sensors, including a thermographic camera (to measure body temperature), an infrared laser (to gauge depth), and a multi-array microphone. By scanning a room, the system could determine the occupants’ age, gender, weight, height, skin color, hair length, facial features, mannerisms, what language they spoke, and whether they had an accent. It could identify pets, furniture, paintings, even a bag of chips. It could track “ambient actions”: eating, exercising, reading, sleeping, cuddling, cleaning, playing a musical instrument. It could probe other devices—to learn what a person might be browsing on the Web, or writing in an e-mail. It could scan for affect, tracking moments of laughter or argument. All this data would then shape the console’s choice of TV ads. A marital fight might prompt an ad for a counsellor. Signs of stress might prompt ads for aromatherapy candles. Upbeat humming might prompt ads “configured to target happy people.” The system could then broadcast the ads to every device in the room.

In 2013, Representative Mike Capuano, of Massachusetts, drafted the We Are Watching You Act, to compel companies to indicate when sensing begins, and to give consumers the right to disable it.

2. DIY Drugs Digital Future

I’ve been asked to do this a dozen times. Every editor said to me, “Can you make us a drug?” and I said, “Yes, but what would the point be?” And they couldn’t give me an answer. [Then] I started talking with Bobby Johnson, an editor at Matter. I said, “What was the point at which, culturally, drugs actually became part of the weave of everyday society?”

My contention would be that that was the birth of LSD and the Beatles and the ’60s. So I thought, What was the first drug experience that the Beatles had? And it was Benzedrine, but I didn’t fancy making Benzedrine [an amphetamine] because it isn’t as unusual.

The legal high story represents a pivotal change in the way that drugs are manufactured, consumed, experienced, and mediated in a society, and I wanted to find a drug that was taken by the man who introduced LSD to the United Kingdom.

There is a picture of the Beatles holding tubes of Preludin. I thought, You know? That’s no different than young kids now posing on Facebook with a pile of mephredrone—except for way that the whole world is so interconnected.

It just tied together a few strings for me: privacy, publicity, the consequences of drug use. [I wanted] to make a legal version of John Lennon’s favorite drug. It’s a great headline, isn’t it?

What was your scariest moment when you were having the drug made?

When I went to collect it, walking through the streets of London with a bag of five grams of white powder. If the police stopped me, I would have to tell them that it was actually a legal version of Preludin that I had had synthesized in a Shanghai laboratory.

I don’t fancy my chances that the police would have believed me. I think I would have been taken to the cells while they sent it off for testing. That was really scary.

3. Spider Tanks

Because, Spider Tanks… These renderings are just stunning in their detail and design.