Canon PowerShot SD4000 IS
ƒ/2.8
4.9 mm
1/100
400

As we walk by the camera, the TV tracks the faces and displays in real-time:

• Emotional attributes of affect: I generally peg the happiness scale

• Age: 38 +/- 12 (their mean error is 6.85 years, and this guess was within 6 years of accuracy)

• Gender: Male (94.3% accuracy)

• ID: Assigned on the fly I presume, to track me over time (91.5% detection rate)

• Uptime: how long it has been tracking me, often unawares as I had conversations off in the distance.

The most eerie instance occurred some time later. I was having a spirited argument with a fellow who designs and sells high end AV cables. “Surely, you have to admit that fancy HDMI cables are a scam, right? It’s a digital standard. At normal cable lengths, it either works or it doesn’t. Are you selling the quality of the 0’s and 1’s?” He disdained double blind tests and insisted that somehow, magically, the blacks on the screen were more grey. I was stupefied.

Then this screen caught my eye off in the distance; it spotted me in the crowd. It said I was Angry and Surprised.

18 responses to “Fraunhofer Face Finder”

  1. Jeff Clow Avatar
    Jeff Clow

    Fascinating stuff, Steve….thanks for sharing.

    Reply
  2. BillS49 Avatar
    BillS49

    Privacy is mostly gone in technically advanced nations, but we all sail that same ship – the price we pay to enjoy the life we live, most of the time it’s a good trade.

    Reply
  3. scleroplex Avatar
    scleroplex

    :-))

    Reply
  4. nhr Avatar
    nhr

    Coming soon to a surveillance camera near you: "Big Sister"  

    Reply
  5. JJ San Avatar
    JJ San

    More proof that the simple personal act of walking to and fro is a potential win for all sorts of big-data-enabled entities…

    Reply
  6. DNSF David Newman Avatar
    DNSF David Newman

    …and now to suit
    our great computer…

    youtu.be/xgvVFv2jiL0

    Reply
  7. AGrinberg Avatar
    AGrinberg

    Remarkable!

    I wonder what kind of cable connects to the Face Finder? 🙂

    Reply
  8. !MimosaMicheMichelle! Avatar
    !MimosaMicheMichelle!

    Spooky! Also fascinating.

    I bet the happiness scale would be strong for you all day as, based on all your photos, you seem to have a big smile permanently etched on your face. I would have like to see you when you are angry and even more so when you are surprised.

    Fun, Steve, Merci!

    Bonjour T!

    Reply
  9. jurvetson Avatar
    jurvetson

    yes… and kids smile a lot more than adults. Big source of endogenous endorphins. (ref. TED Talk)

    [http://www.flickr.com/photos/agrinberg] – Cables are NSA standard issue.

    Miko on fB had an interesting comment: "Think of it as a neural prosthesis for autistic kids."

    Portraits in surveillance, some of my prior posts:Welcome to ShenzhenRepressive Tolerance TSA Far Blue CellAi Weiwei in the MachineMore on the Fraunhofer IIS

    Reply
  10. mikomatsumura Avatar
    mikomatsumura

    facial expression analysis / microexpressions has a grand tradition… citing Dr Paul Ekman’s brilliant Faction Action Coding System work. Pretty soon this technology will be banned in poker tournaments if it isnt already.

    Reply
  11. solerena Avatar
    solerena

    Makes one wonder again…

    Reply
  12. seatonsnet Avatar
    seatonsnet

    Double Wow (seriously)

    Reply
  13. linux-works Avatar
    linux-works

    I don’t know much about video, but I do know that jitter is real, in terms of digital audio cabling. as long as the DAC derives some timing from the spdif signal (which has clock and data components, all on the same wire), and if the DAC does not fully reclock (buffering) then jitter DOES seem to affect audio. I’ve seen it on scopes. I’m not sure my ears are good enough to hear it but I really can see some cables in digital audio being more of a problem than others.

    hdmi may not be the same problem, though; if they have isolated their clocks and data and kept them different then the argument about jitter mostly goes away.

    (no, I’m not a cable voo-doo guy; but recently I did take a visit to an audio lab and actually saw the spectrum display showing jitter ‘mattering’ on a cable).

    Reply
  14. jurvetson Avatar
    jurvetson

    I’m not sure, but when it comes to a digital interface, the data should be easy to gather. No analog judgement calls to be made. And many articles on this.

    Reply
  15. linux-works Avatar
    linux-works

    lately, I’ve been tinkering with i2s audio as a way to get around the problems of spdif (which has clock+data in one waveform encoding, on the wire). it turns out that you cannot extract the clock perfectly when both are combined. but by having a clock line AND a data line (or in i2s, lots of clock lines; word clock, bit clock, etc) you solve the problem at hand.

    now, that’s audio. again, I have no idea about hdmi video and its possible that the 2 problem sets share little in common.

    the audio problem is that you have to ‘dump out’ the d/a value on the wire at the exact right point in time. the clock wire is what determines the ‘when’ of the voltage (or current) value and that’s what is jittery. see, I have this sample I want to squirt out on the wire but WHEN do I squirt it out? the time value I have is a little shakey (jittery) and its impossible to KNOW the precise (down to picoseconds) time to dump that d/a value.

    fwiw, time has to jitter a LOT in order for it to be a major impact on sound. but on test gear, it can ‘hear’ when the timing is off. 50picoseconds matters! even 1/10 of that matters (and is a good value to shoot for).

    but I do wonder if video, by its nature, has to buffer a LOT more stuff before dumping to the wire (so to speak) and also the frame redraw rate governs a lot about the video bit timing. its possible that video jitter has little or none of the problems that audio jitter has.

    Reply
  16. linux-works Avatar
    linux-works

    @ "the data should be easy to gather."

    just to be clear, knowing if its a 0 or 1 has never been hard or in dispute. its trivially easy to extract the data bits and with near 100% confidence.

    @ "No analog judgement calls to be made."

    correct; even a very sloppy digital pulse can be correctly ID’d as a 0 or 1. no data ever gets dropped. but at exactly _which point in time_ do you drop this 8 or 16 or 32bit sample? that’s the tl;dr version, I guess 😉

    Reply
  17. fuzee3d Avatar
    fuzee3d

    So interesting. Can i use this as a title picture for Emotional Awareness as part of Education and Career Guidance for Singapore Polytechnic?

    Reply
  18. jurvetson Avatar
    jurvetson

    Sure. 🙂 I see it was used here today: "AI can now read emotions—but should it?"

    "In its annual report, the AI Now Institute, an interdisciplinary research center studying the societal implications of artificial intelligence, called for a ban on technology designed to recognize people’s emotions in certain cases. Specifically, the researchers said affect recognition technology, also called emotion recognition technology, should not be used in decisions that "impact people’s lives and access to opportunities," such as hiring decisions or pain assessments, because it is not sufficiently accurate and can lead to biased decisions."

    It reminds me of Yuval Noah Harari’s warning in 21 Lessons for the 21st Century:

    “AI is now beginning to outperform humans in the understanding of human emotions.”

    “In particular, AI can be better at jobs that demand intuitions about other people.”

    “Feelings guide not just voters but their leaders as well. This reliance on the heart might prove to be the Achilles’ heel of liberal democracy. For once somebody (whether in Beijing or San Francisco) gains the technological ability to hack and manipulate the human heart, democratic politics will mutate into an emotional puppet show.”

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *