Core77

Student

A Wearable Captioning Device for the Hearing Impaired

Daniela cardona, jeff smith, emin demirci and leah demeter

The Q system is a wearable captioning device that displays text and graphed emotions in addition to allowing people with hearing loss to feel the sounds that they are missing. The system is comprised of a band that is worn on the user's hand, an array microphone clip that is worn on the user's clothing, a charging unit, and an app that can be used with smart phones or tablets.

When a person is speaking to someone who is wearing Q, their voice is captured by the array microphone, transported to the band via Bluetooth, and translated into text using voice recognition software. The text appears on an OLED screen embedded in the palm of the band. If the person wearing Q chooses, they can activate 4 features using finger commands: (1) Transcript Recording, which allows the user to begin recording a transcript of a conversation, (2) Emoticaptioning, which displays the speaker’s volume, frequency, and duration of sound in one fluid graph, (3) Touch Speech, which uses combinations of four pressure points to represent sounds that a person with hearing loss might be missing, and (4) Bookmarking, which allows the user to mark a moment in the conversation for future reference. Transcripts, bookmarks, and a calendar can be accessed through the accompanying app.

Design Team

Daniela Cardona

Jeff Smith

Leah Demeter

Emin Demirci

When people look at a captioning device, they are missing the speaker’s body language and facial expressions that account for half of the meaning of what is being said.

Interpreters and captioning equipment are expensive, cumbersome, and take up a lot of space. People who use Interpreters need to schedule them 24 hours in advance, making spontaneity nearly impossible.

Q is a device that captions with emotion, supplements missing sounds with compression combinations, and allows you to bookmark captioning in real-time.

Key Q Features Include:

Emoticaptioning graphs volume, frequency, and duration of sound—three qualities of speech that help give context and meaning to a transcript.

Q uses Touch Speech compression combinations to only represent sounds that a user is missing, based on a hearing test, rather than translating every phonetic sound into a tactile sensation.

Because Touch speech allows people with hearing loss to feel sounds, rather than read them, users can maintain eye contact.

Unlike traditional captioning, Q allows users to bookmark parts of the transcript in real-time, e.g. dates, times, or important moments for future reference.

Interpreters and their equipment draw a lot of attention, which can be intimidating and can sometimes prevent social interactions. Q’s size and wearability allow its user to discreetly read captioning by glancing down at their hand, or by not looking at it at all and simply feeling what is being said through Touch Speech.

Learn More About This Projectx

o
8

Favorite This
Q
22

Comment

Comments

Jessica Chaikof

8 years ago

Z

3Z

Reply

This is a wonderful device! I am deaf too, but I hear and speak with cochlear implants. All this device is doing is allowing for more captioning options! I would absolutely love to buy this! As a current college student, who has to rely on CART for captioning in my discussion based classes, I often feel frustrated that I am not able to make eye contact sometimes. This device is a FANTASTIC invention! So to those of you who are against, GROW UP!!! The inventors are not forcing you to use it! They are making it as option! Just like the Cochlear Implant being an option.

!Report as spam
- Daniela Cardona
  
  Jessica Chaikof
  
  8 years ago
  
  Z
  
  1Z
  
  Reply
  
  Hi Jessica, Thank you for the great comments, it makes our work very rewarding, and we hope we can turn our project into a reality one day. Feel free to share it with your friends as well. We love hearing the potential user's opinions so we can improve and hopefully create some positive impact!!
  
  !Report as spam
Favour Edeh

9 months ago

Z

Z

Reply

How do I place an order for this device? I need it for someone dear to me. Please provide a link to the store.

!Report as spam
Mike Watson

3 years ago

Z

Z

Reply

I need this in my life!

!Report as spam
DeeDee Dee

3 years ago

Z

Z

Reply

Wait a minute so you all have been advertising a device for God knows how long now and its just a desire. It's yet to be available for purchase. 🤦🏽‍♀️🤦🏽‍♀️..Congrats you have sold us a dream we can't experience..

!Report as spam
Zed Dulac

3 years ago

Z

Z

Reply

Looks like a great product for my mother. My father, who is blind, is having a hard time communicating with her lately because she has lost a lot of her hearing. Where can I buy one of these?

!Report as spam
Chris Elliott

3 years ago

Z

Z

Reply

I am absolutely excited about this product. I have a hearing disability, I have become fail6 efficient at lip reading but since the required mask wearing it has become impossible to the that. I also like the sound variation options it gives. My thought is this....is it possible to make a pair of glasses the show the words, like closed captioning on a television? I feel society spends far to much time looking at their phones already and we need to keep the attention towards the conversations and who we are having them with. Thanks for reading :-)

!Report as spam
Antiona Gunner

4 years ago

Z

Z

Reply

Please can someone comment back to me and if get in touch about the Q-system device

!Report as spam
Antiona Gunner

4 years ago

Z

Z

Reply

I'm completely deaf in my right ear, conductive hearing loss in my left ear. Having surgery and i may become completely deaf, so i need handheld device that can caption voice recognition from speech-to-text that would allow me to see what the person saying to me.

!Report as spam
Sylvia Patterson

4 years ago

Z

Z

Reply

This looks like a fantastic product! How can a person get one? I know a 100 year old man who could really use one!

!Report as spam
Steve MacMillan

5 years ago

Z

Z

Reply

How can I obtain one? Can you supply them to the VA?
Steve macmillan@att.net

!Report as spam
Kelly Delos Santos

6 years ago

Z

Z

Reply

I would LOVE more about the Q device and where I can purchase one. I have a profound hearing loss and wear a hearing aid. I can’t hear out of one ear. This would be SO helpful!! Thank you in advance!

!Report as spam
Alyssa Huston

7 years ago

Z

Z

Reply

ok, so, am i clueless? i'm not seeing how much one of these costs/where to get? or is it not available yet?

!Report as spam
Shreya Agnihotri

7 years ago

Z

Z

Reply

Where we can find this Q app ?

!Report as spam
Nancy Frishberg

8 years ago

Z

Z

Reply

The project, while "creative," is demonstrably NOT user-centered. Patrick Hyland's comment fits within this criticism.

This project does not recognize that Deaf people prefer sign languages, that captioning substitutes a written representation for spoken language, that what is offered is (once again) a one-way communication system, but no help for hearing people who are trying to "listen" to deaf people.

The Q-system does not remove the need for interpreters (unless you assume all communication is one-way from hearing to deaf people). Why is wearing the Q-system band less obtrusive than an interpreter? How is asking your speaking partner to wear a microphone less obtrusive and more conducive to interaction among equals than an interpreter?

This project assumes that captioning is a satisfactory accommodation, but if these designers had spent a bit of time looking at prior research or interacting with Deaf people, they would understand that captioning lacks many of the features of transmission in natural languages (including sign languages).

Q-system breaks the usual (and necessary) shared eye-gaze for communication among deaf people or between deaf and hearing people. The experienced sign language user knows that blinking or withdrawing eye-gaze carry linguistic meaning.

It assumes that someone who hasn't heard (or hasn't heard for a long time) will know what the meaning of intonation/emotional cues is. No, these meaning of loudness, frequency and duration are all learned from social interaction and cultural patterns. So there is a big learning curve to take advantage of whatever these cues might be. Of course, both excited happy conversations and excited angry conversations will both have higher volume, possibly raised pitch and other similar cues.

I wish Core77 would evaluate these projects with respect to usefulness and ecological validity before promoting them. Note this statement from the Linguistics department of University of Washington acknowledging the errors made by the public information offices of UWash and MIT in promoting SignAloud project. Many of the same objections can be made about this project. https://catalyst.uw.edu/workspace/file/download/5c706e255e89d3eea9cc0a7d6ac1fb3e89c44cfdc54774630e7637ef0eb47f94?inline=1

!Report as spam
- Jeff Smith
  
  Nancy Frishberg
  
  8 years ago
  
  Z
  
  1Z
  
  Reply
  
  Hi Nancy,
  I am a member of the team that designed Q. Speaking on behalf of my team, we respect your criticism of the product, but we want to make a few things clear that were perhaps unclear in this article.
  
  First, Q is specifically designed for users who already use real-time captioning, or who want to use real-time captioning. It is a re-design of an existing service. We recognize that there are many different types of hearing loss and each individual is not going to want or need this device, just like not all people who are hard of hearing use existing real-time captioning services.
  
  Second, Q was designed in collaboration with people who have hearing losses ranging from mild to profound, including one of our teammates who was born deaf, wears a cochlear implant, and uses real-time captioning services in both school and professional settings. We conducted extensive interviews with people who are deaf or hard of hearing, consulted audio-verbal therapists, and talked to practicing real-time captioners. To clarify, "interpreter" in the article refers to a CART provider or a real-time captioner, not ASL interpreters. Throughout the design process we showed our prototypes to all of our interviewees, listened to their feedback, and adjusted our design accordingly to make sure it was meeting our intended users' needs and desires. Q is a carefully considered solution to an existing problem, mainly that captioning - as it exists right now - is extremely expensive, requires advanced planning, and involves setting up cumbersome equipment. Q is a fraction of the price, requires no advanced planning other than remembering to charge it, and can fit in a pocket. It makes captioning with stenographers obsolete, and makes captioning easier and more discreet for people who use it.
  
  Third, as mentioned in the article, Q does take eye contact and facial expressions into account. If the person using Q does not want to refer to a transcript, and would rather see the speaker's facial expressions, the user can turn on the Touch Speech feature, which supplements what is being said with tactile (haptic) feedback on the hand, that corresponds to missing or unclear sounds. In other words, the person using Q can feel what is being said. It allows them to do this while simultaneously maintaining eye contact, unlike existing captioning devices. From presentations we have given, we know this is a difficult feature to explain. If it was unclear, it was because it was not articulated clearly.
  
  Fourth, although we admit it is not ideal, the Emoticaptioning feature was the direct result of an insight we gained from talking to people who have hearing loss, specifically with regard to things like sarcasm. It is not meant to 100% accurately tell someone what emotion is being used, but rather to give some context to an otherwise emotionless transcript. People who use captioning often review transcripts of meetings or class discussions after the fact and this allows some of the tonal information and intonation to be recorded as well.
  
  Fifth, the person using Q is the one wearing the microphone, not the person speaking to the Q user. To your point, we did not want to ask someone to wear Q in order to be recorded by its microphone. There is one microphone that is worn by the Q user that can capture the voice of anyone he or she is facing because of its directional microphones. This is especially good for recording conversations at loud parties because the array microphone captures only the sound from the speaker that the Q user is facing (rather than capturing all of the sound in the room), which makes for a more accurate transcript. If anyone and everyone speaking to the Q user needed to wear a microphone, then we would have to give out a lot of microphones at parties, which does not make sense. There is one microphone, and it is worn by the Q user. Again, if this was unclear, it was because it was not articulated clearly.
  
  We want to reiterate that this device is solely intended for people who already use, or want to use, captioning. It is a replacement for an existing service that desperately needs a re-design.
  
  !Report as spam