Artificial Passenger
- 1. ARTIFICIAL PASSENGER (A Sleep Prevention Dialogue Based Car System) By Kini Priyanka
- 2. Introduction IBM (International business machines corporation, NY) has developed a software that holds a conversation with the driver to determine whether the driver can respond alertly enough, called “ Artificial Passenger ”. This was designed to make long solo journeys safer and more bearable.
- 3. Why Such System? According to a national survey in UK and USA, it is observed that the driver fatigue annually causes 10000 crashes 1500 deaths 7100 injuries Majority of off-road accidents observed, were caused by eye closure of half and even 2-3 seconds, where the normal human eye blinks at 0.2-0.3 seconds.
- 4. What is an Artificial Passenger? Natural language e-companion. Sleep preventive device in cars to overcome drowsiness. Life safety system.
- 5. What Does It Do? Detects alarm conditions through sensors. Broadcasts pre-stored voice messages over the speakers. Captures images of the driver.
- 6. Condition Sensor Mobile Indicator Device
- 7. About Artificial Passenger The AP is an Artificial Intelligence based companion that will be resident in software and chips embedded in the automobile dashboard. The system has a conversation planner that holds a profile of you, including details of your interests and profession.
- 8. A microphone picks up your answer and breaks it down into separate words with speech-recognition software. A camera built into the dashboard also tracks your lip movements to improve the accuracy of speech recognition.
- 9. A voice analyzer then looks for signs of tiredness by checking to see if the answer matches your profile. Slow responses and lack of attention are signs of fatigue. If you reply quickly and clearly, the system judges you to be alert and tells the conversation planner to continue the line of questioning.
- 10. If your response is slow or doesn’t make sense, the voice analyzer assumes you are dropping off and acts to get your attention. If driver displays signs of fatigue, the artificial passenger might be programmed to open all the windows, sound a buzzer, increase background music volume, or even spray the driver with ice water.
- 11. Devices Used in Artificial Passenger Eye tracker Voice recognizer or speech recognizer Natural language processor Driver analyzer Conversational planner Alarm Microphone Camera
- 12. Working Components Eye-Tracker Collecting eye movement data requires both hardware and software. Hardware:: Head-Mounted systems or remote systems Both systems measure the corneal reflection of the infrared LED, which illuminates and generates a reflection off the surface of the eye. This action causes the pupil to appear as a bright disk in contrast to surrounding iris and creates a small glint underneath the pupil which is used for calibration and tracking. Head mounted systems use multiple data points to record eye movement and measures pupil glint from multiple angles. Remote systems ,in contrast, measure the orientation of the eye relative to a fixed unit such as camera.
- 14. Software: Data collection and analysis is handled by eye-tracking software. Software catalogs eye-tracking in one of the two ways ERICA’s Eye Gaze software uses a small red x to represent eye movement. In other, data is stored as a series of x/y coordinates related to specific grid points on the computer screen
- 15. Eye Tracker
- 16. Working Components( cont…) Automatic Speech Recognition There are two ASRs used in the system: First one is “speaker independent” and used for decoding voice signals of the driver. Second one operates with voice car media and decodes tapes, audio , telephones, mails etc.
- 17. Working Components( contd…) Natural Language Processor Processes the decoded signals of voice or text data provided from the ASR. Identifies related meanings from the contents of the decoded messages. Produces variant of responses. This output goes to the driver analyzer as an input.
- 18. Working Components( contd…) Driver Analyzer Receives the textual data and voice data. Measures the time response using a clock. Time responses, conclusions about driver’s alertness will be passed on to the conversational planner.
- 19. Working Components( contd…) Conversational Planner It is the heart of the system. Instructs the language generator to produce the response. If the driver is in perfect condition CP instructs the language generator to continue the conversation otherwise it will be instructed to change the conversation.
- 20. The following is a typical scenario involving Artificial Passenger. Imagine, driver “Joe” returning home after late nights. His head starts to nod AP Hey Joe, what did you get your daughter for her birthday? Joe It’s not her birthday! AP You seem a little tired. Want to play a game? Joe Yes. AP You were a wiz at “Name that Tune” last time. I was impressed. Want to try your hand at trivia? Joe ok. AP Pick a category: Hollywood Stars, Magic Moments or Hall of Fame? Joe Hall of Fame.
- 21. Working Components (contd…) Alarm If the CP (conversational planner) receives information that the driver is about to fall asleep an alarm system is activated. Microphone For picking up words and separate them by some internally used software for conversation.
- 22. Working Components (contd…) Camera This will track the lip movements of the driver. Used to improve the accuracy of the speech recognition.
- 23. . Showing the dashboard of the car where the whole artificial system is generally attached.
- 24. Steps Involved in Tracking Automatically initialize lips and eyes using color predicates and connected components. Track lip corners using dark line between lips and color predicate even through large mouth movement like yawning.
- 25. Construct a bounding box of the head. Determine rotation using distances between eye and lip feature points and sides of the face. Determine eye blinking and eye closing using the number and intensity of pixels in the eye region. Determine driver vigilance level using all acquired information.
- 26. Detecting Driver Vigilance Aiming a single camera at a head of the driver. Detecting frequency of up and down nodding and left to right rotations of the head within a selected time period with the camera. Determining frequency of eye blinking and eye closing. Determining frequency of yawning of the driver within the selected time period with the camera Generating an alarm signal in real time if the frequency value of the up and down nodding, the left to right rotations, the eye blinking, the eye closings, the yawning exceeds a selected threshold value.
- 27. Applications Interface with Neighboring Cars - Determines if a driver presents a high safety risk for ex. falling asleep, tired, inexperienced or under the influence of alcohol and signals the cars nearby to be careful of the driver. Medical Application -The system can monitor a driver and detect if they are sick ,for ex. having a stroke or heart attack. In any problem it alerts the vehicles near by , so the driver there can become alert. Opens and closes the doors and windows of the car automatically. It is also used for the entertainment. Also used in cabins in airplanes, trains, boats etc
- 28. Future Implementation Will provide us with shortest time routing based on road conditions changing because of weather and traffic, information about the cars on the route, destination requirement (as-flight has been delayed etc)
- 29. Conclusion Method for monitoring driver alertness Sufficient time to avert an accident. Successful implementation of Artificial passenger would allow use of various services in car like reading emails, navigation, downloading music files, voice games etc without compromising on driver safety
- 30. Questions?
- 31. Thank You…