08-03-2011, 02:36 PM
[attachment=9784]
1. INTRODUCTION
Studies of road safety found that human error was the sole cause in more than half of all accidents .One of the reasons why humans commit so many errors lies in the inherent limitation of human information processing .With the increase in popularity of Telematics services in cars (like navigation, cellular telephone, internet access) there is more information that drivers need to process and more devices that drivers need to control that might contribute to additional driving errors. This paper is devoted to a discussion of these and other aspects of driver safety. Telematics typically is any integrated use of telecommunications and informatics, also known as Information and Communications Technology (ICT). Artificial Intelligence is defined by Webster's as a branch of computer science dealing with the simulation of intelligent behavior in computers. It can also be defined as the capability of a machine to imitate intelligent human behavior. From that we can tell that any program that can strike up a conversation with a human (such as the artificial passenger) would be artificial intelligence.
The Artificial passenger was introduced by Dimitry Kanevsky and Wlodek Zadrozny of IBM labs. IBM got the patent for this in 2001. The artificial passenger is a type of artificial intelligence
Field of invention
The present invention relates to a system and method for determining three dimensional head pose, eye gaze direction, eye closure amount, blink detection and flexible feature detection on the human face using image analysis from multiple video sources.
Additionally, the invention relates to systems and methods that make decisions using passive video analysis of a human head and face. These methods can be used in areas of application such as human-performance measurement, operator monitoring and interactive multi-media.
Background of the invention
Early techniques for determining head-pose used devices that were fixed to the head of the subject to be tracked. For example, reflective devices were attached to the subjects head and using a light source to illuminate the reflectors, the reflector locations were determined.
As such reflective devices are more easily tracked than the head itself, the problem of tracking head-pose was simplified greatly.
Virtual-reality headsets are another example of the subject wearing a device for the purpose of head-pose tracking. These devices typically rely on a directional antenna and radio-frequency sources, or directional magnetic measurement to determine head-pose.
Wearing a device of any sort is clearly a disadvantage, as the user's competence and acceptance to wearing the device then directly affects the reliability of the system. Devices are generally intrusive and will affect a user's behavior, preventing natural motion or operation.
Structured light techniques that project patterns of light onto the face in order to determine head-pose are also known. The light patterns are structured to facilitate the recovery of 3D information using simple image processing. However, the technique is prone to error in conditions of lighting variation and is therefore unsuitable for use under natural lighting conditions.
Summary of the Invention
The present invention provides apparatus and techniques for providing an alarm indication to an owner or driver of a vehicle to indicate potentially hazardous or undesirable conditions. An advantage of the present invention is that it is configured to monitor the environment of a vehicle and provide an alarm indication to an owner or driver of the vehicle regardless of the location of the owner or driver. Additionally, the present invention is configured to have the ability to take preventative and/or corrective actions with respect to the potentially hazardous or undesirable situation.
Accordingly, in a first aspect of the present invention, a situation controller for a vehicle is provided. The situation controller includes a processing device and an image monitor coupled to the processing device, for monitoring images associated with one or more items within the vehicle. The situation controller also includes a device for communicating a message relating to the one or more monitored items wherein the content of the message is determined by the processing device based at least in part on the one or more monitored items. Additionally, a controller coupled to the processing device, for controlling at least one function of the vehicle in response to the one or more monitored items within the vehicle, is included.
In a second aspect of the present invention, a camera system is combined with an artificial passenger system (also referred to herein as a “vehicle system situation controller” or “situation controller”) to monitor an environment of a vehicle and provide an alarm indication to the owner. The camera system identifies the position of keys, for example, and notifies the driver that he or she has left the keys in a particular spot in the vehicle. Thus, the present invention will warn the driver against accidentally locking the keys in the car.
In accordance with a third aspect of the present invention, the artificial passenger is connected to a temperature indicator to analyze the temperature in the vehicle. Thus, in combination with the camera, the artificial passenger is able to determine that a child or pet has been left in a vehicle that it is beginning to get very hot or cold. If the temperature gets too hot or too cool inside the vehicle, the artificial passenger has several options including sending a message to the owner/driver, calling the owner's phone or beeper, calling the police, opening a window or a door, and sounding an alarm to get the attention of people walking by the vehicle (as well as allowing them to open the door to help the occupant). The artificial passenger is able to analyze the situation and execute a corrective action, which includes opening a window or a door to allow the temperature to moderate or to allow the child or pet to leave the vehicle, after the artificial passenger has notified the driver or authorities.
In a fourth aspect of the present invention, the artificial passenger is configured to analyze the situation to determine, for example, whether groceries were left in the vehicle. If the owner did not remove all of the groceries, the artificial passenger will call the owner and tell him or her that the groceries were left in the vehicle. The artificial passenger utilizes an odor detector or sensor as well as the camera to detect whether groceries were left in the vehicle.
In accordance with a fifth aspect of the invention, a communication system that interacts with the owner of the vehicle from a remote location is provided. The communication system utilizes, for example, the Internet and/or a global positioning system (GPS) to locate and communicate with the vehicle owner. Through the communication system, the owner can, for example, open a vehicle door remotely such that a person can enter the locked vehicle.
These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings.
Examples of systems that use this style of technique
examples of systems that use this style of technique can be seen in "A Robust Model-Based Approach for 3D Head Tracking in Video Sequences" by Marius Malciu and Francoise Preteux, and "Robust 3D Head Tracking Under Partial Occlusion" by Ye Zhang and Chandra Kambhamettu, both from Conference of Automatic and Gesture Recognition 2000, Grenoble France.
2. OVERVIEW
The AP is an artificial intelligence–based companion that will be resident in software and chips embedded in the automobile dashboard. The heart of the system is a conversation planner that holds a profile of you, including details of your interests and profession. When activated, the AP uses the profile to cook up provocative questions such “Who was your first teacher?” via a speech generator and in-car speakers. A microphone picks up your answer and breaks it down into separate words with speech-recognition software. A camera built into the dashboard also tracks your lip movements to improve the accuracy of the speech recognition. A voice analyzer then looks for signs of tiredness by checking to see if the answer matches your profile. Slow responses and a lack of intonation are signs of fatigue. If you reply quickly and clearly, the system judges you to be alert and tells the conversation planner to continue the line of questioning. If your response is slow or doesn’t make sense, the voice analyzer assumes you are dropping off and acts to get your attention.
The system, according to its inventors, does not go through a suite of rote questions demanding rote answers. Rather, it knows your tastes and will even, if you wish, make certain you never miss Paul Harvey again. This is from the patent application: “An even further object of the present invention is to provide a natural dialog car system that understands content of tapes, books, and radio programs and extracts and reproduces appropriate phrases from those materials while it is talking with a driver.”
For example, a system can find out if someone is singing on a channel of a radio station. The system will state, “And now you will hear a wonderful song!” or detect that there is news and state, “hear the following and play some news.” The system also includes a recognition system to detect who is speaking over the radio and alert the driver if the person speaking is one the driver wishes to hear.” Just because you can express the rules of grammar in software doesn’t mean a driver is going to use them. The AP is ready for that possibility:
It provides for a natural dialog car system directed to human factor engineering for example, people using different strategies to talk (for instance, short vs. elaborate responses). In this manner, the individual is guided to talk in a certain way so as to make the system work—e.g., “Sorry, I didn’t get it. Could you say it briefly?” Here, the system defines a narrow topic of the user reply (answer or question) via an association of classes of relevant words via decision trees. The system builds a reply sentence asking what are most probable word sequences that could follow the user’s reply.”
1. INTRODUCTION
Studies of road safety found that human error was the sole cause in more than half of all accidents .One of the reasons why humans commit so many errors lies in the inherent limitation of human information processing .With the increase in popularity of Telematics services in cars (like navigation, cellular telephone, internet access) there is more information that drivers need to process and more devices that drivers need to control that might contribute to additional driving errors. This paper is devoted to a discussion of these and other aspects of driver safety. Telematics typically is any integrated use of telecommunications and informatics, also known as Information and Communications Technology (ICT). Artificial Intelligence is defined by Webster's as a branch of computer science dealing with the simulation of intelligent behavior in computers. It can also be defined as the capability of a machine to imitate intelligent human behavior. From that we can tell that any program that can strike up a conversation with a human (such as the artificial passenger) would be artificial intelligence.
The Artificial passenger was introduced by Dimitry Kanevsky and Wlodek Zadrozny of IBM labs. IBM got the patent for this in 2001. The artificial passenger is a type of artificial intelligence
Field of invention
The present invention relates to a system and method for determining three dimensional head pose, eye gaze direction, eye closure amount, blink detection and flexible feature detection on the human face using image analysis from multiple video sources.
Additionally, the invention relates to systems and methods that make decisions using passive video analysis of a human head and face. These methods can be used in areas of application such as human-performance measurement, operator monitoring and interactive multi-media.
Background of the invention
Early techniques for determining head-pose used devices that were fixed to the head of the subject to be tracked. For example, reflective devices were attached to the subjects head and using a light source to illuminate the reflectors, the reflector locations were determined.
As such reflective devices are more easily tracked than the head itself, the problem of tracking head-pose was simplified greatly.
Virtual-reality headsets are another example of the subject wearing a device for the purpose of head-pose tracking. These devices typically rely on a directional antenna and radio-frequency sources, or directional magnetic measurement to determine head-pose.
Wearing a device of any sort is clearly a disadvantage, as the user's competence and acceptance to wearing the device then directly affects the reliability of the system. Devices are generally intrusive and will affect a user's behavior, preventing natural motion or operation.
Structured light techniques that project patterns of light onto the face in order to determine head-pose are also known. The light patterns are structured to facilitate the recovery of 3D information using simple image processing. However, the technique is prone to error in conditions of lighting variation and is therefore unsuitable for use under natural lighting conditions.
Summary of the Invention
The present invention provides apparatus and techniques for providing an alarm indication to an owner or driver of a vehicle to indicate potentially hazardous or undesirable conditions. An advantage of the present invention is that it is configured to monitor the environment of a vehicle and provide an alarm indication to an owner or driver of the vehicle regardless of the location of the owner or driver. Additionally, the present invention is configured to have the ability to take preventative and/or corrective actions with respect to the potentially hazardous or undesirable situation.
Accordingly, in a first aspect of the present invention, a situation controller for a vehicle is provided. The situation controller includes a processing device and an image monitor coupled to the processing device, for monitoring images associated with one or more items within the vehicle. The situation controller also includes a device for communicating a message relating to the one or more monitored items wherein the content of the message is determined by the processing device based at least in part on the one or more monitored items. Additionally, a controller coupled to the processing device, for controlling at least one function of the vehicle in response to the one or more monitored items within the vehicle, is included.
In a second aspect of the present invention, a camera system is combined with an artificial passenger system (also referred to herein as a “vehicle system situation controller” or “situation controller”) to monitor an environment of a vehicle and provide an alarm indication to the owner. The camera system identifies the position of keys, for example, and notifies the driver that he or she has left the keys in a particular spot in the vehicle. Thus, the present invention will warn the driver against accidentally locking the keys in the car.
In accordance with a third aspect of the present invention, the artificial passenger is connected to a temperature indicator to analyze the temperature in the vehicle. Thus, in combination with the camera, the artificial passenger is able to determine that a child or pet has been left in a vehicle that it is beginning to get very hot or cold. If the temperature gets too hot or too cool inside the vehicle, the artificial passenger has several options including sending a message to the owner/driver, calling the owner's phone or beeper, calling the police, opening a window or a door, and sounding an alarm to get the attention of people walking by the vehicle (as well as allowing them to open the door to help the occupant). The artificial passenger is able to analyze the situation and execute a corrective action, which includes opening a window or a door to allow the temperature to moderate or to allow the child or pet to leave the vehicle, after the artificial passenger has notified the driver or authorities.
In a fourth aspect of the present invention, the artificial passenger is configured to analyze the situation to determine, for example, whether groceries were left in the vehicle. If the owner did not remove all of the groceries, the artificial passenger will call the owner and tell him or her that the groceries were left in the vehicle. The artificial passenger utilizes an odor detector or sensor as well as the camera to detect whether groceries were left in the vehicle.
In accordance with a fifth aspect of the invention, a communication system that interacts with the owner of the vehicle from a remote location is provided. The communication system utilizes, for example, the Internet and/or a global positioning system (GPS) to locate and communicate with the vehicle owner. Through the communication system, the owner can, for example, open a vehicle door remotely such that a person can enter the locked vehicle.
These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings.
Examples of systems that use this style of technique
examples of systems that use this style of technique can be seen in "A Robust Model-Based Approach for 3D Head Tracking in Video Sequences" by Marius Malciu and Francoise Preteux, and "Robust 3D Head Tracking Under Partial Occlusion" by Ye Zhang and Chandra Kambhamettu, both from Conference of Automatic and Gesture Recognition 2000, Grenoble France.
2. OVERVIEW
The AP is an artificial intelligence–based companion that will be resident in software and chips embedded in the automobile dashboard. The heart of the system is a conversation planner that holds a profile of you, including details of your interests and profession. When activated, the AP uses the profile to cook up provocative questions such “Who was your first teacher?” via a speech generator and in-car speakers. A microphone picks up your answer and breaks it down into separate words with speech-recognition software. A camera built into the dashboard also tracks your lip movements to improve the accuracy of the speech recognition. A voice analyzer then looks for signs of tiredness by checking to see if the answer matches your profile. Slow responses and a lack of intonation are signs of fatigue. If you reply quickly and clearly, the system judges you to be alert and tells the conversation planner to continue the line of questioning. If your response is slow or doesn’t make sense, the voice analyzer assumes you are dropping off and acts to get your attention.
The system, according to its inventors, does not go through a suite of rote questions demanding rote answers. Rather, it knows your tastes and will even, if you wish, make certain you never miss Paul Harvey again. This is from the patent application: “An even further object of the present invention is to provide a natural dialog car system that understands content of tapes, books, and radio programs and extracts and reproduces appropriate phrases from those materials while it is talking with a driver.”
For example, a system can find out if someone is singing on a channel of a radio station. The system will state, “And now you will hear a wonderful song!” or detect that there is news and state, “hear the following and play some news.” The system also includes a recognition system to detect who is speaking over the radio and alert the driver if the person speaking is one the driver wishes to hear.” Just because you can express the rules of grammar in software doesn’t mean a driver is going to use them. The AP is ready for that possibility:
It provides for a natural dialog car system directed to human factor engineering for example, people using different strategies to talk (for instance, short vs. elaborate responses). In this manner, the individual is guided to talk in a certain way so as to make the system work—e.g., “Sorry, I didn’t get it. Could you say it briefly?” Here, the system defines a narrow topic of the user reply (answer or question) via an association of classes of relevant words via decision trees. The system builds a reply sentence asking what are most probable word sequences that could follow the user’s reply.”
