Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Gilbert (US Patent No.: US 10,525,934 B1) hereinafter Gilbert, in view of Ezzat (Blink‑To‑Live eye‑based communication system for users with speech impairments, Sci Rep 13, 7961 (2023)), hereinafter Ezzat.
Regarding claim 1, Gilbert teaches a computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising: receiving, at a monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), tracking data (This can include some or all of the back end support functions for the vehicle 12, including receipt of the silent alarm signal, tracking of vehicle location, and remotely triggered disabling of the vehicle or certain vehicle functions remotely (such as to slow down/stop the vehicle to help prevent its theft or carjacking of the occupant). Column 4 line 61) from an imaging system (Similarly, use of the camera 60 or gesture input device 62 may be used to recognize certain discrete facial, head, or hand signaling that operates as the silent alarm sequence. Column 12 line 58); executing, via the monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), a tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5); comparing, via the tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5), tracking data with a predefined pattern of the monitoring application (in response to determining that the received input sequence matches the silent alarm sequence, then wirelessly transmitting a silent alarm notification from the vehicle using the vehicle electronics. Column 1 line 41); executing, via the tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5), the protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5) based on the comparison of the tracking data with the predefined pattern (in response to determining that the received input sequence matches the silent alarm sequence, then wirelessly transmitting a silent alarm notification from the vehicle using the vehicle electronics. Column 1 line 41); generating, via the monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), an emergency alert (A system and method for triggering a silent alarm notification in the event the vehicle occupant comes under attack and needs to discretely signal for help. Abstract) based on the executed protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5); receiving, at a back-office server (The silent alarm notification is wirelessly sent to a remote facility for vehicle tracking and emergency response. Abstract), the emergency alert (A system and method for triggering a silent alarm notification in the event the vehicle occupant comes under attack and needs to discretely signal for help. Abstract); and generating, based on the emergency alert received at the back-office server (The silent alarm notification is wirelessly sent to a remote facility for vehicle tracking and emergency response. Abstract), a confirmation notification (Determining a successful receipt by the remote facility 80 may be accomplished by communications handshaking or receipt of a success acknowledgement. Column 14 line 33).
Gilbert does not teach the following limitations as further recited, but Ezzat further teaches receiving, at an eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), eye tracking data (Blink-To-Live is an eye-tracking system based on a modified Blink-To-Speak language and computer vision for patients with speech impairments. Abstract) from an imaging system (A mobile phone camera tracks the patient’s eyes by sending real-time video frames to computer vision modules for facial landmarks detection, eye identification and tracking. Abstract); executing, via the eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), an eye tracking protocol (
PNG
media_image1.png
604
1028
media_image1.png
Greyscale
), the eye tracking protocol including a blink protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and a directional protocol (After the eyeball is detected, its position (in terms of x and y coordinates) is computed and returned as three values mapped to three directions: left (1), right (2), and up (3). Page 7 1st paragraph); comparing, via the eye tracking protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph), key points of the eye tracking data with a predefined pattern of the eye monitoring application (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph); executing, via the eye tracking protocol, one of the blink protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and the directional protocol (After the eyeball is detected, its position (in terms of x and y coordinates) is computed and returned as three values mapped to three directions: left (1), right (2), and up (3). Page 7 1st paragraph) based on the comparison of the key points of the eye tracking data with the predefined pattern (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph); generating, via the eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), an ocular emergency alert based on the executed one of the blink protocol and the directional protocol (
PNG
media_image2.png
604
1028
media_image2.png
Greyscale
).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gilbert to incorporate the teachings of Ezzat to generate, based on the ocular emergency alert received at the back-office server, a confirmation notification and contact emergency services in response to the ocular emergency alert generated by the eye monitoring application in order to generate silent emergency alert based on a blink protocol and a directional protocol.
Regarding claim 2, Gilbert in the combination teaches the method of Claim 1, wherein generating the confirmation notification includes one or more of an icon notification, a light notification, and a haptic notification (This confirmatory indicator may be a predetermined visible (which reads on “an icon notification, a light notification”), audible, or tactile notification that is substantially unnoticeable if at all from outside the vehicle. Column 14 line 36).
Regarding claim 3, Ezzat in the combination teaches the method of Claim 1, further including calibrating, via computer vision models, the eye monitoring application based on facial landmarks (Given that the face region is detected, the next step is detecting the facial landmarks by localizing and labeling the mouth, left and right eyebrows, left and right eyes, nose, and jaw. A set of manually labeled key facial structures in terms of x and y coordinates, along with the pixels’ intensity values and the prior probabilities of the distance of the pixel values corresponding to the facial landmarks pairs, are fed into an ensemble model of regression trees to train a model for facial landmark detector. Page 5 3rd paragraph.
PNG
media_image3.png
420
1022
media_image3.png
Greyscale
).
Regarding claim 4, Ezzat in the combination teaches the method of Claim 1, wherein the key points of the eye tracking data include eye coordinates (The patient’s eyes are detected using the facial landmarks module, which locates the indexes of left and right eyes in the space of 68- (x, y) coordinates. Each eye is detected by 6 coordinates drawn on the created black mask of the same dimension for each reading frame corresponding to a patient’s image. Page 5 last paragraph).
Regarding claim 5, Ezzat in the combination teaches the method of Claim 1, further including executing a training function of the eye monitoring application (A set of manually labeled key facial structures in terms of x and y coordinates, along with the pixels’ intensity values and the prior probabilities of the distance of the pixel values corresponding to the facial landmarks pairs, are fed into an ensemble model of regression trees to train a model for facial landmark detector. Page 5 3rd paragraph).
Regarding claim 6, Ezzat in the combination teaches the method of Claim 5, wherein executing the training function includes calibrating the predefined pattern of the eye tracking protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph).
Regarding claim 7, Gilbert in the combination teaches the method of Claim 1, further including contacting, via the back-office server, emergency services in response to the ocular emergency alert generated by the monitoring application (Then, in step 260, the remote facility 80, if it received the silent alarm notification, can send an alarm notification to the PSAP 82 for dispatch of emergency services. Column 14 line 54).
Ezzat in the combination further teaches in response to the ocular emergency alert generated by the eye monitoring application (
PNG
media_image2.png
604
1028
media_image2.png
Greyscale
).
Regarding claim 8, Ezzat in the combination teaches the method of Claim 1, wherein the predefined pattern includes a blink pattern (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and a directional pattern (
PNG
media_image4.png
870
728
media_image4.png
Greyscale
).
Apparatus claims 9-16 are drawn to the apparatus corresponding to the method of using same as claimed in claims 1-8. Therefore apparatus claims 9-16 correspond to method claims 1-8, and are rejected for the same reasons of obviousness as used above.
Regarding claim 17, Gilbert teaches a monitoring system (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11) for a vehicle (INTEGRATED SILENT VEHICLE ALARM. Title), the monitoring system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (According to another aspect of the invention, there is provided a vehicle having original equipment manufacturer (OEM)-installed vehicle electronics that include a vehicle system module having an electronic processor and memory that is accessible by the processor and that stores a computer program for execution by the processor. Column 2 line 23) comprising: receiving, at a monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), tracking data (This can include some or all of the back end support functions for the vehicle 12, including receipt of the silent alarm signal, tracking of vehicle location, and remotely triggered disabling of the vehicle or certain vehicle functions remotely (such as to slow down/stop the vehicle to help prevent its theft or carjacking of the occupant). Column 4 line 61) from an imaging system (Similarly, use of the camera 60 or gesture input device 62 may be used to recognize certain discrete facial, head, or hand signaling that operates as the silent alarm sequence. Column 12 line 58); executing, via the monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), a tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5); comparing, via the tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5), tracking data with a pattern of the monitoring application (in response to determining that the received input sequence matches the silent alarm sequence, then wirelessly transmitting a silent alarm notification from the vehicle using the vehicle electronics. Column 1 line 41); executing, via the tracking protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5), the protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5) based on the comparison of the tracking data with the pattern (in response to determining that the received input sequence matches the silent alarm sequence, then wirelessly transmitting a silent alarm notification from the vehicle using the vehicle electronics. Column 1 line 41); generating, via the monitoring application (The memory may be a computer-readable medium that stores a computer program having instructions which, when executed by the processor, performs various steps of the method 200, including monitoring for entry of a silent alarm sequence, triggering of the silent alarm in response to detecting the sequence, and initiating the transmission of a silent alarm notification to the remote facility 80, PSAP 82, or elsewhere. Column 8 line 11), an emergency alert (A system and method for triggering a silent alarm notification in the event the vehicle occupant comes under attack and needs to discretely signal for help. Abstract) based on the executed protocol (the one or more manually actuatable input devices includes a microphone, vehicle interior camera, and/or a gesture input device within the vehicle, and wherein the predetermined sequence of activations comprises physical movement or sound from the vehicle occupant that is detected using the microphone, vehicle interior camera, and/or the gesture input device; Column 2 line 5); receiving, at a back-office server (The silent alarm notification is wirelessly sent to a remote facility for vehicle tracking and emergency response. Abstract), the emergency alert (A system and method for triggering a silent alarm notification in the event the vehicle occupant comes under attack and needs to discretely signal for help. Abstract); generating, based on the emergency alert received at the back-office server (The silent alarm notification is wirelessly sent to a remote facility for vehicle tracking and emergency response. Abstract), a confirmation notification (Determining a successful receipt by the remote facility 80 may be accomplished by communications handshaking or receipt of a success acknowledgement. Column 14 line 33); and contacting, via the back-office server, emergency services in response to the emergency alert generated by the monitoring application (Then, in step 260, the remote facility 80, if it received the silent alarm notification, can send an alarm notification to the PSAP 82 for dispatch of emergency services. Column 14 line 54).
Ezzat in the combination further teaches receiving, at an eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), eye tracking data (Blink-To-Live is an eye-tracking system based on a modified Blink-To-Speak language and computer vision for patients with speech impairments. Abstract) from an imaging system (A mobile phone camera tracks the patient’s eyes by sending real-time video frames to computer vision modules for facial landmarks detection, eye identification and tracking. Abstract); executing, via the eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), an eye tracking protocol (
PNG
media_image1.png
604
1028
media_image1.png
Greyscale
), the eye tracking protocol including a blink protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and a directional protocol (After the eyeball is detected, its position (in terms of x and y coordinates) is computed and returned as three values mapped to three directions: left (1), right (2), and up (3). Page 7 1st paragraph); comparing, via the eye tracking protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph), key points of the eye tracking data with a blink pattern (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and a directional pattern of the eye monitoring application (
PNG
media_image4.png
870
728
media_image4.png
Greyscale
); executing, via the eye tracking protocol, one of the blink protocol (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and the directional protocol (After the eyeball is detected, its position (in terms of x and y coordinates) is computed and returned as three values mapped to three directions: left (1), right (2), and up (3). Page 7 1st paragraph) based on the comparison of the key points of the eye tracking data with the blink pattern (The solution implies inspecting the EAR value in the predefined number of video frames (i.e. 13 frames) and if the EAR is still less than 0.2, then it’s the intended language blink. If the number of inspecting frames are less than 13 frames, the blink is very fast and might be a normal one considering that the camera captures 25 frames per second. Page 7 4th paragraph) and the directional pattern (
PNG
media_image4.png
870
728
media_image4.png
Greyscale
); generating, via the eye monitoring application (Blink‑To‑Live eye‑based communication system for users with speech impairments. Title), an ocular emergency alert based on the executed one of the blink protocol and the directional protocol (
PNG
media_image2.png
604
1028
media_image2.png
Greyscale
).
Apparatus claims 18-20 are drawn to the apparatus corresponding to the method of using same as claimed in claims 2-4. Therefore apparatus claims 18-20 correspond to method claims 2-4, and are rejected for the same reasons of obviousness as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights