DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim status
Claims 1 and 53-71 are currently pending for examination.
Double Patenting
3. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-l.jsp.
4. Claims 1, 53-55, 58-71 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-3, 5-9, 12-13, 15-20 of US Patent No. 11,738,756. Although the conflicting claims are not identical, they are not patentably distinct from each other because:
The patent claims include all of the limitations of the instant application claims, respectively. The patent claims also include additional limitations. Hence, the instant application claims are generic to the species of invention covered by the respective patent claims. As such, the instant application claims are anticipated by the patent claims and are therefore not patentably distinct therefrom. (See Eli Lilly and Co. v. Barr Laboratories Inc., 58 USPQ2D 1869, "a later genus claim limitation is anticipated by, and therefore not patentably distinct from, an earlier species claim", In re Goodman, 29 USPQ2d 2010, "Thus, the generic invention is 'anticipated' by the species of the patented invention" and the instant “application claims are generic to species of invention covered by the patent claim, and since without terminal disclaimer, extant species claims preclude issuance of generic application claims”).
5. Claims 56-57 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1 and 19 of US Patent No. 11,738,756. Although the conflicting claims are not identical, they are not patentably distinct from each other because “the one or more processors also receiving incoming data from one or more other sensing devices” and “ the one or more other sensing devices include one or more motion sensors, heart rate monitors, breathing monitors, or a combination thereof” are conventional prior art features (as shown in the art rejection below) and the use of such features in claims 1 and 19 of US Patent No. 11,738,756would have been obvious and would not have involved a patentable invention.
6. Claims 1, 53-58, 60-63, 66-67 and 69-71 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1, 3-4, 6, 9, 13-16 of US Patent No. 12,084,062. Although the conflicting claims are not identical, they are not patentably distinct from each other because:
The patent claims include all of the limitations of the instant application claims, respectively. The patent claims also include additional limitations. Hence, the instant application claims are generic to the species of invention covered by the respective patent claims. As such, the instant application claims are anticipated by the patent claims and are therefore not patentably distinct therefrom. (See Eli Lilly and Co. v. Barr Laboratories Inc., 58 USPQ2D 1869, "a later genus claim limitation is anticipated by, and therefore not patentably distinct from, an earlier species claim", In re Goodman, 29 USPQ2d 2010, "Thus, the generic invention is 'anticipated' by the species of the patented invention" and the instant “application claims are generic to species of invention covered by the patent claim, and since without terminal disclaimer, extant species claims preclude issuance of generic application claims”).
7. Claims 59, 64-65 and 68 and are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1 and 15 of US Patent No. 12,084,062. Although the conflicting claims are not identical, they are not patentably distinct from each other because “the video data is received by a recording service module and associated with one or more identification data; and wherein the one or more identification data includes one or more video labels, timestamps, camera identifiers, user identifiers, or any combination thereof”, “the one or more machine learning networks include one or more convolutional neural networks (CNN), one or more Dlib machine learning algorithms, or any combination thereof”, “the one or more notifications includes one or more calls, text notifications, or both to one or more pre-determined individuals” and “ the one or more safety protocols of the vehicle include automatically engaging driving assistance technology of a vehicle and automatically slowing the vehicle down, braking, stopping, initiating lane control, controlling acceleration, moving the vehicle to a side of a road, moving the vehicle to a nearby parking spot, moving the vehicle to the nearest emergency services location, or a combination thereof” are conventional prior art features (as shown in the art rejection below) and the use of such features in claims 1 and 19 of US Patent No. 12,084,062 would have been obvious and would not have involved a patentable invention.
Claim Rejections - 35 USC § 102
8. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
10. Claims 1, 53, 55, 58-61 and 65 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jeon (US 2008/0291008).
For claim 1, Jeon discloses a method for recognizing and monitoring one or more users for an occurrence of a health event of the one or more users comprising:
a) receiving incoming video data by one or more processors related to the one or more users [E.g. 0010: to achieve the above objects, a drowsy driving prevention apparatus employing a facial recognition technology according to the present invention includes a camera for capturing a driver's face image upon vehicle driving…, 0019: driver's face vector template is generated using the driver's face images captured by a camera of the drowsy driving prevention apparatus while driving and then compared with the driver's vector template, which has been registered at normal times in order to determine whether the driver drives a vehicle while dozing off. If, as a result of the comparison, it is determined that the driver dozes off, various warning methods for preventing drowsy driving are executed. The driver's drowsy looks are sent to a drowsy driving prevention server, which then executes anti-drowsy driving contents, and are also sent to an external helper];
b) preprocessing the video data by the one or more processors [E.g. 0010: a face image acquisition unit for converting an analog image of the driver's face image captured by the camera into a digital image stream and storing the converted digital image stream in a memory…];
c) extracting facial data by the one or more processors from the video data to identify the one or more users [E.g. 0022: The user registration unit 104 stores a vector template of a driver's face (who is the user of the drowsy driving prevention apparatus 100) in a driver DB 120. The driver's face vector template refers to a value in which all characteristic elements constituting the driver's face are extracted and quantified and is used to calculate and analyze the eye's flickering, the face angle, shaking, etc. for recognizing drowsy driving];
d) determining a presence, and/or a probability of a health event in the one or more users by the one or more processors by comparing the facial data with one or more stored facial data models accessible by the one or more processors [E.g. 0026: The drowsy driving analysis unit 112 continuously compares/analyzes the driver's first face vector template generated from the face image reader 110 and the driver's vector template stored in the driver DB 120 in order to confirm the driver's drowsy state. For example, the drowsy driving analysis unit 112 can track and monitor the driver's drowsy driving by continuously comparing the vector template of the captured face image, such as the eye's flickering, a face angle, and shaking, and its previous state and the stored vector template using a facial recognition technology and a statistic process]; and
e) generating one or more notifications by the one or more processors based on recognizing the presence and/or the probability of the health event [E.g. 0047: If it is determined that the driver's face image belongs to drowsy driving in the drowsy driving analysis unit 112, the image of the drowsy driver is transmitted to the drowsy driving prevention server 200 through a wireless Internet communication gateway of the communication unit 116 having communication functions such as Wibro, HSDPA & HSUPA, LTE, UMB, TD-SCDMA, TRS, GPRS (GSM) and CDMA (step S409). At this time, an encrypted face recognition vector template that is previously generated is also sent along with the driver's drowsy driving image information in order to authenticate a user].
For claim 53, Jeon discloses wherein the stored facial data models include a plurality of facial data pre-stored within one or more storage mediums and already associated with one or more irregular health conditions [E.g. 0010: a user registration unit for storing and managing a face vector template of a driver who is a user of the drowsy driving prevention apparatus; a face image acquisition unit for converting an analog image of the driver's face image captured by the camera into a digital image stream and storing the converted digital image stream in a memory] and at least one of the one or more irregular health conditions is associated with the health event [E.g. 0010-0011, 0019].
For claim 55, Jeon discloses wherein the one or more processors receive the incoming video data from one or more cameras [E.g. 0042: The vehicle's camera 102 captures an image of a driver's face and sends the captured image information to the drowsy driving prevention apparatus 100 (step S402). The face image acquisition unit 106 of the drowsy driving prevention apparatus digitalizes the image information, converts the digital image information into a digital image stream, and stores the converted digital image stream in the memory 108 (step S403). At this time, the memory 108 may include cyclic memory in which information is refreshed periodically.].
For claim 58, Jeon discloses wherein the incoming video data includes one or more video files, image files, frames, or any combination thereof [E.g. 0042: The face image acquisition unit 106 of the drowsy driving prevention apparatus digitalizes the image information, converts the digital image information into a digital image stream, and stores the converted digital image stream in the memory 108 (step S403). At this time, the memory 108 may include cyclic memory in which information is refreshed periodically].
For claim 59, Jeon discloses wherein the video data is received by a recording service module and associated with one or more identification data; and wherein the one or more identification data includes one or more video labels, timestamps, camera identifiers, user identifiers, or any combination thereof [E.g. 0042: The vehicle's camera 102 captures an image of a driver's face and sends the captured image information to the drowsy driving prevention apparatus 100 (step S402). The face image acquisition unit 106 of the drowsy driving prevention apparatus digitalizes the image information, converts the digital image information into a digital image stream, and stores the converted digital image stream in the memory 108 (step S403). At this time, the memory 108 may include cyclic memory in which information is refreshed periodically.].
For claim 60, Jeon discloses wherein extracting the facial data includes utilizing one or more face detection models, pose detection models, facial analysis models, or a combination thereof [E.g. 0026: The drowsy driving analysis unit 112 continuously compares/analyzes the driver's first face vector template generated from the face image reader 110 and the driver's vector template stored in the driver DB 120 in order to confirm the driver's drowsy state].
For claim 61, Jeon discloses wherein the one or more face detection models convert facial image data into numeric array representations of the one or more faces of the one or more users [E.g. 0010: a face image acquisition unit for converting an analog image of the driver's face image captured by the camera into a digital image stream and storing the converted digital image stream in a memory].
For claim 65, Jeon discloses wherein the one or more notifications includes one or more calls, text notifications, or both to one or more pre-determined individuals [E.g. 0056: Further, images within a vehicle and a driver's drowsy driving state are automatically informed to external helpers such as families and close acquaintances through a text message or MMS (multi-media message), voice messages, mobile phone and various wireless Internet terminals. Thus, drowsy driving can be eliminated remotely through a call with an external helper].
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. Claim 54 is rejected under 35 U.S.C. 103 as being unpatentable over Jeon in view of Wu et al. (Wu; US 2020/0320319).
For claim 54, Jeon fails to expressly disclose wherein the one or more stored facial data models are determined using supervised machine learning in which the facial data is stored and associated with known irregular health conditions.
However, as shown by Wu, it was well known in the art of driver monitoring that one or more stored facial data models are determined using supervised machine learning in which facial data is stored and associated with known irregular health conditions [E.g. 0035-0037, 0023-0025].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Wu in order to implement a system with enhanced facial expression and pose analysis functionality.
14. Claims 56-57 are rejected under 35 U.S.C. 103 as being unpatentable over Jeon in view of Ricci (US 2014/0309880).
For clam 56, Jeon fails to expressly disclose wherein the method includes the one or more processors also receiving incoming data from one or more other sensing devices.
However, as shown by Ricci, it was well known in the art of driver monitoring that one or more stored facial data models are determined using supervised machine learning in which facial data is stored and associated with known irregular health conditions [E.g. 0425, 0445-0446].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Ricci because providing more than one sensor to monitor the driver health improve accuracy, increase reliability, and provide a more comprehensive picture of the driver's condition, also it is merely combining prior art elements according to known methods to yield predicable results.
For claim 57, Ricci further teaches wherein the one or more other sensing devices include one or more motion sensors, heart rate monitors, breathing monitors, or a combination thereof [E.g. 0425, 0445-0446].
15. Claims 62-63 and 66-71 are rejected under 35 U.S.C. 103 as being unpatentable over Jeon in view of Rau et al. (Rau; US 2015/0092056).
For claim 62, Jeon fails to expressly disclose wherein the one or more pose detection models convert facial image data into one or more measures, predictions, or both related to one or more facial poses, body poses, or both of the one or more users.
However, as shown by Rau, it was well known in the art of driver monitoring to include one or more pose detection models convert facial image data into one or more measures, predictions, or both related to one or more facial poses, body poses, or both of the one or more users [E.g. 0029].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Rau in order to accurately determine the driver condition to safely control the driving of the vehicle for an incapacitated driver, also it is merely combining prior art elements according to known methods to yield predictable results.
For claim 63, Jeon fails to expressly disclose wherein the one or more stored facial data models are determined by one or more machine learning networks.
However, as shown by Rau, it was well known in the art of driver monitoring to include one or more stored facial data models determined by one or more machine learning networks [E.g. 0011].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Rau because using machine learning allow continuous improvement in the driver monitoring system.
For claim 66, Jeon fails to expressly disclose wherein upon detecting the presence and/or the probability of the health event, the method includes identifying and activating one or more response protocols.
However, as shown by Rau, it was well known in the art of driver monitoring to include wherein upon detecting presence and/or probability of health event, identifying and activating one or more response protocols [E.g. 0043, 0062].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Rau in order to enable activation of a response protocol based on the driver health that increase the safety of the driver and others on the road, also it is merely combining prior art elements according to known methods to yield predicable results.
For claim 67, Rau further teaches wherein the one or more response protocols include notifying one or more emergency services, providing a geolocation of the one or more users, engaging one or more safety protocols of a vehicle [E.g. 0043, 0062], or a combination thereof.
For claim 68, Rau further teaches wherein the one or more safety protocols of the vehicle include automatically engaging driving assistance technology of a vehicle and automatically slowing the vehicle down [E.g. 0043, 0062], braking, stopping [E.g. 0043, 0062], initiating lane control, controlling acceleration, moving the vehicle to a side of a road, moving the vehicle to a nearby parking spot, moving the vehicle to the nearest emergency services location, or a combination thereof.
For claim 69, Jeon fails to expressly disclose a system for facial recognition and monitoring for performing the method of claim 1, comprising a recognition device including: i) one or more cameras; and ii) one or more image processing units in communication with the one or more cameras including: a) one or more processors, b) graphics processors, c) storage mediums, and d) internet connections.
However, as shown by Rau, it was well known in the art of driver monitoring to include disclose a system for facial recognition and monitoring comprising a recognition device including: i) one or more cameras [E.g. 0029]; and ii) one or more image processing units in communication with the one or more cameras [E.g. 0029] including: a) one or more processors [E.g. 0034], b) graphics processors [E.g. claim 5], c) storage mediums [E.g. 0065], and d) internet connections [E.g. 0026].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon with the teaching of Rau in order to enable activation of a response protocol based on the driver health that increase the safety of the driver and others on the road, also it is merely combining prior art elements according to known methods to yield predicable results.
For claim 70, Rau further teaches wherein the recognition device is integrated into a vehicle [E.g. 0017].
For claim 71, Rau further teaches wherein the one or more cameras are configured to have a line of sight on one or more drivers, passengers, or both within the vehicle [E.g. 0017].
16. Claim 64 is rejected under 35 U.S.C. 103 as being unpatentable over Jeon in view of Rau and further in view of Levkova et al. (Levkova; US 2018/0126901).
For claim 64, Jeon in view of Rau fails to expressly disclose wherein the one or more machine learning networks include one or more convolutional neural networks (CNN), one or more Dlib machine learning algorithms, or any combination thereof.
However, as shown by Levkova, it was well known in the art of driver monitoring that one or more machine learning networks include one or more convolutional neural networks (CNN), one or more Dlib machine learning algorithms, or any combination thereof [E.g. 0038].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention to modify Jeon in view of Rau with the teaching of Levkova in order to implement a system with enhanced facial expression and pose analysis functionality.
Conclusion
17. The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure: see PTO-892 Notice of Reference Cited.
18. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED BARAKAT whose telephone number is (571)270-3696. The examiner can normally be reached on 9:00am-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Davetta Goins can be reached on (571) 272-2957. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMED BARAKAT/
Primary Examiner, Art Unit 2689
Prosecution Insights