DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Regarding claims 1-19,
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. Claims 1-19 are directed towards a process, machine, manufacture or composition of matter which is/are statutory subject matter. (Step 1: YES)
Step 2A, Prong 1: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim. The broadest reasonable interpretation of “identifying a face from a facial image acquired from a camera; inferring an emotion corresponding to the face to obtain an emotion-based threat score based on the inferred emotion; obtaining an eye-based threat score based on a blink frequency of eyes on the face and a number of pupil movements in the face; and detecting whether a person corresponding to the face is in a threat situation based on a total threat score calculated by combining the emotion-based threat score and the eye-based threat score” (claim 1) fall within the mental process groupings of abstract ideas because they cover concepts performed in the human mind, including observation, evaluation, judgment, and opinion. See MPEP 2106.04(a)(2), subsection III. Similarly, the dependent claim limitations fall within the mental process groupings of abstract ideas because they cover concepts performed in the human mind, including observation, evaluation, judgment, and opinion. The apparatus claims 10-18 and the non-transitory computer-readable storage medium claim 19 necessitate additional limitations in addition to all limitations addressed above, such as a memory and a processor, but these elements do not change the scope of the claims to make them part of a non-mental process. See MPEP 2106.04(a)(2), subsection III. (Step 2A, Prong 1: YES)
Step 2A, Prong 2: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d). Claim 1 recites the additional element of (a) “camera”. The limitation (a) is a mere data gathering device and thus is insignificant extra-solution activity. See MPEP 2106.05(g) (“whether the limitation is significant”). In addition, all uses of the recited judicial exceptions require such data gathering and output, and, as such, these limitations do not impose any meaningful limits on the claim. These limitations amount to necessary data gathering. See MPEP 2106.05. The dependent claims do not recite additional elements, the apparatus claims 10-18 merely recite an additional memory and processor in addition to the limitations described in claims 2-9, and the non-transitory computer-readable storage medium merely recites an additional processor. In totality, claims 1-19 do not recite additional elements that integrate the judicial exception into a practical application. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application (Step 2A, Prong Two: NO), and the claims are directed to the judicial exception. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05. As explained with respect to Step 2A, Prong Two, there is one additional element in claim 1, no additional elements in claims 2-9, and two additional elements in claims 10-19. The additional elements of claim 1 of (a) “camera” are at best mere instructions to “apply” the abstract ideas, which cannot provide an inventive concept. See MPEP 2106.05(f). Element (a) and the elements of a memory and processor were all found to be insignificant extra-solution activity in Step 2A, Prong Two, because they were determined to be insignificant limitations as necessary for data gathering and outputting. However, a conclusion that an additional element is insignificant extra-solution activity in Step 2A, Prong Two should be re-evaluated in Step 2B. See MPEP 2106.05, subsection I.A. At Step 2B, the evaluation of the insignificant extra-solution activity consideration takes into account whether or not the extra-solution activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). As discussed in Step 2A, Prong Two above, the recitation of (a) and the elements of a memory and processor are recited at a high level of generality. These elements amount to collecting or transmitting data over a network and are well understood, routine, conventional activity. See MPEP 2106.05(d), subsection II. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception and insignificant extra-solution activity, which do not provide an inventive concept. (Step 2B: NO).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 7, 9, 10-11, 16, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Merchant (US-11335126-B1) and further in view of Liu et al. (CN-105868694-A).
Regarding claim 1, Merchant teaches:
A method for threat detection (“the output 118 of the security system 102 may be analyzed using the AI 124 to determine whether one or more threats are present.” Col. 7, Lines 24-26) based on facial image analysis performed by a facial image analysis-based threat detection apparatus (“The AI 124 may analyze an emotion conveyed by a facial expression of the person” Col. 9, Lines 15-16), the method comprising:
identifying a face from a facial image acquired from a camera (“the AI 124 may analyze a facial expression of the person 114 (e.g., first person) and determine one or more emotions (e.g., fear, surprise, or the like) expressed by the facial expression of the person,” Col. 15, Lines 8-11);
detecting whether a person corresponding to the face is in a threat situation (“The AI 124 may analyze an emotion conveyed by a facial expression of the person 114, the person 116, or both.” Col. 9, Lines 15-17, and “The processed images 122 may be processed in such a way as to enable threats to be more easily detectable by an artificial intelligence (AI) module 124” Col. 7 Lines 16-19).
Merchant is not relied upon to teach the following limitations. Liu, however, further teaches:
inferring an emotion corresponding to the face to obtain an emotion-based threat score based on the inferred emotion (“Based on the five reference facial expression features in step S3.1, perform a qualitative analysis on the facial expression features in step S3.2, and determine the preliminary emotional state by voting,” Para [0034]);
obtaining an eye-based threat score based on a blink frequency of eyes (“blink frequency”) on the face and a number of pupil movements (“eye movement trajectory”) in the face (“S2b.1 Classify the collected eye movement information according to eye movement trajectory, eye movement time, eye movement direction, eye movement distance, fixation time, fixation frequency, pupil diameter and blink frequency to obtain eight types of eye movement feature vectors;” Para [0027]); and
a total threat score calculated by combining the emotion-based threat score and the eye-based threat score (“The fusion and classification module is used to extract the matched facial expression feature vectors and eye movement feature vectors, and then use SVR to fuse and classify them to obtain the classified emotion feature vectors,” Para [0046]).
Liu is considered to be analogous to the claimed invention because they are both in the field of facial and eye analysis for emotion detection. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Liu into Merchant for the benefit of more accurate emotion analysis and recognition.
Regarding claim 2, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, and Merchant further teaches:
wherein the camera is a near-infrared (NIR) camera (“The cameras 110 in addition to capturing visible light, the cameras 110 may be capable of capturing other types of light (e.g., electromagnetic radiation) that are not visible to the human eye, such as infrared, ultraviolet, thermal imaging, and the like.” Col. 6, Lines 40-44),
and the facial image is acquired from the near-infrared camera (If the output of the security system is infrared data/an infrared image, then “The AI 124 may analyze an emotion conveyed by a facial expression of the person 114, the person 116, or both.” Col. 9, Lines 15-17).
Regarding claim 7, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, and Liu further teaches:
wherein the total threat score (“Z”) is calculated by reflecting a result of applying a first weight (“w<sub>1</sub>”) to the emotion-based threat score (represented by “z<sub>1</sub>”, as shown in Para [0035]) and a result of applying a second weight (“w<sub>2</sub>”) to the eye-based threat score (represented by z<sub>2</sub>”, as shown in Paras [0016-0017]; additional context: “fuse Z<sub>2</sub> with the preliminary emotional state Z<sub>1</sub> according to the fusion ratio to obtain the subject's final emotional state Z,” Para [0072], where the fusion ratio “is w<sub>1</sub>:w<sub>2</sub> (w<sub>1</sub>, w<sub>2</sub> are variables, and the specific relationship is: 0≤w<sub>1</sub>, w<sub>2</sub>≤1, Z＝w<sub>1</sub>Z<sub>1</sub>+w<sub>2</sub>Z<sub>2</sub> and w<sub>1</sub>+w<sub>2</sub>＝1),” Para [0088]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Liu into Merchant for the benefit of more accurate emotion analysis and recognition.
Regarding claim 9, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, and Merchant further teaches:
wherein in the detecting whether a person corresponding to the face is in a threat situation, it is determined that the person is in the threat situation when the total threat score exceeds a threshold (“the output 118 of the security system 102 may be analyzed using the AI 124 to determine whether one or more threats are present. If the AI 124 detects one or more threats and the threats satisfy the threat thresholds 128, then the server 106 may perform the one or more notification actions 132” Col. 9, Lines 24-29).
Regarding claims 10, 11, 16, 18, and 19, the claims contain corresponding subject matter to claims 1, 2, 7, 9, and 10. Therefore, the rejection of these claims applies to claims 10, 11, 16, 18, and 19.
Claim(s) 3-4 and 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Merchant in view of Liu as applied to claims 1 and 10 above, and further in view of Aminian et al. (US-20230140369-A1).
Regarding claim 3, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, and Liu further teaches:
wherein when the inferred emotion belongs to one of preset threat emotion classes (“obtains the preliminary emotional state Z<sub>1</sub> by comparing the subject's facial expression features with multiple sets of standard reference facial expression features collected in advance,” Para [0035]).
Merchant and Liu are not relied upon to teach the following limitations. Aminian, however, further teaches:
the emotion-based threat score is determined dependent on a confidence level for the inferred emotion (“For each instance of a detected facial emotion or sentiment (e.g., facial emotion tag) in a given frame, heuristics tool 175 multiplies the designated weight for that instance's facial emotion or sentiment class by the predicted confidence level for that instance,” Para [0047]).
Aminian is considered to be analogous to the claimed invention because they are both in the field of emotion detection from facial features. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Aminian into Merchant and Liu for the benefit of more accurate emotion detection.
Regarding claim 4, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, but are not relied upon to teach the following limitation. Aminian, however, further teaches:
wherein when the inferred emotion does not belong to any of preset threat emotion classes, the emotion-based threat score is determined to be 0 (“In some embodiments where a user selects one or more facial emotion or sentiment classes of interest, any frame with an associated facial emotion tag that matches a selected class is set to 1, and other frames are set to 0,” Para [0047]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Aminian into Merchant and Liu for the benefit of more accurate emotion detection.
Regarding claims 12 and 13, the claims contain corresponding subject matter to claims 3 and 4. Therefore, the rejection of these claims applies to claims 12 and 13.
Claim(s) 5, 8, 14, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Merchant in view of Liu as applied to claims 1 and 10 above, and further in view of Hu et al (CN-117171708-A).
Regarding claim 5, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, and Liu further teaches “eye blink frequency” and “number of pupil movements” as explained in the rejection of claim 1. Liu is not relied upon to teach the following limitation. Hu, however, further teaches:
wherein the eye-based threat score is calculated by applying a first confidence level to the eye blink frequency and a second confidence level to the number of pupil movements (“Using a target-based personalized fusion model, biometric data is weighted and fused based on confidence scores to obtain fused feature data.,” Para [0033]; also see in Para [n0020] that the biometric data can be “gaze coordinates and trajectory, blinking”).
Hu is considered to be analogous to the claimed invention because they are both in the field of using multiple types of biosignals from a person to determine emotions or cognitive load assessment (para [n0023] of Hu). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Hu into Merchant and Liu for the benefit of more accurate emotion detection.
Regarding claim 8, the rejection of claim 7 is incorporated herein. Merchant in view of Liu teaches the method of claim 7, and Liu further teaches “the inferred emotion, the eye blink frequency, and the number of pupil movements” as explained in the rejection of claim 1. Merchant and Liu are not relied upon to teach the following limitations. Hu, however, further teaches:
wherein the first weight and the second weight are determined based on confidence levels (“Using a target-based personalized fusion model, biometric data is weighted and fused based on confidence scores to obtain fused feature data,” Para [0033]; also see in Para [n0020] that the biometric data can be “gaze coordinates and trajectory, blinking”, and see in Para [n0021] that the biometric data can be “facial expressions”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Hu into Merchant and Liu for the benefit of more accurate emotion detection.
Regarding claims 14 and 17, the claims contain corresponding subject matter to claims 5 and 8. Therefore, the rejection of claims 5 and 8 applies to claims 14 and 17.
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Merchant in view of Liu as applied to claims 1 and 10 above, and further in view of Mezher, "Gaze Tracking for Emotional Detection", University of Debrecen, Short Thesis for the Degree of the Doctor of Philosophy (PhD), December 2020, hereinafter referred to as Mezher.
Regarding claim 6, the rejection of claim 1 is incorporated herein. Merchant in view of Liu teaches the method of claim 1, but are not relied upon to teach the following limitations. Mezher, however, further teaches:
further comprising acquiring movement of the face (“even though the person tries not to move his/her head, a motion of some pixels may occur. Therefore it's far necessary to discover the quantity of head movement, that's calculated using the approach described in section 4.4,” Section 10, Test Result of the Gaze Estimator), wherein the eye-based threat score is calculated by identifying the movement of pupil in the face while taking the movement of the face into account (“The quantity of head motion is subtracted from the tracked eye positions to get the real eye movements at some point of the calibration process,” Section 10, Test Result of the Gaze Estimator, also see Fig. 2 for the overall system that tracks gaze for emotion detection).
PNG
media_image1.png
598
295
media_image1.png
Greyscale
Mezher is considered to be analogous to the claimed invention because they are both in the field of eye gaze detection and tracking for emotion analysis. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mezher into Merchant and Liu for the benefit of more accurate emotion detection.
Regarding claims 15, the claim contains corresponding subject matter to claim 6. Therefore, the rejection of this claim applies to claim 15.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Seo et al. (US-20200104670-A1) teaches a method for emotion prediction of a person using their biometric data.
Rubner et al. (US-20190343382-A1) teaches a method for monitoring human cognition via eye biometric data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL A OMETZ whose telephone number is (571)272-2535. The examiner can normally be reached 6:45am-4:00pm ET Monday-Thursday, 6:45am-1:00pm ET every other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Rachel Anne Ometz/ Examiner, Art Unit 2668 6/29/26
Rachel.ometz@uspto.gov
/VU LE/ Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights