DETAILED ACTION
Information Disclosure Statement
The information disclosure statement filed 12/8/2023 fails to comply with 37 CFR 1.98(a)(1), which requires the following: (1) a list of all patents, publications, applications, or other information submitted for consideration by the Office; (2) U.S. patents and U.S. patent application publications listed in a section separately from citations of other documents; (3) the application number of the application in which the information disclosure statement is being submitted on each page of the list; (4) a column that provides a blank space next to each document to be considered, for the examiner’s initials; and (5) a heading that clearly indicates that the list is an information disclosure statement. The information disclosure statement has been placed in the application file, but the information referred to therein has not been considered.
Response to Arguments
Applicant's arguments filed with respect to claims 1-4, 6, and 8 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine,
manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4, 6 and 8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-4, 6 and 8 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101.
The Examiner will analyze Claim 1, and similar rationale applies to independent Claim 6 and 8.
The rationale, under MPEP § 2106, for this finding is explained below. The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a machine since the claim is directed to an emotion inferring apparatus.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of a controller part detecting fine expressions appearing on the subject's face for each frame image forming a moving image of the subject' face captured and obtained by the image capture part [mental process]; and
inferring an emotion of the subject depending on a ratio of a number of the frame images in which each of kinds of the fine expressions is detected within a plurality of the frame images where the fine expressions are detected [mental process and mathematical] are directed to an abstract.
If/when the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The additional claim limitations an image capture part capturing a subject's face [data gathering] is nothing more than insignificant extra solution activity.
A controller/apparatus [high level of generality] is used to generally apply the abstract idea without limiting how it functions.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The remaining claim elements, considering individually or in combination, do not add significantly more than the abstract idea.
The Examiner finds that Claims 2-4 does not state significantly more since the claim only recites additional steps for analyzing video using machine learning model.
Thus, claims 1-4, 6 and 8 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, all claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date, of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over el Kabiouby et al. (Pub. NO. US 2017/0238860) in view of el Kaliouby et al (Pub. No. US 2011/0263946 hereinafter “el2”).
Regarding claims 1, 6, and 8 Le teaches an emotion inferring apparatus comprising: an image capture part capturing a subject's face [Para. 69]; a controller part detecting fine expressions (micro expressions) appearing on the subject's face for each frame image (frame) forming a moving image (video) of the subject’s face captured and obtained by the image capture part (camera) [Para. 70 “a high frame rate camera is used. A high frame rate camera has a frame rate of sixty frames per second or higher. With such a frame rate, micro expressions can also be captured.”. Para. 74 “For example, the classifiers can be used to determine a probability that a given AU or expression is present in a given frame of a video”]; and inferring an emotion of the subject depending on a ratio and detecting fine expressions (micro expression) in frame images (frame) of a moving image (video) [Para. 70 “With such a frame rate, micro expressions can also be captured.”; Para. 74 “For example, the classifiers can be used to determine a probability that a given AU or expression is present in a given frame of a video”; Para. 81 “The result of the analyzing can be used to infer one or more emotional metrics”; and Para. 108 “the statistics can include a percentage of time smiling, the time at which the smile occurred, the website or media or game for which the smile occurred, an image of the individual for whom the statistical results are being displayed, etc”
However, el doesn’t explicitly teach inferring an emotion of the subject depending on a ratio of a number of the frame images in which each of kinds of the fine expressions is detected within a plurality of the frame images where the fine expressions are detected.
el2 teaches inferring an emotion of the subject depending on a ratio of a number (number of instances detected) of the frame images (frame) in which each of kinds (each action unit) of the fine expressions is detected within a plurality of the frame images where the fine expressions are detected [Para. 98 “The Action Unit log 954 includes a line for each action unit for each frame; each line contains the Action Unit name and the number of instances detected of this Action Unit and the length of each instance (start frame and End Frame), so it is essentially a memory dump of the action unit buffer; alternatively, the Action Unit log file 956 may be structured to only show the action units detected per frame”. Para. 88 “In the first case, the events are aligned across all participants (e.g., all participants watching same advertisement or trailer, so facial expressions are lined up in time across all participants), the aggregate function may be a simple sum or average function that counts number of occurrences of certain states of interest at specific event markers or time stamps”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify el’s video-based facial analysis system by incorporating el2’s per frame action unit log and occurrence-count aggregation to count el’s fine-expression (micro expression) detections by kind within detected expression frames and those counts with el’s ratio (percentage) metric. This modification improves el by converting transient frame-level expression detections into normalized per-kind measurements, thereby improving emotion inference from mixed or co-occurring facial signals.
Regarding claim 2, el teaches wherein the controller part categorizes detected fine expressions (micro expressions) into individual types [Para. 81, 71, and 74]; infers an emotion of the subject depending on a ratio (percentage)of a number of the frame images in which each of the fine expressions categorized into the individual types (categories) [Para. 81, 74, and 108].
However, el doesn’t explicitly teach the rest of claim limitations.
el2 teaches a ratio of a number (number of instances detected) of the frame images in which each of the fine expressions categorized into the individual types is detected within the plurality of the frame images where the fine expressions are detected [Para. 98, “The ActionUnit log 954 includes a line for each action unit for each frame; each line contains the Action Unit name and the number of instances detected of this Action Unit and the length of each instance (start frame and End Frame), so it is essentially a memory dump of the action unit buffer; alternatively, the ActionUnit log file 956 may be structured to only show the action units detected per frame”; “The Mental State log 960 is similar to the Gesture log, but the columns represent the mental states and the rows represent the frame numbers at which the function detect Mental States( ) was invoked. Each cell contains the raw probability output by the classifier”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify el’s video-based facial analysis system by incorporating el2’s per frame action unit log and occurrence-count aggregation to count el’s fine-expression (micro expression) detections by kind within detected expression frames and those counts with el’s ratio (percentage) metric. This modification improves el by converting transient frame-level expression detections into normalized per-kind measurements, thereby improving emotion inference from mixed or co-occurring facial signals.
Claims 6 and 8 are rejected for the same reasons as claim 1. Furthermore, el teaches having a non-transitory computer readable medium and a method to perform the claim limitations stated in claim 1 above [see fig. 1-3, and related description].
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over el Kabiouby et al. (Pub. NO. US 2017/0238860; hereinafter “el”) in view of el Kaliouby et al (Pub. No. US 2011/0263946 hereinafter “el2” ) and further in view of Lee (Pub. No. US 2013/0144937).
Regarding claim 2, el in view of el2 doesn’t explicitly teach wherein the controller part categorizes detected fine expressions into individual types and infers an emotion of the subject depending on ratios of each of fine expressions categorized into the individual types.
However, Lee teaches the controller part categorizes detected fine expressions into individual types and infers an emotion of the subject depending on ratios of each of fine expressions categorized into the individual types (positive or negative) [fig. 3, 4 and related description. Para. 56, 47, claim 3 states “calculate the emotion rate based on a ratio of the positive emotional state to the negative emotional state”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify el in view of el2 to categorize detected expressions into individual types, feature as taught by Lee; because the modification enables the system to quantify and share a user’s emotion reliably by converting recognized emotions into a ratio-based emotion rate for communication between devices
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over el Kabiouby et al. (Pub. NO. US 2017/0238860; hereinafter “el”) el Kaliouby et al (Pub. No. US 2011/0263946 hereinafter “el2” ) and further in view of Zhang (Pub. No. US 2020/0110863) further in view of El Kaliouby et al. (Pub. No. US 2014/0323817 “hereinafter Kalio”).
Regarding claim 3, el teaches the FACS encodes movements of individual muscles of the face, where the muscle movements often include slight, instantaneous changes in facial appearance.
However, el in view of el2 doesn’t explicitly teach the rest of claim limitation.
Zhang teaches wherein the image capture part captures the subject's face with changing the expressions according to guidance (prompt) instructing (requesting) expressions to the subject wherein the expression is not detected [Para. 2, 13, 15, and 31].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify el in view of el2 to giving guidance instruction in order to capture facial expression, feature as taught by Zhang; because the modification enables the system prevents spoofing in face-based authentication by using prompted, live facial expression response to verify the real authorized user.
el in view of el2 further in view of Zhang doesn’t explicitly teach the controller part (14) obtains a changing-rate of expressions of the subject from images of the subject's face captured and obtained by the image capture part and infers an emotion of the subject depending on the changing-rate.
Kalio teaches the controller part obtains a changing-rate (rate of change) of expressions of the subject from images of the subject's face captured and obtained by the image capture part and infers an emotion of the subject depending on the changing-rate [Para. 26 “The categorizing can further be based on a rate of change in the facial expressions by the individual. In embodiments, the rate of change is evaluated during exposure to specific media.”, claims 8, 9 and corresponding description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify el in view of el2 further in view of Zhang to teach the claim limitation feature as taught by Kalio; because the modification enables the system improve emotion determination from captured face image, particularly where static expression detection alone is insufficient.
Regarding claim 4, el teaches wherein a changing-rate pattern table (plurality of mental state event temporal signature) registering patterns of changing rate (rise time, fall time, and duration) of the expression in association with each of the emotion (sadness, stress, happiness etc..) is further comprised, [Para. 65 and 64] and wherein the controller part infers the emotion corresponding to patterns of changing-rates of the expressions obtained from images of the subject's face in the changing-rate pattern table as an emotion of the subject [Para. 92 “The analysis of the facial regions can also include determining probabilities of occurrence of one or more facial expressions”; Para. 94 “One or more classifiers can be used to analyze the facial regions that can include the eyebrows to determine a probability for either a presence or an absence of an eyebrow furrow. The probability can include a posterior probability, a conditional probability, and so on. The probabilities can be based on Bayesian Statistics or another statistical analysis technique. The presence of an eyebrow furrow can indicate [infer] the person from whom the facial data was collected is annoyed, confused, unhappy, and so on [kinds/types]”. Since the term “rate” is not quantified or compared to any number/threshold, el’s teaching reads on the claim limitation].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
Prosecution Insights