DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 11-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. “MDAN: Multi-level Dependent Attention Networks for Visual Emotion Analysis” supplied by applicant in view of Soares (US Patent 11727724).
Regarding claims 1 and 11, Xu et al. teaches a system and method comprising:
receiving a plurality of images of a video, the plurality of images including a face (Section 5.1);
generating, a plurality of feature maps at a plurality of semantic levels based on the
plurality of images (page 2 col.2 first partial paragraph), the plurality of semantic levels including a lowest semantic level and a highest semantic level (page 3 section 2.3); generating, by a first classifier based on a first feature map of the plurality of feature maps associated with the highest semantic level (Section 3.1 on page 4), a first emotion prediction associated with the face over a first set of emotions of a first affective level (); generating, by a second classifier based on the plurality of feature maps, a second emotion prediction associated with the face over the first set of emotions (figure 6 shows second classification based on feature maps).
As shown above Xu teaches all of the limitations of claim 1 except explicitly generating a fine-grain emotion prediction based on the first emotion prediction and the second emotion prediction. However, Soares in the same field of endeavor of emotion detection uses multiple predictions to generate a fine grain emotion prediction (see col.9, lines 25-35 which states better prediction (fine grain) is achieved by using multiple CNNs not just one in isolation). Since both systems are trained to determine emotion from images it would have been obvious to one of ordinary skill at the time of filing to include multiple CNNs into Xu’s system to get better results as noted by Soares. By adding more trained systems accuracy will improve.
Regarding claims 2, 4, 12 and 14, Soares teaches generating, via a plurality of transformer encoders based on an input based on a respective feature map of the plurality of feature maps (meshes), a plurality of vector representations latent vectors), wherein the generating the first emotion prediction is based on a first vector representation of the plurality of vector representations based on the first feature map. See col.9, lines 25-65 of Soares. It should also be noted that Xu teaches that there are multiple features maps at different levels where each level could be considered a different trainable network just on a top to bottom level.
Regarding claims 3 and 13, Xu shows input based on the respective feature map includes a respective feature vector based on a 1x1 convolution and a flattening operation performed on the respective feature map of the plurality of feature maps. See figure 4 which has a 1 x1 convolution.
Allowable Subject Matter
Claims 5-10 and 15-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 20 is allowed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Hu et al. US 11106896 is cited as teaching emotion prediction using CNNs with semantic engines.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER S KELLEY whose telephone number is (571)272-7331. The examiner can normally be reached Mon-Fri 6:30 to 4 pm alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Debbie Reynolds can be reached at 571-272-0734. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER S KELLEY/Supervisory Patent Examiner, Art Unit 2482