DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Examiner is opening for a discussion regarding a potential allowable subject matter per the current Specs, [0103]…”the network learns the correspondence between the audio (i.e. phonemes) and the corresponding face movements, especially the lip movements. Such a trained network would enable to continuously render the facial expressions and specifically the lip movements even when the video quality is low or when part of the face is obstructed to the original video camera”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-16 and 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Chang in view of Vemulapalli et al (US 2020/0151438) and further in view of Gopal OR Kurtz.
Claims 1, 9 and 18, Chang teaches a method, a non-transitory medium and a system for updating a model of a participant of a three dimensional (3D) video conference, the method comprises:
obtaining images of the participant during the 3D video conference; (Chang: Fig. 2,… The facial expression data can be in the form of still image or video data, and can be 2D or 3D images, [0029]);
determine an expression representation for an expression of the participant in the image; (Chang: A baseline facial expression can be established for a user via a set of facial image training data. A variety of other facial expressions (e.g., expressions associated with happiness, interest, confusion, etc.) can also be established based on this training data, [0012]);
determining, by a change detector, that the expression of the participant is an unmodeled expression a trained model by comparing the expression representation with a plurality of predefined expression representation corresponding to the trained model;
Chang: Step 220: detects differences in the user's expression relative to the previous images…. when the user's facial expression has changed relative to a baseline facial expression (e.g., a neutral expression). Or the real-time data 125 is recorded when the detection module 140 detects any change from one expression to another, [0031] and The new facial expression is compared to a set of training data 130. The expression classifier 150 determines whether there is a defined facial expression in the training data 130 that matches the new facial expression. For example, the new facial expression and a defined facial expression can be identified as matching when they have a similarity of 95% or greater, [0032);
modifying the trained model to incorporate the expression using the expression representation in response to the determination that the expression is an unmodeled expression; (Chang: if a matching defined facial expression is not found at step 230, the most similar defined facial expression is selected, [0035], This is illustrated at step 250. The training data 130 is updated to include the recognized facial expression and its associated definition. This allows the expression classifier 150 to be retrained on the updated training data 130 for greater accuracy. [0034]);
generating a representation of the participant using the modified trained model to mimic the expression. (Applicant argues that “Chang is directed to a classification model and, as such, is trained to detect an expression, not to mimic an expression. To that end, incorporating the expression in Chang is directed to incorporating features from which the identifier from which the identifier for the expression can be determined, which is inherently different than incorporating a representation of the expression to mimic the expression. Thus, Chang fails to disclose "modifying the trained model to incorporate the expression using the expression representation in response to the determination that the expression is an unmodeled expression”.
Examiner wishes to provide:
Vemulapalli teaches, “some other mapping between facial expressions and puppet actions can be used that does not necessarily simply re-create the facial expression depicted by the input image 204. For example, a predefined mapping might translate from an embedding 206 descriptive of a smile included in the input image 204 to a dancing action by the puppet 904. Likewise, the mapping might translate from an embedding 206 descriptive of a frown included in the input image 204 to a crying action by the puppet 904, [0130] where the convolutional network 402 and/or other portions of the facial expression model 400 can have been previously trained on other forms of data and can thereafter be repurposed or otherwise retrained for use within the facial expression model 400, [0093] to mimic or otherwise reflect human perceptions or opinions regarding facial expressions, [0096].
While Chang teaches the technical features that met the claimed requirement in various environments… but the conferencing environment.
Gopal teaches, via Fig. 3A and 3B, a flowchart of a method for carrying out scheduled video chat sessions, according to various aspects of the present disclosure based on some rule, [0019].
OR
Kurtz presents a system for aiding family video-conferencing or video communications with one or more remote individuals. Such a system should function as seamlessly as is reasonably possible while being adaptable to the dynamic situations present in a residence. In particular, the system should enable the users to readily manage and maintain their privacy, relative at least to image capture, recording, and transmission. This system should also manage the contextual information of the user and their environments, to provide an effective communication experience.
Therefore it would have been obvious to the ordinary artisan before the effective filing date to incorporate the teaching of Vemulapalli into the teaching of Chang for the purpose of explicitly detailing the training on a modified of an avatar/representation reflecting the user/participant’s latest emotion/expression and also to incorporate the teaching of Gopal or Kurtz into the teaching of Chang for the purpose of expanding the claimed features to include the conferencing benefit where a real time video communication link between two or more locations, and more particularly to an automated method for detecting and characterizing activity in a local environment, and then transmitting or recording video images, for either live or time shifted viewing in a remote location, respectively, depending on both the acceptability of the characterized images and the status of users at the remote viewing system based on the desired privacy rules when sending/receiving images.
Claims 2, 10 and 19, wherein the model of the participant was generated based on training expressions, wherein the captured expression determined to be an unmodeled expression based on a detected difference between the captured expression and the training expressions. (See the independent claims).
Claims 7-8, 15-16 and 20-21, determining one or more parameters of the one or more captured expressions; wherein the determining that the captured expression of the participant is an unmodeled expression is based on values of the one or more parameters; and, wherein the trained model of the participant is retrained using the one or more parameters. (Chang: facial expression, mental state (e.g., interested, happy, excited, bored, etc.)).
17. (Cancelled)
Claims 3 and 11. (Currently Amended) The method according to claim 1, further comprising: sending the images comprising the captured expression to a computerized system under privacy restrictions.
Gopal: [0019] In some instances, the preloading operations may include transmitting audio and/or video from a participant's device to one or more other devices for some period of time prior to the start of the meet-and-greet. For example, a short video clip may be transmitted to a server, which may determine (e.g., via human review, image analysis, etc.) whether the participant's video contains images that violate one or more rules (e.g., prohibited symbols, nudity or inappropriate attire, etc.). Alternatively and/or additionally, a video clip may be transmitted to the celebrity or public figure's device for him or her to decide whether or not to initiate the call, [0019].
Kurtz teaches an enhancement for privacy as seen in the table below:
PNG
media_image1.png
648
656
media_image1.png
Greyscale
Claims 4 and 12, wherein the privacy restrictions restrict a number of one or more images sent to the computerized system. (Kurtz teaches the limitations of the predetermined privacy and contextual settings, [0163]. It is obvious that user can set the rule of image quantity to be transmitted to any destination).
Claims 5 and 13, wherein the privacy restrictions restrict information embedded in the one or more images sent to the computerized system. ((Kurtz teaches the limitations of the predetermined privacy and contextual settings, [0163] and encryption is notoriously well known in the art… It is obvious that user can set condition with ease without much modification to the current references).
Claims 6 and 14, wherein the privacy restrictions prevent sending one or more images that enable a reconstruction of a content of the 3D video conference. (Gopal: preloading operations,” “preloading checks,” and the like may refer to steps in which video and/or audio information captured by the client device during preloading are analyzed and determined to represent images and/or sounds that preclude the user from proceeding with a private one-on-one video chat with the celebrity or public figure. For example, certain profanity, nudity, or other content may be prohibited on the platform, and the preloading stage may be used to prevent such prohibited content from being broadcast to the celebrity or public figure (e.g., using image analysis, video analysis, object recognition, human review, etc.). In other examples, the celebrity or public figure (or a representative of that celebrity or public figure) may be given a chance to preview the transmission from a client device prior to the private one-on-one meet-and-greet with that client device's user to determine for themselves whether to initiate or decline the meet-and-greet. In such cases, a celebrity's or public figure's decision to decline the meet-and-greet may be considered a preloading failure, in that the client device did not meet the requirement of prior approval by the celebrity or public figure, [0069]. Here examiner reads that no image will be sent before the preloading checks.
Response to Arguments
Applicant’s arguments with respect to claim(s) filed on 4/15/26 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant argues that “Chang is directed to a classification model and, as such, is trained to detect an expression, not to mimic an expression. To that end, incorporating the expression in Chang is directed to incorporating features from which the identifier from which the identifier for the expression can be determined, which is inherently different than incorporating a representation of the expression to mimic the expression. Thus, Chang fails to disclose "modifying the trained model to incorporate the expression using the expression representation in response to the determination that the expression is an unmodeled expression”.
Examiner respectfully disagrees as examiner has produced new reference address the applicant’s argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUNG-HOANG J. NGUYEN whose telephone number is (571)270-1949. The examiner can normally be reached on Reg. Sched. 6:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached on 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUNG-HOANG J NGUYEN/ Primary Examiner, Art Unit 2691
Prosecution Insights