DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the response filed October 29, 2025. No claims have been amended. Claims 4-9, 11, 13-18, 20-23, 25-27, and 29-32 are pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-9, 11, 13-18, 20-22, and 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Maeng et al (US Patent Application Publication No. 2021/0327447), hereinafter Maeng, in view of Oz et al (WO 2022/238908), hereinafter Oz.
Maeng teaches speaker detection using lip reading. Regarding claim 4, Maeng teaches a computer-implemented method [fig 1. Fig 4A, Fig 6, Fig 7], the method comprising: receiving first image data [captured video and audio --para 0010; 0011-0015; 0369-0370]; determining that a first face is represented in the first image data [objects P1, P2, P3 detected -- Fig 8; para 0011-0015; 0337-0340; 0347-0354]; processing the first image data using at least on machine learning model representing a first facial expression [models, ANN, deep learning used …system detect objects and feature points –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the mesh representation to determine at least a first value of the first characteristic of the first mouth of the first face [system detect objects and feature points…F1-F10, where the various feature points provide mouth height and width info –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; and using the first value to determine that the first face is speaking [facial recognition ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]. Maeng fails to teach the representation is a mesh representation corresponding to the facial expression without also representing a shape of the first face that is unique to a person speaking. In a similar field of endeavor, Oz teaches a system and method for creating a 3D face mesh model for a generic human face [para 0320] for creating changes in general expressions of the face, One having ordinary skill in the art at the time of the invention would have recognized the advantages of implementing the face mesh representations of Oz, in the lip reading/speaker detection processing of Maeng, and the results would have been predicable, and provided realistic facial expressions, as taught by Oz and thereby enhancing the accuracy of the system as suggested by Maeng [para 0032-0033].
Regarding claim 5, the combination of Maeng and Oz teaches wherein determining the fourth data further comprises determining a standard deviation value associated with the first characteristic [mouth shape variations….system detect objects and feature points…F1-F10, where using the feature points to calculate specific standard deviation values is an obvious step requiring only routine skill in the art ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376], and determining that the first face is speaking further comprises: determining that the standard deviation value satisfies a threshold [mouth shape variations compared with predetermined gap size and predetermined number of times ….system detect objects and feature points…F1-F10, where using the feature points to calculate specific standard deviation values using the exceeding of a predetermined gap size and number of times is an obvious step requiring only routine skill in the art ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]; and in response to determining that the standard deviation value satisfies the threshold, determining that a user associated with the first face is speaking [facial recognition… recognition performed based on direction of each sound source ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0366-0367; 0369-0376].
Regarding claim 6, the combination of Maeng and Oz teaches wherein using the mesh representation to determine the first value further comprises: using the second data to determine first coordinate values corresponding to a top lip of the first mouth [system detect objects and feature points…F1-F10, where the various feature points provide mouth lip data –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the mesh representation to determine second coordinate values corresponding to a bottom lip of the first mouth [system detect objects and feature points…F1-F10, where the various feature points provide mouth lip data –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the first coordinate values and the second coordinate values to determine the first mouth height [system detect objects and feature points…F1-F10, where the various feature points provide mouth height and width info –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the mesh representation to determine third coordinate values corresponding to a first intersection between the top lip and the bottom lip [system detect objects and feature points…F1-F10, where the various feature points provide mouth lip data –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the mesh representation to determine fourth coordinate values corresponding to a second intersection between the top lip and the bottom lip [system detect objects and feature points…F1-F10, where the various feature points provide mouth lip data –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; using the third coordinate values and the fourth coordinate values to determine the first mouth width [system detect objects and feature points…F1-F10, where the various feature points provide mouth width info –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; and determining the third data at least in part by determining the ratio value between the mouth height and the mouth width [system detect objects and feature points…F1-F10, where the various feature points provide mouth height and width info and determining optimum element functional relationships is an obvious step requiring only routine skill in the art –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376].
Regarding claim 7, the combination of Maeng and Oz teaches wherein determining that the user is speaking further comprises detecting a beginning of an utterance during a first time interval [time gaps.. predetermined time…. detected uttering times para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376], the method further comprising: generating audio data representing the utterance, a beginning of the audio data occurring within the first time interval [speech signal processing…. detected uttering times para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376],; causing speech processing to be performed to the audio data; and in response to the speech processing, causing an action to be performed corresponding to the utterance [speaker recognition, speech recognition, speech-to-text, mapping text to mouth shape in video -- para 0010-0015; 0015-0018; 0026-0029; 0327-0340; 0347-0354; 0367; 0372-0376].
Regarding claims 8-9, and 11, Maeng teaches determining that a second face is represented in the first image data [objects P1, P2, P3 detected…multiple people captured in image -- Fig 8; para 0011-0015; 0337-0340; 0347-0354; 0369-0370]; processing first image data using the at least one machine learning model to determine a second facial expression of the second face without also representing a shape of the second face that is unique to a person speaking [models, ANN, deep learning used …system detect objects and feature points –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376 in combination with Oz’s 3D mesh representations]; using the second mesh representation to determine a seventh data representing at least a second value of the second characteristic of the second mouth [system detect objects and feature points…F1-F10, where the various feature points provide mouth height and width info –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376]; determining second data representing a second amount of variation in the second characteristic over time [mouth shape variations….system detect objects and feature points…F1-F10, where using the feature points to calculate specific variation amounts is an obvious step––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]; and determine that the first face is speaking [facial recognition ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]; detecting an utterance [captured video and audio --para 0010; 0011-0015; 0369-0370]; determining a first time interval extending from a beginning of the utterance to an ending of the utterance [time gaps.. predetermined time…. detected uttering times para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0366-0376]; determining a first amount of variation is greater than the second amount of variation [mouth shape variations compared with predetermined gap size and predetermined number of times ….system detect objects and feature points…F1-F10, where using the feature points to calculate specific standard deviation values using the exceeding of a predetermined gap size and number of times is an obvious step requiring only routine skill in the art ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]; and determining that the first face is speaking based on at least the first amount of variation being greater than the second amount of variation [facial recognition… recognition performed based on direction of each sound source ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0366-0367; 0369-0376]; determining the second face is not speaking [facial recognition… recognition performed based on direction of each sound source ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0366-0367; 0369-0376].
Regarding claim 25, the combination of Maeng and Oz teaches the first characteristic comprises a ratio between a mouth height of the first mouth and a mouth width of the first mouth [system detect objects and feature points…F1-F10, where the various feature points provide mouth height and width info –para 0010-0015; 0327-0340; 0347-0354; 0367; 0372-0376].
Regarding claim 26, the combination of Maeng and Oz teaches determining fourth data representing a first amount of variation in the first characteristic over time [Maeng’s mouth shape variations….system detect objects and feature points…F1-F10, where using the feature points to calculate specific variation amounts is an obvious step––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376]; and determining that the first face is speaking based at least in part on the fourth data [Maeng’s facial recognition ––para 0010-0015; 0015-0018; 0327-0340; 0347-0354; 0367; 0372-0376].
Claims 13-18 and 20-22 are system claims reciting processor, memory, and executable instructions [fig 1. Fig 4A, Fig 6, Fig 7], capable of performing steps similar to the steps of method claims 4-9, 11, and 25-26, and are therefore rejected under similar rationale as claims 4-9, 11, and 25-26.
Allowable Subject Matter
Claims 23, 27, and 29-32 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant's arguments filed October 29, 2025 have been fully considered but they are not persuasive.
Applicant argues “there is nothing in Oz that teaches or suggest that the 3D representations of faces that are generated by using a 3D deformation model to deform a 3D face template mesh could or should "represent[] a ... facial expression of [a] face without also representing a shape of the ... face that is unique to a person speaking." The Examiner respectfully disagrees. Oz specifically teaches the template mesh is sufficient to represent the general shape but not wrinkles, microstructures or other fine details. Oz’s general facial shape without details provides a form of shape of the face that is not unique to a person speaking, as wrinkles, microstructures or other fine details that would be present when a person is speaking, would not be provided in the general face shape.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGELA A ARMSTRONG whose telephone number is (571)272-7598. The examiner can normally be reached M,T,TH,F 11:30-8:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ANGELA A. ARMSTRONG
Primary Examiner
Art Unit 2659
/ANGELA A ARMSTRONG/Primary Examiner, Art Unit 2659