Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 5, 6, 8, 9, 11-13, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Berger et al. (U.S. Patent Application Publication No. 2023/0070631), referred herein as Berger, in view of Kim et al. (U.S. Patent Application Publication No. 2022/0139113), referred herein as Kim.
Regarding claim 1, Berger teaches a computer-implemented method for real-time customization of a virtual avatar, comprising:
providing a virtual avatar associated with a user (paragraph 27; paragraph 40, lines 1-5; paragraph 61, the last 6 lines; paragraph 70, lines 1-9; paragraph 119, lines 1-5; a real-time avatar is provided to the user, while the user provides image data comprising the user and surroundings);
extracting a region of interest from an image, the region of interest including the user’s face shown in the image, and providing the region of interest to a data model (paragraph 34, lines 1-10; paragraph 38, lines 1-15 and the last 7 lines; paragraph 40, lines 5-9; paragraph 64; paragraph 119, the last 8 lines; paragraph 120, lines 1-9; facial detection algorithms are applied to identify a region of interest including the user’s face shown in the image, which is processed by a machine learning model);
processing the region of interest, by the data model, and using feature recognition to determine whether the region of interest comprises a facial feature from a set of predefined features, wherein the set of predefined features includes a set of facial features (paragraph 38, the last 16 lines; paragraph 40, lines 9-12; paragraphs 64 and 65; paragraph 122, lines 1-8; paragraph 123, the last 16 lines; paragraph 134, lines 1-14; the machine learning model processes the region of interest and determines whether features correspond to predefined features),
wherein determining whether the region of interest comprises the facial feature from the set of predefined features comprises: assigning, by the data model, a confidence value to each facial feature in the set of predefined features, and in response to determining that the confidence value assigned to the facial feature from the set of predefined features exceeds a predetermined threshold, updating the appearance of the virtual avatar (paragraph 38, lines 1-8; paragraph 39, lines 1-16; paragraph 40, the last 5 lines; paragraph 43; paragraphs 121 and 124; paragraph 134, lines 14-21; a popularity [confidence] value and threshold is determined, and in response to the value exceeding the threshold, the appearance of the avatar is updated in real time based on the results of the processing) by applying the facial feature to the virtual avatar (paragraph 40, lines 1-5 and the last 5 lines; paragraph 61; paragraph 70, lines 1-9; paragraph 134, lines 11-21).
Berger does not explicitly teach determining if the region of interest comprises a type of facial feature from a set of predefined types of facial features, wherein the set of predefined features includes a set of types of the facial feature, assigning a confidence value to each type of facial feature in the set of predefined features, and wherein the confidence value indicates a likelihood that the region of interest includes the type of the facial feature.
However, in a similar field of endeavor, Kim teaches a computer-implemented method comprising applying a facial recognition algorithm to an image, extracting a region of interest of the face shown in the image, and processing the region of interest using feature recognition to determine that the region of interest comprises a predefined facial feature (paragraph 49, lines 1-2; paragraph 52, lines 1-8; paragraph 55, lines 1-8), and further comprising determining if the region of interest comprises a type of facial feature from a set of predefined types of facial features, wherein the set of predefined features includes a set of types of the facial feature, and assigning a confidence value to each type of facial feature in the set of predefined features, wherein the confidence value indicates a likelihood that the region of interest includes the type of the facial feature (paragraph 55, lines 1-8; paragraph 78, lines 6-17; paragraph 81; paragraph 82, lines 1-7; paragraph 85, lines 1-11).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the facial feature type and confidence value processing of Kim with the region of interest processing of Berger because accuracy is of upmost importance in facial recognition technologies, and the probabilities and object type identification process of Kim efficiently increases recognition accuracy, while minimizing computing resource requirements (see, for example, Kim, paragraphs 4 and 5; paragraphs 79 and 114).
Regarding claim 2, Berger in view of Kim teaches the computer-implemented method of claim 1, wherein the set of predefined features includes one or more of: facial hair; ear type; nose type; eye type; and/or skin tone (Berger, paragraph 83; Kim, paragraphs 38 and 42; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 5, Berger in view of Kim teaches the computer-implemented method of claim 1, wherein updating the appearance of the virtual avatar comprises: reviewing an image directory comprising a plurality of image elements to identify an image element that is representative of the type of the facial feature; and applying the image element to the virtual avatar (Berger, paragraph 38, the last 11 lines; paragraph 40, lines 9-12; paragraphs 64 and 65; paragraph 122, lines 1-8; paragraph 123, the last 16 lines; paragraph 134, lines 1-14; Kim, paragraph 52; paragraph 78, lines 1-17; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 6, Berger in view of Kim teaches the computer-implemented method of claim 5, wherein the image directory is associated with the virtual avatar (Berger, paragraph 38, the last 11 lines; paragraph 40; paragraph 134, lines 1-21).
Regarding claim 8, Berger in view of Kim teaches the computer-implemented method of claim 1, wherein the region of interest is centered on the user's face and has a predefined area (Berger, paragraph 34, lines 1-10; paragraph 38, lines 1-15; paragraph 40, lines 5-9; paragraph 64; paragraph 120, lines 1-12).
Regarding claim 9, Berger in view of Kim teaches the computer-implemented method of claim 1, further comprising, prior to providing the region of interest to the data model: scaling the region of interest to a predetermined size (Berger, paragraph 83, lines 1-6 & the last 8 lines; Kim, paragraph 50, lines 1-9; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 11, Berger in view of Kim teaches the computer-implemented method of claim 1, further comprising: applying a facial detection algorithm to detect the user's face in the image (Berger, paragraph 34, lines 1-10; paragraph 38, lines 1-15; paragraph 40, lines 5-9; paragraph 64; paragraph 119, the last 8 lines; paragraph 120, lines 1-9; facial detection algorithms are applied to detect the user’s face in the image); and identifying the user's face from a plurality of faces detected in the image (Berger, paragraph 40, lines 1-10 and the last 5 lines; paragraph 65; paragraph 93, lines 1-14 and the last 7 lines; a particular user is extracted from the image, which includes a plurality of people).
Regarding claim 12, Berger in view of Kim teaches the computer-implemented method of claim 1, further comprising: receiving input audio data; and providing at least a portion of the input audio data to the data model; processing the input audio data by the data model to determine whether the input audio data comprises a feature from a second set of predefined features, and in response to determining that the input audio data comprises a feature from the second set of predefined features, updating the appearance of the virtual avatar (Berger, paragraph 48, lines 1-7, 13-22, and the last 4 lines; paragraphs 65 and 92; paragraphs 96 and 101-103; paragraph 121).
Regarding claim 13, Berger in view of Kim teaches the computer-implemented method of claim 12, wherein the input audio data comprises one or more of: voice or vocal data; environmental audio data; and breathing audio data (Berger, paragraph 48, lines 13-22 and the last 4 lines; paragraphs 101-103).
Regarding claim 18, Berger in view of Kim teaches the computer-implemented method of claim 1, wherein the data model comprises an artificial neural network, ANN, optionally wherein the ANN comprises a convolutional neural network, CNN (Berger, paragraph 85).
Regarding claim 19, Berger in view of Kim teaches a system comprising one or more processors configured to implement the method as claimed in claim 1 (Berger, fig 11, processor(s) 1102; paragraph 144, lines 1-13).
Regarding claim 20, Berger in view of Kim teaches a non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to implement the method as claimed in claim 1 (Berger, fig 11, medium 1104; paragraph 143, lines 1-11).
Claims 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Berger, in view of Kim, and further in view of Gorumkonda et al. (U.S. Patent Application Publication No. 2024/0177390), referred herein as Gorumkonda.
Regarding claim 14, Berger in view of Kim teaches the computer-implemented method of claim 1, further comprising: receiving input audio data; processing the input audio data to determine whether the input audio data includes sound effects; in response to determining that the input audio data includes sound effects, determining at least one property of the audio data; and updating the appearance of the virtual avatar in response to the determined at least one property (Berger, paragraph 48, lines 1-7, 13-22, and the last 4 lines; paragraphs 65 and 92; paragraphs 96 and 101-103; paragraph 121).
Berger in view of Kim teaches determining content related to a music concert and providing music AR data (see, for example, Berger, paragraphs 46 and 137), but does not explicitly teach determining whether the audio data includes music, in response to determining that the audio data includes music, determining at least one property of the music, and updating the appearance of the avatar accordingly.
However, in a similar field of endeavor, Gorumkonda teaches a method for real time customization of a virtual avatar of a user based on extracting regions of interest from an image of the user and their surroundings (paragraph 120, lines 9-15), comprising determining whether the audio data includes music, in response to determining that the audio data includes music, determining at least one property of the music, and updating the appearance of the avatar accordingly (paragraph 119, lines 1-12; paragraph 120, lines 1-6 and the last 6 lines; paragraph 121, lines 1-12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the music determination of Gorumkonda with the audio data processing of Berger in view of Kim because this improves the functionality of the avatar animation system by providing a more realistic response by the avatar to music (such as the music disclosed in Berger), thereby improving the usage of the avatars and driving user engagement (see, e.g., Gorumkonda, paragraph 121, lines 1-5 and the last 4 lines).
Regarding claim 15, Berger in view of Kim teaches the computer-implemented method of claim 1, further comprising: receiving input data identifying audio data; determining or receiving at least one property of the audio data; and updating the appearance of the virtual avatar in response to the determined at least one property (Berger, paragraph 48, lines 1-7, 13-22, and the last 4 lines; paragraphs 65 and 92; paragraphs 96 and 101-103; paragraph 121).
Berger in view of Kim teaches determining content related to a music concert and providing music AR data (see, for example, Berger, paragraphs 46 and 137), but does not explicitly teach identifying a music track that the user is listening to, determining or receiving a property of the music track, and updating the appearance of the avatar accordingly.
However, in a similar field of endeavor, Gorumkonda teaches a method for real time customization of a virtual avatar of a user based on extracting regions of interest from an image of the user and their surroundings (paragraph 120, lines 9-15), comprising identifying a music track that the user is listening to, determining or receiving a property of the music track, and updating the appearance of the avatar accordingly (paragraph 119, lines 1-12; paragraph 120, lines 1-6 and the last 6 lines; paragraph 121, lines 1-12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the music track determination of Gorumkonda with the audio data processing of Berger in view of Kim because this improves the functionality of the avatar animation system by providing a more realistic response by the avatar to music (such as the music disclosed in Berger), thereby improving the usage of the avatars and driving user engagement (see, e.g., Gorumkonda, paragraph 121, lines 1-5 and the last 4 lines).
Regarding claim 16, Berger in view of Kim, further in view of Gorumkonda teaches the computer-implemented method of claim 14, wherein the at least one property includes one or more of: a volume; a tempo; a genre; an artist; a play count; and a duration (Gorumkonda, paragraph 119, lines 1-12; paragraph 121, lines 4-12; the motivation is the same as that discussed in the rejection of claim 14, above).
Regarding claim 17, Berger in view of Kim, further in view of Gorumkonda teaches the computer-implemented method of claim 14, wherein updating the appearance of the virtual avatar in response to the determined at least one property comprises: changing a color or design of at least one accessory worn by the virtual avatar; adding or removing at least one accessory worn by the virtual avatar; and/or applying a skin from a plurality of predefined skins to the virtual avatar (Berger, paragraph 41; paragraph 43, lines 1-16).
Allowable Subject Matter
Claim 7 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, the prior art teaches the method of claim 1, as shown above. In the context of claims 1, 5, and 7 as a whole, however, the prior art does not appear to teach the method of claim 1, wherein updating the appearance of the virtual avatar comprises reviewing an image directory comprising a plurality of image elements to identify an image element that is representative of the type of the facial feature, and applying the image element to the virtual avatar, wherein each feature of the set of predefined features has an associated label and each image element has an associated label, and wherein reviewing the image directory to identify the image element that is representative of the type of the facial feature comprises reviewing the image directory to identify the image element having a label that is the same as, or is closest to, or is associated with, the label of the type of the facial feature.
Response to Arguments
Applicant’s arguments with respect to the prior art rejections have been fully considered, but are moot in view of the new ground(s) of rejection presented above.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Lee (U.S. Patent Application Publication No. 2012/0154619); Augmented reality processing based on eye capture in handheld device.
Li (U.S. Patent Application Publication No. 2020/0294243); Method, electronic device and storage medium for segmenting image.
Kuo (U.S. Patent No. 11,783,612); False positive suppression using keypoints.
Zhang (U.S. Patent Application Publication No. 2022/0368979); Non-occluding video overlays.
Bhargava (U.S. Patent Application Publication No. 2022/0405500); Computationally efficient and robust ear saddle point detection.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights