DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 8-9, and 15-16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Xiong et al. (CN 117425017 A) (Hereinafter referred to as Xiong).
Regarding Claim 1, Xiong discloses A method for processing a digital human expression, applied to an electronic device, the method comprising: (See Abstract, “The invention claims an adjusting method, device and medium of digital human expression, relating to the technical field of artificial intelligence (AI) . . .”)
obtaining first information of a user; obtaining second information of the user; (See Page 12 Paragraph 2, “The face capturing device 110 is used to capture the facial expression data of the face capturing actors and sends it to the live broadcast management device 120. the facial expression data is used for representing the expression action amplitude of multiple parts of the face of the face capturing actor.”
See Page 15 Paragraph 5, “In one possible embodiment, after obtaining the facial expression data of each frame collected by the face capturing device, the facial expression data can be divided according to different parts, such as eyebrow, eye, cheek, nose, mouth, chin and so on.” In this case, one can consider the eye data of the facial expression data to correspond to “first information of a user” while the mouth data of the facial expression data to correspond to “second information of the user”.)
acquiring corresponding expression information based on the second information; and (See Page 5 Paragraph 3, “. . . the facial expression corresponding to the animation of the digital human model is generated based on the adjusted facial expression data of the adjusting parameter.”
See Page 16 Paragraphs 4-5, “Further, the live broadcast management device 120 can adjust a plurality of expression base vector coefficients corresponding to the at least one part in the facial expression data of the digital human model according to the adjustment parameter of the at least one part, so as to realize adjusting the expression action amplitude of at least one part in the facial expression data of the digital human model according to the adjusting parameter of at least one part.
For example, the opening amplitude of the mouth part can be adjusted by adjusting at least one expression vector coefficient corresponding to the mouth part. For example, when at least one expression vector coefficient corresponding to the mouth part is increased, the opening amplitude of the mouth part is increased, and when at least one expression vector coefficient corresponding to the mouth part is decreased, the opening amplitude of the mouth part is decreased.”
In summary, Xiong teaches adjustment parameters used to adjust specified facial features from the facial expression data, resulting in the adjusted facial expression data. In this case, one can consider the adjusted facial expression data to correspond to “expression information”. The adjusted facial expression data would be based on the facial expression data, with Xiong specifically giving an example of adjusting the amplitude of the mouth part (second information).)
generating the digital human expression corresponding to the user according to the first information, the second information, and the expression information. (See Page 5 Paragraph 3, “displaying a second user interface, the second user interface comprises an animation of the digital human model corresponding to the character, and the facial expression corresponding to the animation of the digital human model is generated based on the adjusted facial expression data of the adjusting parameter.”
In this case, the facial expression corresponding to the animation of the digital human model (the digital human expression) would be generated corresponding to user and according to the adjusted facial expression data (expression information), which would be based on the facial expression data (first and second information).)
Regarding Claim 2, Xiong discloses The method according to claim 1, wherein acquiring corresponding expression information based on the second information comprises: acquiring the corresponding expression information based on a mouth shape of the second information. (See Page 15 Paragraph 5, “In one possible embodiment, after obtaining the facial expression data of each frame collected by the face capturing device, the facial expression data can be divided according to different parts, such as eyebrow, eye, cheek, nose, mouth, chin and so on.”
See Page 5 Paragraph 3, “displaying a second user interface, the second user interface comprises an animation of the digital human model corresponding to the character, and the facial expression corresponding to the animation of the digital human model is generated based on the adjusted facial expression data of the adjusting parameter.”
See Page 16 Paragraph 5, “For example, the opening amplitude of the mouth part can be adjusted by adjusting at least one expression vector coefficient corresponding to the mouth part. For example, when at least one expression vector coefficient corresponding to the mouth part is increased, the opening amplitude of the mouth part is increased, and when at least one expression vector coefficient corresponding to the mouth part is decreased, the opening amplitude of the mouth part is decreased.”
One can consider the adjusted facial expression data (the corresponding expression information) to be based on the mouth shape of the second information as the opening amplitude of the mouth can be considered a mouth shape.)
Regarding Claim 8, Xiong discloses An electronic device comprising: a storage device; at least one processor; and the storage device storing one or more programs that, when executed by the at least one processor, cause the at least one processor to: (See Page 8 Paragraphs 12 – 14, “In a ninth aspect, the embodiment of the application provides a live broadcast management device, comprising a memory and a processor; said memory is used for storing the computer program; The processor is configured to execute a computer program stored in the memory, implement a method as described . . .”)
obtain first information of a user; obtain second information of the user; acquire corresponding expression information based on the second information; and generate a digital human expression corresponding to the user according to the first information, the second information, and the expression information. (The above limitations are similar to those of Claim 1 rejection motivation, and are thus rejected under a similar rationale as that of Claim 1.)
Regarding Claim 9, Claim 9 contains similar limitations as to Claim 2 and is thus rejected under a similar rationale as that of Claim 2.
Regarding Claim 15, Xiong discloses A non-transitory storage medium having instructions stored thereon, when the instructions are executed by a processor of an electronic device, the processor is caused to perform a method for processing a digital human expression, wherein the method comprises: (See Page 8 Paragraphs 12 – 14, “In a ninth aspect, the embodiment of the application provides a live broadcast management device, comprising a memory and a processor; said memory is used for storing the computer program; The processor is configured to execute a computer program stored in the memory, implement a method as described . . .” Also see Page 13 Paragraph 4, “a non-volatile memory, such as at least one disk memory device, a flash memory device, a universal flash storage (UFS), and the like.”)
obtaining first information of a user; obtaining second information of the user; acquiring corresponding expression information based on the second information; and generating the digital human expression corresponding to the user according to the first information, the second information, and the expression information. (The above limitations are similar to those of Claim 1 rejection motivation, and are thus rejected under a similar rationale as that of Claim 1.)
Regarding Claim 16, Claim 16 contains similar limitations as to Claim 2 and is thus rejected under a similar rationale as that of Claim 2.
Allowable Subject Matter
Claims 3-7, 10-14, and 17-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 3, the cited prior art does not disclose or render obvious the combination of elements cited in the claims as a whole. Specifically, the cited prior art fails to disclose or render obvious the limitations: wherein acquiring corresponding expression information based on the mouth shape of the second information comprises: obtaining a circle based on the mouth shape; determining whether an overlapping portion between an arc corresponding to the mouth shape and the circle is greater than a first threshold; in response to the overlapping portion being greater than the first threshold, determining whether a center of the circle is located above the arc; in response to the center of the circle being located above the arc, determining whether a radius of the circle is equal to a preset radius; in response to the radius of the circle being equal to the preset radius, determining that an expression of the user corresponds to a digital human associated with the preset radius. Thus Claim 3 contains allowable subject matter.
Regarding Claims 4-7, Claims 4-7 are dependent upon the base Claim of 3 and thus also contains allowable subject matter.
Regarding Claim 10, Claim 10 recites similar subject matter as to Claim 3 and thus contains similar allowable subject matter.
Regarding Claims 11-14, Claims 11-14 are dependent upon the base Claim of 10 and thus also contains allowable subject matter.
Regarding Claim 17, Claim 17 recites similar subject matter as to Claim 3 and thus contains similar allowable subject matter.
Regarding Claims 18-20, Claims 18-20 are dependent upon the base Claim of 17 and thus also contains allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Reference Yamada (US 20120140994 A1) is made of record as an art that teaches determining facial expressions by extracting features such as eye shape and mouth shape (See [0045]) and pattern matching the shapes to already standardized ones to determine the type of facial expression (See [0045]).
Reference Beisel et al. (US 11373446 B1) is made of record as an art that teaches analyzing user images to identify facial expressions (See Col 15 Lines 48 – 50) and using the identified information, such as the shape of a user’s mouth being open or closed, to identify if the user is happy, sad, laughing, etc. (See Col 15 Lines 48 – 57).
Reference Avani et al. (“Geometrical features of lips using the properties of parabola for recognizing facial expression”) is made of record as an art that teaches using reference points on the face to estimate the facial expressions (See Abstract), using an input facial image and localizing and identifying specifically the lips to determine the facial expression (See Page 7 Fig. 1), and using different machine learning methods for facial expression classification based on the lips feature points (See Page 14).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THANG G HUYNH whose telephone number is (571)272-5432. The examiner can normally be reached Mon-Thu 7:30am-4:30pm EST | Fri 7:30am-11:30am EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/T.G.H./Examiner, Art Unit 2611
Prosecution Insights