DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to communication filed on 8/17/2026. Claims 1-20 are pending on this application.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot in view of the new grounds of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 13, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144) in view of Grigoriev et al (WO2020/096368A1) and Hu et al (US20210209347).
Regarding claim 1, Ogawa teaches a non-transitory computer-readable medium storing a computer program that allows a computer to execute a process (para. [0060]) comprising:
acquiring face information of a subject (para. [0093]) detected by a sensor (par. [0082], [0087]);
generating face structure information of the subject on a basis of the acquired face information (para. [0108]-[0109], pseudo-normal image); and
estimating presence or absence of neuropathy of the subject on a basis of a two-dimensional image in the front view as the face information (fig. 9A; para. [0113], [0115]-[0116]).
Ogawa fails to teach complementing missing face information of the subject with respect to the acquired face information on a basis of the generated face structure information.
However Grigoriev teaches generating face structure information of a subject on a basis of acquired face information (page 1 lines 15-18 and page 11 lines 6-10); and
complementing missing face information of the subject (page 7 lines 11-19 and page 13 lines 6-22).
Furthermore, it would be obvious to substitute the complemented face information of Grigoriev in the estimation step of Ogawa.
Therefore taking the combined teachings of Ogawa and Grigoriev as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Grigoriev into the apparatus of Ogawa. The motivation to combine Ogawa and Grigoriev would be to improve accuracy of image resynthesis for synthesizing new views of a person or an object based on at least one input image (page 5 lines 9-11 of Grigoriev).
Ogawa also fails to teach generating a reference three-dimensional model;
generating a latest three-dimensional model of the face of the subject by pasting an acquired latest face information of the subject onto the reference three-dimensional model; and
converting the latest three-dimensional model of the face into a two-dimensional image in a front view.
However Hu teaches generating a reference three-dimensional model (para. [0052], At 304, a three-dimensional (3D) model may be generated. In an embodiment, the circuitry 202 may be configured to generate the 3D model of the subject's face 114 based on at least one color image of the set of color images and the depth information 320; para. [0053], The generation of the 3D model may further include acquisition of a 3D mean-shape model of a reference 3D face);
generating a latest three-dimensional model of the face of the subject (para. [0058]) by pasting an acquired latest face information of the subject onto the reference three-dimensional model (para. [0060], The generation of the set of texture maps may be done based on the refined 3D model 326 and the acquired set of color images; para. [0061], The first texture map 328A may be generated by a direct application of texture (or color information) from the first color image 318A); and
converting the latest three-dimensional model of the face into a two-dimensional image in a front view (para. [0056], To generate each of such projections, the circuitry 202 may be configured to project the 3D model 322 onto a 2D image plane along each of the set of viewpoints; para. [0087]).
Therefore taking the combined teachings of Ogawa and Grigoriev with Hu as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Hu into the apparatus of Ogawa and Grigoriev. The motivation to combine Hu, Ogawa and Grigoriev would be to capture texture with a higher accuracy and with improved quality (para. [0027] of Hu).
Regarding claim 13, the modified invention of Ogawa teaches a non-transitory computer-readable medium further comprising:
in a case where the face information detected by the sensor is face information other than a front view of the subject (page 19 lines 8-10 of Grigoriev), displaying complemented face information in a front view (page 7 lines 11-19 and page 13 lines 6-22 of Grigoriev) on a display unit (para. [0106] of Ogawa).
Regarding claim 18, the modified invention of Ogawa teaches a non-transitory computer-readable medium wherein the sensor is a camera that images a two-dimensional image (311 in fig. 2 and para. [0093] of Ogawa) or a ranging sensor (para. [0082] of Ogawa).
Regarding claim 19, the claim recites similar subject matter as claim 1 and is rejected for the same reasons as stated above.
Regarding claim 20, the claim recites similar subject matter as claim 1 and is rejected for the same reasons as stated above.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Grigoriev et al (WO2020/096368A1) and Hu et al (US20210209347) in view of Yan et al (US20200302623).
Regarding claim 2, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
acquiring reference face structure information of the subject generated in advance from a storage unit; and
generating face structure information of the subject on a basis of the face information detected by the sensor and the reference face structure information stored in the storage unit.
However Yan teaches acquiring reference face structure information of a subject generated in advance from a storage unit (para. [0008], acquiring from the storage unit the two-dimensional image of the face; acquiring from the storage unit a three-dimensional mesh image); and
generating face structure information of the subject on a basis of the face information detected by the sensor and the reference face structure information stored in the storage unit (para. [0008], identifying a plurality of key points on the two-dimensional image of the face, generating a texture map of the two-dimensional image of the face, projecting said texture map of the two-dimensional image of the face onto the three-dimensional mesh image, determining a first set of one or more proportions within the two-dimensional image of the face, determining a second set of one or more proportions within the three-dimensional mesh image, determining a plurality of scaling factors wherein each of said scaling factors is a function of one of said first set of one or more proportions and a corresponding one of said second set of one or more proportions, adjusting the three-dimensional mesh image based on the determined plurality of scaling factors to yield the face image that appears three dimensional).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Yan as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Yan into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Yan, Hu, Ogawa and Grigoriev would be to improve and automate methods for converting a single 2-D image to a corresponding image with increased dimensional depth (para. [0006] of Yan).
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Hu et al (US20210209347), Grigoriev et al (WO2020/096368A1) and Yan et al (US20200302623) in view of Ye (US20200349731).
Regarding claim 3, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium wherein the reference face structure information includes multi-view face information acquired by a sensor in two or more directions with respect to a front face of the face of the subject.
However Yan teaches wherein reference face structure information includes multi-view face information acquired by a sensor in two or more directions (para. [0008]) with respect to a front face of the face of a subject (para. [0036]).
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Ye as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Ye into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Ye, Ogawa and Grigoriev would be to improve the accuracy of 3D face modeling (para. [0071] of Ye).
Claim(s) 4-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Grigoriev et al (WO2020/096368A1), Hu et al (US20210209347) and Yan et al (US20200302623) in view of Geng (US20070258627).
Regarding claim 4, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium wherein the reference face structure information includes three-dimensional face information obtained by acquiring a three-dimensional shape of the face of the subject by a sensor.
However Geng teaches a reference face structure information including three-dimensional face information obtained by acquiring a three-dimensional shape of the face of the subject (para. [0051]) by a sensor (para. [0043]).
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Geng as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Geng into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Geng, Ogawa and Grigoriev would be to provide rapid and accurate face identification (para. [0042] of Geng).
Regarding claim 5, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium wherein the reference face structure information includes face structure information generated on a basis of the multi-view face information acquired by a sensor in two or more directions with respect to a front face of the face of the subject and/or face structure information generated on a basis of the three-dimensional face information obtained by acquiring a three-dimensional shape of the face of the subject by a sensor.
However Geng teaches wherein a reference face structure information includes face structure information generated on a basis of the multi-view face information acquired by a sensor in two or more directions (claims 7 and 8) with respect to a front face of the face of the subject (par. [0109]) and/or face structure information generated on a basis of the three-dimensional face information obtained by acquiring a three-dimensional shape of the face of the subject (para. [0051]) by a sensor (para. [0043]).
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Geng as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Geng into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Geng, Ogawa and Grigoriev would be to provide rapid and accurate face identification (para. [0042] of Geng).
Regarding claim 6, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
storing the acquired face information or the face structure information generated on a basis of the face information in the storage unit as the reference face structure information.
However Geng teaches storing acquired face information or face structure information generated on a basis of the face information (para. [0051]) in the storage unit as the reference face structure information (para. [0015], [0062]).
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Geng as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Geng into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Geng, Ogawa and Grigoriev would be to provide rapid and accurate face identification (para. [0042] of Geng).
Claim(s) 7 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Grigoriev et al (WO2020/096368A1), Hu et al (US20210209347) and Yan et al (US20200302623) in view of Kusens et al (US20200210679).
Regarding claim 7, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
estimating the presence or absence of neuropathy of the subject on a basis of a comparison result between a right half of the face of the subject based on the complemented face information and a right half of the face based on the reference face structure information and a comparison result between a left half of the face of the subject based on the complemented face information and a left half of the face based on the reference face structure information.
However Kusens teaches estimating the presence or absence of neuropathy of a subject (para. [0032], [0041]) on a basis of a comparison result between a right half of the face of the subject based on the complemented face information and a right half of the face based on the reference face structure information (para. [0032], [0041]) and a comparison result between a left half of the face of the subject based on the complemented face information and a left half of the face based on the reference face structure information (para. [0032], [0041]). It would be obvious to apply the complemented face information of Grigoriev to the comparison steps of Kusens.
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Kusens as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kusens into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Kusens, Ogawa and Grigoriev would be to reduce the risk of a future stroke (abstract of Kusens).
Regarding claim 8, the modified invention of Ogawa teaches a non-transitory computer-readable medium further comprising:
in a case where the right half of the face based on the complemented face information is lower than the right half of the face based on the reference face structure information, or in a case where the left half of the face based on the complemented face information is lower than the left half of the face based on the reference face structure information, estimating that the subject has neuropathy (para. [0032], [0041] of Kusens).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Grigoriev et al (WO2020/096368A1) and Hu et al (US20210209347) in view of Li et al (US20180374242).
Regarding claim 9, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
generating face structure information of the subject by inputting the face information detected by the sensor to a learning model subjected to machine learning so as to receive the face information of the subject as an input and output the face structure information of the subject and acquiring the face structure information output by the learning model.
However Li teaches inputting face information detected by the sensor (para. [0036]) to a learning model subjected to machine learning (para. [0027]) so as to receive the face information of the subject as an input (para. [0089]) and output the face structure information of the subject (para. [0027]) and acquiring the face structure information output by the learning model (para. [0102]).
Therefore taking the combined teachings of Ogawa, Hu, Grigoriev and Yan with Li as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Li into the apparatus of Ogawa, Hu, Yan and Grigoriev. The motivation to combine Yan, Hu, Li, Ogawa and Grigoriev would be to generate high-quality and high-resolution renderings of individual's faces (para. [0029] of Li).
Claim(s) 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Grigoriev et al (WO2020/096368A1) and Hu et al (US20210209347) in view of Kusens et al (US20200210679).
Regarding claim 10, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
estimating the presence or absence of neuropathy of the subject on a basis of symmetry of a right half and a left half of the face of the subject based on the complemented face information
However Kusens teaches estimating the presence or absence of neuropathy of a subject on a basis of symmetry of a right half and a left half of the face of the subject based on the complemented face information (para. [0009], [0032], [0041]). It would be obvious to apply the complemented face information of Grigoriev to the comparison steps of Kusens.
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Kusens as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kusens into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Kusens, Hu, Ogawa and Grigoriev would be to reduce the risk of a future stroke (abstract of Kusens).
Regarding claim 11, the modified invention of Ogawa teaches a non-transitory computer-readable medium further comprising:
in a case where the right half of the face is lower than the left half, or in a case where the left half of the face is lower than the right half, estimating that the subject has neuropathy (para. [0032], [0041] of Kusens).
Claim(s) 12 and 15-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Hu et al (US20210209347) and Grigoriev et al (WO2020/096368A1) in view of Kostic et al (US20170007167).
Regarding claim 12, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
storing, in a case where it is estimated that the subject has neuropathy on a basis of face information obtained by complementing the face information detected by the sensor, and then it is estimated that the subject does not have neuropathy without complementing on a basis of the face information detected by the sensor, the face information when estimating that there is neuropathy in the storage unit.
However Kostic teaches storing (24 in fig. 1), in a case where it is estimated that the subject has neuropathy on a basis of face information obtained by complementing the face information detected by a sensor (para. [0074]), and then it is estimated that the subject does not have neuropathy without complementing on a basis of the face information detected by the sensor (para. [0062], [0078]), the face information when estimating that there is neuropathy in the storage unit (para. [0072]-[0073]).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Kostic as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kostic into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Kostic, Ogawa, Hu and Grigoriev would be to better assess whether the user has experienced a stroke or not (para. [0004] of Kostic).
Regarding claim 15, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
acquiring information of the subject by a second sensor different from the sensor in a case where it is estimated that the subject has neuropathy; and
further estimating the presence or absence of neuropathy of the subject on a basis of the information acquired by the second sensor.
However Kostic teaches acquiring information of the subject by a second sensor different from the sensor in a case where it is estimated that the subject has neuropathy (para. [0022]-[0023], [0029], [0062]); and
further estimating the presence or absence of neuropathy of the subject on a basis of the information acquired by the second sensor (para. [0029]).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Kostic as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kostic into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Kostic, Hu, Ogawa and Grigoriev would be to better assess whether the user has experienced a stroke or not (para. [0004] of Kostic).
Regarding claim 16, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
acquiring information of the subject by a second sensor different from the sensor; estimating the presence or absence of neuropathy of the subject on a basis of the information acquired by the second sensor (para. [0022]-[0023], [0029], [0062]); and
in a case where it is estimated that the subject has neuropathy, further estimating the presence or absence of neuropathy of the subject on a basis of the face information acquired by the sensor (para. [0029]).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Kostic as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kostic into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Kostic, Hu, Ogawa and Grigoriev would be to better assess whether the user has experienced a stroke or not (para. [0004] of Kostic).
Regarding claim 17, the modified invention of Ogawa fails to teach a non-transitory computer-readable medium further comprising:
performing a diagnostic test on the subject in a case where it is estimated that the subject has neuropathy.
However Kostic teaches performing a diagnostic test on the subject in a case where it is estimated that the subject has neuropathy (para. [0022]-[0023], [0029], [0062]).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Kostic as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Kostic into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Kostic, Hu, Ogawa and Grigoriev would be to better assess whether the user has experienced a stroke or not (para. [0004] of Kostic).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ogawa et al (US20230238144), Hu et al (US20210209347) and Grigoriev et al (WO2020/096368A1) in view of Sudou et al (US20240394630).
Regarding claim 14, the modified invention of Ogawa teaches a non-transitory computer-readable medium further comprising:
displaying a portion involved in an estimation result of the presence or absence of neuropathy on the display unit together with the face information (para. [0081] of Ogawa).
The modified invention of Ogawa fails to teach displaying the complemented face information and face information of the subject stored in advance in the storage unit on the display unit side by side or in a superimposed manner.
However Suduo teaches displaying complemented face information (402R in fig. 8) and face information of a subject stored in advance (402L in fig. 8) on a display unit (401L in fig. 8) side by side (para. [0121]).
Therefore taking the combined teachings of Ogawa, Hu and Grigoriev with Suduo as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the features of Suduo into the apparatus of Ogawa, Hu and Grigoriev. The motivation to combine Suduo, Hu, Ogawa and Grigoriev would be to know the state of the user in a short time (para. [0123] of Suduo).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEON VIET Q NGUYEN whose telephone number is (571)270-1185. The examiner can normally be reached Mon-Fri 11AM-7PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEON VIET Q NGUYEN/Primary Examiner, Art Unit 2663
Prosecution Insights