DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 16-18, 20-29 is/are rejected under 35 U.S.C. 102(a)(1)as being anticipated by DeTone ( US 20240231102).
Regarding claim 16, DeTone teaches a method for training a machine learning model including a first deep neural network and a second deep neural network, the method comprising the following steps:
(S1) providing two training images, which at least partially include the same or similar image contents, wherein the two training images depict the image contents from different perspectives and/or at different points in time ( 102-1, 102-2 in Fig. 1; 1802 in Fig. 18).
(S2) processing the two training images by the first deep neural network in such a way that image features are extracted from the two training images by setting corresponding respective keypoints in the two training images and describing the corresponding respective keypoints with the same or similar descriptors( 108-1, 110-1, 108-2, 110-2 in Fig. 1; 1804 in Fig. 18).
(S3) matching the extracted image features of the two training images, wherein the image features with the same or similar descriptors are matched to form image feature pairs(106 in Fig. 1; 1806 in Fig. 18); and for each matched image feature pair,
(S4) extracting information of the respective keypoint from one of the two training images and information of the descriptor from the other of the two training( 112, 108 in Fig. 3),
(S5) reconstructing the one of the two training images, for which information of the respective keypoints of the matched image feature pairs was extracted, by the second deep neural network based on the extracted information of the respective keypoints and based on the extracted information of the respective descriptors of the matched image feature pairs ( 130, 132 in Fig. 3),
(S6) quantifying differences between the reconstructed training image and the provided one of the two training images using a cost function ( 134 in Fig. 3; 1808 in Fig. 18) , and
(S7) adjusting parameters of the first and/or the second deep neural network based on the quantified differences, for optimizing the cost function for providing a trained machine learning model ([0053], based on a loss signal/value that indicates some quantity that is to be minimized, and/or based on a correction signal that indicates a specific modification to be made to neural network 100. Modifying neural network 100 may include modifying only interest point detector subnetwork 112, modifying only descriptor subnetwork 114, and/or modifying both of subnetworks 112, 114; 1810 in Fig. 18).
Regarding claim 17, DeTone teaches the method according to claim 16, wherein, for providing the trained machine learning model, steps S2 to S7 are repeated at least partially iteratively until a termination criterion or a threshold value of the cost function is reached( [0053], neural network 100 may be modified based on an error signal/value that indicates a difference between an output and ground truth, based on a loss signal/value that indicates some quantity that is to be minimized).
Regarding claim 18, DeTone teaches the method according to claim 16, wherein the cost function ascertains an error between the reconstructed training image and the provided one of the two training images based on the quantified differences([0053], based on a loss signal/value that indicates some quantity that is to be minimized, and/or based on a correction signal that indicates a specific modification to be made to neural network 100. Modifying neural network 100 may include modifying only interest point detector subnetwork 112, modifying only descriptor subnetwork 114, and/or modifying both of subnetworks 112, 114; 1810 in Fig. 18).
Regarding claim 20, DeTone teaches the method according to claim 16, wherein the respective keypoints are set at the same or similar locations of the two training images( 108-1, 110-1, 108-2, 110-2 in Fig. 1).
Regarding claim 21, DeTone teaches the method according to claim 16, wherein the first deep neural network includes an encoder and the second deep neural network includes a decoder ( [0082], network design for dense prediction may involve an encoder-decoder pair).
Regarding claim 22, DeTone teaches the method according to claim 16, wherein a plurality of two training images, each representing image pairs of the same or similar image content, is provided( 108-1, 110-1, 108-2, 110-2 in Fig. 1).
Regarding claim 23, DeTone teaches the method according to claim 16, wherein the processing of the two training images by the first deep neural network further includes: calculating an image descriptor of the one of the two training images( 110-1, 110-2 in Fig. 1; 110 in Fig. 4); and wherein reconstructing the one of the two training images is also carried out based on the image descriptor associated with the one of the two training images( 144 in Fig. 4).
Regarding claim 24, DeTone teaches the method according to claim 16, wherein each pixel of the two training images defines a keypoint( 108-1, 108-2 in Fig. 1; 108 in Fig. 4) .
Claim 25 recites similar subject matter as in claims 1, thus also rejected.
Claim 26 recites the intended use for claim 25, thus also rejected.
Claims 27, 28 and 29 recite the apparatus, control device and medium for the method in claim 1. Since DeTone also teaches apparatus, device and medium for the method ( [0004], (AR) device, computer vision and image display systems).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over DeTone in view of Benhimane ( US 20160321811)
Regarding claim 19, DeTone teaches the method according to claim 18.
DeTone does not expressly teach wherein the cost function ascertains the error as a sum of pixel differences between the reconstructed training image and the provided one of the two training images as a photometric error.
However Benhimane teaches ascertains the error as a sum of pixel differences between the reconstructed training image and the provided one of the two training images as a photometric error( [0043], estimate of a parameter set x comprising the homography between the current image and the reference image… the photometric error y is computed using the data of the current image and the reference image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of DeTone and Benhimane, by using the homographic transformation in DeTone as photometric error as taught in Benhimane, with motivation for “determining the pose of a camera with regard to at least one real object and/or for determining a three-dimensional structure of the at least one real object” ( Benhimane, [0014]).
However, in the same field of endeavor
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to ***
One of ordinary skill would have been motivated to
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANGENG SUN whose telephone number is (571)272-3712. The examiner can normally be reached 8am to 5pm, EST, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Randolph Vincent can be reached at 571 272 8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIANGENG SUN
Examiner
Art Unit 2661
/Jiangeng Sun/Examiner, Art Unit 2671