DETAILED ACTION
This action is responsive to the Amendments and Remarks received 07/29/2026 in which claims 2, 5, 8, 9, and 13 are cancelled, claims 1, 3, 4, 6, 7, and 10–12 are amended, and no claims are added as new claims.
Response to Arguments
In view of the amendments, the rejection of claims 5 and 9 under 35 U.S.C. 112(a) for lack of enablement is withdrawn. Remarks, 6.
In view of the amendments, the rejection of claims 1–13 for being indefinite under 35 U.S.C. 112(b) is withdrawn. Remarks, 6–7.
On page 9 of the Remarks, Applicant contends the cited prior art fails to teach or suggest the features added by way of amendment drawn to generating ground truth data by transforming a detected object position using a deformation parameter. Examiner disagrees. As explained in the rejection, infra, the skilled artisan had in their possession at the time of Applicant’s filing affine and homography transforms (deformation transform parameter(s)) performed on a detected object in one image to relate its position to the same object in another perspective view image wherein the ground truth can be defined as the coordinates of a bounding box circumscribing the detected object and wherein the object is detected using a pretrained object detection model. Training a pretrained model on feedback data is a conventional approach to fine tuning a model based on training. Furthermore, training a pretrained model is really just stretching the definition of what it means to train a model. If so-called pretraining is not sufficient such that more training is required, then both the pretraining and re-training (post-training) is all just training. This type of semantic distinction is not a technological distinction to one skilled in the art. For all the foregoing reasons, Examiner is unpersuaded of patentability.
Other claims are not argued separately. Remarks, 10.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claim 6 is rejected under 35 U.S.C. 112(a) as failing to comply with the enablement requirement. The claims contain subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention. Specifically, claim 6 recites “so as to” achieve a desired result. Claiming a result rather than a series of steps or structures for how to achieve the result amounts to purely functional claiming covering all ways known and unknown for achieving the result. Such breadth of scope is not enabled by Applicant’s Specification which does not describe all ways of achieving the stated result. Aristocrat, 521 F.3d, 1334 (criticizing claim language that claims an outcome rather than the means for achieving the outcome). Sitrick v. Dreamworks, LLC, 516 F.3d 993, 999, 85 USPQ2d 1826, ____ (Fed. Cir. 2008) (“The scope of the claims must be less than or equal to the scope of the enablement to ensure that the public knowledge is enriched by the patent specification to a degree at least commensurate with the scope of the claims.”) (quotation omitted). Where the specification is deficient in describing every means for achieving the claimed results, the claims are not enabled. Interval Licensing LLC v. AOL, Inc., 896 F.3d 1335 (Fed. Cir. 2018) (claims that “simply demand the production of a desired result . . . without any limitation on how to produce that result” are directed to an abstract idea).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4, 6, 7, and 10–12 are rejected under 35 U.S.C. 103 as being unpatentable over Bylicka (US 2021/0112238 A1), Akiyama (US 2015/0278249 A1), Jiang (US 2021/0103776 A1), and Xin (US 2022/0036525 A1).
Regarding claim 1, the combination of Bylicka, Akiyama, Jiang, and Xin teaches or suggests a model generation apparatus comprising: at least one memory storing processing instructions; and at least one processor configured to execute the processing instructions to: detect an object in a first image using a pretrained object detection model, wherein the object is at a first position in the first image; detect the object in a second image using the pretrained object detection model, wherein: the object is at a second position in the second image, and the second image has a different angle of view of the object relative to the first image (Bylicka, ¶¶ 0031–0032: teaches multi-object multi-view association (MMVA) wherein association is performed between bounding boxes of detected objects in different 2D views and wherein feature points are used to initially perform object detection but are not used for object tracking in favor of a simpler approach of merely tracking the bounding boxes; see original claim 4 regarding why bounding boxes are interpreted to be part of an included embodiment for this claim); calculate a deformation parameter corresponding to the difference in the angles of view between the first and second images (Examiner notes the claimed deformation parameter is an affine parameter in the art; While homography modifies lines at infinity, thus introducing perspective warp (lines that are parallel in 3D can converge); Affine transformation keeps parallel lines parallel and thus the projective vector components are set to zero; Bylicka, ¶ 0027: teaches MMVA is accomplished using a homography transform; Akiyama, ¶ 0103: teaches correspondence between images calculated by matching feature points between images and relating them using a geometric transform, such as a homography or affine transform parameter); generate a transformed position in the second image by transforming the first position using the deformation parameter, wherein the transformed position in the second image corresponds to the first position (Examiner interprets this limitation consistent with Applicant’s published paragraph [0032], which explains the geometric deformation between two images having differing fields of view is reconciled to corresponding points using an affine transform; see also original claim 2; Bylicka, ¶ 0027: teaches MMVA is accomplished using a homography transform; While Bylicka teaches a homography transform, the skilled artisan knows the perspective deformation is a planar homographic transform which is locally tangent to affine transforms; This means that the perspective transform behaves exactly like an affine transformation at a specific point or locally and can be approximated by affine; Areas near the center of projection require less perspective correction and act affinely such that small projective deformations are treated as local affine warps; Akiyama, ¶ 0103: teaches correspondence between images calculated by matching feature points between images and relating them using a geometric transform, such as a homography or affine transform parameter); generate a ground truth position information for the second image based on the transformed position (Examiner interprets this limitation consistent with Applicant’s published paragraph [0032], which explains the geometric deformation between two images having differing fields of view is reconciled to corresponding points using an affine transform; see also original claim 2 and amended claim 6 wherein claim 6 explains the ground truth coordinate information is the position information of the bounding box of the detected object; Akiyama, ¶ 0103: teaches correspondence between images calculated by matching feature points between images and relating them using a geometric transform, such as a homography or affine transform parameter; see also Jiang, cited next); and additionally train the pretrained object detection model by updating parameters of the pretrained object detection model (Bylicka, ¶ 0030: teaches a trained machine learning model to re-identify detected objects across images; However, Bylicka does not appear to explicitly teach that which Jiang teaches regarding training an object detection model with the correspondence information; Jiang, ¶ 0005: teaches an object detection model trained with images of an object from multiple views wherein the training data includes correspondence data between the 2D images of the object), wherein the updated parameters decrease an error between the second position and the ground truth position information (Examiner interprets this claim as covering the scenario in which the feature point matching between off-angle images using a particular affine transform creates matches and outliers due to errors in matching feature points across images; Examiner finds that models fitting feature point correspondences between images is subject to some error due to the correspondences being “fitted” to the affine or homography model; In this field of endeavor, such a model fitting algorithm typically must deal with outliers; Xin, ¶¶ 0038, 0042, and 0044: teach removing outliers from matched feature points while fitting a geometric transform function to the feature point correspondences and further teaches that such approaches, for example RANSAC, are known in the art).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bylicka, with those of Akiyama, because both references are drawn to the same field of endeavor or drawn to solving the same or similar problem such that one wishing to practice mapping feature points across images using a geometric transform would be led to their relevant teachings and because, as Akiyama explains, an affine transform is an obvious substitute for a homography transform. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bylicka and Akiyama used in this Office Action unless otherwise noted.
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bylicka and Akiyama, with those of Jiang, because all three references are drawn to the same field of endeavor or drawn to solving the same or similar problem such that one wishing to practice object detection would be led to their relevant teachings and because, as Jiang explains, it is desirable to train an object detection model using images of the object from multiple views (Jiang, e.g. ¶ 0014). Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bylicka, Akiyama, and Jiang used in this Office Action unless otherwise noted.
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bylicka, Akiyama, and Jiang, with those of Xin, because all four references are drawn to the same field of endeavor or drawn to solving the same or similar problem such that one wishing to match feature points among images would be led to their relevant teachings and because, as Xin explains, it is desirable remove outliers from an image registration algorithm (Xin, e.g. ¶ 0042). Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bylicka, Akiyama, Jiang, and Xin used in this Office Action unless otherwise noted.
Regarding claim 3, the combination of Bylicka, Akiyama, Jiang, and Xin teaches or suggests the model generation apparatus according to claim 1, wherein the at least one processor is configured to execute the processing instructions to extract a first feature point within the first image and a second feature point within the second image corresponding to the first feature point, and calculate the deformation parameter based on the first feature point and the second feature point (Akiyama, ¶ 0103: teaches correspondence between images calculated by matching feature points between images and relating them using a geometric transform, such as a homography or affine transform parameter).
Regarding claim 4, the combination of Bylicka, Akiyama, Jiang, and Xin teaches or suggests the model generation apparatus according to claim 1, wherein the at least one processor is configured to execute the processing instructions to by deforming a box region corresponding to the first position using the deformation parameter, generate the transformed position and generate a box surrounding the transformed position as the ground truth position information for the second image (First, Examiner notes that because affine is locally equivalent (tangent) to homography, it is obvious to geometrically transform (deform) the detected object within its small bounding box rather than transform the whole frame; Akiyama, ¶ 0114: teaches utilizing as few as three feature point pairs, which one skilled in the art would find obvious to fit within a detected object’s bounding box; Furthermore, Bylicka teaches bounding box registration (tracking) within images rather than association of the entire frame; Bylicka, ¶¶ 0031–0032: teaches multi-object multi-view association (MMVA) wherein association is performed between bounding boxes of detected objects in different 2D views and wherein feature points are used to initially perform object detection but are not used for object tracking in favor of a simpler approach of merely tracking the bounding boxes).
Regarding claim 6, the combination of Bylicka, Akiyama, Jiang, and Xin teaches or suggests the model generation apparatus according to claim 4, wherein the at least one processor is configured to execute the processing instructions to additionally train the pretrained object detection model by updating parameters of the pretrained object detection model so as to decrease a loss based on an error between coordinates of the second position that is the position of the object within the second image detected using the pretrained object detection model and coordinates of the ground truth position information composed of a box surrounding a region of the transformed position (see treatment of claim 5 regarding the training using the error between correspondences; Bylicka, ¶¶ 0031–0032: teaches multi-object multi-view association (MMVA) wherein association is performed between bounding boxes of detected objects in different 2D views and wherein feature points are used to initially perform object detection but are not used for object tracking (object correspondence across views) in favor of a simpler approach of merely tracking (finding correspondences between) the bounding box’s position; Jiang, ¶ 0005: teaches an object detection model trained with images of an object from multiple views wherein the training data includes correspondence data between the 2D images of the object).
Claim 7 lists the same elements as claim 1, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 1 applies to the instant claim.
Claim 10 lists the same elements as claim 1, but in CRM form rather than apparatus form. Therefore, the rationale for the rejection of claim 1 applies to the instant claim.
Claim 11 lists the same elements as claim 3, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 3 applies to the instant claim.
Claim 12 lists the same elements as claim 4, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 4 applies to the instant claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sundaresan (US 2019/0114804 A1) teaches object detection within a first image applied to a second image for tracking the object using a trained neural network wherein the object tracker adjusts the location of a bounding box from the first image to the second image and wherein the object is tracked by applying both the first and second object trackers across the first and second images (e.g. Abstract) and teaches bounding boxes geometrically transformed (i.e. tracked) from a first image to a second image (e.g. ¶ 0010).
Pathangay (US 2018/0232584 A1) teaches tracking bounding boxes based on feature points and Scale-invariant feature transform (SIFT) (e.g. ¶ 0025).
Afrasiabi (US 2023/0334671 A1) teaches providing different views of an object from different angles for training more accurate and more robust object detection models (¶ 0029).
Tanii (US 11,640,709 B2) teaches using an affine transform to geometrically transform image coordinates from a first image onto a second image based on coordinates from a specified pair of feature points within the images to create a front-facing image (e.g. claim 1).
Davis (US 10,262,356 B2) teaches affine distortion and translation determination between two images (e.g. a captured image and a reference image) by identifying coordinates of three feature points in one image using feature point detection methods like corner point detection methods wherein point correspondences are determined between the two images for the purposes of dewarping and re-scaling to remove affine distortion and that such tools are common in image processing toolkits including warpAffine in OpenCV.
Rhoads (US 2015/0016712 A1) teaches object recognition using matching against an affine transform of a frontal view of the object (¶ 0035) and improving the frontal-affine matching approach utilizing a model generated from two images (¶ 0036).
Pinel (US 2022/0027599 A1) teaches object detection technology utilizes training datasets including 2D images of objects from multiple views (Abstract).
Savvides (US 12,322,012 B2) teaches pose-correction by generating a more frontal view of the object by applying planar homography by identifying feature points of the object and repositioning the points to a more frontal view and inputting the more frontal view into an object detector to generate a bounding box in a retail setting (Abstract).
Zou (US 11,842,321 B1) teaches planogram data updated to track product inventory on store shelves using computed correspondences of imaged objects between first and second cameras (e.g. Fig. 7) and rectifying the images to be matching pairs using a homography, utilizing bounding boxes, and utilizing a trained classifier.
Singh (US 12,288,294 B2) teaches, in a planogram or inventory management context, rectifying and undistorting camera images, finding point correspondences between a pair of cameras (col. 13, ll. 27–44), transforming between corresponding points to map points across images (col. 14, ll. 8–35).
Sommerfelt (US 2024/0257376 A1) teaches mapping corresponding feature points using an affine transform rather than a homography transform (Sommerfelt, ¶ 0089).
Javan Roshtkhari (US 2020/0372679 A1) teaches AI learning techniques to obtaining a homography between feature point correspondences between two images which is finding a geometric transform that minimizes the projection error (¶ 0007).
Stout (US 2024/0185435 A1) teaches a trained object detection model that identifies objects based on one or more views of an object captured by cameras at different angles (¶ 0050).
Pinel (US 2022/0027599 A1) teaches, in a stock monitoring or checkout-less shopping context, computer vision utilized for object detection and accounting for differences in viewing angle between training images and detected objects (e.g. ¶ 0013).
Ding (US 2015/0317821 A1) teaches matching feature points from one image to another representing different view angles (¶ 0098) and application of Affine SIFT to match feature points from a first image to a second image (¶ 0123).
Pang et al., “Fully affine invariant SURF for image matching,” Neurocomputing 85 (2012). This publication teaches, inter alia, ASIFT.
Yu et al., “ASIFT: An Algorithm for Fully Affine Invariant Comparison,” Image Processing On Line, 1 (2011), pp. 11–38.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael J Hess whose telephone number is (571)270-7933. The examiner can normally be reached Mon - Fri 9:00am-5:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8933.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J HESS/Examiner, Art Unit 2481
Prosecution Insights