DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/08/2026 has been entered.
Response to Amendments
The amendments to claims 1, 4, 11, 14, and 20 are accepted and entered.
The amendment to the Specification has been accepted and entered.
Claims 1-20 are pending regarding this application.
Response to Arguments
Applicant’s arguments, see Remarks, filed 04/14/2026, with respect to the 112(b) rejection applied to claims 4 and 14, have been fully considered and are persuasive. The 112(b) rejection of claims 4 and 14 have been withdrawn.
Applicant’s arguments, see Remarks, filed 04/14/2026, with respect to the objection applied to the specification, have been fully considered and are persuasive. The specification objection has been withdrawn.
Applicant's arguments filed 04/14/2026 have been fully considered but they are not persuasive. Applicant argues that allowable subject matter from claims 4 and 14 have been rewritten in independent form including all of the limitations of the base claim and any intervening claims. However, in the Final Rejection filed 01/21/2026, the following limitation(s) in claims 4 and 14 were indicated as allowable subject matter: “selecting a second set of points from the first template; computing a second transform based at least in part on the second set of selected points and the at least one reference image; applying the nonlinear algorithm to determine a second error associated with the first transform; and if the second error is lower than the first error, setting the image transform as the second transform”. However, none of these limitations were included in the amendment to claims 1, 11, or 20. Therefore, claims 1, 11, and 20 do not include any of the allowable subject matter as defined in the Final Rejection filed on 01/21/2026. Additionally, since the aforementioned allowable subject matter has been amended in the most recent claim set (04/14/2026), it no longer contains the allowable subject matter defined in the Final Rejection filed on 01/21/2026.
The rejection of claims 3, 5-10, 13, and 15-19 have not been argued. As a result, the rejections of these claims have not been altered.
Claim Objections
Claims 1, 11, and 20 are objected to because of the following informalities:
Regarding claim 1, in line 18, replace the word “an” with “the”.
Regarding claim 11, in line 23, replace the word “an” with “the”.
Regarding claim 20, in line 20, replace the word “an” with “the”.
As a result, the claims will be analyzed below assuming these changes were made. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, 10, 11-14, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Rasco et. al. (U.S. Publication No. 20160379388 A1), hereinafter Rasco in view of Mayle et al. (U.S. Publication No. 2016/0012317 A1), hereinafter Mayle and Li et al. (CN 109919984 A, see attached English translation), hereinafter Li.
Regarding claim 1, Rasco teaches a method for image georegistration (Rasco teaches a “method” for extracting structures from satellite imagery data in para. [0061]), the method comprising:
receiving an input image (Rasco teaches receiving input satellite data in para. [0061]);
generating a plurality of templates from the input image, the plurality of templates being associated with a plurality of template scores (Rasco teaches “decomposing, using a processor, at least one input satellite image into a plurality of components (e.g., pixels or groups of pixels) of a first hierarchical data structure (e.g., a space-partitioning data structure for organizing data points, such as a MaxTree)” in para. [0061]; the components here are interpreted as the plurality of templates in the claim language; ***see also applicant’s description of templates as portions of the input image in para. [0039]; Rasco also teaches that the first hierarchical data structure can be weighted (positive or negative) in para. [0121], these weights are interpreted as equivalent to the scores in the claim language. See also para. [0138] which lists different ways the hierarchical data structure can be determined);
generating a template queue based at least in part on the plurality of template scores, the template queue including a set of selected templates (Rasco teaches a first hierarchical data structure in which the components are prioritized based on certain physical aspects, along with crowdsourced data, in para. [0138] and [0061]; the data structure is interpreted as the template queue, and the data structure is based on positive/negative weights as shown in para. [0120]-[0122]. These weights are interpreted as the template scores; ***see applicant’s specification in which they describe the queue as being a data structure in para. [0042]. See also below citation), the generating a template queue including at least:
generating the template queue by selecting the set of selected templates from the plurality of templates based at least in part on the plurality of template scores (Rasco teaches a first hierarchical data structure (template queue) based on categorizing components (templates) of structures of interest and non structures of interest as shown in para. [0061] and [0120]. Here, Rasco states that “at least some components of the first hierarchical data structure” are defined as structures of interest or non-structures of interest (emphasis added) in para. [0120], which directly implies some of the components are not defined as either a structure of interest or a non-structure of interest. these structures/components (templates) “may be appropriately weighted to indicate the relative degree to which corresponding portions of the one or more reference satellite image data sets do or do not indicate urban or built-up areas” as shown in para. [0121]. As a result, the first data structure (template queue) used for training is generated by selecting ONLY CERTAIN components (templates) of the data structure based on the template weight (i.e. deciding whether a component is negative or positive). To summarize, the categorizing step 512 involves categorizing certain components as positive or negative based on a comparison between the reference and the input image (template score) and only utilizing the positive/negative components in the training step. See also para. [0122] where a subgroup of the components are used to train the second hierarchical data structure);
receiving one or more reference images (Rasco teaches a reference data set in para. [0068]. See also the feature elements of the second hierarchical data set as taught in para. [0064] which can be interpreted as reference data);
applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Rasco teaches a process of comparing the reference and the input image components in para. [0067-0068] and [0130] this process is interpreted as the matching algorithm in the claim language; the weights described in para. [0068] are interpreted as the match scores. Additionally, Rasco teaches matching the components (templates) of the first hierarchical data structure in para. [0120] and [0064] with the feature elements (reference data) of the second hierarchical data structure in para. [0064] and [0126]);
select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Rasco teaches that, “only those pixels of each of the successive images that are determined to be the “best” (e.g., as determined by the ascertained error rates) may be included in the resultant image” in para. [0069] and [0130]; here, the selected pixels are interpreted as the templates in the collection of templates. Rasco additionally teaches only assigning a component as a “built-up” structure when it “ha[s] a value within some percentage of the value of a feature element from the trained second hierarchical data structure” as shown in para. [0064] and para. [0126]. Here, the end result is a collection of components (templates) which fall into the category of being a built up structure); and
generating an image transform based at least in part on the collection of templates (Rasco teaches that “built-up structure components may be extracted from successive input images of the geographic area… and mapped into the resultant image to allow for an iterative refinement of the resultant image over a plurality of cycles” in para. [0067] and [0131]; this refinement process using the “built-up structures that are associated with error rates below a threshold error rate into a resultant image” is interpreted as equivalent to the image transform based on the collection of templates as claimed in the claim language. See also Applicant’s description of image transform as a reduction of error in para. [0030] of Applicant’s specification);
wherein the method is performed using one or more processors (Rasco teaches the use of processors to carry out the above mapped information in para. [0070]).
Rasco fails to specifically teach, beyond what is shown in the mapping above, determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image; applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Mayle teaches determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Mayle teaches that “feature point matching may also be used to see if a template is found in an arbitrary image. A template may be a real or an artificial image that expresses a pattern to be found in the image” in para. [0056]; Mayle further teaches that a template image may be treated as multiple templates in para. [0121]; Mayle teaches “third and fourth steps [which] match (or attempt to match) feature points from one image to feature points from another image” in para. [0071]; this process is carried out by “computing the nearest neighbors of each descriptor from a first image to descriptors in a second image” wherein “Lowe describes a ratio test that computes the ratio of the smallest distance from a keypoint in a first image to a keypoint in a second image” in para. [0071-0072]; here, the keypoints are interpreted as the multitude of templates as the keypoints represent portions of the template/input image; additionally, this ratio is interpreted as the match score as shown in the claim language);
evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Mayle teaches that “a large ratio (Lowe used a threshold of 0.8) may be used to indicate that two keypoints in the second image are similar to the keypoint in the first image. When this condition arises, there is no matching keypoint in the second image to the keypoint in the first image. This process is carried out by comparing every keypoint in one image to the keypoints in the second image” in para. [0072]; here, the threshold is interpreted as the selection criteria in the claim language); and
generating an image transform based at least in part on the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]);
wherein the generating an image transform comprises:
selecting a first set of points from a first template in the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates]” in para. [0077]);
computing a first transform based at least in part on the first set of selected points and the at least one reference image (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]. See also para. [0081] which states, for example, “a valid homography is used to establish the coordinate system relationship between points in the template image with the points in the test image”).
Mayle and Rasco are both considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco to incorporate the teachings of Mayle and “determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image”. The motivation for doing so would have been to “eliminate candidate matches via checks of coverage, homography, reprojection, number of match points, determinant, SVD ratio”, as suggested by Mayle in para. [0087]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco with Mayle to obtain the invention specified in the above limitations.
Rasco and Mayle fail to teach applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Li teaches applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform (Li teaches “obtaining a rigid body transformation matrix of point cloud rough matching by using a sampling consistency algorithm” in para. [0072] and “evaluating the quality of the rough registration rigid body transformation matrix by using an error measurement loss function for the preliminarily estimated corresponding point pairs” as shown in para. [0077]-[0078]. Here, the rough registration rigid body transformation matrix is interpreted as equivalent to the first transform, and the quality of the transform is evaluated by using an error metric loss function and a Levenberg-Marquardt algorithm. This function error metric loss function is interpreted as equivalent to the non-linear algorithm).
Mayle, Rasco, and Li are all considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle) to incorporate the teachings of Li and include “applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform”. The motivation for doing so would have been that “the method can obtain initial matching point pairs based on a local fast point feature histogram descriptor and a sampling consistency algorithm, obtain a coarse registration matrix through an error measurement loss function, and then obtain a fine registration rigid body transformation matrix by combining an iterative closest point algorithm, so that the problem that three-dimensional point cloud cannot be automatically registered under different viewing angles can be effectively solved”, as suggested by Li in para. [0045]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco and Mayle with Li to obtain the invention specified in claim 1.
Regarding claim 2, Rasco, Mayle, and Li teach the method of claim 1,
wherein each template of the plurality of templates is associated with a pixel location in the input image (Rasco teaches wherein the components are pixels or groups of pixels in para. [0061], and the components are based on dissimilarity between adjacent nodes in para. [0138]); Rasco further teaches the nodes of the second hierarchical tree being associated with two-dimensional space (location) in para. [0117], which is used to classify the components (templates) in the first hierarchical data structure as shown in para. [0064])),
wherein the generating a template queue based at least in part on the plurality of template scores comprises:
generating the template queue based at least in part on a plurality of pixel locations associated with the plurality of templates (Rasco teaches that the components (templates) are based on groups of pixels, which inherently implies the queues involve pixel locations, since pixel locations must be known in order to determine specific groups of pixels to define as components (see para. [0121]). Since the components involve pixel locations, and the components are analyzed based on pixel similarity (see para. [0126]), it is inherent, that the template queue is also based at least in part on pixel locations associated with the templates).
Regarding claim 3, Rasco, Mayle, and Li teach the method of claim 2, wherein the generating a template queue based at least in part on the plurality of template scores comprises:
generating a data structure including a plurality of template points corresponding to the plurality of pixel locations of the plurality of templates (Rasco teaches a first hierarchical data structure in para. [0064] which consists of components (pixels), which is based on a trained second hierarchical data structure (para. [0064]), and nodes (pixels) rooted in two-dimensional space in para. [0117]; since the pixels are rooted in two-dimensional space, it is inferred that there is a location associated with them); and
selecting the set of selected templates using the data structure (Rasco teaches that the second hierarchical data structure aids in the analysis of the first hierarchical data structure in which the components (templates) of the first hierarchical data are “selected” using a comparative analysis of both the first and second hierarchical data structure to classify/tag the “built up” structures in para. [0064]; here, the selected components with the “built-up” structure are interpreted as the selected templates. See also para. [0066] and [0067]).
Regarding claim 4, Rasco, Mayle, and Li teach the method of claim 1, wherein the generating an image transform further comprises:
selecting a second set of points from the first template (Li teaches “repeating the step S303 until an optimal measurement error result is achieved” in para. [0077] wherein step 303 involves selecting a set of points. Here, since this process occurs repeatedly, it is inherent that there exists at least a second set of points chosen from the first template, wherein the first template is defined in para. [0075]) (See also both Rasco and Mayle’s teaching of the template in claim 1);
computing a second transform based at least in part on the second set of selected points and the at least one reference image (Li teaches repeatedly “randomly selecting a corresponding relation representing sampling points from the points, and preliminarily estimating the points to be corresponding points” in para. [0076] (S303). Here, since, as shown in para. [0077], the process outlined in para. [0076] occurs repeatedly until a criteria is met, it is inherent that there exists at least a second transform, wherein the second transform is interpreted as the corresponding relation (“the rough registration rigid body transformation matrix”);
applying the nonlinear algorithm to determine a second error associated with the second transform (Li teaches repeatedly “evaluating the quality of the rough registration rigid body transformation matrix by using an error measurement loss function for the preliminarily estimated corresponding point pairs” in para. [0077] (S304). Here, since, as shown in para. [0077], the error is calculated repeatedly until a criteria is met, it is inherent that there exists at least a second error for the second transform); and
if the second error is lower than the first error, setting the image transform as the second transform (Since Li teaches that the error is calculated until an optimal measurement error result is met, is implied that, if the second error is lower than the first error, and the second error exhibits an optimal measurement error result, the second transform replaces the first transform as the “coarse registration rigid body transformation matrix” as shown in para. [0077]-[0079]).
Regarding claim 7, Rasco, Mayle, and Li teach the method of claim 1,
wherein the selection criteria include a match score being higher than a predetermined threshold (Mayle teaches “a large ratio (Lowe used a threshold of 0.8) may be used to indicate that two keypoints in the second image are similar to the keypoint in the first image” in para. [0068]; the threshold here is interpreted as equivalent to the selection criteria). Similar motivations as applied to claim 1 can be applied here to claim 7.
Regarding claim 10, Rasco, Mayle, and Li teach the method of claim 1,
wherein the input image is an image taken from a satellite (Rasco teaches receiving input satellite data in para. [0061]).
Regarding claim 11, Rasco teaches a system for image georegistration (Rasco teaches “a system for combining geographical and economic data extracted from satellite imagery” in para. [0010]), the system comprising:
one or more memories having instructions stored thereon (Rasco teaches memory in para. [0083]); and
one or more processors configured to execute the instructions and perform operations (Rasco teaches processors in para. [0083-0086]) comprise:
receiving an input image (Rasco teaches receiving input satellite data in para. [0061]);
generating a plurality of templates from the input image, the plurality of templates being associated with a plurality of template scores (Rasco teaches “decomposing, using a processor, at least one input satellite image into a plurality of components (e.g., pixels or groups of pixels) of a first hierarchical data structure (e.g., a space-partitioning data structure for organizing data points, such as a MaxTree)” in para. [0061]; the components here are interpreted as the plurality of templates in the claim language; ***see also applicant’s description of templates as portions of the input image in para. [0039]; Rasco also teaches that the first hierarchical data structure can be weighted (positive or negative) in para. [0121], these weights are interpreted as equivalent to the scores in the claim language. See also para. [0138] which lists different ways the hierarchical data structure can be determined);
generating a template queue based at least in part on the plurality of template scores, the template queue including a set of selected templates (Rasco teaches a first hierarchical data structure in which the components are prioritized based on certain physical aspects, along with crowdsourced data, in para. [0138] and [0061]; the data structure is interpreted as the template queue, and the data structure is based on positive/negative weights as shown in para. [0120]-[0122]. These weights are interpreted as the template scores; ***see applicant’s specification in which they describe the queue as being a data structure in para. [0042]. See also below citation), the generating a template queue including at least:
generating the template queue by selecting the set of selected templates from the plurality of templates based at least in part on the plurality of template scores (Rasco teaches a first hierarchical data structure (template queue) based on categorizing structures of interest and non structures of interest as shown in para. [0061] and [0120]. Here, Rasco states that “at least some components of the first hierarchical data structure” are defined as structures of interest or non-structures of interest (emphasis added) in para. [0120], which directly implies some of the components are not defined as either a structure of interest or a non-structure of interest. these structures/components (templates) “may be appropriately weighted to indicate the relative degree to which corresponding portions of the one or more reference satellite image data sets do or do not indicate urban or built-up areas” as shown in para. [0121]. As a result, the first data structure (template queue) used for training is generated by selecting ONLY CERTAIN components (templates) of the data structure based on the template weight (i.e. deciding whether a component is negative or positive). To summarize, the categorizing step 512 involves categorizing certain components as positive or negative based on a comparison between the reference and the input image (template score) and only utilizing the positive/negative components in the training step. See also para. [0122] where a subgroup of the components are used to train the second hierarchical data structure);
receiving one or more reference images (Rasco teaches a reference data set in para. [0068]. See also the feature elements of the second hierarchical data set as taught in para. [0064] which can be interpreted as reference data);
applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Rasco teaches a process of comparing the reference and the input image components in para. [0067-0068] and [0130] this process is interpreted as the matching algorithm in the claim language; the weights described in para. [0068] are interpreted as the match scores. Additionally, Rasco teaches matching the components (templates) of the first hierarchical data structure in para. [0120] and [0064] with the feature elements (reference data) of the second hierarchical data structure in para. [0064] and [0126]);
select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Rasco teaches that, “only those pixels of each of the successive images that are determined to be the “best” (e.g., as determined by the ascertained error rates) may be included in the resultant image” in para. [0069] and [0130]; here, the selected pixels are interpreted as the templates in the collection of templates. Rasco additionally teaches only assigning a component as a “built-up” structure when it “ha[s] a value within some percentage of the value of a feature element from the trained second hierarchical data structure” as shown in para. [0064] and para. [0126]. Here, the end result is a collection of components (templates) which fall into the category of being a built up structure); and
generating an image transform based at least in part on the collection of templates (Rasco teaches that “built-up structure components may be extracted from successive input images of the geographic area… and mapped into the resultant image to allow for an iterative refinement of the resultant image over a plurality of cycles” in para. [0067] and [0131]; this refinement process using the “built-up structures that are associated with error rates below a threshold error rate into a resultant image” is interpreted as equivalent to the image transform based on the collection of templates as claimed in the claim language. See also Applicant’s description of image transform as a reduction of error in para. [0030] of Applicant’s specification);
wherein the method is performed using one or more processors (Rasco teaches the use of processors to carry out the above mapped information in para. [0070]).
Rasco fails to specifically teach, beyond what is shown in the mapping above, determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image; applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Mayle teaches determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Mayle teaches that “feature point matching may also be used to see if a template is found in an arbitrary image. A template may be a real or an artificial image that expresses a pattern to be found in the image” in para. [0056]; Mayle further teaches that a template image may be treated as multiple templates in para. [0121]; Mayle teaches “third and fourth steps [which] match (or attempt to match) feature points from one image to feature points from another image” in para. [0071]; this process is carried out by “computing the nearest neighbors of each descriptor from a first image to descriptors in a second image” wherein “Lowe describes a ratio test that computes the ratio of the smallest distance from a keypoint in a first image to a keypoint in a second image” in para. [0071-0072]; here, the keypoints are interpreted as the multitude of templates as the keypoints represent portions of the template/input image; additionally, this ratio is interpreted as the match score as shown in the claim language);
evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Mayle teaches that “a large ratio (Lowe used a threshold of 0.8) may be used to indicate that two keypoints in the second image are similar to the keypoint in the first image. When this condition arises, there is no matching keypoint in the second image to the keypoint in the first image. This process is carried out by comparing every keypoint in one image to the keypoints in the second image” in para. [0072]; here, the threshold is interpreted as the selection criteria in the claim language); and
generating an image transform based at least in part on the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]);
wherein the generating an image transform comprises:
selecting a first set of points from a first template in the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates]” in para. [0077]);
computing a first transform based at least in part on the first set of selected points and the at least one reference image (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]. See also para. [0081] which states, for example, “a valid homography is used to establish the coordinate system relationship between points in the template image with the points in the test image”).
Mayle and Rasco are both considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco to incorporate the teachings of Mayle and “determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image”. The motivation for doing so would have been to “eliminate candidate matches via checks of coverage, homography, reprojection, number of match points, determinant, SVD ratio”, as suggested by Mayle in para. [0087]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco with Mayle to obtain the invention specified in the above limitations.
Rasco and Mayle fail to teach applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Li teaches applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform (Li teaches “obtaining a rigid body transformation matrix of point cloud rough matching by using a sampling consistency algorithm” in para. [0072] and “evaluating the quality of the rough registration rigid body transformation matrix by using an error measurement loss function for the preliminarily estimated corresponding point pairs” as shown in para. [0077]-[0078]. Here, the rough registration rigid body transformation matrix is interpreted as equivalent to the first transform, and the quality of the transform is evaluated by using an error metric loss function and a Levenberg-Marquardt algorithm. This function error metric loss function is interpreted as equivalent to the non-linear algorithm).
Mayle, Rasco, and Li are all considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle) to incorporate the teachings of Li and include “applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform”. The motivation for doing so would have been that “the method can obtain initial matching point pairs based on a local fast point feature histogram descriptor and a sampling consistency algorithm, obtain a coarse registration matrix through an error measurement loss function, and then obtain a fine registration rigid body transformation matrix by combining an iterative closest point algorithm, so that the problem that three-dimensional point cloud cannot be automatically registered under different viewing angles can be effectively solved”, as suggested by Li in para. [0045]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco and Mayle with Li to obtain the invention specified in claim 11.
Regarding claim 12, Rasco, Mayle, and Li teach the system of claim 11,
wherein each template of the plurality of templates is associated with a pixel location in the input image (Rasco teaches wherein the components are pixels or groups of pixels in para. [0061], and the components are based on dissimilarity between adjacent nodes in para. [0138]); Rasco further teaches the nodes of the second hierarchical tree being associated with two-dimensional space (location) in para. [0117], which is used to classify the components (templates) in the first hierarchical data structure as shown in para. [0064])),
wherein the generating a template queue based at least in part on the plurality of template scores comprises:
generating the template queue based at least in part on a plurality of pixel locations associated with the plurality of templates (Rasco teaches that the components (templates) are based on groups of pixels, which inherently implies the queues involve pixel locations, since pixel locations must be known in order to determine specific groups of pixels to define as components (see para. [0121]). Since the components involve pixel locations, and the components are analyzed based on pixel similarity (see para. [0126]), it is inherent, that the template queue is also based at least in part on pixel locations associated with the templates).
Regarding claim 13, Rasco, Mayle, and Li teach the system of claim 12, wherein the generating a template queue based at least in part on the plurality of template scores comprises:
generating a data structure including a plurality of template points corresponding to the plurality of pixel locations of the plurality of templates (Rasco teaches a first hierarchical data structure in para. [0064] which consists of components (pixels), which is based on a trained second hierarchical data structure (para. [0064]), and nodes (pixels) rooted in two-dimensional space in para. [0117]); and
selecting the set of selected templates using the data structure (Rasco teaches that the second hierarchical data structure aids in the analysis of the first hierarchical data structure in which the components (templates) of the first hierarchical data are “selected” using a comparative analysis of both the first and second hierarchical data structure to classify/tag the “built up” structures in para. [0064]; here, the selected components with the “built-up” structure are interpreted as the selected templates. See also para. [0066] and [0067]).
Regarding claim 14, Rasco, Mayle, and Li teach the system of claim 11, wherein the generating an image transform further comprises:
selecting a second set of points from the first template (Li teaches “repeating the step S303 until an optimal measurement error result is achieved” in para. [0077] wherein step 303 involves selecting a set of points. Here, since this process occurs repeatedly, it is inherent that there exists at least a second set of points chosen from the first template, wherein the first template is defined in para. [0075]) (See also both Rasco and Mayle’s teaching of the template in claim 1);
computing a second transform based at least in part on the second set of selected points and the at least one reference image (Li teaches repeatedly “randomly selecting a corresponding relation representing sampling points from the points, and preliminarily estimating the points to be corresponding points” in para. [0076] (S303). Here, since, as shown in para. [0077], the process outlined in para. [0076] occurs repeatedly until a criteria is met, it is inherent that there exists at least a second transform, wherein the second transform is interpreted as the corresponding relation (“the rough registration rigid body transformation matrix”);
applying the nonlinear algorithm to determine a second error associated with the second transform (Li teaches repeatedly “evaluating the quality of the rough registration rigid body transformation matrix by using an error measurement loss function for the preliminarily estimated corresponding point pairs” in para. [0077] (S304). Here, since, as shown in para. [0077], the error is calculated repeatedly until a criteria is met, it is inherent that there exists at least a second error for the second transform); and
if the second error is lower than the first error, setting the image transform as the second transform (Since Li teaches that the error is calculated until an optimal measurement error result is met, is implied that, if the second error is lower than the first error, and the second error exhibits an optimal measurement error result, the second transform replaces the first transform as the “coarse registration rigid body transformation matrix” as shown in para. [0077]-[0079]).
Regarding claim 17, Rasco, Mayle, and Li teach the system of claim 11,
wherein the selection criteria include a match score being higher than a predetermined threshold (Mayle teaches “a large ratio (Lowe used a threshold of 0.8) may be used to indicate that two keypoints in the second image are similar to the keypoint in the first image” in para. [0068]; the threshold here is interpreted as equivalent to the selection criteria). Similar motivations as applied to claim 11 can be applied here to claim 17.
Regarding claim 20, Rasco teaches a method for image georegistration (Rasco teaches a “method” for extracting structures from satellite imagery data in para. [0061]), the method comprising:
receiving an input image (Rasco teaches receiving input satellite data in para. [0061]);
generating a plurality of templates from the input image, the plurality of templates being associated with a plurality of template scores (Rasco teaches “decomposing, using a processor, at least one input satellite image into a plurality of components (e.g., pixels or groups of pixels) of a first hierarchical data structure (e.g., a space-partitioning data structure for organizing data points, such as a MaxTree)” in para. [0061]; the components here are interpreted as the plurality of templates in the claim language; ***see also applicant’s description of templates as portions of the input image in para. [0039]; Rasco also teaches that the first hierarchical data structure can be weighted (positive or negative) in para. [0121], these weights are interpreted as equivalent to the scores in the claim language. See also para. [0138] which lists different ways the hierarchical data structure can be determined);
generating a template queue based at least in part on the plurality of template scores, the template queue including a set of selected templates (Rasco teaches a first hierarchical data structure in which the components are prioritized based on certain physical aspects, along with crowdsourced data, in para. [0138] and [0061]; the data structure is interpreted as the template queue, and the data structure is based on positive/negative weights as shown in para. [0120]-[0122]. These weights are interpreted as the template scores; ***see applicant’s specification in which they describe the queue as being a data structure in para. [0042]. See also below citation), the generating a template queue including at least:
generating the template queue by selecting the set of selected templates from the plurality of templates based at least in part on the plurality of template scores (Rasco teaches a first hierarchical data structure (template queue) based on categorizing structures of interest and non structures of interest as shown in para. [0061] and [0120]. Here, Rasco states that “at least some components of the first hierarchical data structure” are defined as structures of interest or non-structures of interest (emphasis added) in para. [0120], which directly implies some of the components are not defined as either a structure of interest or a non-structure of interest. these structures/components (templates) “may be appropriately weighted to indicate the relative degree to which corresponding portions of the one or more reference satellite image data sets do or do not indicate urban or built-up areas” as shown in para. [0121]. As a result, the first data structure (template queue) used for training is generated by selecting ONLY CERTAIN components (templates) of the data structure based on the template weight (i.e. deciding whether a component is negative or positive). To summarize, the categorizing step 512 involves categorizing certain components as positive or negative based on a comparison between the reference and the input image (template score) and only utilizing the positive/negative components in the training step. See also para. [0122] where a subgroup of the components are used to train the second hierarchical data structure);
receiving one or more reference images (Rasco teaches a reference data set in para. [0068]. See also the feature elements of the second hierarchical data set as taught in para. [0064] which can be interpreted as reference data);
applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Rasco teaches a process of comparing the reference and the input image components in para. [0067-0068] and [0130] this process is interpreted as the matching algorithm in the claim language; the weights described in para. [0068] are interpreted as the match scores. Additionally, Rasco teaches matching the components (templates) of the first hierarchical data structure in para. [0120] and [0064] with the feature elements (reference data) of the second hierarchical data structure in para. [0064] and [0126]);
select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Rasco teaches that, “only those pixels of each of the successive images that are determined to be the “best” (e.g., as determined by the ascertained error rates) may be included in the resultant image” in para. [0069] and [0130]; here, the selected pixels are interpreted as the templates in the collection of templates. Rasco additionally teaches only assigning a component as a “built-up” structure when it “ha[s] a value within some percentage of the value of a feature element from the trained second hierarchical data structure” as shown in para. [0064] and para. [0126]. Here, the end result is a collection of components (templates) which fall into the category of being a built up structure); and
generating an image transform based at least in part on the collection of templates (Rasco teaches that “built-up structure components may be extracted from successive input images of the geographic area… and mapped into the resultant image to allow for an iterative refinement of the resultant image over a plurality of cycles” in para. [0067] and [0131]; this refinement process using the “built-up structures that are associated with error rates below a threshold error rate into a resultant image” is interpreted as equivalent to the image transform based on the collection of templates as claimed in the claim language. See also Applicant’s description of image transform as a reduction of error in para. [0030] of Applicant’s specification);
wherein the method is performed using one or more processors (Rasco teaches the use of processors to carry out the above mapped information in para. [0070]).
Rasco fails to specifically teach, beyond what is shown in the mapping above, determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image; applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Mayle teaches determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images (Mayle teaches that “feature point matching may also be used to see if a template is found in an arbitrary image. A template may be a real or an artificial image that expresses a pattern to be found in the image” in para. [0056]; Mayle further teaches that a template image may be treated as multiple templates in para. [0121]; Mayle teaches “third and fourth steps [which] match (or attempt to match) feature points from one image to feature points from another image” in para. [0071]; this process is carried out by “computing the nearest neighbors of each descriptor from a first image to descriptors in a second image” wherein “Lowe describes a ratio test that computes the ratio of the smallest distance from a keypoint in a first image to a keypoint in a second image” in para. [0071-0072]; here, the keypoints are interpreted as the multitude of templates as the keypoints represent portions of the template/input image; additionally, this ratio is interpreted as the match score as shown in the claim language);
evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria (Mayle teaches that “a large ratio (Lowe used a threshold of 0.8) may be used to indicate that two keypoints in the second image are similar to the keypoint in the first image. When this condition arises, there is no matching keypoint in the second image to the keypoint in the first image. This process is carried out by comparing every keypoint in one image to the keypoints in the second image” in para. [0072]; here, the threshold is interpreted as the selection criteria in the claim language); and
generating an image transform based at least in part on the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]);
wherein the generating an image transform comprises:
selecting a first set of points from a first template in the collection of templates (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates]” in para. [0077]);
computing a first transform based at least in part on the first set of selected points and the at least one reference image (Mayle teaches a step which “takes as input the keypoints from the template image [collection of templates] that match the keypoints in the test image, and computes a geometric relationship between the points in each image” in para. [0077]; here, the geometric relationship can be determined through multiple different types of transformation as further shown in para. [0077]. See also para. [0081] which states, for example, “a valid homography is used to establish the coordinate system relationship between points in the template image with the points in the test image”).
Mayle and Rasco are both considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco to incorporate the teachings of Mayle and “determining a set of match scores for the set of selected templates by applying a matching algorithm to the set of selected templates and at least one reference image of the one or more reference images; evaluating the set of match scores to select a collection of templates, each template of the collection of templates meeting one or more selection criteria; and generating an image transform based at least in part on the collection of templates; wherein the generating an image transform comprises: selecting a first set of points from a first template in the collection of templates; computing a first transform based at least in part on the first set of selected points and the at least one reference image”. The motivation for doing so would have been to “eliminate candidate matches via checks of coverage, homography, reprojection, number of match points, determinant, SVD ratio”, as suggested by Mayle in para. [0087]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco with Mayle to obtain the invention specified in the above limitations.
Rasco and Mayle fail to teach applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform.
However, Li teaches applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform (Li teaches “obtaining a rigid body transformation matrix of point cloud rough matching by using a sampling consistency algorithm” in para. [0072] and “evaluating the quality of the rough registration rigid body transformation matrix by using an error measurement loss function for the preliminarily estimated corresponding point pairs” as shown in para. [0077]-[0078]. Here, the rough registration rigid body transformation matrix is interpreted as equivalent to the first transform, and the quality of the transform is evaluated by using an error metric loss function and a Levenberg-Marquardt algorithm. This function error metric loss function is interpreted as equivalent to the non-linear algorithm).
Mayle, Rasco, and Li are all considered to be analogous to the claimed invention because they are in the same field of performing image transformations through matching. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle) to incorporate the teachings of Li and include “applying a nonlinear algorithm to determine a first error associated with the first transform; and setting the image transform as the first transform”. The motivation for doing so would have been that “the method can obtain initial matching point pairs based on a local fast point feature histogram descriptor and a sampling consistency algorithm, obtain a coarse registration matrix through an error measurement loss function, and then obtain a fine registration rigid body transformation matrix by combining an iterative closest point algorithm, so that the problem that three-dimensional point cloud cannot be automatically registered under different viewing angles can be effectively solved”, as suggested by Li in para. [0045]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco and Mayle with Li to obtain the invention specified in claim 20.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Rasco et. al. (U.S. Publication No. 2016/0379388 A1), hereinafter Rasco in view of Mayle et al. (U.S. Publication No. 2016/0012317 A1), hereinafter Mayle, Li et al. (CN 109919984 A, see attached English translation), hereinafter Li, and further in view of Holz (U.S. Publication No. 2018/0306587 A1).
Regarding claim 5, Rasco, Mayle, and Li teach the method of claim 1.
Rasco, Mayle, and Li fail to teach determining a subset of inlier templates in the collection of templates; and determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates.
However, Holz teaches determining a subset of inlier templates in the collection of templates (Holz teaches “measur[ing] confidence comprising a percentage of occupied cells in the occupancy grid map that are within a threshold distance of lines from the design model that contain the sampled points” in para. [0142]; here, the occupied cells in the map within a threshold distance are interpreted as the subset of inlier templates in the collection of templates); and
determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates (Holz teaches that “the computing system may then determine the transformation based on the measured confidence determined for each candidate transformation” in para. [0142]).
Rasco, Mayle, Li, and Holz are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Holz and include “determining a subset of inlier templates in the collection of templates; and determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates”. The motivation for doing so would have been to “reduce[] errors between cells or points of the occupancy grid map and sampled points from the design model”, as suggested by Holz in para. [0032]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Holz to obtain the invention specified in claim 5.
Regarding claim 15, Rasco, Mayle, and Li teach the system of claim 11.
Rasco, Mayle, and Li fail to teach determining a subset of inlier templates in the collection of templates; and determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates.
However, Holz teaches determining a subset of inlier templates in the collection of templates (Holz teaches “measur[ing] confidence comprising a percentage of occupied cells in the occupancy grid map that are within a threshold distance of lines from the design model that contain the sampled points” in para. [0142]; here, the occupied cells in the map within a threshold distance are interpreted as the subset of inlier templates in the collection of templates); and
determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates (Holz teaches that “the computing system may then determine the transformation based on the measured confidence determined for each candidate transformation” in para. [0142]).
Rasco, Mayle, Li, and Holz are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Holz and include “determining a subset of inlier templates in the collection of templates; and determining a confidence value of the image transformation based at least in part on the subset of inlier templates and the collection of templates”. The motivation for doing so would have been to “reduce[] errors between cells or points of the occupancy grid map and sampled points from the design model”, as suggested by Holz in para. [0032]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Holz to obtain the invention specified in claim 15.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Rasco et. al. (U.S. Publication No. 20160379388 A1), hereinafter Rasco in view of Mayle et al. (U.S. Publication No. 2016/0012317 A1), hereinafter Mayle, Li et al. (CN 109919984 A, see attached English translation), hereinafter Li, and Fan et al. (“Exploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images”), hereinafter Fan.
Regarding claim 6, Rasco, Mayle, and Li teach the method of claim 1.
Rasco, Mayle, and Li fail to teach wherein the applying a matching algorithm includes: generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates; and determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image.
However, Fan teaches wherein the applying a matching algorithm includes:
generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates (Fan “AWOG constructs dense descriptors for each pixel, increasing the description ability of image details”…“AWOG uses an angle weighting strategy to distribute the gradient value only into the two most related orientations, largely increasing the distinguishability of the feature descriptors” Section 3.1; “based on the analysis of AWOGPC in Section 5.1, we set the values of m, n, and the size of the template window image as 3, 8, and 61 × 61 pixels” Section 5.2.2; here, while Rasco teaches the plurality templates (see claim 1), Fan teaches the AWOG representation of the pixels; see also FIG. 5); and
determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image (Fan teaches “using AWOG to match the images” “AWOG is applied to match the roughly geo-rectified image with the reference SAR image” Section 4.(3), and a peak value of the correlation function, which is based on the AWOG representation as shown in Section 3.2, equation (8), (10), and (11); Fan also teaches similarity values, which are interpreted as match scores, with AWOG descriptors in FIG. 6).
Rasco, Mayle, Li, and Fan are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Fan and include “wherein the applying a matching algorithm includes: generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates; and determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image”. The motivation for doing so would have been to improve calculation efficiency, as suggested by Fan in section 3.2, Phase Correlation Matching. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Fan to obtain the invention specified in claim 6.
Regarding claim 16, Rasco, Mayle, and Li teach the system of claim 11.
Rasco, Mayle, and Li fail to teach wherein the applying a matching algorithm includes: generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates; and determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image.
However, Fan teaches wherein the applying a matching algorithm includes:
generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates (Fan “AWOG constructs dense descriptors for each pixel, increasing the description ability of image details”…“AWOG uses an angle weighting strategy to distribute the gradient value only into the two most related orientations, largely increasing the distinguishability of the feature descriptors” Section 3.1; “based on the analysis of AWOGPC in Section 5.1, we set the values of m, n, and the size of the template window image as 3, 8, and 61 × 61 pixels” Section 5.2.2; here, while Rasco teaches the plurality templates (see claim 1), Fan teaches the AWOG representation of the pixels; see also FIG. 5); and
determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image (Fan teaches “using AWOG to match the images” “AWOG is applied to match the roughly geo-rectified image with the reference SAR image” Section 4.(3), and a peak value of the correlation function, which is based on the AWOG representation as shown in Section 3.2, equation (8), (10), and (11); Fan also teaches similarity values, which are interpreted as match scores, with AWOG descriptors in FIG. 6).
Rasco, Mayle, Li, and Fan are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Fan and include “wherein the applying a matching algorithm includes: generating an angle weighted oriented gradients (AWOG) representation of a first template of the plurality of templates; and determining one match score of the set of match scores based at least in part on the AWOG representation of the first template and the at least one reference image”. The motivation for doing so would have been to improve calculation efficiency, as suggested by Fan in section 3.2, Phase Correlation Matching. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Fan to obtain the invention specified in claim 16.
Claim 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Rasco et. al. (U.S. Publication No. 20160379388 A1), hereinafter Rasco in view of Mayle et al. (U.S. Publication No. 2016/0012317 A1), hereinafter Mayle, Li et al. (CN 109919984 A, see attached English translation), hereinafter Li, and Safra et al. (U.S. Publciation No. 2009/0019081 A1), hereinafter Safra.
Regarding claim 8, Rasco, Mayle, and Li teach the method of claim 1.
Mayle further teaches selection criterion associated with templates in the input image. Rasco, Mayle, and Li fail to teach wherein the selection criteria include a criterion associated with a template distribution in the input image.
However, Safra teaches wherein the selection criteria include a criterion associated with a template distribution in the input image (Safra teaches selecting real-world entities, wherein the locations are chosen “according to a uniform distribution” in para. [0076] in order to analyze error; here, the locations of the entities are interpreted as the templates in the claim language).
Rasco, Mayle, Li, and Safra are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Safra and include “wherein the selection criteria include a criterion associated with a template distribution in the input image”. The motivation for doing so would have been to “test [their] methods on data with varying levels of accuracy and incompleteness”, as suggested by Safra in para. [0076]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Safra to obtain the invention specified in claim 8.
Regarding claim 18, Rasco, Mayle, and Li teach the system of claim 11.
Mayle further teaches selection criterion associated with templates in the input image. Rasco, Mayle, and Li fail to teach wherein the selection criteria include a criterion associated with a template distribution in the input image.
However, Safra teaches wherein the selection criteria include a criterion associated with a template distribution in the input image (Safra teaches selecting real-world entities, wherein the locations are chosen “according to a uniform distribution” in para. [0076] in order to analyze error; here, the locations of the entities are interpreted as the templates in the claim language).
Rasco, Mayle, Li, and Safra are both considered to be analogous to the claimed invention because they are in the same field of transforming geographical environments. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Safra and include “wherein the selection criteria include a criterion associated with a template distribution in the input image”. The motivation for doing so would have been to “test [their] methods on data with varying levels of accuracy and incompleteness”, as suggested by Safra in para. [0076]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Safra to obtain the invention specified in claim 18.
Claim 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Rasco et. al. (U.S. Publication No. 20160379388 A1), hereinafter Rasco in view of Mayle et al. (U.S. Publication No. 2016/0012317 A1), hereinafter Mayle, Li et al. (CN 109919984 A, see attached English translation), hereinafter Li, and Amann et al. (U.S. Publication No. 2004/0189677 A1), hereinafter Amann.
Regarding claim 9, Rasco, Mayle, and Li teaches the method of claim 1.
Rasco, Mayle, and Li fail to teach applying the image transform to the input image to generate a registered image; and exporting the registered image.
However, Amann teaches applying the image transform to the input image to generate a registered image (Amann teaches “transforming, with a graphics processing unit, a rendered image from a first image format to a second image format” in para. [0013]; the rendered image here is interpreted as equivalent to the registered image taught in the claim language); and exporting the registered image (Amann teaches “send[ing] scaled and transformed images to the at least one remote device” in para. [0013]).
Rasco, Mayle, Li, and Amann are both considered to be analogous to the claimed invention because they are in the same field of transforming images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Amann and include “applying the image transform to the input image to generate a registered image; and exporting the registered image”. The motivation for doing so would have been to “reduce the effect of jagged lines and aliasing” and correcting depth of field effect, as suggested by Amann in para. [0005]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Amann to obtain the invention specified in claim 9.
Regarding claim 19, Rasco, Mayle, and Li teaches the system of claim 11.
Rasco, Mayle, and Li fail to teach applying the image transform to the input image to generate a registered image; and exporting the registered image.
However, Amann teaches applying the image transform to the input image to generate a registered image (Amann teaches “transforming, with a graphics processing unit, a rendered image from a first image format to a second image format” in para. [0013]; the rendered image here is interpreted as equivalent to the registered image taught in the claim language); and exporting the registered image (Amann teaches “send[ing] scaled and transformed images to the at least one remote device” in para. [0013]).
Rasco, Mayle, Li, and Amann are both considered to be analogous to the claimed invention because they are in the same field of transforming images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Rasco (as modified by Mayle and Li) to incorporate the teachings of Amann and include “applying the image transform to the input image to generate a registered image; and exporting the registered image”. The motivation for doing so would have been to “reduce the effect of jagged lines and aliasing” and correcting depth of field effect, as suggested by Amann in para. [0005]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Rasco, Mayle, and Li with Amann to obtain the invention specified in claim 19.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights