DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Claims 1-9 are pending regarding this application.
Claims 10-12 are cancelled.
The amendments to claims 1-6, in the claims filed 12/06/2023, have been accepted and entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/06/2023 and 06/18/2024 are
considered and attached.
Claim Objections
Claims 1, 4, 5, 7, and 9 are objected to because of the following informalities:
Claim 1 recites “the same observed area” in line 6. The term “same observed area” lacks antecedent basis. Please amend to recite “a same observed area”.
Claim 4 recites “the non-changed area” in lines 8-9. The term “non-changed area” lacks antecedent basis. Please amend to recite “a non-changed area”.
Claim 4 recites “the same observed area” in line 6. The term “same observed area” lacks antecedent basis. Please amend to recite “a same observed area”.
Claim 5 recites “the number of pixels” in lines 3-4. The term “number of pixels” lacks antecedent basis. Please amend to recite “a number of pixels”.
Claim 7 recites “the same observed area” in line 3. The term “same observed area” lacks antecedent basis. Please amend to recite “a same observed area”.
Claim 9 recites “a non-change area” in lines 2-3, however, there is already a non-change area introduced in claim 7, upon which claim 9 is dependent. Please amend to recite “the non-change area” OR amend claim 7 (upon which claim 9 is dependent) to recite “a first non-change area” in line 6 and amend claim 9 to recite “a second non-change area” in lines 2-3.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5 and 7-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
An image processing device comprising:
a memory storing software instructions, and
one or more processors configured to execute the software instructions to
detect a change area, where an object has changed compared to the first image, in the second image obtained by observing the same observed area as an observed area in the first image;
perform a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed; and
output an image pair of the corrected processed second image and the first image.
Step 1: Evaluating whether the claim belongs to one of the statutory categories.
Claim 1 recites at least one step or act. Thus, the claim is directed to a process, which is one of the statutory categories of invention (Step 1: YES)
Step 2A Prong One: Evaluating whether the claim recites a judicial exception (an abstract idea enumerated in 2019 PEG, a law of nature, or a natural phenomenon). If no exception is recited, the claim is eligible. This concludes the eligibility analysis. If the claim recites an exception, go to Step 2A Prong Two.
Claim 1 recites an abstract idea of mental processes. At least steps c and d are recited at a high level of generality such that they could be practically performed in the human mind (The courts consider a mental process (thinking) that “can be performed in the human mind, or be a human using a pen and paper” to be an abstract idea.). These concepts fall into the “mental processes” group of abstract ideas, which is observation, evaluation and/or judgement. The limitations, interpreted under their broadest reasonable interpretation and in consistence with the specification, cover performance of the limitations in the mind or by generic computer components. See MPEP 2106.04 and the 2019 PEG. (Step 2A Prong One YES)
Step 2A Prong Two: Evaluating whether the claim recites additional elements that integrate the exception into a practical application of the exception. This evaluation is performed by (a) identifying whether there are any additional elements recited in the claim beyond judicial exception, and (b) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into practical application. If the answer to (a) is YES and (b) is NO, go to Step 2B; if the answer to (a) and (b) is YES, go to PATHWAY B, i.e., the claim is not directed to a judicial exception and the claim is eligible.
The 2019 PEG defines the phrase “integration into a practical application” to require an additional element or a combination of additional elements in the claim to apple, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception.
Limitations that are indicative of integration into a practical application when recited in a claim with a judicial exception include:
Improvements to the functioning of a computer, or to any other technology or technical field, as discussed in MPEP 2106.05(a);
Applying or using a judicial exception to affect a particular treatment or prophylaxis for disease or medical condition – see Vanda Memo
Applying the judicial exception with, or by use of, a particular machine, as discussed in MPEP 2106.05(b);
Effecting a transformation or reduction of a particular article to a different state or thing, as discussed in MPEP 2106.05©; and
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception, as discussed in MPEP 2106.05(e) and the Vanda Memo issued in June 2018.
Limitations that are not indicative of integration into a practical application when recited in a claim with a judicial exception include:
Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f);
Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g); and
Generally linking the use of the judicial exception to a particular technological environment or field of use, as discussed in MPEP 2106.05(h).
[Examiners should note that revised Step 2A excludes consideration of whether claim elements represent well-understood, routine, conventional activity. The question of whether claim elements represent only well-understood, routine, conventional activity is considered at Step 2B and is not a consideration in Step 2A.]
Steps a, b, and e can be regarded as additional elements recited in claim 1. These additional elements, i.e., memory storing software instructions, one or more processors configured to execute the software instructions, and outputting image pairs, do not integrate the exception into a practical application of the exception. Note even if the specification discloses that the invention pertains to an improvement in the technology, the claim must be evaluated to ensure the claim itself reflects the improvement in technology. It is also important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. Therefore, the additional elements do not recite an improvement. (Step 2A Prong Two NO)
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
Steps a, b, and e can be regarded as additional elements recited in claim 1. These additional elements, i.e., memory storing software instructions, one or more processors configured to execute the software instructions, and outputting image pairs are considered insignificant extra-solution activities and general computer elements. See MPEP, 2106.04 (a) (2) III (C): Performing a mental process on a generic computer.
Additionally, using a processor to execute software instructions and outputting image data is a well-understood, routine, conventional activity in the field. Using the broadest reasonable interpretation of the claim, the additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the abstract idea itself. See MPEP 2106.05. (Step 2B: NO) The claim is not eligible.
Claim 2 recites
“wherein the one or more processors are configured to execute the software instructions to replace pixel values of the change area in the second image with default values”
Steps a is directed to the abstract idea of mental processes which are mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to step a of claim 2. No additional elements are recited. Accordingly, claim 2 does not have eligible subject matter.
Claims 3 recites “wherein the one or more processors are configured to execute the software instructions to replace the change area in the second image with a non- change area in another image obtained by observing the same observed area as the observed area in the first image” which is a process directed to the abstract idea of mental processes which is mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to the steps of claim 3. No additional elements are recited. Accordingly, claim 3 does not have eligible subject matter.
Independent claim 4 is directed to a machine, which is a statutory category of invention. Similar analysis is applicable as applied above to the method of claim 1. However, claim 4 further recites “calculate a matching degree between the first image and the second image based on the size of the non-change area in the second image where the object has not changed; and image pair output means for outputting output an image pair of the second image and the first image when the matching degree exceeds a predetermined value”. The element of calculating a matching degree is a process directed to the abstract idea of mental processes which is mere automation of manual processes. Additionally, outputting images based on an exceeded threshold does not integrate the exception into a practical application of the exception or amount to significantly more than the recited exception. Outputting images based on thresholds is a routine and well-known process in image analysis. The remaining limitations of claim 4 can be analyzed similarly to the limitations of claim 1. Claim 4 is not eligible.
Claims 5 recites “wherein the one or more processors are configured to execute the software instructions to calculate a percentage of the number of pixels in the non-change area to the number of pixels in a total area in the second image as the matching degree” which is a process directed to the abstract idea of mental/mathematical processes which is mere automation of manual/mathematical processes. The similar examination analysis as applied to claim 4 is applied to the steps of claim 5. No additional elements are recited. Accordingly, claim 5 does not have eligible subject matter.
Independent claim 7 is directed to a method, which is a statutory category of invention. Similar analysis is applicable as applied above to the method of claim 1. Claim 7 is not eligible.
Similar analysis as applied to claims 2 and 3 can be applied to corresponding claims 8 and 9, respectively. Claims 8 and 9 are not eligible.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 4 recite the limitation "the first image" and “the second image” in line 5. There is insufficient antecedent basis for this limitation in the claim.
Similarly, claim 7 recites “the first image” in lines 2-3, and “the second image” in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites “the two object presence images” in line 6 and “the two observed images” in line 7. Firstly, neither of these elements have been properly introduced, as the above instances of these elements are the first recitation of these elements and they both lack insufficient antecedent basis. Secondly, even if “the two object presence images” in line 6 and “the two observed images” in line 7 were introduced properly, it remains unclear whether the two object presence images are equivalent to the two observed images, and furthermore whether either of “the two object presence images” and “the two observed images” are equivalent to the first and second images introduced in claim 1, upon which claim 6 is dependent. Applicant’s specification seems to imply the object presence images and the observed images are both derived from the observed images in para. [0023]-[0024]. However, the difference in naming of “the two object presence images” and “the two observed images” in both the claim and specification simultaneously implies there is a distinct difference between the two sets of images.
Claims 2-3, 5-6, and 8-9 are rejected due to their dependency upon rejected claims 1, 4, and 7, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6, 7, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. (U.S. Publication No. 2010/0104185 A1), hereinafter Johnson, in view of Garten (U.S. Publication No. 2011/0103644 A1) and Merkle et al. (“Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images”), hereinafter Merkle.
Regarding claim 1, Johnson teaches an image processing device comprising:
a memory storing software instructions, and one or more processors configured to execute the software instructions (Johnson teaches that the computing device 500 includes at least one processing unit 502 and system memory 504”, wherein “the system memory 504 typically includes an operating system 506, one or more program modules 508, and may include program data 510” in para. [0056]; see FIG. 5) to
detect a change area, where an object has changed compared to the first image, in the second image obtained by observing the same observed area as an observed area in the first image (Johnson teaches “detecting the insertion, removal, and change of objects of interest through the use of two or more images containing a common area of interest” in para. [0030]; here, the common area of interest is equivalent to the claimed same observed areas and “the region identification process 312 spatially partitions the set of GPC likelihoods, created by the feature content analysis process, into a set of variously sized regions where the region sizes are determined by the objects within the imagery” in para. [0040]. Here, these regions are interpreted as equivalent to the claimed change areas).
Johnson fails to teach perform a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed and outputting an image pair of the corrected processed second image and the first image.
However, Garten teaches performing a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed (Garten teaches that “image data may be extracted from the first image and to replace image data of the second image to generate a correct image at block 220” in order “to replace unwanted or disturbing objects with image data associated with a background” in para. [0020]; since data from a first image is being used to replace an object in the second image, it is inherent that would make a change area smaller in the second image. See also FIGs. 2 and 3).
Johnson and Garten are both considered to be analogous to the claimed invention because they are in the same field of analyzing change between images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson to incorporate the teachings of Garten and include “perform a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed”. The motivation for doing so would have been “to allow for removal of one or more undesired objects”, as suggested by Garten in para. [0004]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson with Garten to obtain the invention specified in the above limitation.
Johnson and Garten fail to teach outputting an image pair of the corrected processed second image and the first image.
However, Merkle teaches outputting an image pair of the corrected processed second image and the first image (Merkle teaches a process of matching an optical and SAR image pair in order to correct the optical image through geo-localization to output a pair with higher matching accuracy in FIG. 7 and Section 3.7 Qualitative Results; the SAR image is interpreted as equivalent to the claimed first image, and the optical image is interpreted as equivalent to the claimed second image. See Applicant’s Specification which states “both optical and SAR images are acceptable for an observed image” in para. [0004]).
Johnson, Garten, and Merkle are all considered to be analogous to the claimed invention because they are in the same field of analyzing change between images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson (as modified by Garten) to incorporate the teachings of Merkle and include “outputting an image pair of the corrected processed second image and the first image”. The motivation for doing so would have been to enhance an image to increase matching precision, as suggested by Merkle in Section 3.7 Qualitative Results, and “to generate matching points with high matching accuracy and precision”, as suggested by Merkle in Section 3.9 Strengths. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson and Garten with Merkle to obtain the invention specified in claim 1.
Regarding claim 3, Johnson, Garten, and Merkle teach the image processing device according to claim 1,
wherein the one or more processors are configured to execute the software instructions to replace the change area in the second image with a non- change area in another image obtained by observing the same observed area as the observed area in the first image (Johnson teaches the process of obtaining images of a same observed area in claim 1) (Garten teaches that “image data may be extracted from the first image and to replace image data of the second image to generate a correct image at block 220” in order “to replace unwanted or disturbing objects with image data associated with a background” in para. [0020]; since data from a first image is being used to replace an object in the second image, it is inherent that would make a change area smaller in the second image. See also FIGs. 2 and 3.; FIG. 3 shows a change area (#330) of a second image (#305) being replaced by a non-change area (#340) in a first image (#315)).
Regarding claim 6, Johnson, Garten, and Merkle teach the image processing device according to claim 1, wherein the one or more processors are configured to execute the software instructions to
generate two deformed images by deforming object presence areas in the two object presence images, which are images obtained from each of the two observed images and in which one or more objects are present (Johnson teaches a scene registration process in FIGS. 4 and 5 wherein “all of the one to one and many to one pixel value transformations such as linear rescaling, equalization, and feature extraction. At a block 1002 the original or re-labeled images are transformed into a common reference frame and may produce both a forward and inverse transform which maps the pixel locations in the original image to those in the transformed image and vice versa” in para. [0032]; here, the transformed images are interpreted as the deformed images), based on an observation angle of each of the two observed images (Johnson teaches “the common reference frame may be the original view point of either image or another advantageous view point all together” in para. [0032]) and a size of the object that appears in each of the two observed images (Johnson teaches “the image patterns of the transformed images are aligned and may produce either a mathematical transform or a set transformed images or both, that account for […] other distortions present within the transformed images that were not removed by the previous blocks such as terrain elevation and object height errors” as shown in para. [0033]; here, the transformations based on object height are interpreted as equivalent to creating the transformed (deformed) image based on the size of the object), and
generate a synthesized image by combining the two deformed images (Johnson teaches “At block 1003 the image patterns of the transformed images are aligned” in para. [0033]; the aligned image is interpreted as equivalent to the synthesized image; see FIG. 5, #1003), determine a change of the object between the two object presence images using the synthesized image (Johnson teaches detecting a change likelihood of objects within the two object presence images (FIG. 5) using the aligned imagery (FIG. 5 (#1003))), and generate an image capable of identifying the determined change (Johnson teaches FIG. 5 wherein the synthesized image (#1003) is capable of identifying the determined change as shown in para. [0039]).
Regarding claim 7, Johnson teaches an image processing method (Johnson teaches “ methods for detecting the insertion, removal and change of objects of interest through the comparison of two or more images” in para. [0005]) comprising:
detecting a change area, where an object has changed compared to the first image, in the second image obtained by observing the same observed area as an observed area in the first image (Johnson teaches “detecting the insertion, removal, and change of objects of interest through the use of two or more images containing a common area of interest” in para. [0030]; here, the common area of interest is equivalent to the claimed same observed areas and “the region identification process 312 spatially partitions the set of GPC likelihoods, created by the feature content analysis process, into a set of variously sized regions where the region sizes are determined by the objects within the imagery” in para. [0040]. Here, these regions are interpreted as equivalent to the claimed change areas).
Johnson fails to teach performing a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed and outputting an image pair of the corrected processed second image and the first image.
However, Garten teaches performing a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed (Garten teaches that “image data may be extracted from the first image and to replace image data of the second image to generate a correct image at block 220” in order “to replace unwanted or disturbing objects with image data associated with a background” in para. [0020]; since data from a first image is being used to replace an object in the second image, it is inherent that would make a change area smaller in the second image. See also FIGs. 2 and 3).
Johnson and Garten are both considered to be analogous to the claimed invention because they are in the same field of analyzing change between images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson to incorporate the teachings of Garten and include “performing a correction process to make the change area in the second image smaller relative to a non-change area where the object has not changed”. The motivation for doing so would have been “to allow for removal of one or more undesired objects”, as suggested by Garten in para. [0004]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson with Garten to obtain the invention specified in the above limitation.
Johnson and Garten fail to teach outputting an image pair of the corrected processed second image and the first image.
However, Merkle teaches outputting an image pair of the corrected processed second image and the first image (Merkle teaches a process of matching an optical and SAR image pair in order to correct the optical image through geo-localization to output a pair with higher matching accuracy in FIG. 7 and Section 3.7 Qualitative Results; the SAR image is interpreted as equivalent to the claimed first image, and the optical image is interpreted as equivalent to the claimed second image. See Applicant’s Specification which states “both optical and SAR images are acceptable for an observed image” in para. [0004]).
Johnson, Garten, and Merkle are all considered to be analogous to the claimed invention because they are in the same field of analyzing change between images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson (as modified by Garten) to incorporate the teachings of Merkle and include “outputting an image pair of the corrected processed second image and the first image”. The motivation for doing so would have been to enhance an image to increase matching precision, as suggested by Merkle in Section 3.7 Qualitative Results, and “to generate matching points with high matching accuracy and precision”, as suggested by Merkle in Section 3.9 Strengths. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson and Garten with Merkle to obtain the invention specified in claim 7.
Regarding claim 9, Johnson, Garten, and Merkle teach the image processing method according to claim 7,
wherein in the correction process, the change area in the second image is replaced with a non- change area in another image obtained by observing the same observed area as the observed area in the first image Johnson teaches the process of obtaining images of a same observed area in claim 1) (Garten teaches that “image data may be extracted from the first image and to replace image data of the second image to generate a correct image at block 220” in order “to replace unwanted or disturbing objects with image data associated with a background” in para. [0020]; since data from a first image is being used to replace an object in the second image, it is inherent that would make a change area smaller in the second image. See also FIGs. 2 and 3.; FIG. 3 shows a change area (#330) of a second image (#305) being replaced by a non-change area (#340) in a first image (#315)).
Claims 2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. (U.S. Publication No. 2010/0104185 A1), hereinafter Johnson, in view of Garten (U.S. Publication No. 2011/0103644 A1), Merkle et al. (“Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images”), hereinafter Merkle, and Li et al. (U.S. Publication No. 2021/0241509 A1), hereinafter Li.
Regarding claim 2, Johnson, Garten, and Merkle teach the image processing device according to claim 1.
While Johnson further teaches the one or more processors configured to execute the software instructions in claim 1, Johnson, Garten, and Merkle fail to teach wherein the one or more processors are configured to execute the software instructions to replace pixel values of the change area in the second image with default values.
However, Li teaches wherein the one or more processors are configured to execute the software instructions to replace pixel values of the change area in the second image with default values (Li teaches a method of replacing pixels in target areas which involves “reassigning pixel values for the target area, and replacing, with a pixel value at any position in the second replacement image, the default value of a pixel at a corresponding position in the target area” as shown in para. [0029]; see also FIG. 7 and para. [0068] and [0079] regarding the teaching of the processor).
Johnson, Garten, Merkle, and Li are all considered to be analogous to the claimed invention because they are in the same field of correcting images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson (as modified by Garten and Merkle) to incorporate the teachings of Li and include “wherein the one or more processors are configured to execute the software instructions to replace pixel values of the change area in the second image with default values”. The motivation for doing so would have been to improve the deformation effect and generate a corrected image, as suggested by Li in para. [0039]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson, Garten, and Merkle with Li to obtain the invention specified in claim 2.
Regarding claim 8, Johnson, Garten, and Merkle teach the image processing method according to claim 7.
Johnson, Garten, and Merkle fail to teach wherein in the correction process, pixel values of the change area in the second image are replaced with default values.
However, Li teaches wherein in the correction process, pixel values of the change area in the second image are replaced with default values (Li teaches a method of replacing pixels in target areas which involves “reassigning pixel values for the target area, and replacing, with a pixel value at any position in the second replacement image, the default value of a pixel at a corresponding position in the target area” as shown in para. [0029]).
Johnson, Garten, Merkle, and Li are all considered to be analogous to the claimed invention because they are in the same field of correcting images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson (as modified by Garten and Merkle) to incorporate the teachings of Li and include “wherein in the correction process, pixel values of the change area in the second image are replaced with default values”. The motivation for doing so would have been to improve the deformation effect and generate a corrected image, as suggested by Li in para. [0039]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson, Garten, and Merkle with Li to obtain the invention specified in claim 8.
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. (U.S. Publication No. 2010/0104185 A1), hereinafter Johnson, in view of Merkle et al. (“Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images”), hereinafter Merkle and Abraham et al. (U.S. Publication No. 2010/0303340 A1), hereinafter Abraham.
Regarding claim 4, Johnson teaches image processing device comprising:
a memory storing software instructions, and one or more processors configured to execute the software instructions (Johnson teaches that the computing device 500 includes at least one processing unit 502 and system memory 504”, wherein “the system memory 504 typically includes an operating system 506, one or more program modules 508, and may include program data 510” in para. [0056]; see FIG. 5) to
detect a change area, where an object has changed compared to the first image, in the second image obtained by observing the same observed area as an observed area in the first image (Johnson teaches “detecting the insertion, removal, and change of objects of interest through the use of two or more images containing a common area of interest” in para. [0030]; here, the common area of interest is equivalent to the claimed same observed areas and “the region identification process 312 spatially partitions the set of GPC likelihoods, created by the feature content analysis process, into a set of variously sized regions where the region sizes are determined by the objects within the imagery” in para. [0040]. Here, these regions are interpreted as equivalent to the claimed change areas);
calculate a matching (Johnson teaches that “the feature content analysis process 310 may use a General Pattern Change (GPC) likelihood algorithm, such as the GPC likelihood process 2014 schematically shown in FIG. 6 to determine the likelihood of change for every pixel in the first and second images” in para. [0034], wherein “a total number of pixels in the common image overlap is determined at a block 2026” in para. [0036]. The process of determining whether objects have changed is determined based on the sizes of objects of interest as shown in para. [0037]. Here, the overlap pixels are equivalent to the claimed non-change area).
Johnson fails to specifically teach a matching degree and outputting an image pair of the second image and the first image when the matching degree exceeds a predetermined value.
However, Merkle teaches a matching degree (Merkle teaches that “the matching accuracy is measured as the percentage of matching points” in Section 3.3 Influence of Speckle Filtering) outputting an image pair of the second image and the first image (Merkle teaches a process of matching an optical and SAR image pair in order to correct the optical image through geo-localization to output a more aligned pair in FIG. 7 and Section 3.7 Qualitative Results; the SAR image is interpreted as equivalent to the claimed first image, and the optical image is interpreted as equivalent to the claimed second image. See Applicant’s Specification which states “both optical and SAR images are acceptable for an observed image” in para. [0004]).
Johnson and Merkle are both considered to be analogous to the claimed invention because they are in the same field of detecting changes using UAV/Satellite/SAR images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson to incorporate the teachings of Merkle and include a matching degree and “outputting an image pair of the second image and the first image”. The motivation for doing so would have been to “to generate matching points with high matching accuracy and precision”, as suggested by Merkle in Section 3.9 Strengths. See also Section 3.7 Qualitative Results, regarding additional motivations. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson with Merkle to obtain the invention specified in the above claim limitation.
Johnson and Merkle fail to teach outputting images when the matching degree exceeds a predetermined value.
However, Abraham teaches outputting images when the matching degree exceeds a predetermined value (Abraham teaches that “an image from group A is matched with an image chosen from group B, a stereo pair is formed and a normalized similarity measure Q (0<Q<1) to characterize the degree of their similarity/dissimilarity is produced” as shown in para. [0155] and FIG. 7; here, this pair is only output when the pair exceeds the similarity threshold as shown in para. [0162]).
Johnson, Merkle, and Abraham are all considered to be analogous to the claimed invention because they are in the same field of utilizing registration processes (Johnson, see para. [0031]) and comparing images to detect differences using UAV/Satellite/SAR images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Johnson (as modified by Merkle) to incorporate the teachings of Abraham and include “outputting images when the matching degree exceeds a predetermined value”. The motivation for doing so would have been to facilitate “accurate height/depth estimation and subsequently enable[] volumetric change detection”, as suggested by Abraham in para. [0079]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Johnson and Merkle with Abraham to obtain the invention specified in claim 4.
Regarding claim 5, Johnson, Merkle, and Abraham teach the image processing device according to claim 4.
wherein the one or more processors are configured to (See claim 1) execute the software instructions to calculate a percentage of the number of pixels in the non-change area to the number of pixels in a total area in the second image as the matching degree (Merkle teaches “the matching accuracy is measured as the percentage of matching points” in Section 3.3 Influence of Speckle Filtering wherein the process is carried out on a GPU in Section 3.5).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kobayashi et al. (U.S. Publication No. 2021/0377457 A1) teaches a ghost removing process for pixels of the target image corresponding to the moving object region.
Yamashita et al. (U.S. Publication No. 2011/0085027 A1) teaches a method of correcting images based on a moving subject.
Bian et al. (U.S. Publication No. 2014/0099021 A1) teaches a method of comparing first and second images to detect change areas and replacing the change area.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661