Detailed Action
1. Claims 1-14 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Argument
3. The Applicant’s argument filed 07/06/2026 is fully consider. For Examiner response see discussion below.
a. Regarding claim 1, The Applicant argues “Carlotto determines whether the change (difference) of a pixel is larger than a threshold within a single image pair. However, Carlotto's "p0" is merely a parameter for calculating a threshold that defines the spatial area fraction of pixels considered to have changed within a single difference image. Carlotto neither discloses nor suggests the process of "determining, as a change location, a pixel considered to be a change location candidate in a prescribed percentage or more of the plurality of image pairs”
Examiner respectfully disagrees with the Applicant’s argument for the following reasons:
Carlotto specifically teaches “Our approach to WAS is based on detecting or enhancing patterns of change in imagery that are observed when specific kinds of events or activities occur. Where change detection detects any significant change between pairs of images, our method extracts only those changes that match a given pattern over time( see Section III A, left col. 2nd par., - right col. 2nd par.) and Fig.2).
Regarding the thresholded value p0., as shown in Fig.2 each pairs of image are thresholded using a value p0=5%, i.e., (5% of the pixels in each image were assumed to have changed).
PNG
media_image1.png
492
499
media_image1.png
Greyscale
For example temporal segmentation was performed on seven of the 10 images (from 1/15/87 to 3/4/90) over the same 400 X400 pixel region shown in Figs. 6–8. First, total difference images were computed using the nonlinear prediction algorithm. These images were then thresholded using a value p0=5%, i.e., (5% of the pixels in each image were assumed to have changed), see page 199, Section C : Temporal segmentation.
Further Carlotto teaches a method of using the label vector as a model to specify the pattern of change of interest and define a measure to determine the degree to which changes in imagery over time match the pattern . Specifically patterns of change are represented by an image of label vectors at a particular pixel location, two elements of the label vector are equal if there is no change between the corresponding times at that pixel ; otherwise, the two elements are different (see page 193 section C) .
Regarding claim 12 and 13, method claim 12 and computer program claim 13 correspond to apparatus claim 1; thus, the arguments applied to claim 1 apply equally to claims 12 and 13.
b. Regarding claim 2, The Applicant argues that Carlotto's does not teach “wherein the change index of a pixel is a value in accordance with a ratio of a change in a pixel value of the pixel to a mean of changes in pixel values in a peripheral region of the pixel.”
The Applicant’s argument is persuasive, and the 35 U.S.C 102 rejection based on the Carlotto's is expressly withdrawn. After further search and consideration no prior art is found to anticipate or render the above limitation obvious. Thus claim 2 is in condition of allowance
c Regarding claim 3, The Applicant argues that Carlotto's does not teach “wherein the change candidate extracting unit obtains the change index after adjusting at least one of the first aerial image and the second aerial image so that a mean value and a variance of pixel values of the first aerial image and the second aerial image are consistent.”
The Applicant’s argument is persuasive, and the 35 U.S.C 102 rejection based on the Carlotto is expressly withdrawn. After further search and consideration no prior art is found to anticipate or render the above limitation obvious. Thus claim 3 is in condition of allowance
d. Regarding claim 4, The Applicant argues that Zhong does not teach “wherein the change threshold is set for each pair as a greater value among a prescribed value and a value corresponding to a higher first prescribed percentage of change indices calculated from the first aerial image and the second aerial image.”
The Applicant’s argument is persuasive, and the 35 U.S.C 103 rejection based on the Zhong is expressly withdrawn. After further search and consideration no prior art is found to anticipate or render the above limitation obvious. Thus claim 4 is in condition of allowance
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
4. Claims1, 5 and 9-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mark J. Carlotto ( hereafter Carlotto), “Detection and Analysis of Change in Remotely Sensed Imagery with Application to Wide Area Surveillance” IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 6, NO. 1, pub. JANUARY 1997
As to claim 1, Carlotto teaches An aerial image change detection apparatus ( Abstract, Detection and Analysis of Change in Remotely Sensed Imagery with Application to Wide Area Surveillance), comprising:
an image acquiring unit which acquires N-number of first aerial images and M- number of second aerial images, wherein the first aerial images are aerial images of a same area in a first period, the second aerial images are aerial images of the same area as the first aerial images in a second period subsequent to the first period, and N and M are both integers larger than or equal to 1 (Figs.1 and 2, page 193 left col., 2nd par., A new approach to wide area surveillance (WAS) is described that is based on the detection and analysis of changes across two or more images over time. The Authors approaches to WAS involved the analysis of changes between all pairs of input images. For N input images ,
x
n
i
,
j
,
0
≤
n
≤
N
,
there are
N
(
N
-
1
)
2
total difference images . Consider an example (Fig. 1) where three images acquired at times t1,t2, and t3 and have captured two events (Fig. 1(a)). The and N and M set of of images corresponds to the N input images and N>0);
a change candidate extracting unit which obtains, with respect to a plurality of image pairs among N x M-number of image pairs of the first aerial images and the second aerial images, a change index for each pixel that represents a change in a pixel value between one first aerial image and one second aerial image and (Figs.1 and 2, page 193 section C Temporal Segmentation; For N input images ,
x
n
i
,
j
,
0
≤
n
≤
N
,
there are
N
N
-
1
2
total difference images, where the total difference images is given by
n
m
i
,
j
,
0
≤
m
≤
N
,
0
≤
n
≤
N
, where the total difference images determined based on patterns of change across images over time. The patterns of change are represented by an image of label vectors
⋀
i
,
j
=
λ
n
i
,
j
.At a particular pixel location, two elements of the label vector are equal if there is no change between the corresponding times at that pixel
λ
n
i
,
j
=
λ
m
i
,
j
otherwise, the two elements are different
λ
n
i
,
j
≠
[Symbol font/0x20]
λ
m
i
,
j
.
The change index corresponds to the total difference images
n
m
i
,
j
g
i
v
e
n
b
y
e
q
u
a
t
i
o
n
6
) which adopts a pixel of which the change index is larger than a change threshold as a change location candidate (Equations 6 and 9, page 193 left col., 2nd par., The total difference image
∈
=
∈
F
i
,
j
-
∈
B
i
,
j
, and the thresholds
τ
m
n
are computed from the histograms of the total difference images given by equation (6), where
∈
>
τ
m
n
)
wherein the change threshold is chosen in accordance with the pair of the first aerial image and the second aerial image (Equation 9, page 193 right col., 1st and 2nd pars., although this segmentation algorithm can provide insight into the kinds of changes occurring in different parts of the image, its performance depends on the thresholds that, in turn, depend on the value the histogram given by equation 9); and
a change location extracting unit which determines, as a change location, a pixel considered to be a change location candidate in a prescribed percentage or more of the plurality of image pairs ( Equation 9, page 193 right col., 1st and 2nd pars ., the thresholds are computed from the histograms of the total difference images. Each is chosen to satisfy
p
0
=
∑
∈
>
τ
m
i
n
p
m
n
(
∈
)
, where
p
m
n
(
∈
)
is the histogram of the total difference image
∈
m
n
i
,
j
, and
p
0
is the fraction of the image that is assumed Patterns of change. Thus, the histogram of the total image p0 given by equation 9 and equation 6 measures the change locations candidates ( pixels) )
As to claim 5, Carlotto teaches the change candidate extracting unit adopts a pixel as a change location candidate regardless of a value of the change index when a change of a pixel value of the pixel is larger than a prescribed value (Equations 6 and 9, page 193 left col., 2nd par The patterns of change are represented by an image of label vectors
⋀
i
,
j
=
λ
n
i
,
j
.At a particular pixel location, two elements of the label vector are equal if there is no change between the corresponding times at that pixel
λ
n
i
,
j
=
λ
m
i
,
j
otherwise, the two elements are different
λ
n
i
,
j
≠
[Symbol font/0x20]
λ
m
i
,
j
.
The change index corresponds to the total difference images
n
m
i
,
j
g
i
v
e
n
b
y
e
q
u
a
t
i
o
n
6
. The total difference image
∈
=
∈
F
i
,
j
-
∈
B
i
,
j
, and the thresholds
τ
m
n
are computed from the histograms of the total difference images given by equation (6), where
∈
>
τ
m
n
.
Although this segmentation algorithm can provide insight into the kinds of changes occurring in different parts of the image, its performance depends on the thresholds that, in turn, depend on the value the histogram given by equation 9).
As to claim 9, Carlotto teaches wherein the image acquiring unit: acquires a partial area of a first large-area aerial image in the first period as the first aerial image; and acquires a partial area of a second large-area aerial image in the second period as the second aerial image (page 191, left col. Adaptive techniques use information over larger areas (e.g., within a sliding window) to model and predict two or more images from one another).
As to claim 10, Carlotto teaches the image acquiring unit: acquires the N-number of the first aerial images from a top N-number of the first large-area aerial images with a small cloud coverage in the partial area among more than N- number of first large-area aerial images in the first period; and acquires the M-number of the second aerial images from a top M-number of the second large-area aerial images with a small cloud coverage in the partial area among more than M-number of second large-area aerial images in the second period (Page 197- right col.- Page 198 The two images were taken about a month apart in 3/30/88 and 5/1/88. Except for the clouds, there are relatively few changes on the ground. The clouds in (Fig. 7(b)) significantly increase the scatter in the joint histogram (Fig. 7(c)). They effectively pull the linear model away from the hypothetical regression line, assuming there were no clouds (dotted line) to the actual regression line with clouds (solid line). The absolute value of the total difference image is shown in (Fig. 7(d)) and is dominated by changes dues to the clouds. Fig. 8 shows how the nonlinear prediction algorithm can better adapt to large changes due to cloud.
As to claim 11, Carlotto teaches a change location is detected for each of a plurality of divided areas obtained by dividing an area of interest, and a change in the area of interest between the first period and the second period is obtained ( Page 200 section C, Temporal segmentation was performed on seven of the 10 over the same 400X400 pixel region shown in Figs. 6±8. First, total difference images were computed using the nonlinear prediction algorithm. The labeling algorithm (10) was then applied to the change images of 400X400 pixel region).
As to claim 12, Carlotto teaches an output unit which outputs, in superposition, at least any of the N-number of first aerial images and the M-number of second aerial images and the change locations ( page 200 right col., last par., - page 201 left col. 1st par., Fig. 12 shows the full scene outputs from the Delta filter for two images (Fig. 12(a)) and eight images (Fig. 12(b)) Fig. 12(a) is the total difference image between the 6/24/87 and 3/30/88 images and represents a two image change detection result. Bright areas indicate an increase in the brightness. In dark areas, the total difference has decreased and includes areas where the water level has risen to cover the land and vegetation has appeared or developed more fully. In Fig. 12(b), there are relatively few bright areas where the Delta filter responds strongly other than over the facility itself. The output from the Delta filter after it has been thresholder and smoothed is shown in Fig. 13(a). The largest detection is highlighted and overlaid on the 7/10/90 image in Fig. 13(b).
Claim 13 is rejected the same as claim 1 except claim 13 is directed to a method claim. All the limitations of claim 13 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 13.
As to claim 14, Carlotto teaches A computer-readable medium non-transitorily storing a program for causing a computer to execute the steps (Figs. 3-12, page 193 right col., 2nd par., page 201 right col. 3rd par., Carlotto specifically teaches various computer excitable geocoding ( i.e. computer program) that includes detection algorithm, nonlinear prediction algorithm, image labeling algorithm, linear and nonlinear change detection algorithm, and the experimental result ( computer simulation results) obtained by executing the various geocoding (see Figs. 3-12);
regarding the remaining limitation of claim 14, all the remaining limitations of claim are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to the remaining limitation of claim 14.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claim 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Carlotto
“Detection and Analysis of Change in Remotely Sensed Imagery with Application to Wide Area Surveillance” in view of QUI et al.,(hereafter QUI), CN 107154026 A, pub. 09/12/2017
Regarding claim 6, while Carlotto teaches the limitation of claim 1 but fails to teach the limitation claim 6 .
On the other hand in the same field of image processing, specifically to a shadow eliminating road-based adaptive brightness elevation model of the method of QUI teaches the change candidate extracting unit extracts a pixel obtained by performing area opening processing with respect to a change location candidate as a final change location candidate in the pair of the first aerial image and the second aerial image ( page 3rd par., using the Otsu method, namely the maximum inter-class variance method to calculate and divide the threshold value of the SDI enhanced image; obtaining the binary mask image, the pixel point less than the threshold value is black, representing the shadow area, the part greater than the threshold value is white, representing the non-shadow area; at last, using mathematical morphology opening operation to fill the hole, using closed operation to remove micro burr and debris, after morphological optimization processing, obtaining shadow detection final result)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a well- known morphology opening
operation to fill the hole and closed the operation to remove noise taught by QUI into Carlotto.
The suggestion/motivation for doing so would have been to allow user of Carlotto to
obtain a clear shadow image by removing noise from the shadow and restoring the shape of the shadow image.
As to claim 7, QUI teaches the change candidate extracting unit obtains a shadow area in the pair of the first aerial image and the second aerial image and the shadow area is excluded from the change location candidates (page 3 1st par., shadow land deletion. by analyzing the spectrum of the remote sensing image of the unmanned aerial vehicle, detail and so on, characteristics a method for recovering scene color and texture in shadow area of remote sensing image of unmanned aerial vehicle, so that the shadow area scene and the real color of the ground object have consistency.).
6. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Carlotto
“Detection and Analysis of Change in Remotely Sensed Imagery with Application to Wide Area Surveillance” in view of QUI, CN 107154026 A, farther in view of ANDERSON et al., ( hereafter ANDERSON), WO 2021243360 A1, pub. 12/02/2021
Regarding claim 8, while the combination of Carlotto and QUI teaches the limitation of claim 7 but fails to teach the limitation claim 8 .
On the other hand in the same field of endeavor a methods to perform an automated visual-inspection of components using image processing of ANDERSON teaches the change candidate extracting unit obtains a dark area of which a pixel value in both the first aerial image and the second aerial image is smaller than or equal to a first threshold and obtains a sum area of a first area created by expanding the dark area in the first aerial image to a range in which a pixel value is smaller than or equal to a second threshold that is larger than the first threshold and a second area created by expanding the dark area in the second aerial image to a range in which a pixel value is smaller than or equal to the second threshold ( Fig.6B, [0105] , If a determination is made at operation 607C that the defect is “large,” then erosion and/or dilation steps are performed at operation 607D. The erosion and dilation steps are performed potentially to link pixels identified as a portion of the large defect together. For example, an erosion step increases an area of black regions from the threshold
image (from operation 607A) to link the black regions together to group the black regions as a larger defect. Conversely, the dilation step reduces an area of black regions from the threshold image that are determined not to belong to a larger defect.
Consequently, in this embodiment, dilation increases the size of white areas and erosion increases size of black area)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of increasing an area of black regions from the threshold image to link a plurality of black regions together to group the black regions as a large black region using a well- known dilation operator taught by ANDERSON into modified Carlotto.
The suggestion/motivation for doing so would have been to allow user of modified Carlotto to connect nearby black regions that are separated by small white gaps, and effectively grouping them into a single black region by replacing small white spots inside black regions. Thus, the dilation operation making objects more solid (uniform color) and easier to detect.
Allowable Subject Matter
6. Claims 2-4 are objected to as being dependent upon a rejected base claim1 but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
7. Regarding claim 2 no prior art is found to anticipate or render the following limitation obvious:
“wherein the change index of a pixel is a value in accordance with a ratio of a change in a pixel value of the pixel to a mean of changes in pixel values in a peripheral region of the pixel.”
8. Regarding claim 3 no prior art is found to anticipate or render the following limitation obvious:
“wherein the change candidate extracting unit obtains the change index after adjusting at least one of the first aerial image and the second aerial image so that a mean value and a variance of pixel values of the first aerial image and the second aerial image are consistent.”
9. Regarding claim 4 no prior art is found to anticipate or render the following limitation obvious:
“wherein the change threshold is set for each pair as a greater value among a prescribed value and a value corresponding to a higher first prescribed percentage of change indices calculated from the first aerial image and the second aerial image.”
Change or variation or difference
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/ Primary Examiner, Art Unit 2699
Prosecution Insights