DETAILED ACTION
Claims 1–8 are pending in the instant application.
This Office Action is in response to Applicant’s amendment filed on 08/21/2026.
THIS OFFCIE ACTION IS MADE FINAL.
Response to Arguments
In re Applicant’s remarks, Examiner respectfully considered the amendment to determine whether the instant application is in condition for allowance. However, Examiner found that the amended claim language would not overcome prior art rejection because a new ground of rejection has been arisen during patent prosecution in view of Kottenstette et al. (U.S. 11,568,639 B2).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 8 are rejected under pre-AIA 35 U.S.C. § 103(a) as being unpatentable over Sung (U.S. 11,514,692) in view of Kottenstette (U.S. 11,568,639).
Regarding claim 1, Sung discloses a method performed by a computing device for detecting a change between images, the method comprising:
obtaining reference image data (reference image data construed as reference image model) and comparison target image data (target image data construed as target image model) of a same geographic location taken at different time points;1 and (Per Fig. 2, Sung’s image model building apparatus discloses a reference image model 210 in a trained model where the output is converted based on the target image model 220. Sung col. 7 lines 39–56. The image model building apparatus computes an output based on the reference image model 210 from the conversion image. The image model building apparatus generates an output based on the target image model 220 from an input image of the new training data. Further, the images shown in Figs. 7 and 9 indicate geographic location images.)
inputting a pair of image data including the reference image data and the comparison target image data into a pre-trained artificial neural network model; (Per Fig. 9, Sung’s image model apparatus discloses an additional image model 930 where a target image model 920 and a reference image model 910 are trained to render a plurality of layers. Ibid. col. 14 lines 32–39. The image model building apparatus generates the target image model 920 by connecting an additional layer L0 to a plurality of layers (for example, layers L1 through L4) of the reference image model 910.)
detecting a change in the comparison target image data relative to the reference image data by using the pre-trained artificial neural network model, (Per Fig. 6 at operation 627, Sung’s model building apparatus discloses a change between the reference model output and the target model output such that it robustly maintains a recognition performance. Ibid. col. 11 line 66 – col. 12 line 15. [w]hen a change in the error between the reference model output and the target model output is less than or equal to a threshold, the image model building apparatus determines that the error converges.)
However, Sung fails to specifically disclose generating an analysis result representing the change, wherein one or more first objects in the analysis result where the change is detected are represented in a first pixel value, and one or more second objects in the analysis result where no change is detected are represented in a second pixel value, and wherein the change comprises removal of one or more objects from, or inclusion of one or more objects into the reference image data.
In related art, Kottenstette discloses generating an analysis result representing the change, (Per Fig. 4, Kottenstette’s object detector 404 discloses comparison of object detection after generating labels based on pixel segmentation. Kottenstette col. 19 line 54 – col. 20 line 4. Object detector 404 can then determine a class for each of these center pixels and generate label map 406 based on the determined class.) wherein one or more first objects in the analysis result where the change is detected are represented in a first pixel value (a first pixel value construed as a white color), (Per Fig. 5B, Kottenstette discloses a method where targeted objects are labelled in two ways: foreground labels 506B and 512B are objects of interest—i.e., they are in white. Ibid. col. 20 lines 32–52. [f]oreground labels 506B and 512B indicate an object type of interest, while background labels 508B indicate any other areas that are not the object type of interest.) and one or more second objects in the analysis result where no change is detected are represented in a second pixel value (a second pixel value construed as a black color), (See Fig. 5B of his binary foreground 510B. In this case, the background 516B is in black. Ibid. Binary foreground mask 510B shows foreground 514B and 518B in white, and background 516B in black.) and wherein the change comprises removal of one or more objects from, or inclusion of one or more objects into the reference image data. (Through Figs. 5A–5B, Kottenstette discloses a comparison of object detection of his input image 502A to indicate whether a non-interested object is included in areas. Ibid. col. 20 lines 16–31. Foreground hints 506A and 512A are used to indicate an object type of interest, while background hints 508A are used to indicate any other areas that are not part of the object type of interest.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kottenstette into the teachings of Sung to provide a combined top-down and bottom-up approach in terms of object detection such that class-specific model features are obtained. Ibid. col. 9 lines 21–41.
Regarding claim 8, Sung discloses a computing device, comprising:
a processor including one or more mores; (Fig. 11, 1110 a processor)
a network unit for receiving one or more data; and (Fig. 11, 1100 an image model building apparatus)
a memory, (Fig. 11, 1120 a memory)
wherein the processor is configured to:
obtain reference image data (reference image data construed as reference image model) and comparison target image data (target image data construed as target image model) of a same geographic location taken at different time points; and (Per Fig. 2, Sung’s image model building apparatus discloses a reference image model 210 in a trained model where the output is converted based on the target image model 220. Sung col. 7 lines 39–56. The image model building apparatus computes an output based on the reference image model 210 from the conversion image. The image model building apparatus generates an output based on the target image model 220 from an input image of the new training data.)
input a pair of image data including the reference image data and the comparison target image data into a pre-trained artificial neural network model; (Per Fig. 9, Sung’s image model apparatus discloses an additional image model 930 where a target image model 920 and a reference image model 910 are trained to render a plurality of layers. Ibid. col. 14 lines 32–39. The image model building apparatus generates the target image model 920 by connecting an additional layer L0 to a plurality of layers (for example, layers L1 through L4) of the reference image model 910.)
detect a change in the comparison target image data relative to the reference image data by using the pre-trained artificial neural network model, (Per Fig. 6 at operation 627, Sung’s model building apparatus discloses a change between the reference model output and the target model output such that it robustly maintains a recognition performance. Ibid. col. 11 line 66 – col. 12 line 15. [w]hen a change in the error between the reference model output and the target model output is less than or equal to a threshold, the image model building apparatus determines that the error converges.)
However, Sung fails to specifically disclose generate an analysis result representing the change, wherein one or more first objects in the analysis result where the change is detected are represented in a first pixel value, and ono or more second objects in the analysis result where no change is detected are represented in a second pixel value, and wherein the change comprises removal of one or more objects from, or inclusion of one or more objects into the reference image data.
In related art, Kottenstette discloses generate an analysis result representing the change, (Per Fig. 4, Kottenstette’s object detector 404 discloses comparison of object detection after generating labels based on pixel segmentation. Kottenstette col. 19 line 54 – col. 20 line 4. Object detector 404 can then determine a class for each of these center pixels and generate label map 406 based on the determined class.) wherein one or more first objects in the analysis result where the change is detected are represented in a first pixel value (a first pixel value construed as a white color), (Per Fig. 5B, Kottenstette discloses a method where targeted objects are labelled in two ways: foreground labels 506B and 512B are objects of interest—i.e., they are in white. Ibid. col. 20 lines 32–52. [f]oreground labels 506B and 512B indicate an object type of interest, while background labels 508B indicate any other areas that are not the object type of interest.) and one or more second objects in the analysis result where no change is detected are represented in a second pixel value (a second pixel value construed as a black color), (See Fig. 5B of his binary foreground 510B. In this case, the background 516B is in black. Ibid. Binary foreground mask 510B shows foreground 514B and 518B in white, and background 516B in black.) and wherein the change comprises removal of one or more objects from, or inclusion of one or more objects into the reference image data. (Through Figs. 5A–5B, Kottenstette discloses a comparison of object detection of his input image 502A to indicate whether a non-interested object is included in areas. Ibid. col. 20 lines 16–31. Foreground hints 506A and 512A are used to indicate an object type of interest, while background hints 508A are used to indicate any other areas that are not part of the object type of interest.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kottenstette into the teachings of Sung to provide a combined top-down and bottom-up approach in terms of object detection such that class-specific model features are obtained. Ibid. col. 9 lines 21–41.
Claims 2, 4 and 7 are rejected under 35 U.S.C. § 103 as being unpatentable over Sung in view of Kottenstette and further in view of Brower (U.S. 11,250,296 B2).
Regarding claim 7, Sung discloses obtaining original image data at a single time point and a label corresponding to the original image data; (Per Fig. 8 at step 833, Sung’s model building apparatus discloses a target image in the machine learning model where the new input image is trained and the label is generated. Sung col. 13 line 60 – col. 14 line 7. [t]he image model building apparatus additionally trains the target image model based on the new input image and the label.)
generating transformed image data based on the original image data. (Per Fig. 9, Sung’s model building apparatus discloses an image in other resolution converting the trained input image. Ibid. col. 14 lines 40–64. The image model building apparatus converts the training input image with the third resolution into an image with a second resolution,)
Sung fails to specifically disclose generating a label of the transformed image data by applying the same transformation to the label of the original image data; generating a ground truth label based on the label of the original image data and the label of the transformed image data, wherein the ground truth label represents a change introduced by the transformed image data;
inputting a pair of image data including the original image data and the transformed image data into the change detection model; and
training the change detection model to output the change between the original image data and the transformed image data based on the ground truth label, wherein the change comprises removal or inclusion of one or more objects from the original image data.
In related art, Kottenstette discloses generating a label of the transformed image data by applying the same transformation to the label of the original image data. (Per Fig. 18 at step 1818, Kottenstette applies a transform parameter to convert a first type of image set into a second type of image set. Kottenstette col. 40 line 56 – col. 41 line 4. [a] transformed target second type of image set can be generated by transforming the target second type of image set using the transform parameter so that the transformed target second type of image set is aligned with the target first type of image set.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kottenstette into the teachings of Sung to provide a combined top-down and bottom-up approach in terms of object detection such that class-specific model features are obtained. Ibid. col. 9 lines 21–41.
Sung as modified by Kottenstette, discloses the claimed invention, but fails to specifically disclose generating a ground truth label based on the label of the original image data and the label of the transformed image data, wherein the ground truth label represents a change introduced by the transformed image data;
inputting a pair of image data including the original image data and the transformed image data into the change detection model; and
training the change detection model to output the change between the original image data and the transformed image data based on the ground truth label, wherein the change comprises removal or inclusion of one or more objects from the original image data.
In related art, Brower discloses generating a ground truth label based on the label of the original image data and the label of the transformed image data, (Per Fig. 1B, Brower’s object tracking 134 identifies a location of an object using his label generation 136 for each image 302 and 304 depicted in Fig. 3B. Brower col. 8 line 66 – col. 9 line 40. [t]he class labels 114 may be carried through as ground truth during the label generation 136 for the images 302 and 304.) wherein the ground truth label represents a change introduced by the transformed image data (a change construed as generating new object labels 140 between multiple images.); (Per Fig. 1B, Brower discloses new object labels 140, e.g., a vehicle 310 and a pedestrian 320 in Fig. 3B, in the images 302 and 304. Ibid. [t]o generate new object labels 140 through label generation 136 for the vehicle 310 and the pedestrian 320 in each of the images 302 and 304 (e.g., as illustrated in FIG. 3B).)
inputting a pair of image data including the original image data and the transformed image data into the change detection model; and (Per Fig. 4 at step B408, Brower discloses that a subset of images render a re-arranged ordering of the images corresponding to an object detected image. Ibid. col. 16 lines 1–18. [t]he subset of images (e.g., undetected object frame(s) 132, undetected object image(s) 206) may include a re-arranged ordering of the images from the sequence of images prior to the first image, and the subset of images in the re-arranged ordering may undergo object tracking 134 to track the object through the subset of images.)
training the change detection model to output the change between the original image data and the transformed image data based on the ground truth label, wherein the change comprises removal or inclusion of one or more objects from, or inclusion of one or more objects into the original image data. (Per Fig. 2A, Brower’s object detector 204 discloses new ground truth data in a trained model to determine whether a target object is tracked in image sequence 202. Ibid. col. 13 lines 13–19. [t]he detected object labels in the detected object image 208 may be leveraged to generate to new object labels (e.g., new ground truth data, where the object detector 204 is a machine learning model))
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Brower into the teachings of Sung and Kottenstette to efficiently and accurately to capture images from a different perspective. Ibid. col. 1 lines 28–46.
Regarding claim 2, it has been rejected in the same manner as claim 7.
Regarding claim 4, Sung as modified by Kottenstette and Brower, discloses the method, wherein the generating of the transformed image data based on the original image data includes: generating the transformed image data based on removing one or more objects included in the original image data. (Sung discloses whether a detected object is true or not in the new input image generating label information where its size and location is detected in the target image model. Sung col. 13 lines 34–43. [w]hen the target image model is a model to detect a size and a location of an object in an image, the image model building apparatus generates label information that indicates a size and a location of an object in the new input image.)
Claims 3 and 5–6 are rejected under 35 U.S.C. § 103 as being unpatentable over Sung in view of Kottenstette and Brower and further in view of Kim (U.S. 11,164,046 B1).
Regarding claim 3, Sung as modified by Kottenstette and Brower, discloses the claimed invention, but fails to specifically disclose the method, wherein the generating of the transformed image data based on the original image data includes:
generating the transformed image data based on rotating or inverting a region of at least a portion of the original image data.
In related art, Kim discloses the method, wherein the generating of the transformed image data based on the original image data includes:
generating the transformed image data based on rotating or inverting a region of at least a portion of the original image data. (Per Fig. 3 at step S40, Kim’s labeling server 1000 applies inverse transform function to generate partial labeled images. Kim col. 13 lines 18–26. [i]n response to acquiring the 1-st partial labeled image to the n-th partial labeled image from the labeler, the labeling server 1000 may apply a 1-st inverse transform function to an n-th inverse transform function)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kim into the teachings of Sung, Kottenstette and Brower to recover the original trained image data. Ibid. col. 2 lines 3–10.
Regarding claim 5, Sung as modified by Kottenstette and Brower, discloses the claimed invention, but fails to specifically disclose, wherein the generating of the transformed image data based on the original image data includes:
generating the transformed image data based on synthesizing one or more objects onto the original image data.
In related art, Kim discloses generating the transformed image data based on synthesizing one or more objects onto the original image data. (Per Fig. 4, in a user device 2000, Kim discloses a deep-learning object detection network, where an object is detected in each region in the original image. Kim col. 18 lines 47–59. [t]he labeling server 1000 may use various deep learning networks to classify each region from the original image. Especially, the deep-learning based object detection network may include the partial object detector capable of detecting partial bodies included within an object on the original image.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Kim into the teachings of Sung, Kottenstette and Brower to recover the original trained image data. Ibid. col. 2 lines 3–10.
Regarding claim 6, Sung as modified by Kottenstette, Brower and Kim, discloses the method, wherein the transformed image data is generated on Fourier blending. (Per Fig. 3 at step S40, Kim’s labeling server 1000 applies inverse transform function to generate partial labeled images. Kim col. 13 lines 18–26. [i]n response to acquiring the 1-st partial labeled image to the n-th partial labeled image from the labeler, the labeling server 1000 may apply a 1-st inverse transform function to an n-th inverse transform function)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENEDICT LEE whose telephone number is (571)270-0390. The examiner can normally be reached 10:00-17:00 (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R. Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENEDICT E LEE/Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
1 Examiner found that Sung’s image model apparatus discloses a point in time as his apparatus determines whether a detected object is true based on user’s feedback. Ibid. col. 13 lines 44–56. The image model building apparatus generates label information that is associated with a new input image acquired at a point in time at which a user input, such as a user operation of the brake pad, is acquired, and that indicates that a detected object is true.
Prosecution Insights