DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The supplemental amendment filed on 4/20/26 is not proper because
1) the amendment fails to indicate each claim status, such as, “currently amended”; original, …
2) the amendment fails to either underline for amending or line across for deletion.
3) the amendment is not based on previous filed amendment date 4/13/26, such as, line 5, was amendment to “obtaining first set of images …”. However, the supplemental changes to “obtaining the set of images, …”.
Therefore, the supplemental amendment file on 4/20/26 is not considered.
Examiner only examined the amendment file on 4/13/26.
Please see MPEP for a supplemental amendment filed
(E) A supplemental amendment filed when there is no suspension of action under 37 CFR 1.103(a) or (c), the amendment will be forwarded to the examiner. Such a supplemental amendment is not entered as a matter of right. See 37 CFR 1.111(a)(2)(ii). The examiner will notify the applicant if the amendment is not approved for entry. The examiner may use form paragraph 7.147. See MPEP § 714.03(a).
Applicant’s arguments with respect amended claims 1-2, 4-12, 14-16, and 18-20 filed on 04/13/2026 have been considered but they are moot in view of the new ground (s) of rejection.
Independent claims 10, 15 has been amended similarly to claim 1 and is rejected as the explanation above.
Dependent claims 2-9, 11-14, 16-20 depend on independent claims 1, 10, 15 and rejected as current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10, 15 are rejected under 35 U.S.C. 103 as being unpatentable by Edgar et al. (U.S. 2021/0034920A1) in view of Estrada et al. (U.S. 2018/0158210A1).
Regarding Claim 1 (Currently amended), Edgar discloses one or more non-transitory computer-readable media (Edgar, [0107] “a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects”) storing computer-executable instructions that, when executed by at least one processor, perform a method of automated classification and labeling of objects in a set (Edgar, [0039] “optimizing one or more machine learning models (e.g., machine learning model 110, M1) using accurately annotated/labeled training data samples” and [0040] the machine learning model M1 can include a DNN image analysis model configured to automatically generate an inference classification based on the medical images” Edgar teaches a method (machine learning model) performs automated classification and labeling of objects (data samples) in a set of images (medical images), the method comprising:
obtaining a first set of images including satellite imagery (Edgar, [0047] “the collection component 202 can collect or receive hundreds to thousands to millions (or more) of unannotated medical images” and [0121] “A user enters information into the computer 1502 through input device(s) 1528. Input devices 1528 include satellite dish” Edgar teaches obtaining a first set of images including satellite imagery (unannotated medical images are entered via a satellite dish);
wherein the first set of images depicts a region of interest and the objects within the region of interest; (Edgar, [0050] “the annotation pipeline module 112 can learn that the model consistently generates low confidence diagnosis for medical images from a specific geographic region and [0121] “” Edgar teaches images show a region of interest (referred to as a specific geographic region) and;
reducing the first set of images to a first subset, [[and]] a second subset (Edgar, [0026] “using numerical, optimization techniques that reduce the error between the desired class label and the algorithm's prediction” [0034] “apply the machine learning model to the unannotated data sample to generate an inference result and compare this inference result with the applied annotation to facilitate determining the accuracy of the annotation” and [0057] “the machine learning model (M1) based on application to the new unannotated data samples included in the annotation queue with same or similar attributes and [0096] “At 906, the system can select, a first annotation technique (e.g., a manual annotation technique) for annotating a first subset of the unannotated data samples based on association of the first subset with a first priority level (e.g., a high priority level relative to a defined threshold) of the priority levels At 908, the system can further select, a second annotation technique (e.g., an automated annotation technique) for annotating a second subset of the unannotated data samples based on association of the second subset with a second priority level (e.g., a low priority level relative to a defined threshold) of the priority levels) Edgar teaches reducing the first set of images (the unannotated images) to a first subset of the unannotated data samples based on association with a high priority level and a second subset of the unannotated data samples based on association with a low priority level;
receiving annotations of the objects in the s (Edgar, [0121] “A user enters information into the computer 1502 through input device(s) 1528. Input devices 1528 include satellite dish” and Fig. 4, [0071] “the technique 1 annotation component 302
can be configured to facilitate applying manual annotations to the medical images, resulting in generation of a first subset 402 of manually annotated medical images” Edgar teaches receiving orthorectified medical images (referred to as the medical satellite images) via a satellite dish and applying manual annotations of objects (annotated data samples) to the medical images (Fig. 4).
forward projecting the annotations to each image of a third subset to obtain annotated images (Edgar, [0012] “Fig. 4, example subsets of annotated data samples generated by the annotation component in association with application to medical images” and [0069] “a third subset of annotated data samples generated via annotation technique 3” Edgar teaches forward projecting the annotations to each image e.g., example subsets of annotated data samples generated to medical images of the third subset (Fig. 4, 406) to obtain annotated images.
training a neural network by at least the first subset including the annotated images (Edgar, [0005] “select the first annotation technique for a first subset of the unannotated data samples based on association of the first subset with a first annotation priority level” [0039] “the model development module 108 can facilitate training and/or optimizing one or more machine learning models (e.g., machine learning model 110, M1) using accurately annotated/labeled training data samples” and [0044] “the machine learning model M1 can be a neural network model” Edgar teaches the facilitate training the first subset of unannotated data samples use accurately annotated training data samples by a machine learning model/neural network model; and
classifying and labeling the objects in [[the]] a second [[sub]]set of images by the neural network (Edgar, [0026]“The class labels are then used by the learning algorithm to adapt and change its' internal, mathematical representation (such as the behavior of artificial neural networks) of the data” and [0039] “optimizing one or more machine learning models (e.g., machine learning model 110, M1) using accurately annotated/labeled training data samples” and [0083] “in which the data samples are medical images, the annotation accuracy evaluation component 604 can find annotated training images” Edgar teaches the class labels the objects (data samples) in a second set of images (a set of annotated medical images) by learning algorithm of artificial neural network.
Edgar discloses the annotation management component 204 can facilitate rendering the prioritization information at a device associated with a user responsible for managing and/or controlling annotation of the unannotated data samples ([0052]).
However, Edgar does not explicitly teach wherein the first subset and the second subset include the region of interest and the objects within the region of interest viewed from different angles;
creating orthorectified images by orthorectifying the first subset and the second subset and aggregating the first subset and the second subset to remove
occluded areas based on the different angles;
wherein the third subset comprises images including the region of interest;
Estrada teaches wherein the first subset and the second subset include the region of interest and the objects within the region of interest viewed from different angles (Estrada, [0033] “orthorectified geospatial image” refers to satellite imagery of the earth that has been digitally corrected to remove terrain distortions introduced into the image by either angle of incidence of a particular point from the center of the satellite imaging sensor” and [0073] FIG. 9, consisting of a two-panel figure, 900. The left panel 910 of 900 shows an orthorectified geospatial image that has been scaled and tagged with yellow to allow those who receive the report to confirm a grouping of airliner category objects 912, 913, 914, 915. The right panel of FIG. 9, 920, again shows an orthorectified geospatial image that has been scaled and tagged this time with purple to allow those who receive the report to confirm a grouping of fighter aircraft category objects 922, 923, 924” Estrada teaches the 1st subset (910) and the 2nd subset (920) include the region of interest and the objects (category objects) within the ROI viewed from different angle of orthorectified geospatial images (Fig. 9).
creating orthorectified images by orthorectifying the first subset and the second subset and aggregating the first subset and the second subset to remove
occluded areas based on the different angles (Estrada, [0033] “orthorectified geospatial image” refers to satellite imagery of the earth that has been digitally corrected to remove terrain distortions introduced into the image by either angle of incidence of a particular point from the center of the satellite imaging sensor” and [0071] “FIG. 7 the filter changes all pixels constituent to the cloud to a single color value 702 to clearly demarcate the portion of the image that is obstructed. Further software within the image correction…for object identification 540 based upon a pre-selected threshold 703” and [0073] FIG. 9, consisting of a two-panel figure, 900. The left panel 910 of 900 shows an orthorectified geospatial image… The right panel of FIG. 9, 920, again shows an orthorectified geospatial image…” Estrada teaches creating orthorectified images with a first subset (910) and a second subset (920) (Fig. 9) by remove occlude areas (the cloud, Fig. 7) based on the different angles.
wherein the third subset comprises images including the region of interest (Estrada; [0075] FIG. 12 depicts a comparison panel 1200 comprising three pairs of images: 1201, 1202, 1203; within each pair of images, a circled object of interest 1225, 1245, 1265, 1267. Image pair 1203 illustrates a plurality of synthetic image 1265, 1267”
Estrada teaches the third subset (1203) includes the region of interest (1265, 1267).
Edgar and Estrada are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Edgar to combine with a 1st subset and 2nd subset include ROI and objects (as taught by Estrada) in order to apply a 1st subset and 2nd subset include ROI and objects are view from different angles because Estrada can provide the 1st subset (910) and the 2nd subset (920) include the region of interest and the objects (category objects) within the ROI viewed from different angle of orthorectified geospatial images (Fig. 9) (Estrada , [0033], Fig. 9, [0079]). Doing so, it may allow the use of geospatial imagery to map remote regions, to track deforestation and re-forestation and to detect natural disasters in remote areas of the world (Estrada, [0005]).
Regarding Claim 10 (Currently amended), a combination of Edgar and Estrada discloses a system (Edgar, [0003] “a system”) for automated classification and labeling of objects in a plurality of images (Edgar, [0040] the machine learning model M1 can include a DNN image analysis model configured to automatically generate an inference classification based on the medical images”), the system comprising:
at least one processor (Edgar, [0003] “a processor”);
a datastore (Edgar, [0003] “a memory that stores computer executable components”); and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the at least one processor, perform a method comprising:
obtaining the a first set of images including satellite imagery, wherein the first set of images depicts a region of interest and the
objects within the region of interest
reducing the first set of images to a first subset, [[and]] a second subset.
wherein the first subset and the second subset include the region of interest and the objects within the region of interest viewed from different angles;
creating orthorectified images by orthorectifying the first subset and the
second subset and aggregating the first subset and the second subset
to remove occluded areas in the first subset and the second subset;
receiving annotations of the objects in the orthorectified images;
forward projecting the annotations to each image of a third subset to obtain annotated images,
wherein the third subset includes images of the region of interest;
training a neural network by at least the annotation associated with [[of]] the objects in the annotated images; and
classifying and labeling the objects in [[the]] a second [[sub]]set of images by the neural network.
Claim 10 is substantially similar to claim 1 is rejected based on similar analyses.
Regarding Claim 15 (Currently amended), a combination of Edgar, Estrada and Fathi discloses a method (Edgar, [0029] “computer-implemented methods”) of automated classification and labeling of objects in a plurality of images (Edgar, [0040] the machine learning model M1 can include a DNN image analysis model configured to automatically generate an inference classification based on the medical images”), the method comprising:
obtaining the a first set of images including satellite imagery, wherein the first set and the
objects within the region of interest
reducing the first set .
wherein the first subset and the second subset include the region of interest and the objects within the region of interest viewed from different angles;
creating orthorectified images by orthorectifying the first subset and the
second subset and aggregating the first subset and the second subset
to remove occluded areas in the first subset and the second subset;
receiving annotations of the objects in the orthorectified images; created from the first subset and the second subset (Edgar, Fig. 4, [0071] “the technique 1 annotation component 302 can be configured to facilitate applying manual annotations to the medical images, resulting in generation of a first subset 402 of manually annotated medical images. The technique 2 annotation component 304 can be configured to perform a metadata extraction technique to generate a second subset 404 of automatically annotated medical images” Edgar teaches created annotated medical images from 1st subset and 2nd subset;
forward projecting the annotations to each image of a third subset to obtain annotated images,
wherein the third subset includes images of the region of interest;
training a neural network by at least by the first subset including the annotation of the objects; and
classifying and labeling the objects in [[the]] a second [[sub]]set of images by the neural network.
Claim 15 is substantially similar to claim 1 is rejected based on similar analyses.
Claims 2-4, 6-9, 11, 13-14, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable by Edgar et al. (U.S. 2021/0034920A1) in view of Estrada et al. (U.S. 2018/0158210A1) and further in view of Fathi et al. (U.S. 2020/0082168 A1).
Regarding Claim 2 (Currently amended), the media of claim 1, a combination of Edgar and Estrada does not explicitly teach wherein the method further comprises:
determining the first subset and the second subset by comparing viewing parameters between each image of the first set of images; and selecting one or more pairs of the images at the first subset and the second subset comprising compatible viewing parameters.
However, Fathi teaches determining the first subset and the second subset by comparing viewing parameters between each image of the first set of images; and selecting one or more pairs of the images at the first subset and the second subset comprising compatible viewing parameters (Fathi, [0126] “Image processing techniques as discussed elsewhere herein that suitably compare two or a plurality of images together and highlights the differences and/or track the severity and size change over time can be used” and [0076] “This linkage structure can be interpreted as a generalization scheme and will be based on linking the features from each pair of views. Viewpoint parameters are also represented by T and S” Fathi teaches determining the 1st subset and 2nd subset by comparing two images and highlight the differences of the severity and size change (viewing parameters) and select the features from each pair of views include viewpoint parameters (T, S).
Edgar, Estrada and Fathi are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Edgar to combine with comparing viewing parameters between each image (as taught by Fathi ) in order to determining the first subset and the second subset by comparing viewing parameters between each image because Fathi can provide determining the 1st subset and 2nd subset by comparing two images and highlight the differences of the severity and size change (viewing parameters) and select the features from each pair of views include viewpoint parameters (T, S) (Fathi , [0076], [0126]). Doing so, it may provide image that has the highest SSD for a specific object or surface, image that has the minimum occlusion for a specific object or surface (Fathi , [0240]).
Regarding Claim 3, the media of claim 2, a combination of Edgar and Estrada does not explicitly teach wherein the method further comprises determining a digital surface model from the one or more pairs of the images;
However, Fathi teaches determining a digital surface model from the one or more pairs of the images (Fathi, [0126] “Image processing techniques as discussed elsewhere herein that suitably compare two images together” and [0239] “the capture plan on a 3D rendering of the asset in the form of an orthographic aerial imagery, digital surface model” Edgar teaches determining a digital surface model assets in the form of an aerial imagery and applying to image processing to two images together (a pair of the images).
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 4 (Currently amended), the media of claim 3, Edgar does not explicitly teach wherein the method further comprises orthorectifying the first subset and the second subset based on metadata of satellites used to obtain the first set of images and aggregating an image.
However, Estrada teaches the method further comprises orthorectifying the first subset and the second subset based on metadata of satellites used to obtain the first set of images (Estrada, [0033] ““orthorectified geospatial image” refers to satellite imagery of the earth that has been digitally corrected to remove terrain distortions introduced into the image by either angle of incidence of a particular point from the center of the satellite imaging sensor” and [0068] “orthorectified geospatial image segments that have been labeled to identify feature items which will be used to train the image analysis. The scale or resolution of images to be used in training will be determined from metadata included with the image segments 604.” Estrada teaches orthorectifying the first subset and the second subset based on metadata of satellites (image segments 604) used to obtain the first set of images.
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 6 (Currently amended), the media of claim 1, a combination of Edgar and Estrada does not explicitly teach wherein the objects are buildings, and wherein the annotations are indicative of shapes of the buildings.
However, Fathi teaches the objects are buildings, and wherein the annotations are indicative of shapes of the buildings (Fathi, [0030] “a “physical asset of interest” includes buildings, all or parts (e.g., internal or external) of a building” and [0005] “an assessment of basic information about a physical asset such as the general size, shape, materials used etc.” and [0121] “labeling and classification can be relevant elements in inspection for assessment of a physical asset(s) of interest” Fathi teaches objects (physical assets of interest) are building and labeling/classification of shapes of the buildings.
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 7 (Currently amended), Edgar discloses the media of claim 6, wherein the method further comprises:
classifying and labeling the second [[sub]]set of images by:
inputting of images into the neural network (Edgar, [0026] “The class labels are then used by the learning algorithm to adapt and change its' internal, mathematical representation (such as the behavior of artificial neural networks) of the data” and [0066] “The annotation application can further generate and apply an annotation or label to the medical image based on the user input” Edgar teaches applying the class labels to the second subset of the medical image based on the user input into an artificial neural networks; and
However, a combination of Edgar and Estrada does not explicitly teach training the neural network for building segmentation based at least on the orthorectified images and the annotated images
determining a probability of each pixel containing a building in the second
set of images
Fathi teaches training the neural network for building segmentation based at least on the orthorectified images and the annotated images (Fathi, [0030] “a “physical asset of interest” includes buildings, all or parts (e.g., internal or external) of a building” [0074] “The selected sensor types and data attainable therefrom can include, thesatellite imagery, manned or unmanned aerial orthographic imagery, [0078] “Machine learning-based object identification, segmentation, and/or labeling algorithms can be used to identify a collection physical asset of interest. A directed graph can be used to build the neural networks. Each unit can be represented by a node labeled according to its output” Fathi teaches training (machine learning-based) the neural network for the building segment (object segment of the physical asset) based on the orthorectified images (orthographic imagery) and the annotated images (a node labeled); and
determining a probability of each pixel containing a building in the second
set of images (Fathi, [0030] “a “physical asset of interest” includes buildings” and [0061] “the one or more physical assets of interest can be generated by using probabilistic algorithms” and [0078] “Deep Convolutional Neural Network (DCNNs) can be used to assign a label to one or more portions of an image (e.g., a set of pixels creating a regular or irregular shape) that include a given physical asset of interest” Fathi teaches determining a probability of each pixel (using probabilistic algorithms) containing a building (a physical asset) based on the neural network
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 8 (Currently amended), the media of claim 1, a combination of Edgar and Estrada does not explicitly teach wherein the method further comprises: receiving traffic infrastructure annotation indicative of traffic infrastructure; training the neural network based on the traffic infrastructure; and determining the traffic infrastructure in the second set of images.
However, Fathi teaches receiving traffic infrastructure annotation indicative of traffic infrastructure (Fathi, [0030] “a “physical asset of interest” includes transportation infrastructure (e.g., roads, bridges, ground stockpiles) and [0013] “bridges, roads, etc. can also require the acquisition of 2D and/or 3D data derived from sensors when conducting inspection, repair…” Fathi teaches receiving traffic infrastructure (transportation infrastructure) annotation indicative of traffic infrastructure such as roads, bridges, etc. when conducting inspection, repair.
training the neural network based on the traffic infrastructure (Fathi, [0030] “a “physical asset of interest” includes transportation infrastructure (e.g., roads, bridges, ground stockpiles) and [0078] “Deep Convolutional Neural Networks (DCNNs) can be used to assign a label to one or more portions of an image that include a given physical asset of interest” Fathi teaches training the neural network based on the traffic infrastructure, e.g., roads, bridges; and
determining the traffic infrastructure in the second set of images (Fathi, [0012] “as well as other UAV data capture events, a UAV device outfitted with sensors will gather hundreds of images. With a human operator managing the image acquisition” and [0013] “Other forms of infrastructure components, such as …bridges, roads, etc. can also require the acquisition of 2D and/or 3D data derived from sensors when conducting inspection, repair…” Fathi teaches determining the traffic infrastructure (bridges, road) in the second set of acquisition images (forms of infrastructure components).
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 9 (Currently amended), the media of claim 1, a combination of Edgar and Estrada does not explicitly teach wherein the first set of images comprises one of satellite data, radar data, LiDAR data, scanning laser mapping data, or stereo photogrammetry data.
However, Fathi teaches the first set of images comprises one of satellite data, LiDAR data (Fathi, [0074] “The selected sensor types and data attainable therefrom can include satellite imagery, airborne LiDAR, terrestrial LiDAR” Fathi teaches the first set of images include one of satellite data, LiDAR data.
Edgar, Estrada and Fathi are combinable see rationale in claim 2.
Regarding Claim 11 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the system of claim 10, wherein the method further comprises:
determining the first subset and the second subset by comparing viewing parameters between each image of the first set
creating the first subset and the second subset by selecting one or more pairs of [[the]] images from [[at]] the first subset and the second subset comprising compatible viewing parameters;
determining a digital surface model from the one or more pairs of the images; and
orthorectifying and aggregating the images [0003] “should be directed toward a goal of obtaining improved accuracy (e.g., accurate measurements, enhanced detail, etc.) and thus more complete data collection during an imaging process” and [0074] “the sensor types and associated sensor data characteristics can be selected to allow collection of physical assets of interest to be generated. The selected sensor types and data attainable can include manned or unmanned aerial orthographic imagery” Fathi teaches the method further includes aggregating (collecting data) during an imaging process and an orthographic imagery.
Claim 11 is substantially similar to claims 2 and 3 and is rejected based on similar analyses.
Regarding Claim 13 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the system of claim 10,
wherein the objects are buildings,
wherein the annotation are [[is]] indicative of shapes of the buildings.
Claim 13 is substantially similar to claim 6 is rejected based on similar analyses.
Regarding Claim 14 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the system of claim 13, wherein the method further comprises training the neural network for building segmentation based on the annotated images
inputting the images of the second [[sub]]set of images into the neural network; and
determining a probability of each pixel containing a building based on the neural network.
Claim 14 is substantially similar to claim 7 is rejected based on similar analyses.
Regarding Claim 17, a combination of Edgar, Estrada and Fathi discloses the method of claim 15, wherein the objects are buildings, wherein the annotation is indicative of shapes of the buildings.
Claim 17 is substantially similar to claim 6 is rejected based on similar analyses.
Regarding Claim 18 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the method of claim 17, further comprising training the neural network for building segmentation based on the annotated images first subset and classifying and labeling the second [[sub]]set of images by:
inputting [[the]] images of the annotated images second subset into the neural network; and
determining a probability of each pixel containing the buildings based on the neural network.
Claim 18 is substantially similar to claim 7 is rejected based on similar analyses.
Regarding Claim 19 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the method of claim 15, further comprising: receiving traffic infrastructure annotation indicative of traffic infrastructure; training the neural network based on the traffic infrastructure; and determining the traffic infrastructure in the second set
Claim 19 is substantially similar to claim 8 is rejected based on similar analyses.
Regarding Claim 20 (Currently amended), a combination of Edgar, Estrada and Fathi discloses the method of claim 15, wherein the first set
Claim 20 is substantially similar to claim 9 is rejected based on similar analyses.
Claims 5, 12, 16 are rejected under 35 U.S.C. 103 as being unpatentable by Edgar et al. (U.S. 2021/0034920 A1) in view of Estrada et al. (U.S. 2018/0158210A1) and further in view of Fathi et al. (U.S. 2020/0082168 A1) and further in view of Wu et al. (U.S. 2019/0019324 A1).
Regarding Claim 5 (Currently amended), the media of claim 3, a combination of Edgar, Estrada and Fathi does not explicitly teach wherein the method further comprises:
detecting the occluded areas and shadows in the orthorectified images based on the digital surface model; and
.
aggregating the orthorectified images to minimize the occluded areas further based on the digital surface model.
However, Wu teaches detecting occluded areas and shadows in the digital surface model (Wu [0029] “raw texture images (e.g., from aerial or satellite imager) can be used as textures, raw texture images captured by airborne or terrestrial cameras may suffer from heavy shadow, occlusion, image noise/distortion” and [0033] “a “3D Building Model” can refer to a “Digital Surface Model” (DSM), a single surface that includes buildings, ground, etc.” and [0036] “for 3D building models, textures can be photorealistic (projected from aerial) or abstract (simplified representations of the raw textures). Typically, textures of a DSM are generally photorealistic” Wu teaches detecting occluded areas (occlusion) and shadow in the orthorectified images (referred to as aerial or satellite images) based on the DSM; and
aggregating the orthorectified images to minimize the occluded areas further based on the digital surface model (Wu, [0029] “raw texture images (e.g., from aerial or satellite imager) can be used as textures, raw texture images captured by airborne or terrestrial cameras may suffer from heavy shadow, occlusion, image noise/distortion” and [0038] “FIG. 2, in a photorealistic texture 201 of a building wall or facade, some of the wall texture 201 can be obscured by trees, cars or other buildings and colors may be distorted by shadows and reflections 203” and Fig. 4, [0045] “ the system 100 can apply a morphological “open” and “close” operations to the image mask 411 generated” and [0057] “minimization of deviations just results in an abstract texture that potentially reflects all the errors and distortions of the input images” Wu teaches aggregating an orthorectified image with mask layers e.g., Figs. 2, 4, are orthro/rectangular images with mask layers to minimize occluded areas (closed/black areas) and areas in shadow (white areas).
Edgar, Estrada, Fathi and Wu are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Edgar to combine with detecting occluded areas (occlusion) and shadow in the DSM (as taught by Wu) in order to detect occluded areas (occlusion) and shadow in the DSM because Wu can provide detecting occluded areas (occlusion) and shadow in the DSM (Wu, [0033], [0036], [0039]). Doing so, it may provide efficiently creating textures which realistically represent real world objects such as buildings (Wu, [0001]).
Regarding Claim 12 (Currently amended), a combination of Edgar, Estrada, Fathi and Wu discloses the system of claim 11, wherein the method further comprises:
detecting the occluded areas and shadows in the orthorectified images;
generating [[an]] images with mask layers indicating the occluded areas and the shadows.
Claim 12 is substantially similar to claim 5 is rejected based on similar analyses.
Regarding Claim 16 (Currently amended), a combination of Edgar, Estrada, Fathi and Wu discloses the method of claim 15, further comprising:
detecting the occluded areas and shadows; and
generating an image with [[a]] mask layer indicating the occluded areas and the shadows.
Claim 16 is substantially similar to claim 12 is rejected based on similar analyses.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHOA VU whose telephone number is (571)272-5994. The examiner can normally be reached 8:00- 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHOA VU/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights