DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (2014, “true orthophoto generation using multi-view aerial images”, hereinafter referenced “Chen”) in view of Lv (2024, “True orthophoto generation using multi-view aerial images”, hereinafter referenced “Lv”).
In regards to claim 1. Chen discloses a true orthoimage generation apparatus (Chen “Abstract” section, page 67 and “2.2. Coarse Building outline acquisition” section, page 68; Sections disclose analyzing performance of true orthophoto generating method and use of program package SCOP++ thus use of a computing system to carry out these functions is implicit), comprising:
-a processor; and a memory configured to store instructions executed by the processor (Chen “Abstract” section, page 67 and “2.2. Coarse Building outline acquisition” section, page 68; Sections disclose analyzing performance of true orthophoto generating method and use of program package SCOP++ thus use of a computing system to carry out these functions which would possess a processor and memory capabilities is implicit),
-wherein the processor acquires a point cloud from a plurality of superimposed aerial images (Chen, Fig.1; Reference discloses the true orthophoto method and illustrates the steps of taking overlapped aerial images in which a point cloud is subsequently acquired)
-derives a plurality of local true orthoimages showing a vertical view of the point cloud (Chen, Figs.1 and 3; Reference in Figure 1 discloses the true orthophoto method and illustrates the steps of taking generating ortho-photo for each image. Fig. 3 illustrates the vertical view of the point cloud),
-and then generates a true orthoimage using the local true orthoimages (Chen, “2.4 Visibility analysis and True Orthophoto Generation” section, page 69; Reference discloses afterwards, we generate the true orthophoto by filling up the occluded areas with information from the neighboring orthophotos according to the visibility maps).
Chen does not explicitly disclose but Lv discloses
-(point cloud) through neural radiance field inference (Lv, “2.2 NeRF with Sparse Parametric Encodings” section continued page 4 and “3. Method” section, page 4; References at page 4 discloses. Both explicit and implicit methods require the initial step of SfM to obtain sparse point clouds and camera poses, implicit methods gradually fit to the real scene through implicit neural representation during the training process. Finally, both methods render digital orthophoto images from an orthographic viewpoint. The implicit method is interpreted as the NeRF method in which the point cloud is obtained).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 2. Chen in view of Lv teach the true orthoimage generation apparatus of claim 1.
Chen does not explicitly disclose but Lv teaches
-wherein the processor derives the point cloud using an artificial intelligence model for the neural radiance field inference and performs view synthesis on the point cloud to be viewed at continuous angles (Lv, “2.2 NeRF with Sparse Parametric Encodings” section continued page 3; Reference discloses Mildenhall et al. [5] introduced NeRF, which represents a scene as a continuous neural radiance field. NeRF optimizes a fully connected deep network (i.e. AI model for the neural radiance field inference) as an implicit function to approximate the volume density and view-dependent emitted radiance from 5D coordinates (x, y, z, q, f), with s representing the volume density at a spatial point. To render an image from a specific novel viewpoint, NeRF initially (1) generates camera rays traversing the scene and samples a set of 3D points along these rays, (2) inputs the sampled points and viewing directions into the neural network to obtain a collection of densities RGB values, and (3) employs differentiable volume rendering to synthesize a 2D image (i.e. performs view synthesis on the point cloud to be viewed at continuous angles)).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 3. Chen in view of Lv teach the true orthoimage generation apparatus of claim 2.
Chen does not explicitly disclose but Lv teaches
-wherein the artificial intelligence model learns the superimposed aerial images and a transformation extrinsic matrix that transforms points within a camera from the superimposed aerial images into a world coordinate system (Lv, Fig. 4; Reference discloses the figure illustrates the process of initializing the camera group. Based on the projection relationship of the pinhole camera, the projection pg of the raster’s world center point Pg on the image plane is obtained. The original space is evenly divided into eight regions, and the camera with the highest Sv in each region is found to be the camera group for this raster).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 4. Chen in view of Lv teach the true orthoimage generation apparatus of claim 1.
Chen does not explicitly disclose but Lv teaches
-wherein the processor arranges a plurality of virtual cameras that photograph the point cloud and acquires the local true orthoimages for each virtual camera (Lv, Fig. 1 and “2.2 NeRF with Sparse Parametric Encodings” section page 3; Reference discloses to render an image from a specific novel viewpoint, NeRF initially (1) generates camera rays traversing the scene and samples a set of 3D points along these rays, (2) inputs the sampled points and viewing directions into the neural network to obtain a collection of densities RGB values, and (3) employs differentiable volume rendering to synthesize a 2D image. The Fig. 1 illustrates the digital orthophoto output based on the NeRF process).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 5. Chen in view of Lv teach the true orthoimage generation apparatus of claim 4.
Chen does not explicitly disclose but Lv teaches
-wherein the virtual cameras are arranged at preset intervals at positions perpendicular to the point cloud (Lv, Fig. 1 and “2.2 NeRF with Sparse Parametric Encodings” section page 3; Reference discloses as mentioned in Section 1, digital orthophotos can be rendered with neural approaches. In contrast to the typical pinhole camera imaging model, digital orthophotos are rendered using a set of parallel light rays perpendicular to the ground, as shown in Figure 1.).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 6. Chen in view of Lv teach the true orthoimage generation apparatus of claim 4.
Chen does not explicitly disclose but Lv teaches
-wherein information related to direction and distance about the point cloud are set for the virtual camera (Lv, “3. Method” section, page 4-5; Reference discloses an implicit digital orthophoto generation method typically involves optimizing a group of parameters with posed images (i.e. information related to direction and distance set fir virtual camera). This optimization process often takes several hours or even dozens of hours. Instant NGP [7] represents a speed-optimized radiance field…Both methods rely on the sparse reconstruction results from SfM (i.e. point cloud)).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 7. Chen in view of Lv teach the true orthoimage generation apparatus of claim 6.
Chen does not explicitly disclose but Lv teaches
-wherein the information related to direction includes at least one of an angle formed by the virtual camera and a YZ plane of the point cloud and an angle formed by the virtual camera and an XY plane of the point cloud (Lv, “2.2 NeRF with Sparse Parametric Encodings” section page 3; Reference discloses In recent years, methods for novel view image synthesis on neural rendering have rapidly evolved. Mildenhall et al. [5] introduced NeRF, which represents a scene as a continuous neural radiance field. NeRF optimizes a fully connected deep network as an implicit function to approximate the volume density and view-dependent emitted radiance from 5D coordinates (x, y, z, q, f), with s representing the volume density at a spatial point. To render an image from a specific novel viewpoint, NeRF initially (1) generates camera rays traversing the scene and samples a set of 3D points along these rays, (2) inputs the sampled points and viewing directions into the neural network to obtain a collection of densities RGB values, and (3) employs differentiable volume rendering to synthesize a 2D image).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 8. Chen in view of Lv teach the true orthoimage generation apparatus of claim 4.
Chen does not explicitly disclose but Lv teaches
-wherein the processor generates the local true orthoimages at set patch intervals using the virtual cameras and acquires the true orthoimage by connecting the local true orthoimages generated at the set patch intervals (Lv, “3.1 Explicit Method – TDM” section continued, page 6 and Fig. 5; Reference at page 6 discloses Elevation Propagation: Given rasters with known elevation are considered as seed units gs, and the propagation starts iteratively from these seed units. Each iteration propagates the elevation information Z from the seed raster unit to all raster units within a patch. Fig. 5 illustrates the connecting of the set of patch based on intervals as it details the figure demonstrates the elevation propagation process. The red rectangular raster represents the seed unit. The seed unit with a known elevation and the surrounding eight raster units with unknown elevation form a raster support plane. The raster support plane calculates a matching score based on color consistency. If the score meets the threshold, the elevations of other raster units will be initialized based on the normal vector of the seed raster unit).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 9. Chen discloses a true orthoimage generation method (Chen “Abstract” section, page 67) comprising:
-acquiring, by a processor (Chen “Abstract” section, page 67 and “2.2. Coarse Building outline acquisition” section, page 68; Sections disclose analyzing performance of true orthophoto generating method and use of program package SCOP++ thus use of a computing system to carry out these functions which would possess a processor and memory capabilities is implicit), a point cloud from a plurality of superimposed aerial images (Chen, Fig.1; Reference discloses the true orthophoto method and illustrates the steps of taking overlapped aerial images in which a point cloud is subsequently acquired)
-deriving, by the processor, a plurality of local true orthoimages showing a vertical view of the point cloud (Chen, Figs.1 and 3; Reference in Figure 1 discloses the true orthophoto method and illustrates the steps of taking generating ortho-photo for each image. Fig. 3 illustrates the vertical view of the point cloud);
-and generating, by the processor, a true orthoimage using the local true orthoimages (Chen, “2.4 Visibility analysis and True Orthophoto Generation” section, page 69; Reference discloses afterwards, we generate the true orthophoto by filling up the occluded areas with information from the neighboring orthophotos according to the visibility maps).
Chen does not explicitly disclose but Lv discloses
-(point cloud) through neural radiance field inference (Lv, “2.2 NeRF with Sparse Parametric Encodings” section continued page 4 and “3. Method” section, page 4; References at page 4 discloses. Both explicit and implicit methods require the initial step of SfM to obtain sparse point clouds and camera poses, implicit methods gradually fit to the real scene through implicit neural representation during the training process. Finally, both methods render digital orthophoto images from an orthographic viewpoint. The implicit method is interpreted as the NeRF method in which the point cloud is obtained).
Chen and Lv are combinable because they are in the same field of endeavor regarding orthophoto image generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the multi-view aerial image true orthophoto generation method of Chen to include the implicit and explicit digital orthophoto generation features of Lv in order to provide the user with a method for the generation of true orthophotos from overlapping aerial images of known orientation as taught by Chen while incorporating the implicit and explicit digital orthophoto generation features of Lv to allow for use of implicit methods that rely on neural rendering by obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs) to provide more photorealistic rendering quality and lighting effects, applicable to improving orthophoto rendering methods such as those taught in Chen.
In regards to claim 10, please see the citations and rejection for corresponding apparatus claim 2.
In regards to claim 11, please see the citations and rejection for corresponding apparatus claim 3.
In regards to claim 12, please see the citations and rejection for corresponding apparatus claim 4.
In regards to claim 13, please see the citations and rejection for corresponding apparatus claim 5.
In regards to claim 14, please see the citations and rejection for corresponding apparatus claim 6.
In regards to claim 15, please see the citations and rejection for corresponding apparatus claim 7.
In regards to claim 16, please see the citations and rejection for corresponding apparatus claim 8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: See the Notice of References Cited (PTO-892)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TERRELL M ROBINSON whose telephone number is (571)270-3526. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KENT CHANG can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TERRELL M ROBINSON/Primary Examiner, Art Unit 2614
Prosecution Insights