DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Berg et al. (“Computational Geometry”) in view of Facchin et al. (US 20120069012 A1) and Fiser et al. (US 20240420424 A1).
Regarding Claim 1, Berg teaches A method
performing predetermined first image processing on an acquired target data of a scene data
Berg p. 191:
PNG
media_image1.png
154
442
media_image1.png
Greyscale
Here, the height image/map,
PNG
media_image2.png
156
158
media_image2.png
Greyscale
, is mapped to the first image after grayscale conversion of collected target data of a terrain.
The Examiner’s mapping is consistent with the specification and Applicant’s later claim. Claim 2 recites, “the three-dimensional coordinate system takes a length of the grayscale image as a first axis, a width of the grayscale image as a second axis, and a grayscale value of the grayscale image as a third axis, the first axis and the second axis are in a same foundation plane, and the third axis is perpendicular to the foundation plane,” which is identical or substantially similar to Berg’s description.);
performing, in accordance with a predetermined pixel grouping condition (condition to group pixels into triangles), second image processing on the first image processing result (initial height map), so as to obtain a second image processing result (
PNG
media_image3.png
74
306
media_image3.png
Greyscale
) corresponding to the first image processing result (initial height map), the second image processing comprising three-dimensional projection (projection from 3D height map to planar surface as shown below) and pixel grouping (grouping pixels into triangles), and the second image processing result comprising a plurality of triangular patches (triangular patches in
PNG
media_image3.png
74
306
media_image3.png
Greyscale
) (
Berg p. 192:
PNG
media_image4.png
340
470
media_image4.png
Greyscale
);
performing third image processing on each of the plurality of triangular patches in the second image processing result, so as to obtain a third image processing result (
PNG
media_image5.png
102
178
media_image5.png
Greyscale
) corresponding to all of the plurality of triangular patches (
PNG
media_image3.png
74
306
media_image3.png
Greyscale
), the third image processing comprising determination of vertexes (“We then lift each sample point to its correct height, thereby mapping every triangle in the triangulation to a triangle in 3-space.” Berg p. 192.) for each of the plurality of triangular patches, determination of intersections (vertical lines rising from the vertices in a 2D plane) between a plane (2D flat plane) where each of the plurality of triangular patches is located and a predetermined section (section defined by known heights), and connection (3D triangles) of the intersections (vertical lines) (“We then lift each sample point to its correct height, thereby mapping every triangle in the triangulation to a triangle in 3-space. Figure 9.2 illustrates this. What we get is a polyhedral terrain, the graph of a continuous function that is piecewise linear. We can use the polyhedral terrain as an approximation of the original terrain.”); and
performing, in accordance with the pixel grouping condition (complementing convex hulls by unbounded faces), fourth image processing on non- triangular parts (unbounded area) in the third image processing result, so as to obtain a fourth image processing result corresponding to the third image processing result
PNG
media_image6.png
156
500
media_image6.png
Greyscale
However, Berg does not explicitly disclose; however, Facchin teaches the acquired target data of a scene is an image (“In particular, the image base is the distance which separates the two image centers, which are the points in space in which the two photographs of a stereoscopic image are taken. The height H is the average height between the image base B and the framed surface; in the aerial case, H is the relative flying height, i.e. the difference between the flying height and the average height of terrain in that zone” Facchin ¶ 5.
PNG
media_image7.png
242
368
media_image7.png
Greyscale
PNG
media_image8.png
372
534
media_image8.png
Greyscale
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Facchin’s height/depth map acquiring method with primary reference Berg. One of ordinary skill in the art would be motivated to efficiently and/or accurately acquire a height/depth map of a terrain. Facchin recites “In light of the prior art, it is the object of the present invention to provide a method in which photogrammetric restitution is point cloud assisted to allow a more complete visualization of the area.” Facchin ¶ 6.
Berg in view of Facchin does not explicitly disclose; however, Fiser teaches a method for plane segmentation of a three-dimensional rendering gradient multi-level plane (preamble); the fourth image processing comprising segmentation processing, classification processing, and gradient color rendering, and the fourth image processing result comprising a target rendered image corresponding to the target image (
segmentation processing [Wingdings font/0xE0]“The segments are then used as a basis to form patches as respective single gradient meshes defined using initial vertexes and corresponding color values. The patches, for instance, are generated by the gradient mesh processing system based on variability of colors within the segments, e.g., to represent respective color values within a threshold amount.” Fiser ¶ 23.
segmentation processing [Wingdings font/0xE0]“The gradient mesh processing system 116 first employs, optionally, a segmentation module 204 to generate segments 206 based on the raster digital image 202 (block 904). The segments 206 act as a guide in this example to improve subsequent patch formation by first defining segments as semantically related collections of pixels. ” Fiser ¶ 37.
classification processing [Wingdings font/0xE0] “The segmentation module 204 may also utilize a machine-learning model (e.g., convolutional neural network or other deep-learning technique) that is trained as a classifier to assign tags (e.g., semantic tags) to respective pixels. A variety of other examples are also contemplated. ” Fiser ¶ 39.
gradient color rendering [Wingdings font/0xE0] “The method as described in claim 1, wherein the rendering includes rendering a gradient of color values for a respective said patch based on color values defined for respective said vertexes from the vertex buffer as indexed by the index buffer.” Fiser Claim 11.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Fiser’s image processing method with Berg in view of Facchin. One of ordinary skill in the art would be motivated to process gradient mesh processing more efficiently and/or accurately. Fiser address the problem that it tries to solve, stating “These challenges result in visual artifacts due to the inaccuracies, inefficient use of computational resources when attempting to correct the inaccuracies, and increased power consumption.” Fiser ¶ 1.
Claims 8 and 15 are substantially similar to Claim 1. The rejection analyses for Claim 1 based on Berg in view of Facchin and Fiser are applied to Claims 8 and 15. In addition, Claim 8 recites “An apparatus for plane segmentation of a three-dimensional rendering gradient multi-level plane, comprising: a memory, memorized with an executable code; and a processor, coupled with the memory, wherein the processor invokes the executable code memorized in the memory to perform a method for . . .” (Facchin ¶¶ 18, 21). Claim 15 recites, “A non-transitory computer memory medium, wherein, the non-transitory computer memory medium memorizes computer instructions, and the computer instructions, when invoked, are configured to cause a method for . . . ” (Facchin ¶¶ 18, 21).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Facchin’s computer with Berg in view of Facchin and Fiser. One of ordinary skill in the art would be motivated to conduct computation quickly and to save computation resources.
Allowable Subject Matter
Claims 2-7, 9-14, and 16-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Claim 2 is distinguished from closest prior art combination: Berg et al. (“Computational Geometry”) in view of Facchin et al. (US 20120069012 A1) and Fiser et al. (US 20240420424 A1), which fail to teach Claim’s following limitations:
performing pixel division on all of the plurality of three-dimensional pixels in the two- dimensional view image, so as to obtain a plurality of quadrangles, wherein each of the plurality of quadrangles consists of four of the plurality of three-dimensional pixels and all of the plurality of quadrangles do not overlap each other;
calculating, for each of the plurality of quadrangles, a diagonal distance corresponding to any two diagonal pixels in the respective quadrangle, selecting, in accordance with a predetermined pixel grouping requirement, a target diagonal distance from the two diagonal distances corresponding to the quadrangle, and connecting the two diagonal pixels corresponding to the target diagonal distances as a diagonal connection result corresponding to the quadrangle, the diagonal pixels being the three-dimensional pixels in the quadrangle at diagonal positions; and
determining the diagonal connection results corresponding to all of the plurality of quadrangles as the second image processing result corresponding to the first image processing result.
Berg in view of Facchin and Fiser appears to generally teaches the following limitations of Claim 2:
wherein the performing, in accordance with a predetermined pixel grouping condition, the second image processing on the first image processing result, so as to obtain the second image processing result corresponding to the first image processing result comprises:
constructing a stereo model, and then establishing a three-dimensional coordinate system matched with the stereo model (“In particular, the image base is the distance which separates the two image centers, which are the points in space in which the two photographs of a stereoscopic image are taken. The height H is the average height between the image base B and the framed surface; in the aerial case, H is the relative flying height, i.e. the difference between the flying height and the average height of terrain in that zone” Facchin ¶ 5.), wherein the first image processing result comprises a grayscale image corresponding to the target image (Facchin’s height images), the three-dimensional coordinate system takes a length of the grayscale image as a first axis, a width of the grayscale image as a second axis, and a grayscale value of the grayscale image as a third axis, the first axis and the second axis are in a same foundation plane, and the third axis is perpendicular to the foundation plane (description of the height images);
projecting (3D representation of the height image) the first image processing result onto the three-dimensional coordinate system, so as to obtain a three-dimensional projection result corresponding to the first image processing result, the three-dimensional projection result consisting of a plurality of three-dimensional pixels (Berg p. 191:
PNG
media_image1.png
154
442
media_image1.png
Greyscale
), the three-dimensional projection result comprising a three-dimensional projection image (Facchin :
PNG
media_image8.png
372
534
media_image8.png
Greyscale
) and a two- dimensional view image (Berg:
PNG
media_image2.png
156
158
media_image2.png
Greyscale
) both corresponding to the grayscale image, and the two-dimensional view image being obtained by projecting all of the plurality of three-dimensional pixels onto the foundation plane (Similar to Berg’s
PNG
media_image3.png
74
306
media_image3.png
Greyscale
);
Although Berg in view of Facchin and Fiser fail to teach Claim 2’s performing, calculating, and determining steps, Berg has a related teaching of edge flipping:
PNG
media_image9.png
456
770
media_image9.png
Greyscale
They are different algorithmically:
Edge flipping is differentiated from quadrangle [Wingdings font/0xE0] triangle sequence
In Berg, angles are determined and compared, not diagonal distances.
In Berg, additional and essential steps are included.
Claims 9 and 16 are substantially to Claim 2, and other claims depend on Claim 2, 9, or 16.
Conclusion.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Agarwal et al. (US 11941760 B2), the teaching of which is generally related to the claimed invention.
“At step 302, the system 100 segregates the planar point cloud and non-planar point cloud, from each of the images in the RGBD image sequence, by performing a plane segmentation. At this step, a segmented plane is generated. Further, the planar and non-planar point clouds are generated from the segmented plane, non-planar objects identified, and pose information. The system 100 may use any suitable technique for the plane segmentation.” Agarwal col. 7 lines 3-11.
Han et al. (CN 114882118 A), the teaching of which is generally related to the claimed invention.
“FIG. 6 shows a schematic diagram of an alignment process for generating a standard mesh and point cloud intensity of a display gray scale representative code. The row and column grid reference points respectively represented by dots and triangulation points. The point in the r field for calculating cost is further thickened and displayed by gray scale, the grey value is directly proportional to the intensity value.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHENGXI LIU whose telephone number is (571)270-7509. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHENGXI LIU/ Primary Examiner, Art Unit 2611
Prosecution Insights