DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 13 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 13 does not appear to require performance of the steps recited in Claim 1. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
1. Claim 13 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 13 recites “A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the obstruction detection method according to claim 1.”, which appears to be an incomplete recitation of the features of claims 1, and is indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claims 1 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable JP 2021085828 A by Shingo Hattori, (herein after “Hattori”) in view of “Wikipedia Article on Box blur”.
Regarding claim 1, An obstruction detection method for detecting an obstruction, the obstruction detection method comprising (Hattori, §Abstract: “The obstacle detection device-”):
determining an exclusion depth pixel to be excluded from obstruction candidate pixels among a plurality of depth pixels of a depth image obtained by sensing (Hattori, P[0032]: “an object is extracted using points P whose Z coordinate is greater than or equal to the threshold, from among candidate points P whose surface normal vector is greater than or equal to the first threshold.”), using a depth sensor (Hattori, P[0037]: “The sensor may be a combination of multiple sensors, such as a LiDAR and a stereo camera.”), an object group having an irregular plane (Hattori, P[0032]: “an object is extracted using points P whose Z coordinate is greater than or equal to the threshold, from among candidate points P whose surface normal vector is greater than or equal to the first threshold.”); and
detecting an obstruction, based on one or more depth pixels other than the exclusion depth pixel among the plurality of depth pixels (Hattori, P[0032]: “the obstacle detection device 31 extracts an object using point P. Point P represents a part of the surface of an object. Therefore, a point cloud, which is a collection of points P, can be treated as a single object. The obstacle detection device 31 can extract objects by clustering multiple discrete points P into a single point cloud. In this embodiment, an object is extracted using points P whose Z coordinate is greater than or equal to the threshold, from among candidate points P whose surface normal vector is greater than or equal to the first threshold.”), wherein
the determining includes performing, for each of the plurality of depth pixels, the following:
processing (ii) for calculating a normal line (Hattori, P[0023]: “a surface normal vector is derived for each point P that was not thinned out in step S2.”), based on the average value associated with the depth pixel and an average value associated with each of a plurality of second surrounding depth pixels within a second pixel number from the depth pixel, the second pixel number indicating a predetermined number of pixels (Hattori, P[0023]: “a surface normal vector is derived for each point P that was not thinned out in step S2.”, where the second number of pixels are the points, P, that were not removed. );
processing (iii) for determining whether an angle formed by the normal line calculated and a normal line of a predetermined reference plane is smaller than or equal to a predetermined angle (Hattori, P[0027]: “Candidate points are extracted by determining whether the angle of the surface normal vector with respect to the XY plane, represented by the X and Y axes, is less than a predetermined first threshold.”); and
processing (iv) for determining the depth pixel as the exclusion depth pixel when the angle is determined to be smaller than or equal to the predetermined angle (Hattori, P[0027]: “The obstacle detection device 31 extracts points P as candidate points in which the angle of the surface normal vector with respect to the XY plane is less than a predetermined first threshold. In other words, the obstacle detection device 31 determines that a point P whose angle of the surface normal vector with respect to the XY plane is greater than or equal to a predetermined first threshold is a point P that represents a part of an object different from the obstacle O.”).
Hattori does not explicitly disclose processing (i) for calculating an average value related to the depth pixel, based on a depth-pixel value of the depth pixel and a depth-pixel value of each of a plurality of first surrounding depth pixels within a first pixel number from the depth pixel, and associating the average value with the depth pixel, the first pixel number indicating a predetermined number of pixels; However, this limitation describes a well-known image smoothing technique in image processing, specifically named “Box blur”, where neighboring pixels are averaged together and repeated application of a box blur will approximate a Gaussian blur, which is disclosed in the provided “Wikipedia Article on Box blur” document.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Hattori to utilize Box blurring on the image, as taught by the “Wikipedia Article on Box blur”, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have provided the benefit of reducing noise in the depth image and improve computational speed for image processing.
Claims 13 as best understood and 14 recite features nearly identical to those recited in claim 1. Claims 13 and 14 are rejected for reasons analogous to those discussed above in conjunction with claim 1.
3. Claims 2, 4, 6, and 8 are rejected under 35 U.S.C. 103 as being unpatentable in view of Hattori in view of “Wikipedia Article on Box blur” in further view of “Revisiting Deep Convolutional Neural Networks for RGB-D Based Object Recognition” by Lorand Madai-Tahy et al., (herein after “Madai-Tahy”).
Regarding claim 2, wherein
For what Hattori and the “Wikipedia Article on Box blur” may lack to explicitly disclose, Madai-Tahy discloses utilizing multiple iterations of a box blur filter on their depth images to improve object-based recognition.
in the processing (i), an average depth-pixel value of the depth-pixel value of the depth pixel and the depth-pixel value of each of the plurality of first surrounding depth pixels within the first pixel number from the depth pixel is calculated, and the average depth-pixel value is associated with the depth pixel, which describes multiple iterations of applying a box blur filter to a depth image as disclosed by Madai-Tahy in Fig. 6, “Surface normal images after multiple iterations of unique box blur applied on a depth map. The procedure eliminates the discontinuities induced by the 3D sensor.”, and
in the processing (ii), the normal line is calculated based on the average depth-pixel value associated with the depth pixel and an average depth-pixel value associated with each of the plurality of second surrounding depth pixels within the second pixel number from the depth pixel is disclosed by Madai-Tahy in Fig. 6, where they disclose that the normal line is calculated for the images considered to be normal to the surface, “Surface normal images after multiple iterations of unique box blur applied on a depth map. The procedure eliminates the discontinuities induced by the 3D sensor.”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Hattori and the Wikipedia Article on Box blur to apply box blurring to a depth image, as taught by Madai-Tahy, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have provided the benefit of improving object-based detection in depth images.
Regarding claim 4, wherein in the processing (ii), the normal line is calculated based on the average depth-pixel value associated with the depth pixel and an average depth-pixel value associated with each of valid second surrounding depth pixels among the plurality of second surrounding depth pixels is disclosed by Hattori in P[0023]: “a surface normal vector is derived for each point P that was not thinned out in step S2.”, where the points that are not thinned out are the valid points/pixels (averaged depth values through iterative box blurring provided by Madai-Tahy), which are used for calculating the normal line.
Regarding claim 6, wherein
in the processing (i), an average coordinate value of a coordinate value out of three-dimensional coordinate values obtained through point cloud transformation of the depth-pixel value of the depth pixel and a coordinate value out of three-dimensional coordinate values obtained through the point cloud transformation of the depth-pixel value of each of the plurality of first surrounding depth pixels within the first pixel number from the depth pixel is calculated, and the average coordinate value is associated with the depth pixel (Hattori, P[0002]: “Sensors capable of deriving three-dimensional coordinates, such as stereo cameras, LiDAR, and millimeter-wave radar, are used. In this type of obstacle detection device, the road surface may be extracted as an object, and it is necessary to distinguish whether the extracted object is the road surface or an obstacle.”, and P[0003]: “The obstacle detection device described in Patent Document 1 comprises a conversion unit that converts pixels of a distance image into three-dimensional coordinates, and a surface normal vector derivation unit that extracts three points from pixels of a distance image and derives a surface normal vector.”, and “Wikipedia Article on Box blur” discloses averaging local pixel values, which have coordinates, and therefore the coordinates are also averaged and related to the depth values as disclosed by Madai-Tahy in Fig. 6, “Surface normal images after multiple iterations of unique box blur applied on a depth map. The procedure eliminates the discontinuities induced by the 3D sensor.”), and
in the processing (ii), the normal line is calculated based on the average coordinate value associated with the depth pixel and an average coordinate value associated with each of the plurality of second surrounding depth pixels within the second pixel number from the depth pixel is disclosed by Madai-Tahy in Fig. 6, “Surface normal images after multiple iterations of unique box blur applied on a depth map. The procedure eliminates the discontinuities induced by the 3D sensor.”, where the average value for the coordinate and depth values are based on one another where each coordinate is the cell value of the pixel and the depth value is the depth associated with each cell value of the pixel.
Regarding claim 8, in the processing (ii), the normal line is calculated based on the average coordinate value associated with the depth pixel and an average coordinate value associated with each of valid second surrounding depth pixels among the plurality of second surrounding depth pixels is disclosed by Madai-Tahy in Fig. 6, “Surface normal images after multiple iterations of unique box blur applied on a depth map. The procedure eliminates the discontinuities induced by the 3D sensor.”, where the average value for the coordinate and depth values are based on one another where each coordinate is the cell value of the pixel and the depth value is the depth associated with each cell value of the pixel.
Allowable Subject Matter
4. Claims 3, 5, 7, and 9-12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
5. Any inquiry concerning this communication or earlier communications from the examiner should be directed to TY M BEATTY whose telephone number is (703)756-5370. The examiner can normally be reached Mon-Fri: 8AM-4PM EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272 - 3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TY MITCHELL BEATTY/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights