DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 8 and 20 are objected to because of the following informalities:
Claim 8 should be amended as “wherein determining [[an]]the obstacle raster point in each set of obstacle raster points as [[a]]the new obstacle's center of mass comprises” in line 1-2. The terms “obstacle raster point” and “new obstacle’s center of mass” are previously introduced in claim 6.
Claim 20 should be amended as “wherein determining [[an]]the obstacle raster point in each set of obstacle raster points as [[a]]the new obstacle's center of mass comprises” in line 1-2. The terms “obstacle raster point” and “new obstacle’s center of mass” are previously introduced in claim 18.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 and 12-21 are rejected under 35 U.S.C. 101.
Claims 1-10 and 12-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claims 1, 12, and 13 recite an abstract idea in the form of mental processes without significantly more.
Regarding eligibility step 1, the claimed invention of claims 1, 12, and 13 falls into at least one of the enumerated categories of processes and apparatuses. Therefore, claims 1, 12, and 13 pass step 1.
Proceeding to eligibility step 2A, the claimed invention of claims 1, 12, and 13 is directed to a judicial exception, such as an abstract idea. If a claim limitation under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the mental process grouping of an abstract idea. The claimed invention of claims 1, 12, and 13 is directed to processes that perform clustering on the obstacle raster points and generate an obstacle object in the rater map using generic computer components which can be performed in the human mind, or by a human using a pen and paper. For example, the clustering may be performed by a human by identifying regions of raster points based on a certain value or pattern using visual interpretation. Also, the obstacle object in the raster map may be generated by a human using a pen on a printed raster map. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claims 1, 12, and 13 further recite steps of obtaining data and processing the data by fusing the data such that it amounts to no more than insignificant extra solution activity of data gathering and well-known process of data fusion. Accordingly, the limitation does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Proceeding to eligibility step 2B, claims 1, 12, and 13 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the limitation of obtaining data and processing the data by fusing the data amounts to no more than insignificant extra solution activity of data gathering and well-known process of data fusion. Insignificant data gathering and well-known process of data fusion cannot provide an inventive concept. Therefore, claims 1, 12, and 13 are not patent eligible.
Dependent claims 1-10 and 14-21, when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The additional elements, if any, in the dependent claims are not sufficient to amount to significantly more than the judicial exception for the same reasons as with claims 1, 12, and 13.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6-8, 12, 13, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li (CN 107064955 A). The rejections below are based on the machine translation of Li, a copy of which is attached to this Office Action as also indicated in the 892 form.
Regarding claim 1, Li discloses an obstacle detection method for a robot, comprising:
obtaining three-dimensional point cloud data of an obstacle in a first coordinate system (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; pg. 5, ln. 17-19: “the obstacle clustering method may be applied to an obstacle detection system. In the local area network, the 3D laser radar transmits the collected 3D point cloud data in the form of User Datagram Protocol (UDP) broadcast packet”), the first coordinate system being a coordinate system established by the robot at a current position with the robot as an origin (Li at pg. 5, ln. 35-36: “As shown in FIG. 2A, a three-dimensional lidar can be installed above the vehicle, the lidar coordinate system is centered on the three-dimensional lidar”);
fusing the three-dimensional point cloud data in a robot-centered raster map to determine obstacle raster points in the raster map (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; FIG. 3 and pg. 6, ln. 18-19: “The autonomous vehicle 10 is located in the middle”; pg. 6, ln. 41: “the barrier point is identified based on the mapping point coordinates and the grid map”);
performing clustering on the obstacle raster points to obtain at least one set of obstacle raster points (Li at pg. 7, ln. 7-9: “comparing the obstacle points in the grid map one by one and then comparing them. When the distance between the two obstacle points is less than the distance threshold, it is considered that the two obstacle points belong to the same object point”; pg. 7, ln. 16-17: “one obstacle is chosen as the cluster center of the clustering cluster of the obstacle respectively from the clustering clusters of the K obstacles, so as to form corresponding K cluster centers”); and
generating an obstacle object in the raster map based on the set of obstacle raster points (Li at pg. 9, ln. 23: “an obstacle is identified based on the obstacle clustering result”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”).
Regarding claim 6, Li discloses the method according to claim 1.
Li further discloses wherein performing clustering on the obstacle raster points to obtain at least one set of obstacle raster points comprises:
selecting a predetermined number of obstacle raster points as the obstacle's centers of mass (Li at pg. 3, ln. 17-20: “An average category center determining submodule, configured to respectively calculate an average category centroid of each obstacle clustering cluster, wherein the average category centroid coordinates are mean values of coordinates of respective obstacle points in the corresponding cluster of the obstacle, and the obstacle Point coordinates are coordinates of the obstacle point in the grid map”);
calculating a distance value between each obstacle raster point and each of the obstacle's centers of mass (Li at pg. 3, ln. 56-57: “the loading convergence condition is that the distance between the updated cluster center of the obstacle cluster cluster and the pre-update cluster center is less than a preset distance threshold”);
including each of the obstacle raster points in a set of obstacle raster points in which the obstacle's center of mass with a smallest distance value therebetween is located (Li at pg. 7, ln. 17-19: “The clustering center may be selected randomly or may be the cluster with the closest obstacle to the mean coordinate point of all the obstacle points in the cluster of the obstacle as the cluster of the obstacle clustering center”);
determining an obstacle raster point in each set of obstacle raster points as a new obstacle's center of mass (Li at pg. 3, ln. 22-23: “The cluster center re-determining submodule is configured to determine the average cluster center as the cluster center of the corresponding obstacle cluster cluster”); and
returning to perform the step of calculating a distance value between each obstacle raster point and each of the obstacle's centers of mass until an iterative calculation number reaches a preset iteration number or a distance between the new obstacle's center of mass and an old obstacle's center of mass is less than a first distance threshold (Li at pg. 3, ln. 8-13: “A judging module configured to judge whether a cluster center of each obstacle clustering cluster satisfies a preset convergence condition and a cluster center that does not satisfy the preset convergence in the cluster centers of the K clusters of obstacles And triggering the obstacle point reclustering module to calculate the similarity between the obstacle point and each of the clustering centers and dividing the obstacle point into the highest similarity with the obstacle point Clustering the obstacle in the cluster center until the clustering centers of the K obstacle clustering clusters satisfy the preset convergence condition.”; pg. 3, ln. 56-57: “the loading convergence condition is that the distance between the updated cluster center of the obstacle cluster cluster and the pre-update cluster center is less than a preset distance threshold”), and obtaining at least one set of obstacle raster points (Li at pg. 3, ln. 22-23: “The cluster center re-determining submodule is configured to determine the average cluster center as the cluster center of the corresponding obstacle cluster”).
Regarding claim 7, Li discloses the method according to claim 6.
Li further discloses wherein before arbitrarily selecting a predetermined number of obstacle raster points as the obstacle's centers of mass, the method further comprises:
traversing each obstacle raster point in the raster map, and determining a probability for the obstacle raster point, the probability being used to characterize a confidence level that the obstacle raster point is used to reflect the obstacle (Li at pg. 8, ln. 48-50: “when there is no other obstacle point in the preset area centered on the obstacle point, and the number of radar lines scanned to the actual position point corresponding to the obstacle point is less than the second preset line threshold, the obstacle is determined Point for a single point of noise”);
in response to determining that the probability is less than or equal to a probability threshold (Li at pg. 8, ln. 41-44: “a few radar fault reflections occasionally appear in the identified fault points, and these false reflections tend to be isolated single points, single point noise. Therefore, single point noise can be filtered out after recognition of obstacle points. In this way, sensor noise and environmental noise can be effectively suppressed”; The determination that there is no other obstacle point in the preset area is used as “probability threshold” for identifying fault points as single point noise), determining the obstacle raster point as a noise raster point and excluding the noise raster point from the raster map (Li at pg. 8, ln. 52: “single point noise is removed from the grid map”).
Regarding claim 8, Li discloses the method according to claim 6.
Li further discloses wherein determining an obstacle raster point in each set of obstacle raster points as a new obstacle's center of mass comprises:
calculating a mean value of the coordinates of the obstacle raster points in each of the sets of obstacle raster points to obtain mean value coordinates (Li at pg. 9, ln. 48-51: “an average class center determining sub-module 10061 configured to respectively calculate an average class centroid of each obstacle clustering cluster, wherein the average class centroid Coordinates are mean values of coordinates of respective obstacle points in a cluster of corresponding obstacles”); and
determining the obstacle raster point in each of the sets of obstacle raster points whose coordinates are the closest to the mean value coordinates as the new obstacle's center of mass (Li at pg. 9, ln. 51-52: “coordinates of the obstacle points are coordinates of the obstacle points in the grid map”).
Regarding claim 12, Li discloses a non-transitory computer-readable storage medium storing a computer program thereon, wherein the computer program, when loaded and executed by a processor, causes the processor to realize an obstacle detection method for a robot, and the method comprises:
obtaining three-dimensional point cloud data of an obstacle in a first coordinate system (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; pg. 5, ln. 17-19: “the obstacle clustering method may be applied to an obstacle detection system. In the local area network, the 3D laser radar transmits the collected 3D point cloud data in the form of User Datagram Protocol (UDP) broadcast packet”), the first coordinate system being a coordinate system established by the robot at a current position with the robot as an origin (Li at pg. 5, ln. 35-36: “As shown in FIG. 2A, a three-dimensional lidar can be installed above the vehicle, the lidar coordinate system is centered on the three-dimensional lidar”);
fusing the three-dimensional point cloud data in a robot-centered raster map to determine obstacle raster points in the raster map (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; FIG. 3 and pg. 6, ln. 18-19: “The autonomous vehicle 10 is located in the middle”; pg. 6, ln. 41: “the barrier point is identified based on the mapping point coordinates and the grid map”);
performing clustering on the obstacle raster points to obtain at least one set of obstacle raster points (Li at pg. 7, ln. 7-9: “comparing the obstacle points in the grid map one by one and then comparing them. When the distance between the two obstacle points is less than the distance threshold, it is considered that the two obstacle points belong to the same object point”; pg. 7, ln. 16-17: “one obstacle is chosen as the cluster center of the clustering cluster of the obstacle respectively from the clustering clusters of the K obstacles, so as to form corresponding K cluster centers”); and
generating an obstacle object in the raster map based on the set of obstacle raster points (Li at pg. 9, ln. 23: “an obstacle is identified based on the obstacle clustering result”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”).
Regarding claim 13, Li discloses an electronic device comprising a processor and a memory, wherein the memory stores computer program instructions executable by the processor, and when the processor executes the computer program instructions, operations performed by an obstacle detection method for a robot are realized, and the method comprises:
obtaining three-dimensional point cloud data of an obstacle in a first coordinate system (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; pg. 5, ln. 17-19: “the obstacle clustering method may be applied to an obstacle detection system. In the local area network, the 3D laser radar transmits the collected 3D point cloud data in the form of User Datagram Protocol (UDP) broadcast packet”), the first coordinate system being a coordinate system established by the robot at a current position with the robot as an origin (Li at pg. 5, ln. 35-36: “As shown in FIG. 2A, a three-dimensional lidar can be installed above the vehicle, the lidar coordinate system is centered on the three-dimensional lidar”);
fusing the three-dimensional point cloud data in a robot-centered raster map to determine obstacle raster points in the raster map (Li at pg. 5, ln. 13-15: “the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are determined”; FIG. 3 and pg. 6, ln. 18-19: “The autonomous vehicle 10 is located in the middle”; pg. 6, ln. 41: “the barrier point is identified based on the mapping point coordinates and the grid map”);
performing clustering on the obstacle raster points to obtain at least one set of obstacle raster points (Li at pg. 7, ln. 7-9: “comparing the obstacle points in the grid map one by one and then comparing them. When the distance between the two obstacle points is less than the distance threshold, it is considered that the two obstacle points belong to the same object point”; pg. 7, ln. 16-17: “one obstacle is chosen as the cluster center of the clustering cluster of the obstacle respectively from the clustering clusters of the K obstacles, so as to form corresponding K cluster centers”); and
generating an obstacle object in the raster map based on the set of obstacle raster points (Li at pg. 9, ln. 23: “an obstacle is identified based on the obstacle clustering result”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”).
Regarding claim 18, Li discloses the electronic device according to claim 13.
Li further discloses wherein performing clustering on the obstacle raster points to obtain at least one set of obstacle raster points comprises:
selecting a predetermined number of obstacle raster points as obstacle's centers of mass (Li at pg. 3, ln. 17-20: “An average category center determining submodule, configured to respectively calculate an average category centroid of each obstacle clustering cluster, wherein the average category centroid coordinates are mean values of coordinates of respective obstacle points in the corresponding cluster of the obstacle, and the obstacle Point coordinates are coordinates of the obstacle point in the grid map”);
calculating a distance value between each obstacle raster point and each of the obstacle's centers of mass (Li at pg. 3, ln. 56-57: “the loading convergence condition is that the distance between the updated cluster center of the obstacle cluster cluster and the pre-update cluster center is less than a preset distance threshold”);
including each of the obstacle raster points in a set of obstacle raster points in which the obstacle's center of mass with a smallest distance value therebetween is located (Li at pg. 7, ln. 17-19: “The clustering center may be selected randomly or may be the cluster with the closest obstacle to the mean coordinate point of all the obstacle points in the cluster of the obstacle as the cluster of the obstacle clustering center”);
determining an obstacle raster point in each set of obstacle raster points as a new obstacle's center of mass (Li at pg. 3, ln. 22-23: “The cluster center re-determining submodule is configured to determine the average cluster center as the cluster center of the corresponding obstacle cluster cluster”); and
returning to perform the step of calculating a distance value between each obstacle raster point and each of the obstacle's centers of mass until an iterative calculation number reaches a preset iteration number or a distance between the new obstacle's center of mass and an old obstacle's center of mass is less than a first distance threshold (Li at pg. 3, ln. 8-13: “A judging module configured to judge whether a cluster center of each obstacle clustering cluster satisfies a preset convergence condition and a cluster center that does not satisfy the preset convergence in the cluster centers of the K clusters of obstacles And triggering the obstacle point reclustering module to calculate the similarity between the obstacle point and each of the clustering centers and dividing the obstacle point into the highest similarity with the obstacle point Clustering the obstacle in the cluster center until the clustering centers of the K obstacle clustering clusters satisfy the preset convergence condition.”; pg. 3, ln. 56-57: “the loading convergence condition is that the distance between the updated cluster center of the obstacle cluster cluster and the pre-update cluster center is less than a preset distance threshold”), and obtaining at least one set of obstacle raster points (Li at pg. 3, ln. 22-23: “The cluster center re-determining submodule is configured to determine the average cluster center as the cluster center of the corresponding obstacle cluster cluster”).
Regarding claim 19, Li discloses the electronic device according to claim 18.
Li further discloses wherein the method further comprises:
traversing each obstacle raster point in the raster map, and determining a probability for the obstacle raster point, the probability being used to characterize a confidence level that the obstacle raster point is used to reflect the obstacle (Li at pg. 8, ln. 48-50: “when there is no other obstacle point in the preset area centered on the obstacle point, and the number of radar lines scanned to the actual position point corresponding to the obstacle point is less than the second preset line threshold, the obstacle is determined Point for a single point of noise”);
in response to determining that the probability is less than or equal to a probability threshold (Li at pg. 8, ln. 41-44: “a few radar fault reflections occasionally appear in the identified fault points, and these false reflections tend to be isolated single points, single point noise. Therefore, single point noise can be filtered out after recognition of obstacle points. In this way, sensor noise and environmental noise can be effectively suppressed”; The determination that there is no other obstacle point in the preset area is used as “probability threshold” for identifying fault points as single point noise), determining the obstacle raster point as a noise raster point and excluding the noise raster point from the raster map (Li at pg. 8, ln. 52: “single point noise is removed from the grid map”).
Regarding claim 20, Li discloses the electronic device according to claim 18.
Li further discloses wherein determining an obstacle raster point in each set of obstacle raster points as a new obstacle's center of mass comprises:
calculating a mean value of the coordinates of the obstacle raster points in each of the sets of obstacle raster points to obtain mean value coordinates (Li at pg. 9, ln. 48-51: “an average class center determining sub-module 10061 configured to respectively calculate an average class centroid of each obstacle clustering cluster, wherein the average class centroid Coordinates are mean values of coordinates of respective obstacle points in a cluster of corresponding obstacles”); and
determining the obstacle raster point in each of the sets of obstacle raster points whose coordinates are the closest to the mean value coordinates as the new obstacle's center of mass (Li at pg. 9, ln. 51-52: “coordinates of the obstacle points are coordinates of the obstacle points in the grid map”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 4, 5, 14, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Su et al. (CN 116246157 A, hereinafter “Su”). The rejections below are based on the machine translation of Su, a copy of which is attached to this Office Action as also indicated in the 892 form.
Regarding claim 2, Li discloses the method according to claim 1.
However, Li does not explicitly state:
wherein obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system comprises:
obtaining light strip information collected by the robot at the current position and projected on the obstacle by a linear laser;
determining candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information; and
filtering the three-dimensional point cloud data from the candidate three-dimensional point cloud data based on a predetermined height threshold.
In the same field of endeavor, Su teaches:
wherein obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system comprises:
obtaining light strip information collected by the robot at the current position and projected on the obstacle by a linear laser (Su at pg. 5, ln. 43-46: “Fig. 1 shows the optical path schematic diagram of line laser 10, and O is laser light source, and LR is the line (ie reflection line) that the light that line laser sends forms on the reflection point on the baffle, M is the central point of reflection line, reflection LOR Indicates the surface formed by all the optical paths of the line laser, that is, the optical path surface”; pg. 5, ln. 48: “The line laser collects point cloud data at a fixed frequency”; pg. 5, ln. 54-56: “For a line laser, each frame of point cloud data may include spatial position information of a group of reflection points (hereinafter simply referred to as points) on the baffle. The spatial position information can be represented by coordinates of a preset three-dimensional rectangular coordinate system”);
determining candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information (Su at pg. 7, ln. 26-20: “detecting whether there are feature points in the surrounding environment of the electronic device according to the first frame of point cloud data, so as to determine whether there is a possibility of obstacles in the surrounding environment of the electronic device”); and
filtering the three-dimensional point cloud data from the candidate three-dimensional point cloud data based on a predetermined height threshold (Su at pg. 7, ln. 32-34: “the first frame of point cloud data may be searched for a point meeting the condition of the feature point. If a point meeting the condition of the feature point can be found, it is considered that the feature point has been detected, and there may be obstacles around the electronic device”; pg. 7, ln. 44-46: “a feature point can be a point that at least meets one of the following conditions: 1) The vertical distance from the point to the ground (for example, the Z-axis coordinate of the point) is greater than the third predetermined value”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Li by adding the light strip information of Su with a reasonable expectation of success. The motivation to modify the method of Li in view of Su is to improve obstacle detection performance of robots.
Regarding claim 4, Li in view of Su teaches the method according to claim 2.
Li further discloses wherein determining the candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information comprises:
determining, based on a mapping relationship from a camera coordinate system to a pixel coordinate system, obstacle point cloud coordinates of (Li at pg. 5, ln. 32-33: “the obstacle detection system converts the 3D point cloud data into coordinate points under the lidar coordinate system according to the distance and angle information analyzed above”; pg. 5, ln. 41-42: “the three-dimensional point cloud data may be transformed into coordinate points in the lidar coordinate system”; pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 5, ln. 58-60: “As shown in FIG. 2A, the vehicle body coordinate system takes the direction of the linear motion of the vehicle as the XC axis, the horizontal axis of the vehicle as the YC axis, and the direction perpendicular to the horizontal ground as the ZC axis”); and
determining the candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the obstacle point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 5, ln. 58-60: “As shown in FIG. 2A, the vehicle body coordinate system takes the direction of the linear motion of the vehicle as the XC axis, the horizontal axis of the vehicle as the YC axis, and the direction perpendicular to the horizontal ground as the ZC axis”).
Su further teaches:
a light strip pixel center of the light strip information in the camera coordinate system (Su at pg. 5, ln. 43-46: “Fig. 1 shows the optical path schematic diagram of line laser 10, and O is laser light source, and LR is the line (ie reflection line) that the light that line laser sends forms on the reflection point on the baffle, M is the central point of reflection line, reflection LOR Indicates the surface formed by all the optical paths of the line laser, that is, the optical path surface”; pg. 5, ln. 54-56: “For a line laser, each frame of point cloud data may include spatial position information of a group of reflection points (hereinafter simply referred to as points) on the baffle. The spatial position information can be represented by coordinates of a preset three-dimensional rectangular coordinate system”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Li in view of Su by adding the light strip pixel center of Su with a reasonable expectation of success. The motivation to modify the method of Li in view of Su is to improve obstacle detection performance of robots.
Regarding claim 5, Li in view of Su teaches the method according to claim 2.
Li further discloses wherein said obtaining the three- dimensional point cloud data of the obstacle in the first coordinate system further comprises:
obtaining position change information of the robot moving from a previous position to the current position (Li at pg. 5, ln. 13-15: “In step 101, the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the 13 mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are 14 determined”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The vehicle body coordinate must acquire position change information to detect obstacles while the vehicle is traveling);
obtaining three-dimensional point cloud data of the obstacle in a second coordinate system, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as the origin (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The coordinate system must be updated to detect obstacles while the vehicle is traveling); and
transforming the three-dimensional point cloud data of the obstacle in the second coordinate system into the three-dimensional point cloud data of the obstacle in the first coordinate system based on the position change information (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The three-dimensional point cloud data of the obstacle must be transformed to follow the updated coordinate system to detect obstacles while the vehicle is traveling).
Regarding claim 14, Li discloses the electronic device according to claim 13.
However, Li does not explicitly state:
wherein obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system comprises:
obtaining light strip information collected by the robot at the current position and projected on the obstacle by a linear laser;
determining candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information; and
filtering the three-dimensional point cloud data from the candidate three-dimensional point cloud data based on a predetermined height threshold.
In the same field of endeavor, Su teaches:
wherein obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system comprises:
obtaining light strip information collected by the robot at the current position and projected on the obstacle by a linear laser (Su at pg. 5, ln. 43-46: “Fig. 1 shows the optical path schematic diagram of line laser 10, and O is laser light source, and LR is the line (ie reflection line) that the light that line laser sends forms on the reflection point on the baffle, M is the central point of reflection line, reflection LOR Indicates the surface formed by all the optical paths of the line laser, that is, the optical path surface”; pg. 5, ln. 48: “The line laser collects point cloud data at a fixed frequency”; pg. 5, ln. 54-56: “For a line laser, each frame of point cloud data may include spatial position information of a group of reflection points (hereinafter simply referred to as points) on the baffle. The spatial position information can be represented by coordinates of a preset three-dimensional rectangular coordinate system”);
determining candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information (Su at pg. 7, ln. 26-20: “detecting whether there are feature points in the surrounding environment of the electronic device according to the first frame of point cloud data, so as to determine whether there is a possibility of obstacles in the surrounding environment of the electronic device”); and
filtering the three-dimensional point cloud data from the candidate three-dimensional point cloud data based on a predetermined height threshold (Su at pg. 7, ln. 32-34: “the first frame of point cloud data may be searched for a point meeting the condition of the feature point. If a point meeting the condition of the feature point can be found, it is considered that the feature point has been detected, and there may be obstacles around the electronic device”; pg. 7, ln. 44-46: “a feature point can be a point that at least meets one of the following conditions: 1) The vertical distance from the point to the ground (for example, the Z-axis coordinate of the point) is greater than the third predetermined value”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Li by adding the light strip information of Su with a reasonable expectation of success. The motivation to modify the device of Li in view of Su is to improve obstacle detection performance of robots.
Regarding claim 16, Li in view of Su teaches the electronic device according to claim 14.
Li further discloses wherein determining the candidate three- dimensional point cloud data of the obstacle in the first coordinate system based on the light strip information comprises:
determining, based on a mapping relationship from a camera coordinate system to a pixel coordinate system, obstacle point cloud coordinates of (Li at pg. 5, ln. 32-33: “the obstacle detection system converts the 3D point cloud data into coordinate points under the lidar coordinate system according to the distance and angle information analyzed above”; pg. 5, ln. 41-42: “the three-dimensional point cloud data may be transformed into coordinate points in the lidar coordinate system”; pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 5, ln. 58-60: “As shown in FIG. 2A, the vehicle body coordinate system takes the direction of the linear motion of the vehicle as the XC axis, the horizontal axis of the vehicle as the YC axis, and the direction perpendicular to the horizontal ground as the ZC axis”); and
determining the candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the obstacle point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 5, ln. 58-60: “As shown in FIG. 2A, the vehicle body coordinate system takes the direction of the linear motion of the vehicle as the XC axis, the horizontal axis of the vehicle as the YC axis, and the direction perpendicular to the horizontal ground as the ZC axis”).
Su further teaches:
a light strip pixel center of the light strip information in the camera coordinate system (Su at pg. 5, ln. 43-46: “Fig. 1 shows the optical path schematic diagram of line laser 10, and O is laser light source, and LR is the line (ie reflection line) that the light that line laser sends forms on the reflection point on the baffle, M is the central point of reflection line, reflection LOR Indicates the surface formed by all the optical paths of the line laser, that is, the optical path surface”; pg. 5, ln. 54-56: “For a line laser, each frame of point cloud data may include spatial position information of a group of reflection points (hereinafter simply referred to as points) on the baffle. The spatial position information can be represented by coordinates of a preset three-dimensional rectangular coordinate system”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Li in view of Su by adding the light strip pixel center of Su with a reasonable expectation of success. The motivation to modify the device of Li in view of Su is to improve obstacle detection performance of robots.
Regarding claim 17, Li in view of Su teaches the electronic device according to claim 14.
Li further discloses wherein obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system further comprises:
obtaining position change information of the robot moving from a previous position to the current position (Li at pg. 5, ln. 13-15: “In step 101, the three-dimensional point cloud data sent by the three-dimensional lidar is acquired, and the 13 mapping point coordinates of the three-dimensional point cloud data in the vehicle body coordinate system are 14 determined”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The vehicle body coordinate must acquire position change information to detect obstacles while the vehicle is traveling);
obtaining three-dimensional point cloud data of the obstacle in a second coordinate system, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as the origin (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The coordinate system must be updated to detect obstacles while the vehicle is traveling); and
transforming the three-dimensional point cloud data of the obstacle in the second coordinate system into the three-dimensional point cloud data of the obstacle in the first coordinate system based on the position change information (Li at pg. 5, ln. 55-56: “Next, the coordinate points in the laser radar coordinate system are mapped to the body coordinate system according to the rotation matrix and the translation matrix”; pg. 9, ln. 25-27: “the obstacle detection system may recognize an obstacle based on the obstacle clustering result obtained in the above steps 101 to 107 to prepare for obstacle avoidance while the autonomous vehicle is traveling”; The three-dimensional point cloud data of the obstacle must be transformed to follow the updated coordinate system to detect obstacles while the vehicle is traveling).
Claims 3 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Su further in view of Hickerson et al. (US 2015/0168954 A1, hereinafter “Hickerson”).
Regarding claim 3, Li in view of Su teaches the method according to claim 2.
However, Li in view of Su does not explicitly state:
wherein said obtaining the light strip information collected by the robot at the current position and projected on the obstacle by the linear laser comprises:
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an ON state as a first image;
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an OFF state as a second image; and
performing differential processing on the first image and the second image to obtain the light strip information projected on the obstacle by the linear laser.
In the same field of endeavor, Hickerson teaches:
wherein said obtaining the light strip information collected by the robot at the current position and projected on the obstacle by the linear laser comprises:
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an ON state as a first image (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”);
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an OFF state as a second image (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”); and
performing differential processing on the first image and the second image to obtain the light strip information projected on the obstacle by the linear laser (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”; para. [0023]: “The structured light projected by the light source on the path before the robot may include focused points of light or lines of light arrayed horizontally, vertically, or both”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Li in view of Su by adding the differential processing of Hickerson with a reasonable expectation of success. The motivation to modify the method of Li in view of Su further in view of Hickerson is to easily detect illumination pattern for detecting obstacles.
Regarding claim 15, Li in view of Su teaches the electronic device according to claim 14.
However, Li in view of Su does not explicitly state:
wherein obtaining the light strip information collected by the robot at the current position and projected on the obstacle by the linear laser comprises:
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an ON state as a first image;
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an OFF state as a second image; and
performing differential processing on the first image and the second image to obtain the light strip information projected on the obstacle by the linear laser.
In the same field of endeavor, Hickerson teaches:
wherein obtaining the light strip information collected by the robot at the current position and projected on the obstacle by the linear laser comprises:
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an ON state as a first image (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”);
obtaining an image of the obstacle acquired by the robot at the current position when the linear laser is in an OFF state as a second image (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”); and
performing differential processing on the first image and the second image to obtain the light strip information projected on the obstacle by the linear laser (Hickerson at para. [0016]: “the detectability of the pattern can be enhanced by taking two images in rapid succession, one with the pattern source on and another with the pattern source off, and detecting the pattern in the difference of the two images”; para. [0023]: “The structured light projected by the light source on the path before the robot may include focused points of light or lines of light arrayed horizontally, vertically, or both”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Li in view of Su by adding the differential processing of Hickerson with a reasonable expectation of success. The motivation to modify the device of Li in view of Su further in view of Hickerson is to easily detect illumination pattern for detecting obstacles.
Claims 9 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Mita et al. (JP 2003162718 A, hereinafter “Mita”) further in view of Liu et al. (US 2007/0041638 A1, hereinafter “Liu”). The rejections below are based on the machine translation of Mita, a copy of which is attached to this Office Action as also indicated in the 892 form.
Regarding claim 9, Li discloses the method according to claim 6.
However, Li does not explicitly state:
wherein the method further comprises:
in a set of obstacle raster points, in response to determining that a variance of the distance between the respective obstacle raster points and the obstacle's center of mass is greater than a variance threshold, dividing the any set of obstacle raster points into at least two sets of obstacle raster points; and
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points.
In the same field of endeavor, Mita teaches:
wherein the method further comprises:
in a set of obstacle raster points, in response to determining that a variance of the distance between the respective obstacle raster points and the obstacle's center of mass is greater than a variance threshold, dividing the any set of obstacle raster points into at least two sets of obstacle raster points (Mita at para. [0022]: “the variance within each cluster is calculated. The variance is obtained by dividing the center of gravity of the cluster and the square of the distance between the pixels belonging to the cluster by the number of pixels in the cluster”; para. [0023]: “when the maximum value of the variance is equal to or greater than the threshold value T4, the cluster having the maximum variance is divided”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Li by adding the variance of the distance of Mita with a reasonable expectation of success. The motivation to modify the method Li in view of Mita is to provide efficient image segmentation.
However, Li in view of Mita does not explicitly state:
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points.
In the same field of endeavor, Liu teaches:
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points (Liu at para. [0074]: “According to an aspect of the invention, the closest clusters can be iteratively merged until the desired number is reached. According to another aspect of the invention, at each step, the distance between centroids of current clusters can be used as the merging criterion”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Li in view of Mita by adding the second distance threshold of Mita with a reasonable expectation of success. The motivation to modify the method of Li in view of Mita further in view of Liu is to recognize desired target objects.
Regarding claim 21, Li discloses the electronic device according to claim 18.
However, Li does not explicitly state:
wherein the method further comprises:
in a set of obstacle raster points, in response to determining that a variance of the distance between the respective obstacle raster points and the obstacle's center of mass is greater than a variance threshold, dividing the any set of obstacle raster points into at least two sets of obstacle raster points; and
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points.
In the same field of endeavor, Mita teaches:
wherein the method further comprises:
in a set of obstacle raster points, in response to determining that a variance of the distance between the respective obstacle raster points and the obstacle's center of mass is greater than a variance threshold, dividing the any set of obstacle raster points into at least two sets of obstacle raster points (Mita at para. [0022]: “the variance within each cluster is calculated. The variance is obtained by dividing the center of gravity of the cluster and the square of the distance between the pixels belonging to the cluster by the number of pixels in the cluster”; para. [0023]: “when the maximum value of the variance is equal to or greater than the threshold value T4, the cluster having the maximum variance is divided”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Li by adding the variance of the distance of Mita with a reasonable expectation of success. The motivation to modify the device Li in view of Mita is to provide efficient image segmentation.
However, Li in view of Mita does not explicitly state:
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points.
In the same field of endeavor, Liu teaches:
in response to determining that the distance between the obstacle's centers of mass of two sets of obstacle raster points is less than a second distance threshold, combining the any two sets of obstacle raster points into one set of obstacle raster points (Liu at para. [0074]: “According to an aspect of the invention, the closest clusters can be iteratively merged until the desired number is reached. According to another aspect of the invention, at each step, the distance between centroids of current clusters can be used as the merging criterion”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Li in view of Mita by adding the second distance threshold of Mita with a reasonable expectation of success. The motivation to modify the device of Li in view of Mita further in view of Liu is to recognize desired target objects.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Choudhary et al. (US 2019/0026567 A1, hereinafter “Choudhary”).
Regarding claim 10, Li discloses the method according to claim 6.
However, Li does not explicitly state:
wherein generating the obstacle object in the raster map based on the set of obstacle raster points comprises:
determining a raster region in which each set of obstacle raster points is distributed in the raster map; and
determining the raster region as an obstacle object in response to determining that a size of the raster region exceeds a predetermined threshold and a density of obstacle raster points in the raster region exceeds a density threshold.
In the same field of endeavor, Choudhary teaches:
wherein generating the obstacle object in the raster map based on the set of obstacle raster points comprises:
determining a raster region in which each set of obstacle raster points is distributed in the raster map (Choudhary at para. [0175]: “Motion information may be obtained by subtracting successive video frames and applying suitable thresholding to identify moving "blobs" of objects”); and
determining the raster region as an obstacle object in response to determining that a size of the raster region exceeds a predetermined threshold (Choudhary at para. [0177]: “Reject blobs with an area smaller than 'x' % (e.g., a size threshold) of the total image (e.g., video frame) area. For example, in a 640x360 image with x=-0.02, the processing logic may reject blobs smaller than an area of 460. The present algorithm can work for any other values of x”) and a density of obstacle raster points in the raster region exceeds a density threshold (Choudhary at para. [0178]: “Reject blobs that have a density 'd' that is below a density threshold. In one embodiment, density is defined as the percentage of nonzero pixels inside a convex hull that envelops a blob. In one example, blobs with a density below d=0.4 can be rejected. Any suitable value of density 'd' may be used”) (Since blobs, which are detected object candidates, are rejected based on the blob size being smaller than a size threshold and the density of pixels being below than a density threshold, the remaining blobs are determined as objects (i.e., “obstacle object”) based on the blob size being equal or greater than the size threshold (i.e., “predetermined threshold”) and the density of pixels being equal to or greater than the density threshold (i.e., “density threshold”)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JISUN CHOI whose telephone number is (571)270-0710. The examiner can normally be reached Mon-Fri, 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Browne can be reached at (571)270-0151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JISUN CHOI/Examiner, Art Unit 3666
/SCOTT A BROWNE/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights