DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
The following is a Final Office action. In response to Examiner’s Non-Final Rejection of 1/16/26, Applicant, on 4/6/26, amended claims. Claims 1-9 are pending in this application and have been rejected below.
Response to Amendment
Applicant’s amendments are acknowledged.
Objection to the Specification is withdrawn in light of the new title: “Information Processing Device For a Ship Using Lidar”.
The 101 rejections are withdrawn as the claim is viewed as improving another technology (MPEP 2106.05(a)) for improved navigation of a ship and meaningful limitations (MPEP 2106.05(e)) for performing driving assistance of the ship itself, similar to Diamond v. Diehr.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4, and 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Qingqing, et al., “Detecting Water Reflection Symmetries in Point Clouds for
Camera Position Calibration in Unmanned Surface Vehicles,” 2019 19th International Symposium on Communications and Information Technologies, pages 507-512, in view of Hiromatsu (US 2024/0085553).
Concerning claim 1, Qingqing discloses:
An information processing device (Qingqing – see page 509, Col. 2, Section A - The first step is to pre-process data and apply some filters to the point cloud in order to reduce the number of points and reduce the computational team necessary to run our proposed algorithm. This is essential to enable real-time operation. Taking into account that a considerable part of the point cloud
does not appear in the reflection, we use GNSS or lidar data to estimate the distance to the lakeside or river banks; See FIG. 2 – paper focuses on information from Lidar in “distance to coastline,” “water surface plane estimation”, and “stereo camera orientation”) comprising:
a memory configured to store … (Qingqing – see page 511, Col. 1, 2nd paragraph – hardware installed is Intel IoT edge gateway with CPU, and 4GB RAM); and
a processor configured to execute…(Qingqing – see page 511, Col. 1, 2nd paragraph – hardware installed is Intel IoT edge gateway with CPU, and 4GB RAM).
To any extent Qingqing is unclear on processor executing stored instructions from memory, Hiromatsu discloses:
a memory configured to store “instructions”; a processor configured to “execute the instructions” (Hiromatsu – see par 68-69, FIG. 4 – memory is storage device; main memory used as a work area of the processor; processes stored in memory 6 developed on processor 5; par 71 - The auxiliary storage device 21 stores the obstacle detection program, programs run by the processor 5, data to be used when running the programs, and so on. A specific example of the auxiliary storage device 21 is a Hard Disk Drive (HDD) or a Solid-State Drive (SSD). The auxiliary storage device 21 may be a portable recording medium; claim 11 - . A non-transitory computer-readable recording medium recorded with an obstacle detection program that causes a computer).
Qingqing and Hiromatsu disclose:
acquire point cloud data generated by a measurement device provided on a ship (Qingqing – see abstract - We propose a method to estimate the water plane based on the detection of local symmetry planes that naturally occur when objects are reflected in the water. By using a point cloud generated with a
stereo camera, we are able to accurately estimate the water level and, at the same time, calibrate the camera position and improve the localization of obstacles; see page 510, FIG. 2 – sensors installed on vessel (See FIG. 3 – ship) – includes Intel Realsense camera sensor; see page 510, col. 1, 1st paragraph - We only take into account points above the water surface and try to find their reflection in the point cloud. A KD-tree search algorithm is used to find the nearest point in the point cloud to the expected reflection; see page 511, Col. 1 – Lidar, Intel RealSense stereo camera installed on-board ship; use on-board computer and lidars… and Realsense’s point cloud
see also Hiromatsu – see par 55-58 – data D(k) – water surface separation unit 3d takes as input, point cloud data);
estimate a self-position of the ship based on the point cloud data (Applicant’s [0050] as published states “ the self-position includes the attitude angle such as the yaw angle of the target ship.”
[0064] as published states “As shown in FIG. 3, the self-position in the plane defined on the three-dimensional orthogonal coordinates of xyz is represented by the coordinates “(x,y,z)”, the roll angle “φ”, the pitch angle “θ”, and the yaw angle (azimuth) “ψ” of the target ship.”
Qingqing see page 510, FIG. 2 – distance to coastline; water surface plane estimation; absolute position; See page 510, FIG. 3 – illustration of pitch, roll, and yaw on a vessel; page 511, col. 1, last paragraph - As one of the lidars is tilted 9 degrees towards the ship bow, we apply a transformation to publish 3D point cloud data from the 2D lidar data. We use this information and the Realsense’s point cloud to estimate the waterline and water surface level with PCL, which are then published with their own respective topics;
Hirmatsu – see par 29 - the three-dimensional LiDAR 2 radiates laser light, receives reflected light from an object, and outputs three-dimensional point cloud data collectively representing three-dimensional position information of reflection points 14. That is, the three-dimensional LiDAR 2 outputs information including position information of the reflection points 14);
acquire a position of a shore (Qingqing see page 509, col. 2, 1st paragraph - The first step is to pre-process data and apply some filters to the point cloud in order to reduce the number of points and reduce the computational team necessary to run our proposed algorithm. This is essential to enable real-time operation. Taking into account that a considerable part of the point cloud
does not appear in the reflection, we use GNSS or lidar data to estimate the distance to the lakeside or river banks. Then, this distance is used to filter points so that we only take into accounts objects near the coastline, which is where reflections occur. Based on this distance, points are filtered in two ways:
by absolute distance to the boat, and by vertical distance. If the ship is near the coastline, then only objects with relatively small height are considered, as there would not be enough field of view to perceive the reflection of tall objects. The opposite applies when the ship is far from the water-land boundary, as
there might be too much noise around it with objects being too small to be identifiable while big objects have more clear reflections.);
extract the point cloud data acquired in a position which is a first predetermined distance away from the shore and which is within a second predetermined distance from the self-position of the ship, as a water-surface reflection data measured by reflection at a water surface, based on the self-position of the ship and the position of the shore (Qingqing – see page 509, col. 2, section A - Taking into account that a considerable part of the point cloud does not appear in the reflection, we use GNSS or lidar data to estimate the distance to the lakeside or river banks. Then, this distance is used to filter points so that we only take into
accounts objects near the coastline, which is where reflections occur. Based on this distance, points are filtered in two ways: by absolute distance to the boat, and by vertical distance. The opposite applies when the ship is far from the water-land boundary, as there might be too much noise around it with objects being
too small to be identifiable while big objects have more clear reflections.
see also Hiromatsu – see par 31, FIG. 2 - measures a time taken until the laser light 12 hitting reflection points 14 of the water surface 10 or an obstacle 13 within the measurement range and returning to the laser light-receiving unit as reflected light is detected; see par 43 - The three-dimensional LiDAR 2 [0044] scans the measurement range with a determined resolution, [0045] takes a range to scan within a certain period of time as one frame, [0046] measures the distance R and intensity I of each reflection point 14 included in one frame, and [0047] outputs the distance R, the intensity I, and ϕ and θ to the obstacle detection device 3; see par 104, FIG. 6; par 110-112 – determine whether each plane detected in step s1 is a pseudo water surface on basis of deviation from horizontal/height, and so on of the plane, and detects point cloud data P .sub.w,v located in vicinity of pseudo water surface and removes P.sub.w,v from P; see par 177 - If P.sub.w,v exists, when, about all boundary reflection points, in a case where a distance between a plane corresponding to P.sub.w,v and the boundary reflection point is equal to or smaller than a boundary water surface threshold value, the boundary detection unit 35 determines the boundary reflection point as a point on the water surface 10);
Hirmomatsu FIG. 2 -
PNG
media_image1.png
572
854
media_image1.png
Greyscale
a water-surface height calculation means configured to calculate a water-surface height based on the water-surface reflection data (Qingqing – see page 509, Col. 2, “Camera Height estimation”, 1st paragraph ; page 509, Col. 2, last paragraph - Then, given a pre-defined interval (hl; hh) where the camera is assumed to be, we choose a step _h and calculate, for each height h = hmin+k_h, the reflection of all points such that xiy > h and xiy h < hth. The constant hth 2 R+ is a threshold defined as a function of the distance to the coastline, and
PNG
media_image2.png
24
186
media_image2.png
Greyscale
is an integer that used to sample the interval (hl; hh). We estimate the water level height hw to be defined by:
PNG
media_image3.png
134
424
media_image3.png
Greyscale
See page 511, col. 2, last paragraph - We have applied the water height estimation in (1) to the point cloud data included in Figures 4 and 5. The computation
of the error according to (1) is shown in Figures 6 and 7;
see also Hiromatsu – see par 31 - measures a time taken until the laser light 12 hitting reflection points 14 of the water surface 10 or an obstacle 13 within the measurement range and returning to the laser light-receiving unit as reflected light is detected); and
perform driving assistance of the ship based on the calculated water-surface height (Applicant’s [0049] as published states “ the information processing device 1 performs driving assistance such as autonomous driving control of the target ship on the basis of the estimation result of the self-position. The driving assistance includes berthing assistance such as automatic berthing. Here, “berthing” includes not only the case of berthing the target ship to the wharf but also the case of berthing the target ship to a structural body such as a pier. The information processing device 1 may be a navigation device provided in the target ship or an electronic control device built in the ship.”
Qingqing – see page 508, col. 1, 3rd paragraph - The methods we design and develop can be applied to autonomous navigation in environments where water is reflective enough and there are objects nearby that can be detected by
cameras on-board the vessel; see page 512, Section V, 1st paragraph - the situational awareness and orientation of individual cameras can have a significant impact on the performance of navigation and collision avoidance algorithms in autonomous vessels.).
Qingqing and Hiromatsu are analogous art as they are directed to analyzing point cloud data and Lidar on water (see Qingqing Abstract; Hiromatsu Abstract, FIG. 2). 1) Qingqing discloses having a computer to execute operations (See page 511). Hiromatsu improves upon Qingqing by disclosing having a computer and memory/storage for storing instructions for executing processes. 2) Qingqing discloses filtering points from point cloud based on distance to the boat, so that when the ship is “far from the water-land boundary” the opposite approach is taken (See page 509, col. 2). Hiromatsu improves upon Qingqing by disclosing measuring a distance R for each frame, looking at deviation from horizontal, to have a distance less than a threshold value (par 110-112, 177, FIG. 2). One of ordinary skill in the art would be motivated to further include specific computer implementation and including storage for instructions and having a distance threshold for what to include for reflections from the water surface to efficiently improve upon the computer and RAM and the filtering of points based on distance to the boat relative to how “far” from the water-land boundary in Qingqing.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the camera height estimation with respect to water surface (See page 509, Col. 2, section B) in Qingqing to further use storage/memory for computer instructions and distance for only using some points for water reflections as disclosed in Hiromatsu, since the claimed invention is merely a combination of old elements, and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable and there is a reasonable expectation of success.
Concerning independent claim 8, Qingqing and Hiromatsu disclose:
A control method executed by a computer (Qingqing – see page 511, Col. 1, 2nd paragraph – hardware installed is Intel IoT edge gateway with CPU, and 4GB RAM
see also Hiromatsu – see par 68-69, FIG. 4 – memory is storage device; main memory used as a work area of the processor; processes stored in memory 6 developed on processor 5; par 71 - The auxiliary storage device 21 stores the obstacle detection program, programs run by the processor 5, data to be used when running the programs, and so on. A specific example of the auxiliary storage device 21 is a Hard Disk Drive (HDD) or a Solid-State Drive (SSD). The auxiliary storage device 21 may be a portable recording medium; claim 11 - . A non-transitory computer-readable recording medium recorded with an obstacle detection program that causes a computer).
The remaining limitations are similar to claim 1 above and are rejected for the same reasons.
It would have been obvious to combine Qingqing and Hiromatsu for the same reasons as claim 1.
Concerning claim 9, Qingqing and Hiromatsu disclose:
A non-transitory computer-readable program causing a computer to execute processing of (Qingqing – see page 511, Col. 1, 2nd paragraph – hardware installed is Intel IoT edge gateway with CPU, and 4GB RAM;
see also Hiromatsu – see par 68-69, FIG. 4 – memory is storage device; main memory used as a work area of the processor; processes stored in memory 6 developed on processor 5; par 71 - The auxiliary storage device 21 stores the obstacle detection program, programs run by the processor 5, data to be used when running the programs, and so on. A specific example of the auxiliary storage device 21 is a Hard Disk Drive (HDD) or a Solid-State Drive (SSD). The auxiliary storage device 21 may be a portable recording medium; claim 11 - . A non-transitory computer-readable recording medium recorded with an obstacle detection program that causes a computer).
The remaining limitations are similar to claim 1 above and are rejected for the same reasons.
It would have been obvious to combine Qingqing and Hiromatsu for the same reasons as claim 1.
Concerning claim 2, Qingqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph).
Hiromatsu discloses:
The information processing device according to claim 1, wherein the processor calculates an average of the values of the water-surface reflection data in a height direction as the water-surface height (Hiromatsu – see par 121 - The height of the point cloud data is, in a specific example, an average of heights of all points belonging to the point cloud data corresponding to the plane, or a height of the tallest point or the lowest point, among the points belonging to the point cloud data; see par 173 - , the boundary detection unit 35 [0174] finds an average value of the reflection intensities of all points included in P.sub.b,r(m.sub.r), as an average reflection intensity).
It would have been obvious to combine Qingqing and Hiromatsu for the same reasons as claim 1. Qinqqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph). Hiromatsu improves upon Qingqing by calculating an average for height. One of ordinary would be motivated to further include average values for height to efficiently improve upon the height camera estimation in Qingqing.
Concerning claim 4, Qingqing and Hiromatsu disclose:
The information processing device according to claim 1, wherein the processor is further configured to set a search range based on the measurement position and the water-surface height, and detect an obstacle … based on the point cloud data belonging to the search range (Qingqing – see page 510, FIG. 2 – Object Detection and Collision Avoidance; see page 510, col. 1, 1st paragraph - The value hth is chosen so that only reflections of objects near the water surface are taken into account when the ship is near the coastline, and bigger objects are accounted for otherwise. A KD-tree search algorithm is used to find the nearest point in the point cloud to the expected reflection. see page 512, col. 1, 2nd paragraph - In the situation where the ship is close to the river bank, we have used a height threshold hth = 0:5 m as reflections are more clear for objects close to the water level. In the case of the ship bow pointing towards the center of the river, we have used hth=3 m to account for taller trees and their reflections.
Analogously, we have used absolute threshold distances of 5 and 8m, respectively, taking into account the distance to the river bank estimated from the lidar data. These constants have an impact on the number of points that are used to calculate the error).
Qingqing discloses that waves from water surface can have impact on performance of different methods, based on reflective properties of still waters (See page 2, col. 1, 2nd paragraph).
Hiromatsu disclose:
The information processing device according to claim 1, wherein the processor is further configured to set a search range based on the measurement position and the water-surface height, and detect an obstacle and “a ship- wave” based on the point cloud data belonging to the search range (Hiromatsu – see par 93-101 - The plane detection unit 31 selects a maximum of K pieces of points belonging to a vicinity of a certain plane from the point cloud data P, and generates point cloud data P.sub.v(1) (1≤1≤I.sub.MAX where I.sub.MAX is an upper-limit number of planes to be detected by the plane detection unit 31; definition of distance to plane model; see par 151 - The wave peak detection unit 34 takes into account the fact that a point cloud group at a high position is highly unlikely to represent reflection points 14 of the water surface 10, and extracts a wave peak having a height estimated to represent a wave. see par 205 - the pseudo water surface determination process S2 and the wave peak detection unit 34 detect the reflection point 14 on the water surface 10 or on the wave peak 16 from the point cloud data acquired from the three-dimensional LiDAR 2, by fitting the reflection point 14 to the wave peak model 19, and [0207] the pseudo water surface determination process S2 selects point cloud data located at the lowest position in the height direction; see par 185-188 - An obstacle detection device 3 [0186] receives information on observation point cloud data formed of a plurality of reflection points observed within a measurement range including a water surface 10, the information including position information of each reflection point, [0187] and is provided with [0188] a plane detection unit 31 to detect planes extending near a plurality of reflection points included in the observation point cloud data).
It would have been obvious to combine Qingqing and Hiromatsu for the same reasons as claim 1 and claim 2. Qinqqing discloses that waves from water surface can have impact on performance of different methods, based on reflective properties of still waters (See page 2, col. 1, 2nd paragraph) and discloses a system that includes “object detection and collision avoidance” (See page 510, FIG. 2). Hiromatsu improves upon Qingqing by detecting “waves” and obstacles using various points and data within a vicinity/range (See par 93-101, 185-188). One of ordinary would be motivated to further include detecting waves and obstacles using various points and data within a vicinity/range to efficiently improve upon the “object detection and collision avoidance” in Qingqing.
Claims 3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Qingqing, et al., “Detecting Water Reflection Symmetries in Point Clouds for Camera Position Calibration in Unmanned Surface Vehicles,” 2019 19th International Symposium on Communications and Information Technologies, pages 507-512, in view of Hiromatsu (US 2024/0085553), as applied to claims 1-2, 4, and 8-9 above, and further in view of Chao (CN 109917414).
Concerning claim 3, Qingqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph).
Hiromatsu discloses:
The information processing device according to claim 1, wherein the processor calculates an average and a variance of the values in the height direction of the water-surface reflection data (Hiromatsu – see par 121 - The height of the point cloud data is, in a specific example, an average of heights of all points belonging to the point cloud data corresponding to the plane, or a height of the tallest point or the lowest point, among the points belonging to the point cloud data), and calculates the average as the water-surface height when the variance … (Hiromatsu see par 144 - FIG. 8 is a diagram illustrating examples of the wave peak 16 to be detected by the wave peak detection unit 34, in which white circles represent points in a point cloud group. Note that (a) of FIG. 8 illustrates an example of a feature of a wave peak 16a which forms a linearly long wave, (b) of FIG. 8 illustrates an example of a feature of a wave peak 16b which forms a pyramidal wave generated by synthesis of waves coming from a plurality of directions, and (c) of FIG. 8 illustrates an example of a wave peak model 19 expressed by statistic such as an average and a variance. The wave peak model 19 may be formed by another method.)
It would have been obvious to combine Qingqing and Hiromatsu for the same reasons as claim 1 and claim 2.
Chao discloses:
The information processing device according to claim 1, wherein the processor calculates an average and a variance of the values in the height direction of the water-surface reflection data, and calculates the average as the water-surface height when the variance “is smaller than a predetermined value.” (Chao – see page 6, 1st paragraph - If the variance is greater than the threshold value, it is proved that the corresponding area there is ship, area of other non-ship through which is surface area. If the two differ by less than the threshold of variance, which demonstrates that there is no ship, then it selects the portion of relatively small variance calculating water surface height. See page 8, 4th paragraph - calculating the average value of the x coordinate and the z coordinate of each line namely obtaining the mean of the mean line and the horizontal direction vertical direction lines. in the vertical direction line for example, any one of the line as the reference line, searching in turn line satisfying the following condition is composed of vertical plane: (1) the line does not belong to any surface; (2) the x coordinate and the z coordinate of the line with the average value of the obtained the difference is within a certain range).
Qingqing, Hiromatsu, and Chao are analogous art as they are directed to analyzing point cloud data and Lidar on water (see Qingqing Abstract; See Hiromatsu Abstract; Chao Abstract, page 8, 1st paragraph). Qinqqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph). Hiromatsu discloses calculating an average for height and a variance (par 121, 144). Chao improves upon Qingqing and Hiromatsu by using an average value of a z-coordinate (i.e. height) when differences in data gathered is within a certain range or a small variance (See page 6, or page 8). One of ordinary skill in the art would be motivated to further include having considering smaller differences in a range of height values for then just using an “average” value to efficiently improve upon the height camera estimation in Qingqing and the average and variance for height calculated in Hiromatsu.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the camera height estimation with respect to water surface (See page 509, Col. 2, section B) in Qingqing, the average and variance for height in Hiromatsu, and to further use average value for height when variance/range is small as disclosed in Chao, since the claimed invention is merely a combination of old elements, and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable and there is a reasonable expectation of success.
Concerning claim 7, Qingqing, Hiromatsu, and Chao disclose:
The information processing device according to claim 4 wherein the processor changes a height range of the search range based on a variance of a value in a height direction of the water-surface reflection data.
(Qingqing – see page 510, FIG. 2 – Object Detection and Collision Avoidance; see page 510, col. 1, 1st paragraph - The value hth is chosen so that only reflections of objects near the water surface are taken into account when the ship is near the coastline, and bigger objects are accounted for otherwise. A KD-tree search algorithm is used to find the nearest point in the point cloud to the expected reflection. see page 512, col. 1, 2nd paragraph - In the situation where the ship is close to the river bank, we have used a height threshold hth = 0:5 m as reflections are more clear for objects close to the water level.
Hiromatsu see par 144- FIG. 8 is a diagram illustrating examples of the wave peak 16 to be detected by the wave peak detection unit 34, in which white circles represent points in a point cloud group. Note that (a) of FIG. 8 illustrates an example of a feature of a wave peak 16a which forms a linearly long wave, (b) of FIG. 8 illustrates an example of a feature of a wave peak 16b which forms a pyramidal wave generated by synthesis of waves coming from a plurality of directions, and (c) of FIG. 8 illustrates an example of a wave peak model 19 expressed by statistic such as an average and a variance. The wave peak model 19 may be formed by another method..
Chao – see page 6, 1st paragraph - If the variance is greater than the threshold value, it is proved that the corresponding area there is ship, area of other non-ship through which is surface area. If the two differ by less than the threshold of variance, which demonstrates that there is no ship, then it selects the portion of relatively small variance calculating water surface height. See page 8, 4th paragraph - calculating the average value of the x coordinate and the z coordinate of each line namely obtaining the mean of the mean line and the horizontal direction vertical direction lines. in the vertical direction line for example, any one of the line as the reference line, searching in turn line satisfying the following condition is composed of vertical plane: (1) the line does not belong to any surface; (2) the x coordinate and the z coordinate of the line with the average value of the obtained the difference is within a certain range.
It would have been obvious to combine Qingqing, Hiromatsu, and Chao for the same reasons as claim 1 and claim 3.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Qingqing, et al., “Detecting Water Reflection Symmetries in Point Clouds for Camera Position Calibration in Unmanned Surface Vehicles,” 2019 19th International Symposium on Communications and Information Technologies, pages 507-512, in view of Hiromatsu (US 2024/0085553), as applied to claims 1-2, 4, and 8-9 above, and further in view of Muhovic, et al., “Obstacle tracking for unmanned surface vessels using 3-D point cloud,” 2019, IEEE Journal of Oceanic Engineering, Vol. 45, No. 3, pages 786-798.
Concerning claim 5, Qingqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph).
Hiromatsu discloses:
The information processing device according to claim 4, wherein the processor determines whether or not the point cloud data belonging to the search range forms a linear shape, and detects a linear point cloud data as a ship-wave … (Hiromatsu – see par 144 - FIG. 8 is a diagram illustrating examples of the wave peak 16 to be detected by the wave peak detection unit 34, in which white circles represent points in a point cloud group. Note that (a) of FIG. 8 illustrates an example of a feature of a wave peak 16a which forms a linearly long wave).
Muhovic discloses:
The information processing device according to claim 4, wherein the processor determines whether or not the point cloud data belonging to the search range forms a linear shape, and detects a linear point cloud data as a ship-wave “when the linear point cloud data exists for a predetermined time” (Muhovic – see page 792, col. 1, 1st paragraph - On many occasions, false stereo matches occur due to repeated vertical structures such as multiple boat masts in the marina, as shown in Fig. 11; To filter out these inconsistencies, temporal consistency verification must be included, to track only objects that are consistently appearing in multiple sequential frames. 2nd paragraph- To track the detections between sequential frames, we need to connect ones representing the same object as well as detect nonmatched objects and newly seen ones).
Qingqing, Hiromatsu, and Muhovic are analogous art as they are directed to analyzing point cloud data on water (see Qingqing Abstract; See Hiromatsu Abstract; Muhovic Abstract). Qingqing discloses that filters to point cloud used to reduce number of points and reduce computational needs and points can be filtered based on both absolute distance to the boat and vertical distance (See page 509, col. 2, section A) and that there can be a pre-defined interval of the height of the camera (See page 509, col. 2, section B, 3rd paragraph). Hiromatsu improves upon Qingqing by calculating a wave that is “linearly” long. Muhovic improves upon Qingqing and Hiromatsu by having object detection that looks for temporal consistency in sequential frames, to filter out inconsistencies. One of ordinary skill in the art would be motivated to further include further check for temporal consistency when analyzing the data to efficiently improve upon the filtering of data in Qingqing and the detection for linear shapes in Hiromatsu.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the camera height estimation with respect to water surface (See page 509, Col. 2, section B) in Qingqing, the average and variance for height and detection of waves with linear shapes in Hiromatsu, and to further check for temporal consistency when analyzing the data as disclosed in Muhovic, since the claimed invention is merely a combination of old elements, and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable and there is a reasonable expectation of success.
Concerning claim 6, Qingqing discloses that it can realize point cloud acquisition, filtering and “segmentation” (See page 511, Col. 2).
Hiromatsu and Muhovic disclose:
The information processing device according to claim 4, wherein the processor determines whether or not the point cloud data belonging to the search range forms a cluster, and detects a point cloud data forming the cluster as an obstacle (Hiromatsu – see par 130- The grouping unit 33 executes a pre-process for extracting a wave peak. see par 131 - The grouping unit 33 generates point cloud data P.sub.b(m) (1≤m) formed of points included in P.sub.r,2, by connecting points that are at short distances from each other. see par 137 - The grouping unit 33 may divide the plane using one or more types of polygons, instead of dividing the plane using square sections 15).
Muhovic discloses:
The information processing device according to claim 4, wherein the processor determines whether or not the point cloud data belonging to the search range “forms a cluster,” and detects a point cloud data forming the cluster as an obstacle “when the point cloud data forming the cluster exists for a predetermined time” (Muhovic – see page 792, col. 1, 1st paragraph - On many occasions, false stereo matches occur due to repeated vertical structures such as multiple boat masts in the marina, as shown in Fig. 11; To filter out these inconsistencies, temporal consistency verification must be included, to track only objects that
are consistently appearing in multiple sequential frames. 2nd paragraph- To
track the detections between sequential frames, we need to connect
ones representing the same object as well as detect nonmatched
objects and newly seen ones. see page 793, Col. 1, Section D - The obstacle detection method sometimes fragments large obstacles (e.g., large ships, sailboats, and piers) into many 3-D obstacles. This can happen due to local sparsity of the point cloud caused by the lighting conditions on the water surface, or by repeatable structures in the images. When projected back into the 2-D image, these detections can overlap (because of the projection to 2-D image plane) and cause errors. We used unsupervised clustering to address such problems, specifically the affinity propagation algorithm [26]. The algorithm clusters a set of data points based on a distance metric. It detects representative elements called exemplars and clusters the data points based on the distance to the nearest exemplar. The number of exemplars is determined automatically. In our case, data points are detection bounding boxes and the distance metric is the negative squared distance between bounding box centers. After the exemplars are detected, the bounding box with the largest area in each cluster is kept).
It would have been obvious to combine Qingqing, Hiromatsu, and Muhovic for the same reasons as claim 1. In addition, Hiromatsu discloses “grouping” points together. Muhovic improves upon Qingqing and Hiromatsu by further clustering points together based on affinity for tracking objects/obstacles.
Response to Arguments
Applicant's arguments filed 4/6/26 have been fully considered but they are not persuasive and/or are moot in view of the new rejections.
Applicant argues that Qingqing does not disclose “using the point cloud data far from the shore.” Remarks, pages 8-9. In response, Examiner respectfully disagrees. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., that one only uses “data far from the shore”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Instead, the claim only recites that one “extracts” data that is any distance “away” from the shore. It is unclear how the claim covers the subjective standard of being “away” from the shore. Either way, Qingqing discloses the limitation “extract the point cloud data acquired in a position which is a first predetermined distance away from the shore and which is within a second predetermined distance from the self-position of the ship, as a water-surface reflection data measured by reflection at a water surface, based on the self-position of the ship and the position of the shore” as Qinging discloses filtering points for by absolute distance to the boat. Hiromatsu discloses the limitation as well – as it also is having data points that are some distance from land and from the boat (See revised rejection). The claim covers any distance being “distance from the shore”, so long as it is some dimensional distance from the ship. Examiner suggests considering other claim language, such as considering the “direct” versus “indirect” reflection disclosures in the Application.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Bovcon, et al., “Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation,” 2018, Robotics and Autonomous Systems, Vol. 104, pages 1-13 – directed to using boat pitch and roll measurements with image segmentation to estimate the horizon when on water (See Abstract, see page 4, FIG. 4)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IVAN R GOLDBERG whose telephone number is (571)270-7949. The examiner can normally be reached 830AM - 430PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Coupe can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IVAN R GOLDBERG/ Primary Examiner, Art Unit 3619
Prosecution Insights