DETAILED ACTION
This is a response to Applicant’s submissions filed on 6/26/2026. Claims 1-14 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 6/26/2026 have been fully considered but they are not persuasive.
It is noted that Applicant’s amendments to the claims have overcome the previous rejection under 35 U.S.C. § 112.
In response to Applicant’s argument that “WiFi” is not a trademark or trade name (Applicant’s Remarks; p. 9), it is noted that a person of ordinary skill in the art would recognize the term “WiFi”, used to designate a protocol of a wireless type in paragraph 94, is a misspelling of the trademark WI-FI, which is active under US registration number 2525795. Misspelling a trademark is insufficient to avoid the proprietary nature of the mark. If Applicant intended to act as his or her own lexicographer to specifically define the term “WiFi” contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). Merely disclosing “WiFi” as an exemplary wireless protocol is not sufficient to put one reasonably skilled in the art on notice that the applicant intended to so redefine the term “WiFi” to mean something other than WI-FI, which is a family of wireless network protocols. See objection below.
In response to Applicant’s argument that the claimed invention cannot practically be performed mentally because it operates on data and is implemented in an automated driving system of a vehicle (Applicant’s Remarks; p. 11), the Examiner respectfully disagrees. Although the claims are directed to a process performed in a vehicle equipped with an automated driving system, the claims do not disclose any interaction between the automated driving system and the process, therefore, the claimed process could be executed in any generic vehicle that can provide a plurality of images from a generic image capturing device. A claim can recite a mental process, even if the mental processes are claimed as being performed on a computer, when there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper, see MPEP § 2106.04(a)(2)(III). The claims are directed to using received images and 3D point data to track the motion of 3D points associated with an object by observing changes in the positions of corresponding 2D points in the images. The human mind is clearly capable of determining changes in the positions of points in 2D visual information, determining 3D positions of the points using stereoscopic depth information, and tracking the motion of points of an object in 3D based on the visual and depth information. Although the claims disclose using a depth map, rather than stereoscopic depth information, to obtain the 3D points in the scene, paragraph 51 discloses the depth map may comprise a depth value for each pixel of the image, therefore a person could also use said depth map to determine the depths of points within the image. See rejection below.
In response to Applicant’s argument that the invention provides a specific technological improvement to vehicle perception systems by reducing temporal noise while simultaneously reducing computation burden (Applicant’s Remarks; pp. 11-13), it is noted that neither noise nor computational burden are recited in the claims, nor are they inherently reduced by the claimed invention. Although specific implementations of the Applicant’s disclosure may provide noise and processing power reductions when compared to other specific implementations, the same advantages cannot be generally claimed over the entire field of vehicle perception systems given the high level of generality of the recited steps. Further, paragraphs 7 and 10 disclose temporal noise is reduced when motion data is determined using a sequence of 2D images instead of 3D point cloud data, therefore, the same reductions would be realized in prior art systems such as Zhang et al. (US 11,774,983). The claims are also directed to generic computer hardware, a generic image capturing device, and a generic vehicle. Although the scope of the claims is limited to vehicles with automated driving systems, there is no claimed interaction between the method steps and the automated driving systems. See rejection below.
In response to Applicant’s argument that Zhang does not explicitly disclose or suggest utilizing the projection of an optical flow tracked 2D point to determine a subsequent 3D point for the purpose of deriving dynamic motion data for an object in the scene (Applicant’s Remarks; pp. 13-14), the Examiner respectfully disagrees. Zhang discloses, in column 24, lines 53-64, when a depth sensor is used, 2D-3D correspondence information for the optical flow tracked 2D features is obtained by directly using 2D-2D correspondences from optical flow tracking results. Zhang further discloses, in column 25, lines 18-20, that the 3D map is populated based on the 2D features and correspondence information. See rejection below.
In response to Applicant’s argument that because Kocamaz relies on 3D-centric temporal modeling, a person of ordinary skill in the art would not find it obvious to arrive at the claimed 2D-to-3D projection sequence (Applicant’s Remarks; pp. 14-15), the Examiner respectfully disagrees. Kocamaz’s disclosure is directed to joint 2D and 3D object tracking and Kocamaz explicitly discloses, in paragraphs 5-6, that the joint 2D and 3D object tracking offers advantages over tracking in 3D alone, similar to Applicant’s argument. Kocamaz further discloses converting between 2D and 3D spaces in paragraph 46, tracking object locations based on transition vectors in paragraph 53, and performing optical flow to process radar data. See rejection below.
Drawings
The amended specification received on 6/26/2026 has resolved the previous objections to the drawings. The drawings submitted on 12/21/2024 are acceptable.
Specification
The amendments to the abstract and specification were received on 6/26/2026.
The abstract of the disclosure is objected to because there is an extra comma preceding the semicolon in line 8. This appears to be a typographical error. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
The disclosure is objected to because of the following informalities:
In paragraph 94, line 7, the term “WiFi” appears to be a misspelling of the term WI-FI, which is a trade name or a mark used in commerce, and should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM, or ® following the term. Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The determination of whether a claim recites patent ineligible subject matter is a two-step inquiry.
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), see MPEP § 2106.03, or
STEP 2: the claim recites a judicial exception, e.g., an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis: see MPEP § 2106.04
STEP 2A (PRONG ONE): Does the claim recite an abstract idea, law of nature, or natural phenomenon? see MPEP § 2106.04(II)(A)(1)
STEP 2A (PRONG TWO): Does the claim recite additional elements that integrate the judicial exception into a practical application? see MPEP § 2106.04(II)(A)(2)
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? see MPEP § 2106.05
Claims 1-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
101 Analysis – Step 1
Claim 12 is directed to a device (i.e., a machine). Therefore, claim 12 is within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong One
Regarding Prong One of the Step 2A analysis, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. see MPEP § 2106(A)(II)(1) and MPEP § 2106.04(a)-(c). Independent claim 12 includes limitations that recite an abstract idea (emphasized below [with the category of abstract idea in brackets]) and will be used as a representative claim for the remainder of the analysis. Claim 12 recites:
A device comprising control circuitry configured to:
obtain an image sequence captured by an image capturing device of a vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances;
obtain a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene;
determine motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by performing the steps of [mental process/step]
obtaining a 2D point in an image plane of an image of the image sequence corresponding to the 3D point; applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image [mental process/step];
determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene; and determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point [mental process/step]; and
assign the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point [mental process/step].
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “determine motion data…”, “…applying an optical flow…”, “determining a subsequent 3D point…”, and “assign the set of 3D points … to a free space estimation..” in the context of this claim encompasses a person tracking an objects position through a video. Accordingly, the claim recites at least one abstract idea.
101 Analysis – Step 2A, Prong Two
Regarding Prong Two of the Step 2A analysis, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract idea into a practical application. see MPEP § 2106.04(II)(A)(2) and MPEP § 2106.04(d)(2). It must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” [with a description of the additional limitations in brackets], while the bolded portions continue to represent the “abstract idea”):
A device comprising control circuitry configured to [applying the abstract idea using generic computer components]:
obtain an image sequence captured by an image capturing device of a vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances [pre-solution activity (data gathering) using a generic sensor];
obtain a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene [pre-solution activity (receiving data)];
determine motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by performing the steps of
obtaining a 2D point in an image plane of an image of the image sequence corresponding to the 3D point; applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image;
determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene; and determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point; and
assign the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitation(s) of “obtain an image sequence…” and “obtain a set of 3D points…”, the examiner submits that the limitation(s) is/are insignificant extra-solution activities that merely use a computer (a device comprising control circuitry) to perform the process. In particular, the image sequence step is recited at a high level of generality (i.e., as a general means of receiving video), and amounts to merely gathering data using a generic sensor, which is a form of insignificant extra-solution activity. The 3D points step is recited at a high level of generality (i.e., as a general mean of receiving location data), and amounts to merely receiving data, which is a form of insignificant extra-solution activity. The “device comprising control circuitry” is/are also recited at a high level of generality (i.e., as generic computer components performing the generic computer function(s) of receiving and processing image data) such that it amounts to no more than mere instructions to apply the exception using a generic computer component.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception. see MPEP § 2106.05. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
Regarding Step 2B of the Revised Guidance, representative independent claim 12 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a device comprising control circuitry to perform the “determin[ing] motion data…”, “…applying an optical flow…”, “determining a subsequent 3D point…”, and “assign the set of 3D points … to a free space estimation..” amounts to nothing more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Also discussed above with respect to integration of the abstract idea into a practical application, the examiner submits that the additional limitation(s) of “obtain an image sequence…” and “obtain a set of 3D points…” is/are insignificant extra-solution activities. Hence, the claim is not patent eligible.
Claim(s) 1 is/are substantially the same subject matter as claim 12 except drawn to a method (i.e., a process) which falls under one of the statutory categories in step 1. Therefore, claim(s) 1 is/are rejected under step 2 for the same reasons above.
Dependent claim(s) 2-11 and 13-14 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of the dependent claims are directed toward additional aspects of the judicial exception. Although claims 10 and 13 disclose a trajectory planning module configured to generate candidate trajectories of the vehicle, the claims do not recite controlling the vehicle using one of the candidate trajectories. Therefore, dependent claims 2-11 and 13-14 are not patent eligible under the same rationale as provided for in the rejection of claims 1 and 12.
Therefore, claims 1-14 is/are ineligible under 35 U.S.C 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7 and 9-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 11,744,983) in view of Kocamaz et al. (US 2023/0360255), hereinafter Zhang and Kocamaz, respectively.
Regarding claims 1, 12 and 14, Zhang discloses a computer-implemented method, performed in a vehicle equipped with an automated driving system (Zhang; col. 5, ll. 32-34: the technology disclosed can be applied to autonomous vehicle guidance technology), the method comprising: obtaining an image sequence captured by an image capturing device of the vehicle, wherein the image sequence comprises a plurality of images depicting a scene at a respective time instance of a plurality of time instances (Zhang; col. 16, ll. 34-36: One feature extractor 1102 implementation employs optical flow methods to calculate the motion between two image frames, taken at times t and t+Δt at each voxel position.); obtaining a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene (Zhang; col. 49, ll. 2-4: where a depth sensor is used as an auxiliary sensor, a depth map is also received at the Control Unit); determining motion data associated with each 3D point of the set of 3D points, wherein the motion data is indicative of an estimated motion of an object in the scene associated with the 3D point, wherein the motion data associated with each 3D point is determined by performing the steps of obtaining a 2D point in an image plane of an image of the image sequence corresponding to the 3D point (Zhang; col. 16, ll. 13-14: Optical flow gives 2D-2D correspondence between previous image and a current image.); applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point in an image plane of the subsequent image (Zhang; col. 25, ll. 46-47: At step 1510, a new frame is captured by the camera at the new location.; col. 24, ll. 59-64: Based on the current image observation, the information needed for propagation, e.g., features, poses, map points, etc. is prepared. Then 2D-3D correspondence information for the optical flow tracked 2D features is obtained by directly using 2D-2D correspondences from optical flow tracking results.); determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene (Zhang; col. 25, ll. 58-65: At step 1540, depth values for the remaining features on the list of features are retrieved from the table of depth values. At step 1545, a weighted average depth value is calculated for each remaining feature in the feature list with the depth values of all the available pixel coordinates for that feature. At step 1550, a 3D map is populated with the remaining list of features on the list of the features with their corresponding depth value.); and assigning the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point (Zhang; col. 33; ll. 47-54: Monocular-auxiliary sensor prepares an occupancy map 1955 by reprojecting feature points 1901, 1911, 1941, 1951, 1922 onto a 2D layer corresponding to the floor of the room 1900. In some implementations, second and possibly greater occupancy maps are created at differing heights of the robot 1925, enabling the robot 1925 to navigate about the room 1900 without bumping its head into door soffits, or other obstacles above the floor.).
Although Zhang discloses determining motion data associated with the 3D points by determining the optical of the 2D points, Zhang does not appear to explicitly disclose determining the motion data based on a difference between the three-dimensional position of the 3D point and the subsequent 3D point.
Kocamaz, in the same field of endeavor (autonomous vehicle image processing), discloses determining motion data based on a difference between a three-dimensional position of a 3D point and a subsequent 3D point (Kocamaz; para. 107: The method 1100, at block B1106, may include determining, based at least on the first 3D detected location, a 3D predicted location associated with the tracked object. For instance, the tracking component 108 may determine the 3D predicted location associated with the tracked object at the second instance in time. In some examples, to determine the 3D predicted location, the tracking component 108 may use the first 3D detected location along with one or more states associated with the tracked object, such as the velocity, the acceleration, the orientation, and/or the like.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined, using the 3D locations, the velocity and acceleration of the surrounding detected objects, as disclosed by Kocamaz, in the control unit of Zhang, to yield the predictable result of accurately determining the trajectory of an object that could impact the vehicle.
Regarding claim 7, Zhang, as modified, discloses the motion data comprises information pertaining to a velocity of the object in the scene (Kocamaz; para. 67: the prediction component 110 may further use the state data 112 to determine additional state information, such as a predicted shape, velocity, acceleration, and/or the like of the object).
Regarding claim 9, Zhang, as modified, discloses assigning the set of 3D points with associated motion data to the free-space estimation of the scene comprises: selecting a subset of the set of 3D points corresponding to a free-space boundary of the free-space estimation (Zhang; col. 33, ll. 25-28: Monocular-auxiliary sensor determines feature points 1901, 1911, 1941, 1951, 1922, and so forth for the walls, corners and door 1923 of room 1900 from the information in the captured image frames.), and assigning aggregated motion data to the free-space boundary based on the subset of 3D points (Kocamaz; para. 63: prediction component 110 may use the bounding shape 302 and a transition vector, where the transition vector includes at least the final scalar change and the final translation, to determine a new bounding shape 416 (e.g., a predicted and/or estimated location) for the object).
Regarding claims 10 and 13, Zhang, as modified, discloses generating candidate trajectories of the vehicle, based on the free-space estimation (Zhang; col. 34, ll. 13-17: the robot uses the occupancy grid 1955 in order to plan a trajectory 1956 from its current location to another location in the map using the technology described herein above in the Mapping sections).
Regarding claim 11, Zhang, as modified, discloses a non-transitory computer readable storage medium storing instructions, which when executed by a computing device, causes the computing device to carry out the method according to claim 1 (Zhang; para. 49, ll. 22-24: a non-transitory computer readable storage medium storing instructions executable by a processor to perform any of the methods described above).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Pazhayampallil et al. (US 2021/0261157), hereinafter Pazhayampallil.
Regarding claim 2, Zhang, as modified, discloses determining the motion data further comprises obtaining an estimation of a ground plane of the scene depicted in the sequence of images (Zhang; para. 29, ll. 32-39: The occupancy grid map can be a plane oriented substantially perpendicular to the direction of gravity. Each layer has a specific height. (The layer on the floor is typically 0 according to one convention). FIG. 17 illustrates an example of an occupancy grid map in one implementation. Occupancy grid 1700 of FIG. 17 indicates a single layer, such as a floor layer 1702, mapped by an implementation of the monocular-auxiliary sensor.; para. 32, ll. 10-12: Obtain at least 3 points on the ground-plane from the descriptive point cloud to estimate the ground-plane's normal in the original coordinate.).
It is unclear if Zhang, as modified, explicitly discloses the motion data is determined as a motion parallel to the estimated ground plane.
However, Pazhayampallil, in the same field of endeavor (autonomous vehicle collision avoidance), explicitly discloses motion data is determined as a motion parallel to an estimated ground plane (Pazhayampallil; para. 82: Generally, motion of ground-based objects (e.g., vehicles, pedestrians), may occur approximately with in a horizontal plane (i.e., parallel to a ground plane), including linear motion along an x-axis, linear motion along a y-axis, and rotation about a z-axis normal to the horizontal plane, which may be represented as a linear velocity in the horizontal plane and an angular velocity about an axis normal to the horizontal plane. This variation of the method S100 is thus described below as executed by the autonomous vehicle to derive tangential, angular, and total velocities of an object within a horizontal plane given radial velocities and positions (e.g., ranges and angles) of points on the object in the horizontal plane.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined, with respect to the occupancy grid map layers, the velocity and acceleration of the surrounding detected objects in the control unit of Zhang as modified, by projecting their motion onto a horizontal plane parallel to a ground plane, as disclosed by Pazhayampallil, to yield the predictable result of ignoring object trajectories that will not intersect with the vehicle's path.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of McIntosh (US 2025/0054167).
Regarding claim 3, Zhang, as modified, discloses the invention substantially as claimed as describe above.
Zhang, as modified, does not explicitly disclose the depth map is determined from an output of a machine learning model configured to determine the depth map based on the image sequence as input.
McIntosh, in the same field of endeavor (autonomous vehicle image processing), discloses a depth map is based on an output of a machine learning model configured to determine the depth map based on an image sequence as input (McIntosh; para. 23: a machine learning (ML) model is used to generate a dense depth map based on one or more frames/images of a video).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have modified the depth map generated by the depth sensor of Zhang, as modified, to be generated from the images by a machine learning model, as disclosed by McIntosh, to yield the predictable result of accurately determining the depth information when there are many objects in the surrounding environment.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Shugrina et al. (US 12,555,260), hereinafter Shugrina.
Regarding claim 4, Zhang, as modified, discloses the invention substantially as claimed as described above.
Although Zhang, as modified, discloses the imaging system can use LiDAR (Zhang; col. 5, ll. 63-67), Zhang, as modified, does not appear to explicitly disclose the depth map is based on a LIDAR point cloud of the scene.
Shugrina, in the same field of endeavor (autonomous vehicle navigation), discloses a depth map is based on a LIDAR point cloud of the scene (Shugrina; col. 5, ll. 19-20: a depth map for an image is measured with one or more LiDAR sensors when said image is captured).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have modified the depth map generated by the depth sensor of Zhang, as modified, to be generated from LiDAR measurements, as disclosed by Shugrina, to yield the predictable result of accurately measuring distances to the objects in the surrounding environment.
Claim(s) 5-6 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kocamaz as applied to claim 1 above, and further in view of Nastroshvili et al. (US 2019/0049580), hereinafter Nastroshvili.
Regarding claim 5, Zhang, as modified, discloses the motion data is indicative of the estimated motion of the object in the scene associated with the 3D point (Kocamaz; para. 67: the prediction component 110 may further use the state data 112 to determine additional state information, such as a predicted shape, velocity, acceleration, and/or the like of the object).
It is unclear if Zhang, as modified, explicitly discloses estimating motion of the object relative a motion of the vehicle.
However, Natroshvili, in the same field of endeavor (autonomous vehicle navigation), discloses estimating motion of an object relative a motion of a vehicle (Natroshvili; para. 56: the velocity information associated with sensor measurements may be relative to the velocity of the ego vehicle).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined the velocity of the surrounding detected objects in the control unit of Zhang, as modified, relative to the vehicle's velocity, as disclosed by Natroshvili, to yield the predictable result of accurately determining the approach speed of an object that may collide with the vehicle.
Regarding claim 6, Zhang, as modified, discloses the motion data is further indicative of the estimated motion of the object in the scene associated with the 3D reference point relative the ground (Zhang; para. 29, ll. 32-39: The occupancy grid map can be a plane oriented substantially perpendicular to the direction of gravity. Each layer has a specific height. (The layer on the floor is typically 0 according to one convention). FIG. 17 illustrates an example of an occupancy grid map in one implementation. Occupancy grid 1700 of FIG. 17 indicates a single layer, such as a floor layer 1702, mapped by an implementation of the monocular-auxiliary sensor.; para. 32, ll. 10-12: Obtain at least 3 points on the ground-plane from the descriptive point cloud to estimate the ground-plane's normal in the original coordinate.).
It is unclear if Zhang, as modified, explicitly discloses the motion data is determined further based on vehicle motion data of the vehicle.
However, Natroshvili discloses motion data is determined based on vehicle motion data of a vehicle (Natroshvili; paras. 133-135: FIG. 7 shows an example of grid cells velocity propagation as vehicles and other objects move in the environment. In turn, FIG. 7 illustratively shows the rationale for formula (7). In FIG. 7, diagram 710 represents the movement of a vehicle v. The rectangle 712 represents the vehicle v at a time t, while the rectangle 714 represents the vehicle v at a time t+1. The arrow 716 illustratively shows the velocity of the vehicle v at time t. Lacking any other information on the movements of the vehicle v, it may be assumed that the velocity of vehicle v will stay constant in the future. In FIG. 7, diagram 720 represents the consequences of the assumption displayed in diagram 710 on the estimation of grid cells velocities.).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have determined the velocity of the surrounding detected objects in the control unit of Zhang, as modified, relative to the vehicle's velocity, as disclosed by Natroshvili, to yield the predictable result of accurately determining the approach speed of an object that may collide with the vehicle.
Regarding claim 8, Zhang, as modified, discloses assigning the set of 3D points with associated motion data to the free-space estimation of the scene comprises: assigning each 3D point of the set of 3D points to a cell of a plurality of cells in an occupancy grid of the free-space estimation, based on the three-dimensional position of the 3D points (Zhang; col. 37, ll. 39-40: sensory input is used to update an internal 3D map that is used to update an occupancy grid; col. 38, ll. 14-22: FIG. 27A illustrates an example collision map 2700 for an area coverage application in one implementation. Collision map 2700 is generated by planner to determine a “safe” path around an obstacle 2702 that lies at the center cell of the collision map 2700. When finding a path about an obstacle, the planner can build the collision map 2700 with cells representing the cell size in the occupancy grid and plot known obstacles in the collision map.).
Zhang, as modified, does not appear to explicitly disclose assigning, to each cell in the occupancy grid, aggregated motion data based on the motion data associated with the 3D points assigned to the respective cell.
Natroshvili discloses assigning, to each cell in an occupancy grid, aggregated motion data based on the motion data associated with points assigned to the respective cell (Natroshvili; para. 136: The grid cell 732, which corresponds to Cell n in formula (7), has a single occupancy hypothesis, illustratively represented by the circle 742, with a degree of belief of occupancy b732 and a velocity computed by the function vt(Cell 732).).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, with a reasonable expectation of success, to have assigned velocities to the grid cells of the occupancy grid of Natroshvili, for cells of the multilayer occupancy grid, in the control unit of Zhang, as modified, with the motivation of aggregating velocity data for a plurality of particles thereby reducing the computation required to compute the position of objects around the vehicle (Natroshvili; para. 204).
Supplemental References
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lv et al. (US 12,632,974) disclose determining 3D motions of objects around a vehicle by applying a depth map and optical flow to a series of 2D images from a monocular camera, and using the motions and distances to generate an occupancy grid.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH THOMPSON whose telephone number is (571)272-3660. The examiner can normally be reached Mon-Thurs 9:00AM-3:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Bishop can be reached at (571)270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH THOMPSON/Examiner, Art Unit 3665
/Erin D Bishop/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights