DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The present application’s status as a continuation of US Application 17/565,837 is acknowledged.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In January, 2019 (updated October 2019), the USPTO released new examination guidelines setting forth a two-step inquiry for determining whether a claim is directed to non-statutory subject matter. According to the guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claim 17 is directed toward non-statutory subject matter, as shown below:
STEP 1: Does claim fall within one of the statutory categories? Yes. The claim is directed toward a Process which falls within one of the statutory categories.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? Yes, the claim is directed to an abstract idea.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Example: iv. organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721.
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
See claim 17 language below:
A method comprising:
applying, to a neural network, sensor data obtained using one or more sensors, first data representative of first information associated with one or more objects, and second data representative of second information associated with one or more lanes;
determining, based at least on the neural network processing the sensor data, the first data, and the second data, one or more associations between the one or more objects and the one or more lanes; and
performing, based at least on the one or more associations, one or more planning, control, or navigation operations associated with a machine.
The Process in claim 17, specifically the limitations bolded above, is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of applying data, determining associations, and performing planning. This is equivalent to looking at an image of a vehicle’s surroundings, a rasterized image of the objects in the vehicle’s surroundings, and a rasterized image of the lanes in the vehicle’s surroundings, identifying using the imagery which objects lie on which lanes, and mentally planning a course of action for the vehicle to avoid colliding with the objects.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? No, the claim does not recite additional elements, underlined above, that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim 17 does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
The limitation of “to a neural network” merely describe how to generally “apply” the otherwise mental judgments in a generic or general-purpose computing environment. The neural network is recited at a high level of generality without placing any limits on how the neural network functions. The limitations do not include details about how the “apply” step are accomplished with the neural network.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claim does not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claim 17 does not recite any specific limitation or combination of limitations that are not well- understood, routine, conventional (WURC) activity in the field.
Mere data communication steps that can be performed entirely on any one or more generic computer/-s have also been previously identified by the courts as an abstract idea (i.e. a judicial exception): (A) Receiving and/or transmitting data is considered to be well-understood, routine, or conventional at least as evidenced by MPEP § 2106.05(d)(II)(i) "Receiving or transmitting data over a network", and (iv) "Storing and retrieving information in memory" and (B) Comparing the received data to other data is considered to be well-understood, routine or conventional at least as evidenced by MPEP§ 2106.05(d)(II)(ii) "Performing repetitive calculations".
CONCLUSION
Thus, since claim 17 is: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that claim 17 is directed towards non-statutory subject matter.
Additionally, Claims 1-16 and 18-20:
fall within one of the statutory categories (Claims 1-16: Machine, 18-20: Process)
directed toward an abstract idea (mental process),
do not recite additional elements that integrate the judicial exception into a practical application, and
do not recite additional elements that amount to significantly more than the judicial exception.
Therefore, it is clear that Claims 1-16 and 18-20 are directed towards non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 9-11, 14, and 16-20 are rejected under 35 U.S.C 102(a)(2) as being anticipated by Marchetti-Bowick et al. (US 20210004012 A1).
Regarding Claim 9, Marchetti-Bowick teaches A system (FIG. 1, [0047]: “system”) comprising:
one or more processors (see at least FIG. 12: processors 1012, 1032) to:
obtain first data (see at least [0033]: “object features (e.g., object state data)”; [0079]: “the object information can capture the object's current and past behavior as object state data”; [0086]: “the state data for each object can describe an estimate of the object's: current location (also referred to as position)”; [0086]: “the perception system can determine state data 130 for each object”) representative of one or more first locations associated with one or more objects within an environment and second data (see at least [0033]: “path features for a candidate path”; [0024]: “a machine-learned occupancy model of an autonomy computing system for an autonomous vehicle can utilize map data that includes information associated with spatial relationships of lanes of a road network to more accurately predict the future state of an object. These spatial relationships of the lanes can be used to develop a set of candidate paths”; [0056]: “map data 132 can include information regarding: … the location … of traffic lanes”) representative of one or more second locations associated with one or more lanes within the environment;
determine, based at least on a neural network processing the first data, the second data, (see at least FIG. 10 step 708: “combine path, object, and scene features using … prediction neural network”; [0097]: “At 708, method 700 can include combining path features, object features, and scene feature using one or more first layers of a prediction neural network of the machine learned occupancy prediction model.”) and sensor data (see at least [0032]: “a first neural network such as a base convolutional neural network (CNN) that is configured to obtain scene information regarding the environment external to the autonomous vehicle and generate scene features. For example, the base CNN can obtain raster data including a bird's eye view scene raster of the environment. The scene raster can be generated from sensor data generated by a set of sensors of the autonomous vehicle”) representative of the environment, one or more associations between (see at least FIG. 5 step 460: “for each candidate path, predict future occupancy of each cell based on one or more objects”; step 462: “generate prediction data associated with the object(s) based on the predicted occupancies for each candidate path”) the one or more objects and the one or more lanes; and
perform, based at least on the one or more associations, one or more planning (see at least FIG. 5 step 464: “generate one or more motion plans based on prediction data”), control (see at least [0065]: “The vehicle computing system 110 can cause the vehicle 102 to initiate a motion control in accordance with at least a portion of the motion plan 162.”), or navigation operations associated with a machine (see at least FIG. 1: vehicle 102).
Regarding Claim 10, Marchetti-Bowick teaches The system of claim 9, wherein the one or more processors are further to input the first data and the second data (see at least FIG. 10 step 708: “combine path, object, and scene features using … prediction neural network”; [0097]: “At 708, method 700 can include combining path features, object features, and scene feature using one or more first layers of a prediction neural network of the machine learned occupancy prediction model.”) along with the sensor data ([0032]: “a first neural network such as a base convolutional neural network (CNN) that is configured to obtain scene information regarding the environment external to the autonomous vehicle and generate scene features. For example, the base CNN can obtain raster data including a bird's eye view scene raster of the environment. The scene raster can be generated from sensor data generated by a set of sensors of the autonomous vehicle”) into the neural network for processing.
Regarding Claim 11, Marchetti-Bowick teaches The system of claim 9, wherein:
the first data represents one or more bounding shapes (see at least [0060]: “the perception data 142 for each object can describe (e.g., for a given time, time period) an estimate of the object's: current and/or past location (also referred to as position); … size/footprint (e.g., as represented by a bounding shape)”) indicating the one or more first locations associated with the one or more objects; and
the second data represents one or more lane graphs (see at least [0033]: “path features for a candidate path”; [0081]: “generating the set of candidate paths can include generating a lane graph”) indicating the one or more second locations associated with the one or more lanes.
Regarding Claim 14, Marchetti-Bowick teaches The system of claim 9, wherein the one or more processors are further to generate at least one of the first data (see at least [0060]: “the vehicle computing system 110 (e.g., the perception system 140) can process the sensor data 118 … to obtain perception data 142.”; [0027]: “perception data including the state data (e.g., object detection data)”) or the second data based at least on processing the sensor data.
Regarding Claim 16, Marchetti-Bowick teaches The system of claim 9,
wherein the system is comprised in at least one of:
a control system for an autonomous (see at least [0025]: “An autonomous vehicle … can include various systems and devices configured to control the operation of the vehicle.”) or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using a collaborative content creation platform for 3D assets;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding Claim 17, Marchetti-Bowick teaches A method comprising:
applying (see at least FIG. 10 step 708: “combine path, object, and scene features using … prediction neural network”; [0097]: “At 708, method 700 can include combining path features, object features, and scene feature using one or more first layers of a prediction neural network of the machine learned occupancy prediction model.”), to a neural network, sensor data (see at least [0032]: “a first neural network such as a base convolutional neural network (CNN) that is configured to obtain scene information regarding the environment external to the autonomous vehicle and generate scene features. For example, the base CNN can obtain raster data including a bird's eye view scene raster of the environment. The scene raster can be generated from sensor data generated by a set of sensors of the autonomous vehicle”) obtained using one or more sensors, first data (see at least [0033]: “object features (e.g., object state data)”; [0079]: “the object information can capture the object's current and past behavior as object state data”; [0086]: “the perception system can determine state data 130 for each object”) representative of first information associated with one or more objects, and second data (see at least [0033]: “path features for a candidate path”; [0024]: “a machine-learned occupancy model of an autonomy computing system for an autonomous vehicle can utilize map data that includes information associated with spatial relationships of lanes of a road network to more accurately predict the future state of an object. These spatial relationships of the lanes can be used to develop a set of candidate paths”) representative of second information associated with one or more lanes;
determining, based at least on the neural network processing the sensor data, the first data, and the second data, one or more associations (see at least FIG. 5 step 460: “for each candidate path, predict future occupancy of each cell based on one or more objects”; step 462: “generate prediction data associated with the object(s) based on the predicted occupancies for each candidate path”) between the one or more objects and the one or more lanes; and
performing, based at least on the one or more associations, one or more planning (see at least FIG. 5 step 464: “generate one or more motion plans based on prediction data”), control (see at least [0065]: “The vehicle computing system 110 can cause the vehicle 102 to initiate a motion control in accordance with at least a portion of the motion plan 162.”), or navigation operations associated with a machine (see at least FIG. 1: vehicle 102).
Regarding Claim 18, Marchetti-Bowick teaches The method of claim 17, wherein:
the first information indicates one or more first locations (see at least [0086]: “the state data for each object can describe an estimate of the object's: current location (also referred to as position)”) associated with the one or more objects; and
the second information indicates one or more second locations (see at least [0056]: “map data 132 can include information regarding: … the location … of traffic lanes”; [0024]: map data [Wingdings font/0xE0] path features) associated with the one or more lanes.
Regarding Claim 19, Marchetti-Bowick teaches The method of claim 17, wherein:
the first information indicates one or more bounding shapes (see at least [0060]: “the perception data 142 for each object can describe (e.g., for a given time, time period) an estimate of the object's: current and/or past location (also referred to as position); … size/footprint (e.g., as represented by a bounding shape)”) associated with the one or more objects; and
the second information indicates one or more lane graphs (see at least [0033]: “path features for a candidate path”; [0081]: “generating the set of candidate paths can include generating a lane graph”) associated with the one or more lanes.
Regarding Claim 20, Marchetti-Bowick teaches The method of claim 17,
further comprising processing the sensor data to generate at least one of the first data (see at least [0060]: “the vehicle computing system 110 (e.g., the perception system 140) can process the sensor data 118 … to obtain perception data 142.”; [0027]: “perception data including the state data (e.g., object detection data)”) or the second data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5-8, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Marchetti-Bowick et al. (US 20210004012 A1) in view of Choe et al. (US 20200026282 A1).
Regarding claim 1, Marchetti-Bowick teach An autonomous (see at least [0025]: “An autonomous vehicle … can include various systems and devices configured to control the operation of the vehicle.”) or semi-autonomous machine comprising:
one or more
one or more sensors (see at least [0055]: “the sensor(s) 116 can be configured to acquire image(s) and/or other two- or three-dimensional data within a field of view of one or more of the vehicle sensor(s) 116.”) having one or more fields of view or one or more sensory fields,
wherein the autonomous or semi-autonomous machine is to perform one or more planning (see at least FIG. 5 step 464: “generate one or more motion plans based on prediction data”), control (see at least [0065]: “The vehicle computing system 110 can cause the vehicle 102 to initiate a motion control in accordance with at least a portion of the motion plan 162.”), or navigation operations based at least on one or more associations between one or more objects and one or more lanes, wherein the one or more associations (see at least FIG. 5 step 460: “for each candidate path, predict future occupancy of each cell based on one or more objects”; step 462: “generate prediction data associated with the object(s) based on the predicted occupancies for each candidate path”) are determined based at least on a neural network processing (see at least FIG. 10 step 708: “combine path, object, and scene features using … prediction neural network”; [0097]: “At 708, method 700 can include combining path features, object features, and scene feature using one or more first layers of a prediction neural network of the machine learned occupancy prediction model.”) first data (see at least [0033]: “object features (e.g., object state data)”; [0079]: “the object information can capture the object's current and past behavior as object state data”; [0086]: “the state data for each object can describe an estimate of the object's: current location (also referred to as position)”; [0086]: “the perception system can determine state data 130 for each object”) associated with the one or more objects, second data (see at least [0033]: “path features for a candidate path”; [0024]: “a machine-learned occupancy model of an autonomy computing system for an autonomous vehicle can utilize map data that includes information associated with spatial relationships of lanes of a road network to more accurately predict the future state of an object. These spatial relationships of the lanes can be used to develop a set of candidate paths”; [0056]: “map data 132 can include information regarding: … the location … of traffic lanes”) associated with the one or more lanes, and sensor data (see at least [0032]: “a first neural network such as a base convolutional neural network (CNN) that is configured to obtain scene information regarding the environment external to the autonomous vehicle and generate scene features. For example, the base CNN can obtain raster data including a bird's eye view scene raster of the environment. The scene raster can be generated from sensor data generated by a set of sensors of the autonomous vehicle”) obtained using the one or more sensors.
However, Marchetti-Bowick does not explicitly teach one or more central processing units (CPUs);
one or more graphics processing units (GPUs);
one or more hardware accelerators.
Choe teach one or more central processing units (CPUs) (see at least [0084]: “Processor 1501 may represent one or more general-purpose processors such as … a central processing unit (CPU)”);
one or more graphics processing units (GPUs) (see at least [0084]: “Processor 1501 may also be one or more special-purpose processors such as … a graphics processor”);
one or more hardware accelerators (see at least [0027]: “throttle unit 202 (also referred to as an acceleration unit… Throttle unit 202 is to control the speed of the motor or engine that in turn controls the … acceleration of the vehicle.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to include CPUs and GPUs. Doing so would “achieve fast network inference rate”, as recognized by Choe in paragraph [0052].
Regarding claim 2, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Marchetti-Bowick further teaches wherein:
the first data represents one or more first locations (see at least [0086]: “the state data for each object can describe an estimate of the object's: current location (also referred to as position)”) associated with the one or more objects; and
the second data represents one or more second locations (see at least [0056]: “map data 132 can include information regarding: … the location … of traffic lanes”) associated with the one or more lanes.
Regarding claim 3, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Marchetti-Bowick further teaches wherein:
the first data represents one or more bounding shapes (see at least [0060]: “the perception data 142 for each object can describe (e.g., for a given time, time period) an estimate of the object's: current and/or past location (also referred to as position); … size/footprint (e.g., as represented by a bounding shape)”) associated with the one or more objects; and
the second data represents one or more lane graphs (see at least [0033]: “path features for a candidate path”; [0081]: “generating the set of candidate paths can include generating a lane graph”) associated with the one or more lanes.
Regarding claim 5, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Marchetti-Bowick further teaches wherein:
the sensor data represents the one or more objects (see at least [0060]: “The vehicle computing system 100 (e.g., the autonomy computing system 130) can identify one or more objects that are proximate to the vehicle 102 based at least in part on the sensor data 118”)
the first data (see at least [0033]: “object features (e.g., object state data)”) represents the one or more objects without representing the one or more lanes; and
the second data (see at least [0033]: “path features for a candidate path”) represents the one or more lanes without representing the one or more objects.
Choe further teaches the sensor data represents the one or more second locations associated with the one or more lanes (see at least FIG. 6: “camera view of an ADV for an example driving scenario having lane lines”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to consider sensor data including lane lines. Doing so would provide “a reliable vision-based perception system of ADVs when LIDAR sensors may not be available”, as recognized by Choe in paragraph [0003].
Regarding claim 6, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Marchetti-Bowick further teaches wherein the autonomous or semi-autonomous machine is further to generate at least one of the first data (see at least [0060]: “the vehicle computing system 110 (e.g., the perception system 140) can process the sensor data 118 … to obtain perception data 142.”; [0027]: “perception data including the state data (e.g., object detection data)”) or the second data based at least on processing the sensor data.
Regarding claim 7, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Marchetti-Bowick further teaches wherein the sensor data represents the one or more objects (see at least [0060]: “The vehicle computing system 100 (e.g., the autonomy computing system 130) can identify one or more objects that are proximate to the vehicle 102 based at least in part on the sensor data 118”) a
Choe further teach wherein the sensor data represents the one or more lanes (see at least FIG. 6: “camera view of an ADV for an example driving scenario having lane lines”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to consider sensor data including lane lines. Doing so would provide “a reliable vision-based perception system of ADVs when LIDAR sensors may not be available”, as recognized by Choe in paragraph [0003].
Regarding claim 8, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
Choe further teach wherein:
the second data represents one or more classifications (see at least [0067]: “lane detection/tracking module associates these lane markers with several lane line objects having corresponding relative spatial label (e.g., left(L0), right(R0), next left(L1), next right(L2), etc.)”) relating the one or more lanes with respect to a lane associated with the autonomous or semi-autonomous machine; and
the autonomous or semi-autonomous machine is further to perform the one or more planning, control, and navigation operations based at least on (see at least FIG. 9 step 904, [0067]: “The lane line objects are then used by the ADV to generate a lateral control (e.g., steer left or steer right) command for the ADV to keep the ADV within the contour edges of a lane.”) the one or more classifications.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to classify lane information and plan navigation based on classifications. Doing so would provide “a reliable vision-based perception system of ADVs when LIDAR sensors may not be available”, as recognized by Choe in paragraph [0003].
Regarding claim 13, Marchetti-Bowick teach The system of claim 9, wherein:
the sensor data represents the one or more first locations (see at least [0060]: “the vehicle computing system 110 (e.g., the perception system 140) can process the sensor data 118 … to obtain perception data 142.”; [0027]: “perception data including the state data (e.g., object detection data)”; [0060]: “the perception data 142 for each object can describe (e.g., for a given time, time period) an estimate of the object's: current and/or past location (also referred to as position)”) associated with the one or more objects
the first data (see at least [0033]: “object features (e.g., object state data)”) does not represent the one or more second locations associated with the one or more lanes; and
the second data (see at least [0033]: “path features for a candidate path”) does not represent the one or more first locations associated with the one or more objects.
However, Marchetti-Bowick does not explicitly teach and the one or more second locations associated with the one or more lanes.
Choe teach the sensor data represents the one or more second locations associated with the one or more lanes (see at least FIG. 6: “camera view of an ADV for an example driving scenario having lane lines”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to consider sensor data including lane lines. Doing so would provide “a reliable vision-based perception system of ADVs when LIDAR sensors may not be available”, as recognized by Choe in paragraph [0003].
Regarding claim 15, Marchetti-Bowick teach The system of claim 9.
However, Marchetti-Bowick does not explicitly teach wherein:
the second data further represents one or more classifications relating the one or more lanes with respect to a lane associated with the machine; and
the one or more planning, control, and navigation operations are further performed based at least on the one or more classifications.
Choe teach wherein:
the second data further represents one or more classifications (see at least [0067]: “lane detection/tracking module associates these lane markers with several lane line objects having corresponding relative spatial label (e.g., left(L0), right(R0), next left(L1), next right(L2), etc.)”) relating the one or more lanes with respect to a lane associated with the machine; and
the one or more planning, control, and navigation operations are further performed based at least on (see at least FIG. 9 step 904, [0067]: “The lane line objects are then used by the ADV to generate a lateral control (e.g., steer left or steer right) command for the ADV to keep the ADV within the contour edges of a lane.”) the one or more classifications.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Choe to classify lane information and plan navigation based on classifications. Doing so would provide “a reliable vision-based perception system of ADVs when LIDAR sensors may not be available”, as recognized by Choe in paragraph [0003].
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Marchetti-Bowick et al. (US 20210004012 A1) in view of Choe et al. (US 20200026282 A1) and Yu et al. (US 20200149916 A1).
Regarding claim 4, the combination of Marchetti-Bowick and Choe teach The autonomous or semi-autonomous machine of claim 1.
However, Marchetti-Bowick does not explicitly teach wherein:
the first data represents one or more first images indicating the one or more first locations associated with the one or more objects; and
the second data represents one or more second images indicating the one or more second locations associated with the one or more lanes.
Yu teach wherein:
the first data represents one or more first images (see at least FIG. 1A: segmentation image 100c) representing the one or more objects; and
the second data represents one or more second images (see at least FIG. 1A: AR path Overlay image 100d) representing the one or more lanes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Yu to consider object and path images in vehicle navigation. Doing so would help provide for an “enhanced guidance indicator” that “is generated and displayed responsive to real-time images of the upcoming route at the current time and at the current location of the system,” as recognized by Yu in paragraph [0020].
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Marchetti-Bowick et al. (US 20210004012 A1) in view of Yu et al. (US 20200149916 A1).
Regarding claim 12, Marchetti-Bowick teach The system of claim 9.
However, Marchetti-Bowick does not explicitly teach wherein:
the first data represents one or more first images indicating the one or more first locations associated with the one or more objects; and
the second data represents one or more second images indicating the one or more second locations associated with the one or more lanes.
Yu teach wherein:
the first data represents one or more first images (see at least FIG. 1A: segmentation image 100c) indicating the one or more first locations associated with the one or more objects; and
the second data represents one or more second images (see at least FIG. 1A: AR path Overlay image 100d) indicating the one or more second locations associated with the one or more lanes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Marchetti-Bowick to incorporate the teachings of Yu to consider object and path images in vehicle navigation. Doing so would help provide for an “enhanced guidance indicator” that “is generated and displayed responsive to real-time images of the upcoming route at the current time and at the current location of the system,” as recognized by Yu in paragraph [0020].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ros Sanchez et al. (US 20190370666 A1) teaches a system that separates objects, lanes, and surroundings in imagery of synthetic images and then blends the separated images together to improve quality of the imagery (see paragraph [0006], FIG. 2).
Rauch (DE 102013208521 A1) teaches a vehicle system that creates raster images of road markings (see paragraph [0058]) to create a roadway model (see paragraph [0122]).
Vorobiov et al. (DE 102018203583 B4) teaches a system identifies vehicle positions on roadway segments using a neural network (see FIG. 1, paragraph [0047]) for use with a deriver assist system (see paragraph [0053]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE ALCORN whose telephone number is (571) 270-3763. The examiner can normally be reached M-F, 9:30 am – 6:30 pm est.
Examiner Interview are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571) 270-3415. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE A ALCORN III/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights