DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This office action is in response to the application filed on April 14,2025. Claims 1-11 are presently pending and are presented for examination.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. EP24173085.2, filed on April 29, 2024.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on April 14, 2025 and June 13, 2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The Examiner has identified method Claim 1 as the claim that represents the claimed invention for analysis. Claim 1 recites the limitations of (additional elements emphasized in bold and are considered to be parsed from the remaining abstract idea):
A method for determining a surrounding representation of a surrounding of a vehicle, comprising:
providing input data, wherein the input data comprises sensor data and feedback data, wherein the sensor data results from a detection of at least one sensor of the vehicle, and wherein the sensor data represents a detection of the surrounding of the vehicle;
providing a machine-learning model, wherein the machine-learning model comprises a pre-processing module and at least one task-specific module;
providing the feedback data, wherein the feedback data comprises at least one historical output of the at least one task-specific module and/or at least one historical output of the pre- processing module, and wherein the historical output has been determined by the at least one task-specific module and/or the pre-processing module at least one iteration prior to a current iteration;
extracting features from the input data by way of the pre-processing module; and
determining, by way of the at least one task-specific module, a respective output based on the features extracted by the pre-processing module and/or the at least one historical output of the at least one task-specific module and/or the at least one historical output of the pre-processing module for the current iteration in order to determine the surrounding representation of the surrounding of the vehicle.
which is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) as a Mental process (concept performed in the human mind) but for the recitation of generic computer elements. For example, a person could mentally obtain the information about the surrounding of the vehicle, record/remember prior actions taken based on the surrounding information, and determine what features are important in the current surroundings.
With respect to Step 2A, Prong II, this judicial exception is not practically integrated. The claim recites the additional elements of “processor and memory” multiple times. These elements are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. Accordingly, these elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
With respect to Step 2B, the aforementioned additional elements are all generic computer elements have been held to be not significantly more than the abstract idea by Alice. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the additional elements of using the processors to receive information, make decisions, and supply instructions amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Furthermore, the limitation step of “extracting features from a machine learning model”, is not more than the judicial exception, because as detailed in Electric Power Group, additional elements that are used to simply output results do not amount to significantly more than the abstract idea itself.
Claims 9-11 cite the same limitations as that in claim 1, with the exception of adding more generic computer components, and are therefore also rejected under 35 USC § 101.
Claims 2-6 further define characteristics of the system. However, these characteristics do not add limitations that would integrate the abstract idea into a practical application and are therefore also rejected under 35 USC § 101.
Claim 8 recites limitations that include a neural network, a transformer, or point-processing network without specific details of how the data is utilized so they too can also be performed in the human mind and do not integrate the abstract idea into a practical application. Therefore, these claims are also rejected under 35 USC § 101.
Claim 7 recites the limitation of initiating a controlling of the vehicle based on the determined respective output, which does integrate the abstract idea into a practical application. This integration renders this claim as eligible if it were part of the independent claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-11 are rejected under 35 U.S.C. 103 as being unpatentable over Popov et al., US 20250327895 A1 (Hereinafter, “Popov”) in view of Zamir et al., “Feedback Networks” (Hereinafter “Zamir)(NPL Line U).
Regarding Claim 1 and 9-11, Popov discloses a method for determining a surrounding representation of a surrounding of a vehicle, comprising: providing input data, wherein the input data comprises sensor data wherein the sensor data results from a detection of at least one sensor of the vehicle, and wherein the sensor data represents a detection of the surrounding of the vehicle; See Fig.1, 13A/B and [0027], “Systems and methods are disclosed relating to object detection for autonomous machines using deep neural networks (DNNs). Systems and methods are disclosed that use object detection techniques to identify or detect instances of moving or stationary obstacles (e.g., cars, trucks, pedestrians, cyclists, etc.) and other objects within environments for use by autonomous vehicles.” Also [0036-0041], “FIG. 2 is a data flow diagram illustrating an example process for pre-processing 104 sensor data 102 for machine learning model(s) 108 in an object detection system, in accordance with some embodiments of the present disclosure. In this example, sensor data 102 may include RADAR detections, which may be accumulated 210 (which may include transforming to a single coordinate system), ego-motion-compensated 220, and/or encoded 230 into a suitable representation such as a projection image of the RADAR detections, with multiple channels storing different reflection characteristics … More specifically, sensor detections such as RADAR detections may be accumulated 210 from multiple sensors, such as some or all the surrounding RADAR sensor(s) 1360 from different locations of the autonomous vehicle 1300, and may be transformed to a single vehicle coordinate system.” and [0059-0062], “The process … executed by the perception component(s), which may feed up the layers of the drive stack 122 to the world model manager … defined, at least in part, based on affordances for obstacles, paths, and wait conditions that can be perceived in real-time or near real-time by the obstacle perceiver, the path perceiver, the wait perceiver, and/or the map perceiver. The world model manager 126 may continually update the world model based on newly generated and/or received inputs (e.g., “feedback” data) from … components of the autonomous vehicle control system.”
providing a machine-learning model, wherein the machine-learning model comprises a pre-processing module (310) and at least one task-specific module; providing ; See Fig.2 and [0037], “process 100 may include one or more machine learning models 108 configured to detect objects such as instances of obstacles from sensor data 102 such as RADAR detections generated from RADAR sensors 101. The sensor data 102 may be pre-processed 104 into input data with a format that machine learning model(s) 108 understands-such as a RADAR data tensor 106—and the input data may be fed into machine learning model(s) 108 to detect objects 116 represented in the input data … predicts a class confidence tensor 110 and an instance regression tensor 112 (task specific module), which may be post-processed 114 into object detections 116 comprising bounding boxes, closed polylines, or other bounding shapes identifying the locations, sizes, and/or orientations of the detected objects.” In Fig.3 and [0048], “Feature extractor trunk 310 (pre-processing module)may be implemented using encoder and decoder components with skip connections (e.g., similar to a Feature Pyramid Network, U-Net, etc.). For example, feature extractor trunk 310 may accept input data such as RADAR data tensor 106 and apply various convolutions, pooling, and/or other types of operations to extract features into some latent space.”
extracting features from the input data by way of the pre-processing module; and determining, by way of the at least one task-specific module, a respective output based on the features extracted by the pre-processing module aSee [0029], “The architecture of the DNN may enable features to be extracted from the RADAR data tensor, and may enable class segmentation and/or instance regression to be executed on the extracted features.” Also [0037-0040] and [0047-0050].
Popov discloses a method for determining the surrounds of a vehicle, but does not explicitly disclose using feedback data. However, Zamir teaches an object detection model using feedback networks in which the output of a previous iteration of the neural network is used a feedback for a subsequent/current iteration in [abstract], “…This is usually actualized through feedforward multilayer neural networks, e.g.ConvNets, where each layer forms one of such successive
representations. However, an alternative that can achieve the same goal is a feedback based approach in which the representation is formed in an iterative manner based on a feedback received from previous iteration’s output… We observe that feedback develops a considerably different representation compared to feedforward counterparts, in line with the aforementioned advantages. We provide a general feedback based learning architecture, instantiated using existing RNNs, with the endpoint results on par or better than existing feedforward networks and the addition of the above advantages.” + Introduction section: “Feedback is defined to occur when the (full or partial)
output of a system is routed back into the input as part of an iterative cause-and-effect process”
As both are in the same field of endeavor, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Popov’s device with the feedback data limitations disclosed in Zamir with reasonable expectation of success. The motivation for doing so would have been allow for earlier object classification/prediction, episodic learning, and taxonomical compliance classification of image into an object group/class thereby allowing for more accurate and detailed classification compared to feed-forward only , see Zamir [conclusion].
Regarding Claim 2, Popov discloses the following limitation dependent on Claim 1:
further comprising: providing a task-specific analysis of the surrounding of the vehicle based on the determined output of the at least one task-specific module and/or the at least one historical output of the pre-processing module. See [0093], “LIDAR point cloud may be orthographically projected to form a LIDAR projection image (e.g., an overhead image) corresponding to the RADAR projection image contained in the RADAR data tensor … The LIDAR projection image may be annotated … with labels identifying the locations, sizes, orientations, and/or classes of the instances of the relevant objects in the LIDAR projection image … LIDAR labels may be synthetically produced (e.g., generated from computer models or renderings), real produced (e.g., designed and produced from real-world data), machine-automated (e.g., using feature analysis and learning to extract features from data and then generate labels) … FIG. 9B is an illustration of an example LIDAR frame with white ground truth bounding boxes.”
Regarding Claim 3, Popov discloses the following limitation dependent on Claim 1:
wherein: the feedback data further comprises historical sensor data from the at least one iteration prior to the current iteration and/or historical processed input data from the at least one iteration prior to the current iteration, and the extraction is further performed based on the historical sensor data and/or the historical processed input data. See [0061], “world model manager 126 may continually update the world model based on newly generated and/or received inputs (e.g., data) from the obstacle perceiver, the path perceiver, the wait perceiver, the map perceiver, and/or other components of the autonomous vehicle control system”. Also [0201-0206] for driver feedback systems.
Popov discloses a method for determining the surrounds of a vehicle, but does not explicitly disclose using feedback data from a previous iteration compared to the current interation. However, Zamir teaches an object detection model using feedback networks in which the feedback data is from the previous iteration in [abstract], “an alternative that can achieve the same goal is a feedback based approach in which the representation is formed in an iterative manner based on a feedback received from previous iteration’s output.”
As both are in the same field of endeavor, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Popov’s device with the feedback data limitations disclosed in Zamir with reasonable expectation of success. The motivation for doing so would have been to develop a coarse-to-fine representation that is meaningfully and considerably different from feedforward representations, see Zamir [conclusion].
Regarding Claim 5, Popov discloses the following limitation dependent on Claim 1:
wherein: at least two sensors are provided, and the at least two sensors are at least two different types of sensors. See [0092], “a scene may be observed with RADAR and LIDAR sensors (e.g., RADAR sensor(s) 1360 and LIDAR sensor(s) 1364 of autonomous vehicle 1300 of FIGS. 13A-13D) to collect a frame of RADAR data and LIDAR data for a particular time slice.”
Regarding Claim 6, Popov discloses the following limitation dependent on Claim 1:
wherein the at least one task-specific module is configured for a detection and/or classification task. See [0057], “machine learning model(s) 108 may accept sensor data such as a projection image (e.g., of accumulated, ego-motion compensated, and orthographically projected RADAR detections) and predict classification data and/or object instance data, which may be post-processed to generate bounding boxes, closed polylines, or other bounding shapes identifying the locations, sizes, and/or orientations of detected object instances in the projection image. FIG. 5A is an illustration of an example orthographic projection of accumulated RADAR detections and corresponding object detections (i.e., the white bounding boxes, in this example) in accordance with some embodiments of the present disclosure. For visualization purposes, FIG. 5B is an illustration of the object detections projected into corresponding images from three cameras.”
Regarding Claim 7, Popov discloses the following limitation dependent on Claim 1:
further comprising: initiating a visual or audible notification in the vehicle based on the determined respective output; or initiating a controlling of the vehicle based on the determined respective output. See [0035], “the training data may be used to train the DNN to detect moving and stationary obstacles and other objects from RADAR data, and the object detections may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.” And [0121], “One or more of the controller(s) 1336 may receive inputs (e.g., represented by input data) from an instrument cluster 1332 of the vehicle 1300 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 1334, an audible annunciator, a loudspeaker, and/or via other components of the vehicle 1300 … the HMI display 1334 may display information about the presence of one or more objects (e.g., a street sign, caution sign, traffic light changing, etc.), and/or information about driving maneuvers the vehicle has made, is making, or will make (e.g., changing lanes now, taking exit 34B in two miles, etc.). “
Regarding Claim 8, Popov discloses the following limitation dependent on Claim 1:
wherein the pre-processing module is configured as a convolutional neural network, a transformer or point-processing network, or a combination of these types of networks. See Fig.3 and [0046-0048] where they disclose, “machine learning model(s) 108 may be implemented using a DNN, such as a convolutional neural network (CNN). Although certain embodiments are described with machine learning model(s) 108 being implemented using neural network(s), and specifically CNN(s), this is not intended to be limiting. For example, and without limitation, machine learning model(s) 108 may include any type of machine learning model.” And [0048], “Feature extractor trunk 310 may be implemented using encoder and decoder components with skip connections (e.g., similar to a Feature Pyramid Network, U-Net, etc.). For example, feature extractor trunk 310 may accept input data such as RADAR data tensor 106 and apply various convolutions, pooling, and/or other types of operations to extract features into some latent space. In FIG. 3, feature extractor trunk 310 is illustrated with an example implementation involving an encoder/decoder with an encoding (contracting) path down the left side and an example decoding (expansive) path up the right.”
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Popov in view of Zamir, in further view of Saha et al., “Adaptive Control of Camera Modality” NPL (Hereinafter, “Saha”).
Regarding Claim 4, Popov and Zamir teach a method for determining the surrounds of a vehicle using feedback data, but does not explicitly disclose using a physical/learned model to utilize feedback data. However, Saha teaches following: wherein the provision of the feedback data further comprises: transforming the historical output using a physical model, wherein the physical model describes at least one movement of the vehicle and/or of at least one object detected by the at least one task-specific module. See page 3, figure 3/the ROI predictor based on the detected objects and their movement in a detected frame/iteration this is used to predict the corresponding expected bounding box (region of interest) location for the next iteration; SECTION 3.2. RoI predictor “For each object detected in current frame, its position in the next frame is predicted using current observed position and predicted position from previous frame. Assuming constant linear velocity, state of each object is modeled using four coordinates of the bounding box and their derivatives(velocities).”
As both are in the same field of endeavor, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Popov and Zamir’s device with the physical/learned model limitations disclosed in Saha with reasonable expectation of success. The motivation for doing so would have been for data driven feedback control of spatial modality for efficient object detection and tracking for resource constrained edge devices, see Saha [Conclusions].
Additional Relevant Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and may be found on the accompanying PTO-892 Notice of References Cited:
US Publication US 20180300964 A1 by LAKSHAMANAN et al.
US Publication US20250171051A1 by Li et al.
US Publication US20250242836A1 by Weng et al.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN KEITH PALMARCHUK whose telephone number is (571)272-6261. The examiner can normally be reached M-F 7 AM - 5 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, NAVID MEHDIZADEH can be reached at 571-272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.K.P./Examiner, Art Unit 3669
/KENNETH M DUNNE/Primary Examiner, Art Unit 3669
Prosecution Insights