Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/04/2025 is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea accomplishable by mental processes without significantly more. The claims recite the abstract idea of gathering, organizing, analyzing, and displaying data, which is analogous to mental work with the aid of generic computer equipment. This judicial exception is not integrated into a practical application and the claim(s) do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 1: Is the claim directed to a process, machine, manufacture, or composition of matter?
Yes, The claims are directed to a method.
Step 2A; is the claim directed to a law of nature, a natural phenomenon, or an abstract idea?
Yes, claims 1-24 are directed to the abstract idea of gathering, organizing, analyzing and displaying data.
In essence, the independent claims recite; receiving sensor data, organizing which data to analyze it based on trigger thresholds, displaying the selected data for analysis, and further receiving a selection/analysis from a user.
Prong One; Is the claim directed to a law of nature, a natural phenomenon, or an abstract idea?
Yes, as understood in their broadest reasonable interpretation, the independent claims are directed to receiving sensor data, organizing which data to analyze it based on trigger thresholds, displaying the selected data for analysis, and further receiving a selection/analysis from a user. In addition, the remaining claim limitations either work to develop the abstract idea further [such as how to apply a trigger threshold to organize data], or to implement the idea onto generic computer components [such as generic sensors, computer models].
Prong Two; Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the elements are generically recited. In particular, elements of a vehicle controller and sensors merely use generically recited features as tools to perform the abstract idea.
A plurality of sensors merely serve as the source of data, and are generically recited such that they merely amount to insignificant extra solution activity.
Graphical user interface merely serves as a display to assist a user in analyzing data, which amounts to insignificant extra solution activity that does not improve computer functionality.
The recited machine learning model is recited generically, such that it amounts to a using a conventional, routine machine learning model well known in the art.
Therefore, this abstract idea is not integrated into a practical application because there are no meaningful limits on practicing the abstract idea. Therefore, Claims 1-24 are directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Claims 1-24 do not include additional elements that amount to significantly more than the judicial exception.
In Step 2B of the 2019 PEG, a claim is to be evaluated as to whether the claim, as a whole, amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B. Here, the receiving steps and the displaying step were considered to be insignificant extra-solution activity in Step 2A, and thus they are re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The description does not recite any information that would indicate that the sensors are anything but conventional sensors mounted on the vehicle. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). Further, the Federal Circuit in Trading Techs. Int’l v. IBG LLC, 921 F.3d 1084, 1093 (Fed. Cir. 2019), and Intellectual Ventures I LLC v. Erie Indemnity Co., 850 F.3d 1315, 1331 (Fed. Cir. 2017), for example, indicated that the mere displaying of data is a well understood, routine, and conventional function. Accordingly, a conclusion that the collecting step is well-understood, routine, conventional activity is supported under Berkheimer.
Thus, claims 1-24 are ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-14 and 16-24 are rejected under 35 U.S.C. 103 as being unpatentable over Palmer (US20150105934A1) in view of Karpathy (US20210271259A1).
Regarding claim 1, Palmer US20150105934A1 teaches;
A method for labeling events of interest in vehicular sensor data, the method comprising:
accessing sensor data captured by a plurality of sensors disposed at a vehicle (taught as sensors generating output signals conveying information related to the operation and context of the vehicle, paragraph 0028, which are transmitted to a vehicle event playback apparatus, paragraph 0029);
providing a trigger condition comprising a [[plurality of]] threshold values (taught as a trigger, which includes one or more logical criteria determined to have been met, to determine a specific vehicle event to be recorded based on the information conveyed by the output signals and the obtained pre-determined vehicle event profiles, paragraph 0029, an example of a vehicle event includes a condition where vehicle speed exceeds a threshold, paragraph 0025, or with an accelerometer trigger exceeding .5 g, paragraph 0038), wherein the trigger condition is satisfied when values derived from the sensor data satisfy each of the [[plurality of ]] threshold values (taught as detecting vehicle events [trigger signals] with more complex triggers relying on, for example, accelerometer threshold and timing thresholds, exemplified in paragraph 0038);
identifying, via processing the sensor data, an event of interest when the trigger condition is satisfied (taught as triggering the vehicle events to be recorded [i.e. event suspected/confirmed for recording] based on the information of the sensors, paragraph 0029);
displaying, on a graphical user interface, a visual indication of the event of interest (taught as vehicle event playback system, presented to interact on a graphical user interface, paragraph 0049, e.g. Fig 2, for example, timeline pip markers, paragraph 0052);
displaying, on the graphical user interface, visual elements derived from a portion of the sensor data representative of the event of interest (taught as derived information from sensor data, such as acceleration data, paragraph 0080, see Fig 7);
receiving, from a user of the graphical user interface, a label for the event of interest (taught as an operator/reviewer of vehicle event data interacting with the presented vehicle event data playback systems, paragraph 0059, and provide notation to the event, paragraph 0101); and
storing, at a database, the label and the portion of the sensor data representative of the event of interest (taught as appending the notations to the event record, paragraph 0101).
However, Palmer does not explicitly teach; providing a trigger condition comprising a plurality of threshold values (emphasis added).
Karpathy teaches; providing a trigger condition comprising a plurality of threshold values, wherein the trigger condition is satisfied when values derived from the sensor data satisfy each of the [[plurality of ]] threshold values (taught as trigger threshold(s) and required conditions for the trigger classifier, paragraph 0053; and further including trigger filters used to restrict the retention of sensor data to described conditions, paragraph 0069; indicating that multiple thresholds are used to identify a trigger relating to identifying sensor data for use cases).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize multiple threshold conditions as a trigger as suggested in Karpathy in the system taught by Palmer in order to improve event/use case identification. The use of multiple threshold triggers/filters allow for further nuanced conditions to restrict the retention of sensor data, as suggested in Karpathy (paragraph 0069), such that only relevant data is saved.
Regarding claim 2, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein the plurality of sensors comprises at least one [interpreted to indicate only one of the sensors is required] selected from the group consisting of (i) a camera (taught as sensors including a video camera, paragraph 0028), (ii) a radar sensor (taught as sensors including a radar detector, paragraph 0028), (iii) a lidar sensor, and (iv) a GPS sensor (taught as sensors including a GPS device, paragraph 0028).
Regarding claim 3, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein the plurality of sensors comprises a plurality of cameras (indicated in a plurality of video cameras, capturing different views of the vehicle, paragraph 0057).
Regarding claim 4, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein accessing sensor data captured by the plurality of sensors disposed at the vehicle comprises recording the sensor data using the plurality of sensors while the plurality of sensors are disposed at the vehicle (taught as an active safety system installed in the vehicle including sensors, paragraph 0026, listed examples in 0028, which communicate information to the vehicle event playback apparatus via the vehicle event recorder, paragraph 0029).
Regarding claim 5, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein the visual indication of the event of interest comprises a visual indication of a particular point in time captured by the sensor data (taught as pip markers, which are associated with an instant in time/time period associated/subset of the event period, paragraph 0052).
Regarding claim 6, Palmer as modified by Karpathy teaches;
The method of claim 5 (see claim 5 rejection). Palmer further teaches; wherein the visual indication of the particular point in time comprises a marker on a timeline, and wherein the timeline represents a length of a recording of sensor data (taught as pip markers, which are associated with an instant in time/time period associated/subset of the event period, paragraph 0052).
Regarding claim 7, Palmer as modified by Karpathy teaches;
The method of claim 5 (see claim 5 rejection). However, Palmer does not explicitly teach; further comprising training a model using the label and the portion of the sensor data representative of the particular point in time.
Karpathy teaches; further comprising training a model using the label and the portion of the sensor data representative of the particular point in time (taught as a human revieing data, annotating and labeling it, for use for training or verification of a trained machine learning model, paragraph 0083, wherein a machine learning model is trained on the prepared data, paragraph 0085).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train a machine learning model based on the labeled data as taught by Karpathy in the system taught by Palmer in order to better apply the labeled data into active autonomous vehicle control. As taught by Karpathy, the performance of a deep learning system is limited in part by the quality of the training set (paragraph 0004). Thus, extending the curated and manually labeled event data taught in Palmer into a training dataset would constitute as establishing a ground-truth correspondence to enable better detection/prediction of vehicle events.
Regarding claim 8, Palmer as modified by Karpathy teaches;
The method of claim 7 (see claim 7 rejection). However, Palmer does not explicitly teach; wherein the model comprises a machine learning model.
Karpathy teaches; wherein the model comprises a machine learning model (taught as a human revieing data, annotating and labeling it, for use for training or verification of a trained machine learning model, paragraph 0083, wherein a machine learning model is trained on the prepared data, paragraph 0085).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train a machine learning model based on the labeled data as taught by Karpathy in the system taught by Palmer in order to better apply the labeled data into active autonomous vehicle control. As taught by Karpathy, the performance of a deep learning system is limited in part by the quality of the training set (paragraph 0004). Thus, extending the curated and manually labeled event data taught in Palmer into a training dataset would constitute as establishing a ground-truth correspondence to enable better detection/prediction of vehicle events.
Regarding claim 9, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). However, Palmer doesn’t explicitly teach; wherein the label comprises a ground truth associated with the portion of the sensor data representative of a particular point in time captured by the sensor data.
Karpathy teaches; wherein the label comprises a ground truth associated with the portion of the sensor data representative of a particular point in time captured by the sensor data (taught as a human reviewer confirming whether the sensor data represent the targeted use case, paraph 0083).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to establish that the human reviewer labels the event data with ground truth [confirmation] as taught by Karpathy int eh system taught by Palmer to more explicitly retain and use the data for further use, such as training. As taught by Karpathy, annotations made by human reviewers can be used for training/verification of a machine learning model (paragraph 0083); and validation of training data is obviously essential to ensuring quality training data sets.
Regarding claim 10, palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein one of the plurality of threshold values represents a number of objects [based on paragraph 0015, this indicates that a number of objects is greater than 0 to trigger a trigger or rule, paragraph 0015] detected based on the sensor data (indicated in that an object is detected, such as a pedestrian, paragraph 0025).
Regarding claim 11, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein the event of interest comprises a hazardous [hazardous is undefined in the specification, and thus the interpretation reasonably includes any object that requires a safety measure] object in a path of the vehicle (taught as vehicle events including a pedestrian detection/collision warning, paragraph 0025).
Regarding claim 12, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein displaying the visual elements derived from the portion of the sensor data representative of the event of interest comprises displaying a frame of image data captured by a camera (taught as displaying a video playback view, shown in Figs 1a-g).
Regarding claim 13, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein displaying the visual elements derived from the portion of the sensor data representative of the event of interest comprises displaying a waveform representative of the values derived from the sensor data (exemplified in Fig 7, showing an acceleration data as a graph, paragraph 0080).
Regarding claim 14, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). Palmer further teaches; wherein the method further comprises:
receiving, from the user of the graphical user interface, an interaction indication indicating a user interaction with the graphical user interface (taught as the graphical user interface being interactive and responsive to user inputs, such as point and click, paragraph 0043);
responsive to receiving the interaction indication, displaying, on the graphical user interface, a second visual indication of a second point in time of a second event of interest relative to a length of a recording of sensor data (indicated in that the timeline control may automatically update the visual state to represent data captured at the moment of the event period, with various timeline replay indicators including click-and-drag, touch, play, rewind etc. paragraph 0053, which all interact with and move the timeline from a point in time to another associated with the event); and
displaying, on the graphical user interface, second visual elements derived from a second portion of the sensor data representative of the second point in time (indicated in that the timeline control may automatically update the visual state to represent data captured at the moment of the event period, with various timeline replay indicators including click-and-drag, touch, play, rewind etc. paragraph 0053).
Regarding claims 16-24, it has been determined that no further limitations exist apart from those previously addressed in claims 1-4, 6, 8, and 12-13. Therefore, claims 16-24 are rejected under the same rationale as claims 1-4, 6, 8, and 12-13, wherein;
Claims 16 and 21 correspond to claim 1,
Claim 17 corresponds to claim 8,
Claims 18-20 corresponds to claim 2-4
Claim 22 corresponds to claim 6,
And claims 23-24 correspond to claims 12-13.
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over Palmer (US20150105934A1) in view of Karpathy (US20210271259A1) and further in view of Lee (US12482303B2).
Regarding claim 15, Palmer as modified by Karpathy teaches;
The method of claim 1 (see claim 1 rejection). However, Palmer does not explicitly teach; wherein the event of interest represents an automatic emergency braking event by the vehicle.
Lee teaches; wherein the event of interest represents an automatic emergency braking event by the vehicle (taught as, triggering an event data recording upon emergency braking being activated, column 8 lines 32-35).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include event data recording for various autonomous functions like emergency braking as taught by Lee in the system taught by Palmer in order to help diagnose and train future models. Such a system allows for better identification of potential errors in the recognition-judgement-control process in a vehicle, as suggested by Lee (column 1 lines 38-47).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For further data labeling/selection pipeline for the creation of training data from sensors, US11769318B2
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL ANFINRUD whose telephone number is (571)270-3401. The examiner can normally be reached M-F 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571)270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GABRIEL ANFINRUD/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights