DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the application filed on 4/4/2025.
No claims have been amended.
No claims have been added.
No claims have been cancelled.
Claims 1-9 are currently pending and have been examined.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement(s) (IDS(s)) submitted on 4/4/2025 and 6/14/2025 have been received and considered.
Drawings
The drawings are objected to under 37 CFR 1.83(a) because Figure 13 fails to sufficiently show the details of a flowchart as described in the specification. Any structural detail that is essential for a proper understanding of the disclosed invention should be shown in the drawing. The drawing as presented only shows blocks and numbers, requiring the specification for interpretation of the numbers, and does not on its own provide details of the intended method with any degree of clarity. Therefore, the drawing is objected to. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 5 is objected to because of the following informalities: "[…] an encoding for each of the other road users, of the movement of the other road user and encodings of traffic lane nodes […]" uses inconsistent language in listing “an encoding” for “each of the other road user” and “traffic lane nodes” but not for “movement of the other road user.” Appropriate correction is required. For the purposes of examination, these elements will be interpreted as separate, distinct encodings.
Claims 7, 8, and 9 respectively recite “a vehicle control device,” “A computer program,” and “a computer-readable medium” and introduce new embodiments; therefore, they are independent claims. However, language such as “to carry out a method according to claim 1” is indicative of dependent-type claims in the new “vehicle control device,” “computer program,” and “computer-readable medium” embodiments. Since claim 1 explicitly recites “A method” embodiment, it is considered a separate and distinct embodiment from the “vehicle control device,” “computer program,” and “computer-readable medium.”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitation "the quantitative characteristics," emphasis added, in line 2. There is insufficient antecedent basis for this limitation in the claim, as no quantitative characteristics have been previously claimed.
Claim Rejections - 35 USC § 101
Claim(s) 8 & 9 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter. The claims are directed to a computer readable medium. As explained in U.S. Patent & Trademark Office, Subject Matter Eligibility of Computer-Readable Media, 1351 Off. Gaz. Pat. Office 212 (Feb. 23, 2010):
The United States Patent and Trademark Office (USPTO) is obliged to give claims their broadest reasonable interpretation consistent with the specification during proceedings before the USPTO. See In re Zietz, 893 F.2d 319 (Fed. Cir. 1989) (during patent examination the pending claims must be interpreted as broadly as their terms reasonably allow). The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter) and Interim Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C. § 101, Aug. 24, 2009; p. 2.
The USPTO recognizes that applicants may have claims directed to computer readable media that cover signals per se, which the USPTO must reject under 35 U.S.C. § 101 as covering both non-statutory subject matter and statutory subject matter. In an effort to assist the patent community in overcoming a rejection or potential rejection under 35 U.S.C. § 101 in this situation, the USPTO suggests the following approach. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claim. Cf Animals - Patentability, 1077 Off Gaz. Pat. Office 24 (April 21, 1987) (suggesting that applicants add the limitation "non-human" to a claim covering a multi¬ cellular organism to avoid a rejection under 35 U.S.C. § 101). Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se. The limited situations in which such an amendment could raise issues of new matter occur, for example, when the specification does not support a non-transitory embodiment because a signal per se is the only viable embodiment such that the amended claim is impermissibly broadened beyond the supporting disclosure. See, e.g., Gentry Gallery, Inc. v. Berkline Corp., 134 F.3d 1473 (Fed. Cir. 1998).
Accordingly, claim(s) 8, being explicitly directed toward a computer program, is rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter.
Accordingly, claim(s) 9, is rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter because the computer-readable medium is not in a non-transitory form.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3 and 5-9 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Joshi et al (US 20250128734, hereinafter “Joshi,”).
Regarding Claim 1, Joshi teaches:
A method for predicting trajectories of road users, comprising: representing a traffic scene as an agent interaction graph, (Joshi ¶ 0016 lines 25-27 “the backbone machine learned model may be trained in a self-supervised manner to operate on a driving scene constructed as a graph containing nodes and edges and may output a representation (for example, in the form of an embedding) for a node associated with a particular object or feature within the environment.,”)
each having a node for a road user corresponding to a target vehicle and for one or more other road users (Joshi ¶ 0026 lines 1-8 “A graphical representation (graph) 120 of the environment may be generated based at least in part on the data 110. The graph 120 may comprise a plurality of nodes 121. Each node of the plurality 121 may be associated with one or more of a vehicle operating in the environment (for example, first vehicle 102), […], an additional vehicle (for example, second vehicle 104),”)
and having a plurality of edges, wherein each edge between two of the nodes is associated with a respective edge type, which indicates a type of movement of the road users represented by the nodes relative to each other on a respective roadway; (Joshi ¶ 0065 lines 1-9 “The graph 120 may also comprise a plurality of edges 122, where there is at least one edge between a first node and another node of the plurality. An edge may be representative of a relationship between the features associated with the nodes that the edge connects, such that the graph 120 is constructed within semantic space and defines features of the environment with respect to one another based on their meaning (relevance) to one another. For instance, an edge may be directional,” teaching the edges representing relationships between nodes, and teaching distinguishing directionality (i.e., edge types) of relationships)
processing the agent interaction graph by a graph transformer (Joshi ¶ 0019 “The backbone machine learned model may be a transformer-based model which enables predictable scaling using simple architecture, allowing for a tradeoff between latency, memory usage, and performance,”)
to determine embeddings of the target vehicle and the one or more other road users, (Joshi ¶ 0031 lines 1-4 “The SSL model 130 may be a neural network, for example, a neural network configured to comprise a transformer architecture, trained to learn embeddings related to particular features of an environment,”)
wherein the graph transformer has an attention mechanism (Joshi ¶ 0031 lines 4-8 “The transformer architecture of the SSL model 130 may comprise an encoder 132. The encoder 132 may comprise a plurality of attention layers 133 (also referred to as a self-attention mechanism) and a plural,”)
which takes into account the edge types of the edges of the agent interaction graph; (Joshi ¶ 0066 lines 1-6 “The flow of data and the edge directionality representative thereof may be representative of how the attention layers 133 may process the graph 120 and the data relating to each node, for instance, the directionality may affect the determination of the structure of one or more attention matrices using in attention operations by the attention layers 133,” teaching the attention layers taking into account the directionality of the edges)
and predicting at least one trajectory of the target vehicle from the embeddings. (Joshi ¶ 0036 lines 1-3 “In the example of FIG. 1, the plurality of embeddings 140 is transmitted to a downstream machine learned model 150 associated with the first vehicle 102,” and ¶ 0039 lines 1-7 “The downstream model 150 may be a part of […]a prediction vehicle system (configured to generate predicted trajectories of objects in an environment),”)
Regarding Claim 2, Joshi teaches the elements of Claim 1 as described above and further teaches:
wherein the attention mechanism takes into account the edge types of the edges of the agent interaction graph by having a respective set of attention mechanism parameters for each edge type, (Joshi ¶ 0066 lines 7-22 “For example, an N×N attention matrix may have a mask applied that down-weights certain parts of data (e.g. nodes associated with dynamic features) with respect to other parts of data (e.g. nodes associated with static features), to reflect the directionality of the edges (in this case, that information does not flow from dynamic features to static features). This means nodes that are associated with dynamic features may be attended based on data of nodes associated with static features (e.g., features defining the road network (lane width, curvature and gradient, etc.), but not vice versa. In examples, the attention matrix may have a mask that down-weights data relating to nodes that are not deemed to be adjacent to a given node, since nodes that are further away may have less of an effect on a given node's behavior, where adjacency may be defined based on position or another attribute, relative to the given node,” teaching that the attention matrix accounts for directionality as well as example parameters regarding features relating to node relationships)
wherein the sets of attention mechanism parameters are individually trainable. (Joshi ¶ 0032 lines 1-6 “The attention layers 133 of the SSL model 130 may receive the graph 120 as input. The attention layers 133 may be trained to weight the importance of the different nodes in the graph and determine the relationship between and relevance thereof of a feature of the environment associated with a given node,” and ¶ 0048 “During a training phase, training data for the SSL model 130 may comprise the graph 120 (generated as described in relation to FIG. 2). A portion of the training data corresponding to a node 121 of the graph 120 may be masked, so that the training data comprises a masked portion 125 of training data and an unmasked portion of training data. The masking operation may be carried out by setting a portion of training data to a constant. Different ways of masking data are described in more detail in relation to FIG. 3. Masking one or more nodes is a way of training the SSL model 130 to reconstruct the scene of the environment by predicting what was missing from the input training data (that is, the masked portion) using the unmasked portion of training data, since the unmasked portion may comprise data that suggests there may be another feature of the environment (for example, an additional vehicle) that was not present in the input data,” describing a masking process to train specific, targeted portions of data enabling and individual training of parameters, analogously teaching individual training)
Regarding Claim 3, Joshi teaches the elements of Claim 1 as described above and further teaches:
wherein each of the edges has one or more edge attribute values indicating the quantitative characteristics of the movement of the road users represented by the nodes relative to each other, (Joshi ¶ 0059 “The graph 120 may be representative of data relating to the environment 100 over the series of time steps T.sub.0 to T.sub.2. The data relating to the environment may comprise a number of features, also referred to herein as entities, within the environment, such as static and dynamic features. Examples include a vehicle, a pedestrian, and a road feature, where the road feature comprises at least one of a speed limit, a curvature of the road, a gradient of the road and a junction or intersection with another road. A static feature may be fixed over time, such as the curvature, width, speed limit of a road, whereas a dynamic feature may have some properties that change (or have potential to change) over time, such as vehicle or pedestrian, which may change direction, speed, and acceleration,” teaching several quantitative characteristics such as speed, curvature, speed limits, etc. which are included in the data represented in the graph, representing the features which are defined in the graph by nodes, and ¶ 0065 lines 1-9 “The graph 120 may also comprise a plurality of edges 122, where there is at least one edge between a first node and another node of the plurality. An edge may be representative of a relationship between the features associated with the nodes that the edge connects, such that the graph 120 is constructed within semantic space and defines features of the environment with respect to one another based on their meaning (relevance) to one another,” teaching that such quantitative data as mentioned previously is represented in the edges as defining the relative relationship between the nodes, which can include dynamic nodes relative to one another)
and which the attention mechanism takes into account. (Joshi ¶ 0066 lines 1-6 “The flow of data and the edge directionality representative thereof may be representative of how the attention layers 133 may process the graph 120 and the data relating to each node, for instance, the directionality may affect the determination of the structure of one or more attention matrices using in attention operations by the attention layers 133,” teaching the attention layers taking into account the directionality of the edges)
Regarding Claim 5, Joshi teaches the elements of Claim 1 as described above and further teaches:
wherein the trajectories are further determined from at least one of an encoding (Joshi ¶ 0095 lines 3-9 “As described in relation to FIGS. 1-5, the SSL model 130 may be configured to output a plurality of embeddings based at least in part on an input comprising a graph 120 representing an environment 100 through which a vehicle is traversing, where a node of the graph 120 comprising, as an embedding, a vector encoding,” teaching node vector encodings within the node embeddings applying to the remainder of examples in this claim, and ¶ 0036 lines 1-3 describing trajectory prediction as previously established)
of the movement of the target vehicle, (Joshi ¶ 0060 lines 3-5 “In the example of FIG. 2, the first vehicle 102 is associated with node 121a (colored black),” teaching a node for a target vehicle, and ¶ 0059 lines 11-14 “a dynamic feature may have some properties that change (or have potential to change) over time, such as vehicle or pedestrian, which may change direction, speed, and acceleration,” teaching node properties relating to movement of the vehicle)
an encoding for each of the other road users, (Joshi ¶ 0060 lines 5-6 “the second vehicle 104 is associated with the node 121b (colored black),” teaching a specific example of a node for another road user, in addition to the more general case previously established in ¶ 0026 lines 1-8)
of the movement of the other road user (Joshi ¶ 0059 lines 11-14 as previously established)
and encodings of traffic lane nodes (Joshi ¶ 0063 “node 121e is associated with the second lane 101B,” teaching a specific example of a traffic lane node)
of one or more graphs representing one or more traffic lanes of the traffic scene. (Joshi ¶ 0060-0063 “As described in relation to FIG. 1, the graph 120 may comprise a plurality of nodes 121, each of which may be associated with an entity. […] node 121e is associated with the second lane 101B,”)
Regarding Claim 6, Joshi teaches the elements of Claim 1 as described above and further teaches:
further comprising controlling a vehicle, taking into account the at least one predicted trajectory. (Joshi ¶ 0143 “The planning component 922 may receive a location and/or orientation of the vehicle 902 from the localization component 920 and/or perception data from the perception component 918 and may determine instructions for controlling operation of the vehicle 902 based at least in part on any of this data. The planning component 922 may correspond to the downstream ML model 150 described in relation to FIG. 1. In some examples, determining the instructions may comprise determining the instructions based at least in part on a format associated with a system with which the instructions are associated (e.g., first instructions for controlling motion of the autonomous vehicle may be formatted in a first format of messages and/or signals (e.g., analog, digital, pneumatic, kinematic) that the system controller(s) 930 and/or drive component(s) 912 may parse/cause to be carried out, second instructions for the emitter(s) 908 may be formatted according to a second format associated therewith),” emphasis added, describing controlling a vehicle by determining instructions based on the downstream model previously established by ¶ 0036 lines 1-3 to predict trajectories)
Regarding Claim 7, Joshi teaches the elements of Claim 1 as described above and further teaches:
A vehicle control device configured to carry out a method according to claim 1. (Joshi ¶ 0143 lines 1-6 “The planning component 922 may receive a location and/or orientation of the vehicle 902 from the localization component 920 and/or perception data from the perception component 918 and may determine instructions for controlling operation of the vehicle 902 based at least in part on any of this data.”)
Regarding Claim 8, Joshi teaches the elements of Claim 1 as described above and further teaches:
A computer program with instructions that, when executed by a processor, cause the processor to carry out a method according to claim 1. (Joshi ¶ 0198 “At least some of the processes discussed herein are illustrated as logical flow charts, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, cause a computer or autonomous vehicle to perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.”)
Regarding Claim 9, Joshi teaches the elements of Claim 1 as described above and further teaches:
A computer-readable medium that stores instructions that, when executed by a processor, cause the processor to carry out a method according to claim 1. (Joshi ¶ 0198 “At least some of the processes discussed herein are illustrated as logical flow charts, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, cause a computer or autonomous vehicle to perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Joshi in view of Iglesias et al (US 20230047336, hereinafter “Iglesias”).
Regarding Claim 4, Joshi teaches the elements of claim 1 as described above. Joshi does not teach:
wherein the type of movement is one of side-by-side, back-to-back, and intersecting.
Within the same field of endeavor as Joshi, Iglesias teaches:
wherein the type of movement is one of side-by-side, back-to-back, and intersecting. (Iglesias ¶ 0023 lines 3-9 “Depending on the context of the autonomous vehicle's driving environment, including the relative locations and direction of movement of the object and the autonomous vehicle (e.g. alongside, oncoming, crossing, etc.) as well as the type of the object (e.g. vehicle, bicyclist, pedestrian), the time gaps may be applied differently,” teaching categorizations of relative motion between the autonomous vehicle and other objects including vehicles of different types including alongside (analogous to side-by-side), oncoming (analogous to back-to-back), and crossing (analogous to intersecting), that are taken into account by the avoidance system in different ways analogous to different treatment )
Joshi and Iglesias are considered analogous because they both relate to trajectory predictions for scenes around autonomous vehicles. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the graph edges with data indicating relative motion between vehicles of Joshi with the simple addition of the context-dependent treatment of nearby object motion dependent on relative location and direction of movement of the object including alongside, oncoming, and crossing of Iglesias. This modification would be made with a reasonable expectation of success as motivated by increasing the comfort of passengers and nearby road users as well as increasing safety by contextually treating different types of movement (Iglesias ¶ 0021) according to MPEP 2143(I)(G). Furthermore, this modification would be motivated by the application of a known technique (Iglesias’s context-dependent treatment of different types of movement including alongside, oncoming, and crossing, i.e. categorization of movement types) to a known device (Joshi’s trajectory predictor using a graph method with edges including relative motion data between agent nodes) ready for improvement (Iglesias’s movement types with different treatments would improve Joshi’s machine per Iglesias ¶ 0021 above) to yield results that would be predictable by one of ordinary skill in the art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
WO 2024148057 describes a trajectory prediction system with a similar attention mechanism to the one claimed.
US 20250128731 and US 20240199083 describe trajectory prediction systems with a similar node-and-edge graph, transformer, and embedding architectures.
US 20210370990 describes a trajectory prediction system with a similar node-and-edge and embedding architecture.
US 12497079 describes a trajectory prediction system with similar agent embedding and encoding architecture.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZACHARY E GLADE whose telephone number is (703)756-1502. The examiner can normally be reached 4-5-9 7:30-16:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at (571) 270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZACHARY E. F. GLADE/Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights