DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is in response to Applicant’s case, no. 18/748,781, with an effective filing date of 6/20/2024. Claims 1-3, 5-13, and 15 are currently pending. Claims 4 and 14 are canceled by the Applicant. Claim 7 has had allowable subject matter acknowledged to the Applicant in the previous action but is still objected to as it depends on a rejected base claim.
Response to Arguments
Examiner acknowledges that the necessary changes were made regarding the Drawings/Specification and Claim Objection sections in Applicant’s arguments, see page 16, and subsequently withdraws the previous objections to said sections. However, new objections have been made to the abstract and to claim 1 as detailed below.
Examiner acknowledges that the necessary changes were made regarding the 35 USC § 112(b) rejections to claims 12-15 in Applicant’s arguments, see page 17, and subsequently withdraws the 35 USC § 112(b) rejection to said claims.
Examiner acknowledges the changes made regarding 35 USC § 101 to claims 1, 12, and 15 found in Applicant’s arguments, see pp. 5. The Examiner has considered the amended claim limitation controlling the host vehicle based on the at least one pattern related to the environment of the host vehicle from the joined spatial-temporal data set and the amendments properly integrate the judicial exception into a practical application. Therefore, the rejection under 35 USC § 101 to claims 1-15 is withdrawn.
Regarding the 35 USC § 102(a)(1) rejection of claims 1 and 8-15 as being anticipated by Heck et al. (US Pat. Pub. No. 2023/0166743 A1) [hereinafter referred to as Heck], the Applicant has elected to amend the claims. Therefore, the Examiner’s rejection in the previous Office Action based on 35 USC § 102(a)(1) is rendered moot. However, due to said amendments, new reference Long et al., an article titled “Unified spatial-temporal neighbor attention network for dynamic traffic prediction” [hereinafter referred to as Long] has been necessitated. Therefore, a new rejection based on 35 USC § 103 has been made and is discussed in detail below.
In regards to independent claim 1, Applicant argues (see pages 17-18) that Heck does not disclose the limitations wherein the pattern is provided as an abstract feature map which is stored in a grid map, wherein the weights of the attention algorithm are applied to values generated by employing a union of the elements of the set of current input data and the assigned elements of the set of memory data in order to provide the joined spatial-temporal data set as an output of the attention algorithm, and wherein tasks implemented as machine learning algorithms are applied to the abstract feature map in order to derive information regarding a position and/or dynamics of a respective object in the environment of the host vehicle, and/or in order to perform a grid segmentation of the environment of the host vehicle. However, Long teaches on pg. 1516 left column ¶2 s. 6-8 to right column s.2, that the self-attention mechanism is leveraged to capture spatial-temporal correlations synchronously by assigning different correlation weights to spatial-temporal neighbors. Furthermore, a gated fusion module is proposed to fuse external factors with spatial-temporal features according to the different influences of different external factors. The gated fusion module first classifies external factors by types, then embeds them into the same space using neural network for fusion, and finally weighted fuses them with spatial-temporal features with the aid of a nonlinear function. By doing so, our proposed model can perceive the differences of influences caused by different external factors. Furthermore, see Table 2 in the 35 USC §103 rejection below which details a summary of datasets where grid maps are utilized to store and retrieve information. Therefore, this argument is moot
In regards to independent claims 12 and 15, Applicant argues, while differing in scope, these claims recite similar features to claim 1 and their rejections should likewise be withdrawn. However, this argument is unpersuasive for the same reasons as given above.
Applicant argues the dependent claims are patentable by virtue of their dependency. This argument is unpersuasive as each independent claim has been fully rejected for the reasons as given above.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
The abstract of the disclosure is objected to because: contains phraseology that may be implied (e.g., line 1 “is provided”). A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1 line 7 contains a typographical error where the characteristics should be corrected to the respective sets of characteristics.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1 and 8-13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Heck et al. (US Pat. Pub. No. 2023/0166743 A1), hereinafter referred to as Heck, and Long et al. article “Unified spatial-temporal neighbor attention network for dynamic traffic prediction”, hereinafter referred to as Long.
Regarding claim 1, Heck discloses:
A computer implemented method for determining patterns related to an environment of a host vehicle from sequentially recorded data ([0565] second to last sentence, a processor is configured to identify patterns of a circumstance such as environment and these images are time-series data which is inherently sequential), the method comprising:
determining respective sets of characteristics (Fig. 14 below determining characteristics detected by perception system and [0006] a first sensor configured to provide a first input associated with an environment outside a vehicle) detected by a perception system of the host vehicle in the environment of the host vehicle for a current point in time ([0003] Sensors (e.g., cameras, radars, lidars, etc.) have been used in vehicles to capture images of road conditions outside the vehicles which is construed by the Examiner as a perception system to detect the environment of the host vehicle for a current point in time) and for a predefined number of previous points in time ([0360 sentence (s.) 2 where a predefined number of previous time points are discloses (e.g., 5 sec. prior or 10 sec. prior, etc.)), wherein the characteristics include a current position, a current velocity and an object class for a plurality of objects ([0032] time series data that may pertain to distance to lead vehicle, speed of the vehicle, information regarding identified object, object position, object moving direction of, object speed), and
PNG
media_image1.png
283
430
media_image1.png
Greyscale
via a processing unit of the host vehicle ([0246] hardware of the processing unit may include one or more processors and/or more or more integrated circuits and [0247] non-transitory medium is configured to store data relating to operation of the processing unit):
generating a set of current input data associated with the set of characteristics for the current point in time (see [0006] and Fig. 14 as disclosed above),
generating a set of memory data by aggregating the sets of characteristics for the previous points in time ([0557] processor system 1600 also includes a main memory, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus for storing information and instructions to be executed by the processor),
applying an attention algorithm to the set of current input data and to the set of memory data in order to generate a joined spatial-temporal data set (see Fig. 14 above and [0159-161] where objects and regions of interest are determined and those regions have a variable geometry corresponding to the road, [0298] where algorithms are utilized as a processing technique to achieve goals of the processing system, and [0500] s.2, algorithm that processes the time-to-collision risk signals),
wherein the attention algorithm includes weights which are defined by relating elements of the set of current input data and assigned elements of the set of memory data ([0019 and 0022] the processing unit is configured to calculate the risk score by applying a weights to the probabilities to obtain a weighted probabilities), and
determining at least one pattern for the environment of the host vehicle from the joined spatial-temporal data set (see Fig 14 above and [0532] s.1-2, inputs may form certain patterns and the model can be trained to make decisions based on said patterns)
controlling the host vehicle based on the at least one pattern related to the environment of the host vehicle from the joined spatial-temporal data set ([0031] device comprises vehicle control for a vehicle, [0138] processing unit is configured to provide the control signal to cause the device to control the vehicle if the estimated time it will take for the predicted collision to occur is below the threshold, [0262] s.4-5 the vehicle control may automatically apply the brake of the vehicle, automatically disengage the gas pedal, automatically activate hazard lights, automatically steer the vehicle, or any combination of the foregoing and thus the control signal may be fed to a braking system (e.g., automatic emergency braking system), a steering system, a lane control system, a level 3 automation system, etc., or any combination of the foregoing, and [0388] last sentence, generate a control signal to operate a device to provide a warning to the driver, and/or to operate a device to automatically control the vehicle),
but Heck does not explicitly disclose:
wherein the pattern is provided as an abstract feature map which is stored in a grid map,
wherein the weights of the attention algorithm are applied to values generated by employing a union of the elements of the set of current input data and the assigned elements of the set of memory data in order to provide the joined spatial-temporal data set as an output of the attention algorithm, and
wherein tasks implemented as machine learning algorithms are applied to the abstract feature map in order to derive information regarding a position and/or dynamics of a respective object in the environment of the host vehicle, and/or in order to perform a grid segmentation of the environment of the host vehicle.
However, Long teaches on pg. 1516 left column ¶2 s. 6-8 to right column s.2, self-attention mechanism is leveraged to capture spatial-temporal correlations synchronously by assigning different correlation weights to spatial-temporal neighbors. Furthermore, a gated fusion module is proposed to fuse external factors with spatial-temporal features according to the different influences of different external factors. The gated fusion module first classifies external factors by types, then embeds them into the same space using neural network for fusion, and finally weighted fuses them with spatial-temporal features with the aid of a nonlinear function. By doing so, our proposed model can perceive the differences of influences caused by different external factors. Furthermore, see Table 2 which is a summary of datasets where a grid map is utilized to store and retrieve information.
PNG
media_image2.png
208
616
media_image2.png
Greyscale
Therefore it would have been obvious to one of ordinary skill in the art of autonomous navigation and vehicle controls before the effective filing date of the current invention to modify the vehicle control method of Heck, by incorporating the grid map and weighted attention algorithm teachings of Long, such that the combination would provide for the predictable result of, as acknowledged by long in the abstract, improving the accuracy of a prediction based upon the dynamic and various spatial-temporal correlations in a traffic network.
Claims 12-13 and 15 recite a system, a vehicle, and a non-transitory computer-readable medium, respectively, having substantially the same features of claim 1 above, therefore claims 12-13 and 15 are rejected for the same reasons as claim 1.
Claims 4 and 14 have been canceled by the Applicant and are subsequently no longer being considered.
Regarding claim 8, Heck, as modified by Long, discloses:
The method according to claim 1, wherein: information regarding respective distances with respect to a spatial
reference point is associated with the set of current input data and with the set of memory data in order to track movements of objects between one of the previous points in time and the current point in time ([0370] collision predictor 218 may obtain a sequence of sensor information indicating distances between the subject vehicle and the leading vehicle over a period).
Regarding claim 9, Heck, as modified by Long, discloses:
The method according to claim 8, wherein:
the information regarding the respective distances associated with the set of memory data is determined via a motion model which includes a velocity estimation for the objects ([0370] By analyzing the change in distance over the period, the collision predictor may determine the relative speed between the subject vehicle and the leading vehicle).
Regarding claim 10, Heck, as modified by Long, discloses:
The method according to claim 1, wherein:
positional information is associated with elements of the set of current input data and assigned elements of the set of memory data in order to estimate a velocity of an object in the environment of the host vehicle (see claims 8 and 9 as well as a collision predictor would necessarily utilize positional information associated with elements of the data when performing distance and velocity calculations and [0368] where the widths of the detected vehicles and their corresponding positions with respect to the coordinate system of the images may be used by the processing unit for mapping purposes).
Regarding claim 11, Heck, as modified by Long, discloses:
The method according to claim 1, wherein:
the at least one pattern is provided as an input to an algorithm for object detection which determines at least one of a position, a velocity and coordinates of a bounding box associated with one of a plurality of objects located in the environment of the host vehicle, and/or to an algorithm for segmenting the environment of the host vehicle (see claim 1 Fig. 14 regarding coordinates of a bounding box associated with one of a plurality of objects in an environment and a segmenting algorithm, and see claims 8-10 regarding positional, distance, and speed elements as inputs into an algorithm).
_____________________________________
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Heck et al. (US Pat. Pub. No. 2023/0166743 A1), hereinafter referred to as Heck, Long et al. article “Unified spatial-temporal neighbor attention network for dynamic traffic prediction”, hereinafter referred to as Long, and Afrouzi et al. (US Pat. No. 11,274,929), hereinafter referred to as Afrouzi.
Regarding claim 2, Heck, as modified by Long, discloses:
The method according to claim 1, wherein:
the set of current input data and the set of memory data (see claim 1 regarding current and memory data),
but Heck does not explicitly disclose that these datasets:
are associated with respective grid maps defined for the environment of the host vehicle.
However, Afrouzi teaches in column (col) 97 lines (ln) 46 and 51 where nodes are used to represent an area on a grid map and those nodes have information designated to them. Further in col 110 ln 5-14 the processor generates a new grid map with new characteristics associated with each or a portion of the cells of the grid map at each work session. For instance, each unit tile may have associated therewith a plurality of environmental characteristics, like classifications in an ontology or scores in various dimensions. The processor also compiles the map generated at the end of a work session with an aggregate global map (Fig.113E and col 95 ln 33-35) based on a combination of maps generated during each or a portion of prior work sessions.
Therefore it would have been obvious to one of ordinary skill in the art of autonomous navigation before the effective filing date of the current invention to modify the vehicle control method of Heck as modified by the grid map and weighted attention algorithm teachings of Long, by incorporating the grid map teachings of Afrouzi, such that the combination would provide for the predictable result of, as acknowledged by Afrouzi in col 69 ln 54-55, allowing other types of information may be used to improve accuracy of localization.
PNG
media_image3.png
390
381
media_image3.png
Greyscale
Figure 113 E
Regarding claim 3, Heck, as modified by Long and Afrouzi, discloses:
The method according to claim 2, wherein:
when applying the attention algorithm (see claim 1), a matching is determined
between a cell of the grid map associated with the set of current input data and a plurality of cells of the grid map associated with the set of memory data.
However, as discussed in claim 2, the current working sessions is integrated into a global map which is interpreted as matching between the cell of current input with a plurality of cells of the grid map associated with memory data.
________________________________________
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Heck et al. (US Pat. Pub. No. 2023/0166743 A1), hereinafter referred to as Heck, Long et al. article “Unified spatial-temporal neighbor attention network for dynamic traffic prediction”, hereinafter referred to as Long, and Xu et al. (US Pat. Pub. No. 2024/0362923 A1), hereinafter referred to as Xu.
Regarding claim 5, Heck, as modified by Long, discloses:
The method according to claim 4, wherein:
relating the elements of the set of current input data and the assigned elements of the set of memory data (see claim 1),
but Heck as modified by Long, dose not explicitly disclose:
generating a query vector by employing the elements of the set of current input data,
generating a key vector by employing the assigned elements of the set of memory data and the elements of the set of current input data, and
estimating a dot product of the query vector and the key vector from which the weights of the attention algorithm are estimated.
However, Xu teaches in [0016] s.2, The attention algorithm comprised by the machine learning algorithm may include so-called set attention blocks (SAB) which rely on an attention function defined by a pairwise dot product of query and key vectors in order to measure how similar the query and the key vectors are.
Therefore it would have been obvious to one of ordinary skill in the art of autonomous navigation before the effective filing date of the current invention to modify the vehicle control method of Heck, by incorporating the grid map teachings of Long and attention algorithm of Xu, such that the combination would provide for the predictable result of, as acknowledged by Xu in [0012], improved reliability for predicting the future trajectories of the road users.
Regarding claim 6, Heck, as modified by Long and Xu, discloses:
The method according to claim 5,
but Heck, as modified by Long, does not explicitly disclose:
the key vector is generated based on a concatenation of the assigned elements of the set of memory data and the elements of the set of current input data.
However, Xu teaches in [0016] s.3, that each set attention block may include a so-called multi-head attention which may be defined by a concatenation of respective pairwise attention functions, wherein the multi-head attention includes learnable parameters.
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see:
Cella et al. (CA Pat. Pub. No. 3238745 A1) is directed towards transportation and related methods and systems including vehicle operating states, an identity management system, an intelligent digital twin system that creates, manages, and provides digital twins for transportation systems using sensor data and other data, quantum computing methods and systems, including a set of quantum computing services, and biology-based systems and methods for communicating and/or handling data; and
Liang et al. (U.S. Pat. No. 11,548,533 B2) is directed towards an object perception and prediction of object motion apparatus.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEITH ALLEN VON VOLKENBURG whose telephone number is (703)756-5886. The examiner can normally be reached Monday-Friday 8:30 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin D. Bishop can be reached at (571) 270-3713. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Keith A von Volkenburg/Examiner, Art Unit 3665
/Erin D Bishop/ Supervisory Patent Examiner, Art Unit 3665