DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 3, 9, 11, 13 and 15 have been amended. Claims 7 and 19 have been canceled. Claims 21-22 have been added. Claims 1-6, 8-18 and 20-22 have been examined.
Response to Arguments/Amendments
The prior rejections under 35 USC 101 and 112 are withdrawn in view of the 8/7/2026 claim amendments.
Applicant's arguments on pp. 9-12 filed 8/7/2026 have been fully considered but they are not persuasive.
On pp. 9-11 of the 8/7/2026 remarks, Applicant argues with respect to claim 1 that cited art of record Zhao fails to fully disclose the claimed “measured environment data” and argues that “the Zhao environment is not a type of environment that can be measured.” However, Zhao generally teaches the use of a “general driving data store” as well as a “driver-specific driving data store” to train anomaly detection models (e.g. see Fig. 1 elements 112 and 132). Data collection is described in Zhao ¶ 0036: “The engine start indicates a start of a new driving episode and may initiate recording of the sensor signals of the sensors 102. The engine stop indicates an end of the driving episode and may terminate the recording of the sensor signals of the sensors 102.” Notably, ¶ 0024 describes the use of sensors for data measurement, e.g. “any one of mechanical sensors that measure mechanical properties such as inertia, force and displacement, physical sensors that measure physical properties such as magnetic field and other sensors that measures other properties.” Thus, Zhao clearly describes a measurable environment. It is noted that Applicant narrowly focuses upon Zhao’s “local environment” in the course of the argument, but a broad but reasonable interpretation of “environment” is not limited to the Zhao’s “local environment.” That is, Zhao’s sensor data provides a measurement of the environment in which the vehicle is operated. Applicant’s argument is not persuasive.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “atmospheric environment” – see p. 10, 2nd full paragraph) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On pp. 11-12 of the remarks, Applicant references the Interview Summary and argues that inherency has not been established regarding the claimed “converting information.” Applicant argues that “Zhao describes only the acquisition and storage of sequential sensor vectors” and fails to disclose conversion of “time-series data into a fixed-length feature representation.” However, Zhao ¶ 0029 teaches storage of sensor data as vectors of a plurality of recorded values. The recorded values are taken from a variety of sensors (¶ 0025). That is, the recorded values are produced using sensors which are then collected and converted into vectors. The vector does not exist without conversion from the various recorded values. Additionally, Zhao ¶ 0030 explicitly teaches conversion of recorded values in order to provide vector data, e.g. “conversion to obtain the process value …”
On p. 12 of the remarks, Applicant continues and argues that Zhao fails to disclose feature vectors having a predetermined length. However, as noted above, Zhao’s vectors include values “selected from a group” as described in ¶ 0029. Zhao’s group provides a representation for a vector having a predetermined length by way of the predetermined list of values provided in ¶ 0029. Thus, Zhao’s vector provides disclosure according to a broad but reasonable interpretation of the claim language.
Applicant’s arguments, see pp. 13, filed 8/7/2026, with respect to the rejection(s) of claim(s) 1 and 13 under 35 USC 103, have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of U.S. Patent Application Publication 20230376687 by Morariu et al.
Applicant’s arguments on pp. 14-17 of the remarks are similar to essentially similar to previous arguments and are treated similarly as indicated above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-5, 8, 13-14, 16-17 and 20-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 20200380354 by Zhao et al. ("Zhao") in view of U.S. Patent Application Publication 20230376687 by Morariu et al. (“Morariu”) and U.S. Patent Application Publication 20220012988 by Avadhanam et al. ("Avadhanam").
In regard to claim 1, Zhao discloses:
1. A computer-implemented method for defect detection, comprising: Zhao, Fig. 1, broadly depicting a method.
determining a first residual score in a … detector by comparing a first predicted system state, determined according to previously measured environment data, to an actual system state; Zhao Fig. 1 element 114 and ¶ 0037 “general anomaly detection model 114.” Also ¶ 0020-0021, “… a general driving data store 112 for storing a collection of driving data of a variety of drivers, which is used to train the general anomaly detection model 114. … a driver-specific driving data store 132 for storing a collection of driving data obtained from a driver (may be a single driver) in local environment, which is used for training the driver-specific anomaly detection model 134.”
Zhao does not expressly disclose: cross-attention based. However, this is taught by Morariu. See Morariu, ¶ 0002, “To accomplish the multi-granular aspect, the machine learning model analyzes components of a document at different granularities (e.g., page, region, token, etc.) by generating an input to the model that includes features extracted from the different granularities. For example, the input to the multi-modal multi-granular model includes a fixed length feature vector including features and bounding box information extracted from a page-level, region-level, and token-level of the document . With regard to the multi-modal aspect, a machine learning model analyzes different types of features (e.g., textual, visual features, and/or other features) associated with the document. As one example, the machine learning model analyzes visual features obtained from a convolutional neural network (CNN) and textual features obtained using optical character recognition (OCR), transforming such features first based on self-attention weights (e.g., within a single modality or type of feature) and then based on cross-attention weights (e.g., between modalities or types of features). These transformed feature vectors can then be provided to other machine learning models to perform various tasks (e.g., document classification, entity recognition, token recognition, etc.).” Also see ¶ 0039-0040, e.g. “cross-attention.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Morariu’s cross-attention processing with Zhao’s system states in order to provide multi-modal analysis as suggested by Morariu.
Zhao also discloses:
determining a second residual score by comparing a second predicted system state, determined according to previously measured system state data, to the actual system state; Zhao Fig. 1 element 134 and ¶ 0041 “driver-specific anomaly detection model 134.”
converting information about detected … [data] in the previously measured environment data to feature vectors of predetermined length. Zhao, ¶ 0029, “One driving data may include a sequence of vectors in time order (i.e., time series), in which each vector is recorded at a predetermined recording frequency. Each vector includes a plurality of recorded values selected from a group including accelerations of x, y and z axes, angular velocities of roll, pitch and yaw axes, a vehicle speed, a steering angle, latitude, longitude, height, an azimuth, a depression amount and a depression speed of an accelerator pedal, a position of a throttle valve, a depression amount and a depression speed of a brake pedal, and processed values obtained therefrom.” Also ¶ 0030, “The processed values may be obtained by processing any one of the aforementioned recorded values. The way of processing or conversion to obtain the process value may include, but not limited to, differentiation, integration, computing difference, moving average, Fourier transform, to name but a few.” Note that even if Zhao’s vectors fail to fully disclose a predetermined length, Morariu teaches a “fixed length feature vector” as cited above.
Zhao does not expressly disclose: objects. This is taught by Avadhanam. See ¶ 0039, “The processing circuitry may concatenate these three feature vectors to be fitted to a support-vector machine classifier (“SVM”) 318 to predict whether a pedestrian is performing a specific action (e.g., texting, calling, other, etc.)” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Avadhanam’s feature vectors with Zhao’s data in order to predict actions and assess safety and risk as suggested by Avadhanam (e.g. see ¶ 0045).
Zhao also discloses:
generating a defect score based on a difference between the first residual score and the second residual score; and Zhao, ¶ 0022, “a driver change detection module 140 for performing driver change detection based upon the general anomaly detection model 114 and the driver-specific anomaly detection model 134.” Also ¶ 0061, “In one or more preferable embodiments, as a form of the aforementioned relation, the detection score is calculated as a ratio of the driver-specific anomaly score to the general anomaly score or a difference between the driver-specific anomaly score and the general anomaly score.”
performing an automatic action responsive to a determination that the defect score indicates a defect in system behavior. Zhao, ¶ 0049, “In a particular embodiment, the result of the driver change detection may be an alert or notification indicating that driving by a driver different from a usual driver (the target driver learned by the driver-specific anomaly detection model 134) has detected.”
In regard to claim 2, Zhao does not expressly disclose:
2. The method of claim 1, wherein determining the first residual score includes processing the previously measured environment data with a temporal attention and comparing a prediction based on the temporal attention to the previously measured system state data. Zhao, ¶ 0029, “One driving data may include a sequence of vectors in time order (i.e., time series), in which each vector is recorded at a predetermined recording frequency.”
In regard to claim 4, Zhao also discloses:
4. The method of claim 1, wherein determining the second residual score includes processing the previously measured system state data with a time series detector … Zhao, ¶ 0029, “One driving data may include a sequence of vectors in time order (i.e., time series), in which each vector is recorded at a predetermined recording frequency.”
Zhao does not expressly disclose: with a long-short term memory model. This is taught by Avadhanam. See Avadhanam ¶ 0044, “The processing circuitry may implement body pose convolutional neural network(s) 332 which input to Long-Short Term Memory “LTSM” recurrent neural network(s) 334. The LTSM(s) may learn body motion pattern and torso orientation. The outputs from the LTSM(s) are input into a classifier 336 (e.g., a SoftMax classifier) which may predict whether a pedestrian is crossing, or performing other activities.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Avadhanam’s LTSM model with Zhao’s system state in order to predict activities as suggested by Avadhanam.
In regard to claim 5, Zhao also discloses:
5. The method of claim 1, wherein generating the defect score includes subtracting the second residual score from the first residual score to compute a difference. Zhao, ¶ 0030, “The way of processing or conversion to obtain the process value may include, but not limited to, differentiation, integration, computing difference, moving average, Fourier transform, to name but a few.”
In regard to claim 8, Zhao also discloses:
8. The method of claim 1, wherein performing the automatic action includes an action selected from the group consisting of
…
performing an autonomous action to avoid a hazard that an earlier defect created. Zhao, ¶ 0049, “In a particular embodiment, the result of the driver change detection may be an alert or notification indicating that driving by a driver different from a usual driver (the target driver learned by the driver-specific anomaly detection model 134) has detected.”
Avadhanam teaches the following limitations which are not expressly disclosed by Zhao:
disabling an autonomous driving function of the system, Avadhanam, ¶ 0119, “Certain functions are available to the driver only when the vehicle is operating in an autonomous mode, and are disabled otherwise.”
performing an autonomous action to compensate for an earlier defect, and Avadhanam, ¶ 0138, “emergency braking, collision avoidance …”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Avadhanam’s autonomous control with Zhao’s models in order to effect a reduction of the risk level as suggested by Avadhanam (see ¶ 0003).
In regard to claim 13, Zhao discloses:
13. A system for defect detection, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to: Zhao, Fig. 9, elements 12 and 16, depicting a system.
All further limitations of claim 13 have been addressed in the above rejection of claim 1
In regard to claims 14, 16-17 and 20, parent claim 13 is addressed above.
All further limitations of claims 14, 16-17 and 20 have been addressed in the above rejections of claims 2, 4-5 and 8, respectively.
In regard to claim 21, Zhao also discloses:
21. The method of claim 1, wherein the difference between the first residual score and the second residual score identifies an anomaly score, and in response to the anomaly score being above a threshold, determining that the anomaly is the defect. See Zhao, ¶ 0061, “In one or more preferable embodiments, as a form of the aforementioned relation, the detection score is calculated as a ratio of the driver-specific anomaly score to the general anomaly score or a difference between the driver-specific anomaly score and the general anomaly score.” Also ¶ 0066, “In a particular embodiment, the detection threshold (Sthres) may be set to the maximum of the detection scores calculated for the driver-specific driving data store 132.”
In regard to claim 22, Zhao also discloses:
22. The method of claim 1, wherein converting information to feature vectors of predetermined length further includes:
dividing … [data] around a vehicle into … regions, including the detected objects within the … regions in the defect score, and ignoring the detected objects outside the … regions from the defect score; Zhao, ¶ 0064, “In a particular embodiment, remaining parts of the input driving data other than the highlight parts (P) are ignored in calculating the detection score.”
Zhao does not expressly disclose:
dividing an area around a vehicle into spatial regions, … assigning spatial attributes to the detected objects in the spatial regions; and generating features for each spatial region based on the spatial attributes. This is taught by Avadhanam. See Avadhanam, Figs. 1A-1D along with associated text, e.g. ¶ 0027, “The processing circuitry, based on the locations of the teenage boy getting closer to the crosswalk, determines a directionality consistent with traversing the crosswalk. Additionally, the processing circuitry may determine, based on these images, the velocity of the object.” Also, ¶ 0038, “The machine learning model may implement a head bounding box to be passed to a head pose estimation.” ¶ 0039, “Firstly, body pose convolutional neural network(s) 312 may input a pedestrian bounding box of the pedestrian. The body pose convolutional neural network is utilized to determine the specific kinematic values, locomotion and/or posture of the pedestrian. Secondly, object convolutional neural network(s) 314 may input a hand bound box of the pedestrian. The object convolutional neural network is utilized to determine a specific object interacting with the pedestrian. The hand bounding box may be estimated based on wrist joint position computed by body pose and its size may be proportional to the pedestrian's height. Thirdly, head pose convolutional neural network(s) 316 may input a head bounding box of the pedestrian. The head pose convolutional neural network is utilized to determine the pedestrian's head orientation in order to estimate their attention level.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Avadhanam’s spatial attributes with Zhao’s data in order to determine a risk level surrounding a vehicle as suggested by Avadhanam (see ¶ 0003).
Claim(s) 3 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Morariu and Avadhanam as applied above, and further in view of U.S. Patent Application Publication 20200134804 by Song et al. ("Song").
In regard to claim 3, Zhao does not expressly disclose:
3. The method of claim 2, wherein processing the previously measured environment data includes generating a set of weights, further comprising generating the prediction by multiplying the previously measured system state data by the set of weights. This is taught by Song. See Song, Fig. 5 element 565 depicting multiplication. Also ¶ 0063, “Temporal self-attention module 420 then compares the similarity Dcos 555 between query q(tk) (545) and memory m(tk) (540) feature vectors and generates the attention weights by normalizing across time steps using a softmax function σ (560). The attended feature map Ht(l) 572 is a weight average of the feature maps at different time steps.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Song’s temporal attention weights with Zhao’s prediction in order to facilitate learning of dependencies as suggested by Song.
In regard to claim 15, parent claim 14 is addressed above.
All further limitations of claim 15 have been addressed in the above rejection of claim 3.
Claim(s) 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Morariu and Avadhanam as applied above, and further in view of U.S. Patent Application Publication 20200057940 by Hu et al. ("Hu").
In regard to claim 6, Zhao does not expressly disclose:
6. The method of claim 5, wherein the defect score is the greater of the difference and zero. This is taught by Hu. See Hu ¶ 0055, “If the difference is zero, then this may indicate that the reconstruction accurately determined the pixel value for that coordinate of image 202. If the difference is greater than zero, then this may indicate that the reconstruction did not accurately determine the pixel value.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Hu’s relative score analysis with Zhao’s score in order to provide an indication of accuracy as suggested by Hu.
In regard to claim 18, parent claim 17 is addressed above.
All further limitations of claim 18 have been addressed in the above rejection of claim 6.
Claim(s) 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Morariu and U.S. Patent Application Publication 20220383224 by Sinha et al. ("Sinha").
In regard to claim 9, Zhao discloses:
9. A computer-implemented method for training a model, comprising: Zhao, Fig. 1, element 110, broadly depicting a method for training a model.
determining a first residual score in a … detector by comparing a first predicted system state, determined by a first model according to environment data from a training dataset, to an actual system state from the training dataset; Zhao, ¶ 0037, “The general model training module 110 is configured to train an anomaly detection model using the driving data stored in the general driving data store 112 to prepare the general anomaly detection model 114 (fg).” Also Fig. 1 element 114 and ¶ 0037 “general anomaly detection model 114.” Also ¶ 0021, “a collection of driving data obtained from a driver (may be a single driver) in local environment, which is used for training the driver-specific anomaly detection model 134.”
Zhao does not expressly disclose: cross-attention based. However, this is taught by Morariu. See Morariu, ¶ 0002, “To accomplish the multi-granular aspect, the machine learning model analyzes components of a document at different granularities (e.g., page, region, token, etc.) by generating an input to the model that includes features extracted from the different granularities. For example, the input to the multi-modal multi-granular model includes a fixed length feature vector including features and bounding box information extracted from a page-level, region-level, and token-level of the document . With regard to the multi-modal aspect, a machine learning model analyzes different types of features (e.g., textual, visual features, and/or other features) associated with the document. As one example, the machine learning model analyzes visual features obtained from a convolutional neural network (CNN) and textual features obtained using optical character recognition (OCR), transforming such features first based on self-attention weights (e.g., within a single modality or type of feature) and then based on cross-attention weights (e.g., between modalities or types of features). These transformed feature vectors can then be provided to other machine learning models to perform various tasks (e.g., document classification, entity recognition, token recognition, etc.).” Also see ¶ 0039-0040, e.g. “cross-attention.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Morariu’s cross-attention processing with Zhao’s system states in order to provide multi-modal analysis as suggested by Morariu.
Zhao also discloses:
determining a second residual score by comparing a second predicted system state, determined by a second model according to system state data from a training dataset, to the actual system state; Zhao, ¶ 0037, “The general model training module 110 is configured to train an anomaly detection model using the driving data stored in the general driving data store 112 to prepare the general anomaly detection model 114 (fg).” Also Fig. 1 element 134 and ¶ 0041 “driver-specific anomaly detection model 134.”
adjusting parameters of the first model … adjusting parameters of the second model … Zhao, ¶ 0081, “The DyBM has learnable parameters that are updated during training, …”
Zhao does not expressly disclose: … to minimize a first objective function based on a difference between the first predicted system state and the actual system state; and … to minimize a second objective function based on a difference between the second predicted system state and the actual system state. This is taught by Sinha. See Sinha ¶ 0110, “Accordingly, during the training process, the parameters and weights of a data processing network are adjusted to increase the accuracy of the result (i.e., by minimizing a loss function which corresponds in some way to the difference between the current result and the target result).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Sinha’s loss function with Zhao’s training in order to increase the accuracy of the result as suggested by Sinha.
In regard to claim 10, parent claim 9 is addressed above.
All further limitations of claim 10 have been addressed in the above rejection of claim 2.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Morariu and Sinha as applied above, and further in view of Song.
In regard to claim 11, parent claim 10 is addressed above.
All further limitations of claim 11 have been addressed in the above rejection of claim 3.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Morariu and Sinha as applied above, and further in view of Avadhanam.
In regard to claim 12, parent claim 9 is addressed above.
All further limitations of claim 12 have been addressed in the above rejection of claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent Application Publication 20220351716 by Kim et al. See Figs. 8-9 and ¶ 0035-0037, e.g. “FIG. 8 illustrates an embodiment of utilizing cross-attention mechanism between these two groups. … To address such a challenge, the system may include developed general models that can generate a structure-aware representation for each entity while mapping entities into their representation. Many different types of representations can be considered for the models including vectors and graphs. … In addition, additional techniques can be used to further enhance the model's outcome while focusing on similar neighborhoods and giving them higher weights such as attention mechanism . For examples, cross-attention mechanism can be used between these two groups (left and right) vectors as shown in FIG. 8. … FIG. 9A is an illustration of a first group of concatenations of fixed-length vectors. … enhance the model's outcome while focusing on similar neighborhoods and giving them higher weights such as attention mechanism”
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James D Rutten whose telephone number is (571)272-3703. The examiner can normally be reached M-F 9:00-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at (571)272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/James D. Rutten/Primary Examiner, Art Unit 2121
Prosecution Insights