Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2022-0165948, filed on 12/01/2022..
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: sensor part in claim 1, 2, 5, 8, 10 and 18, controller in claims 1-8, and 10, a storage device in claims 1 and 11, model-learning part in claims 8 and 9 and data logging part in claim 9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 8-13, 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Park (US 20240020953 A1) and further in view of Tarrant (US 20220100868 A1).
Regarding claims 1, 11 and 19 Park discloses a sensor part including a plurality of cameras having fields of view different from each other ("[0015] FIG. 7B is an example of camera locations and fields of view for the example autonomous vehicle of FIG. 7A, in accordance with some embodiments of the present disclosure.
[0021] The BEV feature map may be generated based at least on relative positions of the sensors in the environment (e.g., camera positions) corresponding to the plurality of views."); and
a controller configured to process image data obtained by the sensor part ([0079] One or more of the controller(s) 736 may receive inputs (e.g., represented by input data) from an instrument cluster 732 of the vehicle 700 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 734, an audible annunciator, a loudspeaker, and/or via other components of the vehicle 700.),
wherein the controller is configured to:
train an image recognition model by inputting training data to the image recognition model ("[0037] In at least one embodiment, the image encoder 102 may be implemented using one or more machine learning models (MLMs).
[0054] To train the MLM(s) 108, the predicted 3D bounding shapes may be projected into image space to determine corresponding predicted 2D bounding shapes, which may be compared to the ground truth 2D bounding shapes for updating the parameters of the MLM(s) 108."),
wherein the image recognition model is configured to output a training feature map associated with the training data ([0004]: One or more machine learning models (MLMs)— such as one or more deep neural networks (DNNs)— may use the feature map to predict one or more outputs corresponding to the environment.);
extract a feature map by inputting the image data obtained by the sensor part to the trained image recognition model ([0005] Generating a feature map may be based at least on assigning the transformed feature values to one or more bins corresponding to the feature maps.);
transmit, to a storage device, the logging data ("[0030] The MLM 108 may further be configured to use the fused features to generate output data, such as output data 128. For example, the MLM 108 may use the fused features predict one or more outputs corresponding to the environment. The output data may be used by a machine, such as the vehicle 700, to perform one or more operations, such as one or more control operations and/or ADAS operations.
[0050] As described herein, the MLMs 108 may be configured to use the fused features 126 to generate output data, such as the output data 128. In at least one embodiment, the MLMs 108 include one or more MLMs to extract high-level features {circumflex over (F)}.sup.bev from the fused features 126 (e.g., a fused BEV feature map corresponding to each of the image features 122). "). Park implicitly discloses train a training ("[0046] The feature fuser 106 may aggregate F.sub.polar.sup.c with dimensions h×C into the global BEV feature map F.sub.bev. The features F.sub.polar.sup.c may have the same number of radial distance bins ĥ, such that each cell in F.sub.polar.sup.c corresponds to a radial distance bin, from which the perspective transformer 104 may sample multiple depth values and compute the corresponding angular values using the fitted polynomial function ƒ.sup.c(.). For computational efficiency, the perspective transformer 104 may sample, by way of example, and not limitation, one depth value per radial distance bin. Thus, for each cell in the features F.sub.polar.sup.c, an exact location in the grid G.sup.bev can be pre-computed and stored in a lookup table for efficient training and inference.
Rather than concatenating the BEV features maps for different images, the feature fuser 106 may combine the commonly mapped features, such as by using a sum operation and/or other statistical operation. Combining one or more of the mapped features may increase the robustness of the process 100 to camera dropout as the number of BEV feature maps may be invariant to the number of active senor inputs or channels.
[0052] If d.sub.i is a negative candidate, the loss function L(d.sub.i) may include a (focal) binary cross-entropy loss—pulling an objectness score to zero. If d.sub.i is a positive candidate, the loss function L(g.sub.i+d.sub.i) may capture classification and regression losses.");
determine a vulnerability score by inputting the extracted feature map to the trained vulnerability assessment model ([0119] Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections.);
determine, based on the vulnerability score satisfying a threshold, to store logging data associated with the image data obtained by the sensor part ([0046] Thus, for each cell in the features F.sub.polar.sup.c, an exact location in the grid G.sup.bev can be pre-computed and stored in a lookup table for efficient training and inference.).
In a similar field of endeavor of systems for scanning and remedying security vulnerabilities, Tarrant teaches train a vulnerability assessment model by inputting the training feature map to the vulnerability assessment model, wherein the vulnerability assessment model is configured to output a training vulnerability score associated with the training feature map ([0047] If the input vulnerability is mapped to the ML classifier 179, the feature vector creation process would be triggered, the feature vectors would be subsequently created for the input vulnerability, and the ML model would be loaded and invoked for processing the feature vectors to classify the input vulnerability. If the input vulnerability is mapped to the deterministic classifier 175, the classification engine 106 would further map this input vulnerability to one of the predetermined set of types of vulnerabilities and a corresponding ATP.);
determine a vulnerability score by inputting the extracted feature map to the trained vulnerability assessment model (claim 10: extracting features from the electronic document for each potential vulnerability issue; determining a vector based on the extracted features; selecting one of a plurality of vulnerability-scoring models based on the vector, the vulnerability-scoring models selected from the machine learning model library; and determining a vulnerability accuracy score based on the vector using the selected one of the vulnerability-scoring models.);
determine, based on the vulnerability score satisfying a threshold, to store logging data associated with the image data obtained by the sensor part (claim 10: extracting features from the electronic document for each potential vulnerability issue; determining a vector based on the extracted features; selecting one of a plurality of vulnerability-scoring models based on the vector, the vulnerability-scoring models selected from the machine learning model library; and determining a vulnerability accuracy score based on the vector using the selected one of the vulnerability-scoring models.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Regarding claim 2, Park discloses receive the image data obtained by the sensor part ([0035] Each camera and/or view may provide one or more images for input to the image encoder 102.);
extract the feature map by inputting the image data obtained by the sensor part to the trained image recognition model ([0036] As described herein, the image encoder(s) 102 (e.g., one or more 2D image encoders) may be configured to encode, using the sensor data 120, the image features 122 (e.g., image feature maps).).
Park implicitly determine the ([0052] The training loss for the MLMs 108 may be computed based at least on finding a one-to-one matching between G and P. A final loss may be computed based at least on the matching result. The set of positive object candidates matched to the ground truth objects may be referred to as P.sub.pos and the set of negative objects may be referred to as P.sub.neg.).
In a similar field of endeavor of systems for scanning and remedying security vulnerabilities, Tarrant teaches determine the vulnerability score by inputting the extracted feature map to the trained vulnerability assessment model (claim 10: extracting features from the electronic document for each potential vulnerability issue; determining a vector based on the extracted features; selecting one of a plurality of vulnerability-scoring models based on the vector, the vulnerability-scoring models selected from the machine learning model library; and determining a vulnerability accuracy score based on the vector using the selected one of the vulnerability-scoring models.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Regarding claim 3, Park discloses wherein the controller is configured to train, based on at least one convolutional neural network (CNN), the image recognition model and the ([0052] If d.sub.i is a negative candidate, the loss function L(d.sub.i) may include a (focal) binary cross-entropy loss—pulling an objectness score to zero. If d.sub.i is a positive candidate, the loss function L(g.sub.i+d.sub.i) may capture classification and regression losses.
[0106] The DLA(s) may quickly and efficiently execute neural networks, especially CNNs, on processed or unprocessed data for any of a variety of functions, including, for example and without limitation: a CNN for object identification and detection using data from camera sensors; a CNN for distance estimation using data from camera sensors; a CNN for emergency vehicle detection and identification and detection using data from microphones; a CNN for facial recognition and vehicle owner identification using data from camera sensors; and/or a CNN for security and/or safety related events.).
Tarrant teaches a vulnerability assessment model (claim 10: extracting features from the electronic document for each potential vulnerability issue; determining a vector based on the extracted features; selecting one of a plurality of vulnerability-scoring models based on the vector, the vulnerability-scoring models selected from the machine learning model library; and determining a vulnerability accuracy score based on the vector using the selected one of the vulnerability-scoring models.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Regarding claims 4 and 13, Park discloses intersection over union (IOU) ([0056] The radius loss L.sub.fsp.sup.iou({circumflex over (r)},r) may include a polar intersection-over-union loss (IoU) and may be computed between the prediction and ground truth labels); and
an object image of which object recognition result for the extracted feature map is a false positive (FP), a false negative (FN), or a true positive (TP) ([0119] The DLA may be used to run any type of network to enhance control and driving safety, including for example, a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections. In an automatic emergency braking (AEB) system, false positive detections would cause the vehicle to automatically perform emergency braking, which is obviously undesirable. Therefore, only the most confident detections should be considered as triggers for AEB.).
Regarding claims 8, Park discloses wherein the model-learning part is configured to train the image recognition model by inputting the training data that is stored in a learning database (DB) to the image recognition model ("[0037] In at least one embodiment, the image encoder 102 may be implemented using one or more machine learning models (MLMs).
[0054] To train the MLM(s) 108, the predicted 3D bounding shapes may be projected into image space to determine corresponding predicted 2D bounding shapes, which may be compared to the ground truth 2D bounding shapes for updating the parameters of the MLM(s) 108."),
and transmit the trained image recognition model to the data logging part (and transmit the trained image recognition model to the data logging part),
wherein the data logging part is configured to:
receive and store the trained image recognition model ("[0052] The training loss for the MLMs 108 may be computed based at least on finding a one-to-one matching between G and P. A final loss may be computed based at least on the matching result. The set of positive object candidates matched to the ground truth objects may be referred to as P.sub.pos and the set of negative objects may be referred to as P.sub.neg.
[0054] In at least one embodiment, LIDAR data may be used to generate ground truth for the 3D cuboids and/or 3D bounding shapes. To provide ground truth for objects outside of LIDAR range (e.g., 300 meters), the bounding shapes may be labeled in image space using 2D bounding shapes. To train the MLM(s) 108, the predicted 3D bounding shapes may be projected into image space to determine corresponding predicted 2D bounding shapes, which may be compared to the ground truth 2D bounding shapes for updating the parameters of the MLM(s) 108.")
while the vehicle is driving, extract the feature map by inputting the image data obtained by the sensor part to the trained image recognition model ("[0005] Generating a feature map may be based at least on assigning the transformed feature values to one or more bins corresponding to the feature maps.
[0067] At block B508, the method 500 includes performing one or more operations using the one or more BEV features maps. For example, the vehicle 700 may perform, using the MLM(s) 108 and based at least on the one or more BEV feature maps, one or more operations for the vehicle 700.
[0170] In some examples, the output of the ADAS system 738 may be fed into the primary computer's perception block and/or the primary computer's dynamic driving task block. For example, if the ADAS system 738 indicates a forward crash warning due to an object immediately ahead, the perception block may use this information when identifying objects."),
based on a determination that logging is required, transmit the logging data to the learning DB ("[0030] The MLM 108 may further be configured to use the fused features to generate output data, such as output data 128. For example, the MLM 108 may use the fused features predict one or more outputs corresponding to the environment. The output data may be used by a machine, such as the vehicle 700, to perform one or more operations, such as one or more control operations and/or ADAS operations.
[0050] As described herein, the MLMs 108 may be configured to use the fused features 126 to generate output data, such as the output data 128. In at least one embodiment, the MLMs 108 include one or more MLMs to extract high-level features {circumflex over (F)}.sup.bev from the fused features 126 (e.g., a fused BEV feature map corresponding to each of the image features 122). ").
Park does not explicitly disclose but Tarrant teaches train the vulnerability assessment model by inputting the training feature map to the vulnerability assessment model ([0034] The feature vectors 173 may be utilized as input to the pre-trained ML model 141, predetermined programming rules 150 and/or blanket rules 174 in order to determine whether the cleaned vulnerability 127 is a threat or not. The classification engine 106 may determine whether a vulnerability 127 is a threat or not through at least three different methods: blanket rules 174, programming rules 150 and/or ML models 141.),
and transmit the trained vulnerability assessment model to the data logging part ([0037] An embodiment of the output engine 107 is also in FIG. 5. The output from the output engine 107 may include initial findings received from the trained model 141 for the predictions of whether labelled vulnerabilities 187 are a threat or not. The trained model 141 may be stored in the trained model database 119. In some embodiments, the trained model 141 may be transmitted to the probabilistic classifier 179. The classification engine 106 may generate a list of labelled vulnerabilities 187, and/or predictions thereof, that may be stored and later reviewed by the system 100.), and
receive and store the trained vulnerability assessment model ([0037] An embodiment of the output engine 107 is also in FIG. 5. The output from the output engine 107 may include initial findings received from the trained model 141 for the predictions of whether labelled vulnerabilities 187 are a threat or not. The trained model 141 may be stored in the trained model database 119. In some embodiments, the trained model 141 may be transmitted to the probabilistic classifier 179. The classification engine 106 may generate a list of labelled vulnerabilities 187, and/or predictions thereof, that may be stored and later reviewed by the system 100.),
determine the vulnerability score by inputting the extracted feature map to the trained vulnerability assessment model (claim 10: extracting features from the electronic document for each potential vulnerability issue; determining a vector based on the extracted features; selecting one of a plurality of vulnerability-scoring models based on the vector, the vulnerability-scoring models selected from the machine learning model library; and determining a vulnerability accuracy score based on the vector using the selected one of the vulnerability-scoring models.),
determine whether logging is required based on the vulnerability score ([0034] In certain embodiments, blanket rules 174 may be applied to vulnerabilities 127 routed through the vulnerability router 148, and the ML model 141 may not be required. Such a vulnerability 127 may be selected based on historical data that consistently indicates that the vulnerability 127 is exploitable. As such, it may be reasonable to automatically assume that the identified vulnerability 127 may be exploitable again.
[0038] For example, the review engine 108 may be implemented to include a process for an output review (block 600) and a process for a vulnerability review and a model update (block 601). Through these processes, the review engine 108 may review the vulnerabilities 127 that the system 101 determined as being exploitable, and may use such vulnerabilities 127 to retrain the model 141 for future usage. This review may be transmitted back into the model 141 in order to further train the model 141.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Regarding claim 9, Park discloses wherein the model-learning part is configured to train, based on a convolutional neural network (CNN), the image recognition model and the ("[0038] In at least one embodiment, the image encoders 102 are implemented using a convolutional architecture for real-time or near real-time performance. The image encoder 102 may include a Convolutional Neural Network (CNN) backbone including a series of CNN blocks.
[0054] In at least one embodiment, LIDAR data may be used to generate ground truth for the 3D cuboids and/or 3D bounding shapes. To provide ground truth for objects outside of LIDAR range (e.g., 300 meters), the bounding shapes may be labeled in image space using 2D bounding shapes. To train the MLM(s) 108, the predicted 3D bounding shapes may be projected into image space to determine corresponding predicted 2D bounding shapes, which may be compared to the ground truth 2D bounding shapes for updating the parameters of the MLM(s) 108.").
Tarrant teaches the vulnerability assessment model (claim 10: vulnerability-scoring models).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Regarding claims 10 and 18, Park discloses retrain, based on the logging data, the trained image recognition model ([0062] In at least one embodiment, a first epoch, the task loss weights {w.sub.t}.sub.t=1.sup.T may be set manually (e.g., w.sub.t=1 for t∈[1, T]) as the loss sum statistics are unavailable from a previous epoch, and w.sub.t may be updated after each epoch. In at least one embodiment, a multi-stage approach may be used where all {c.sub.t} are set to 1. Then the multi-task results may be compared to single task results, and {c.sub.t} may be inferred accordingly based at least on Key Performance Indicator (KPI) improvements and/or regressions. Using the determined {c.sub.t}, the multi-task network may be retrained in subsequent training trials.);
extract, based on the retrained image recognition model, a second feature map associated with second image data obtained by the sensor part ("[0062] In at least one embodiment, a first epoch, the task loss weights {w.sub.t}.sub.t=1.sup.T may be set manually (e.g., w.sub.t=1 for t∈[1, T]) as the loss sum statistics are unavailable from a previous epoch, and w.sub.t may be updated after each epoch. In at least one embodiment, a multi-stage approach may be used where all {c.sub.t} are set to 1. Then the multi-task results may be compared to single task results, and {c.sub.t} may be inferred accordingly based at least on Key Performance Indicator (KPI) improvements and/or regressions. Using the determined {c.sub.t}, the multi-task network may be retrained in subsequent training trials.
[0067] At block B508, the method 500 includes performing one or more operations using the one or more BEV features maps. For example, the vehicle 700 may perform, using the MLM(s) 108 and based at least on the one or more BEV feature maps, one or more operations for the vehicle 700.
[0089] Cameras with a field of view that include portions of the environment to the rear of the vehicle 700 (e.g., rear-view cameras) may be used for park assistance, surround view, rear collision warnings, and creating and updating the occupancy grid. A wide variety of cameras may be used including, but not limited to, cameras that are also suitable as a front-facing camera(s) (e.g., long-range and/or mid-range camera(s) 798, stereo camera(s) 768), infrared camera(s) 772, etc.), as described herein."); and
control, based on the extracted second feature map, autonomous driving of the vehicle ([0067] At block B508, the method 500 includes performing one or more operations using the one or more BEV features maps. For example, the vehicle 700 may perform, using the MLM(s) 108 and based at least on the one or more BEV feature maps, one or more operations for the vehicle 700.).
Regarding claim 12, Park discloses wherein the training of the image recognition model comprises training, based on a first convolutional neural network (CNN), the image recognition model ("[0050] For example, a CNN backbone may be used to extract the high-level features {circumflex over (F)}.sup.bev. In at least one embodiment, the high-level features {circumflex over (F)}.sup.bev may be consumed by one or more downstream perception modules.
[0052] If d.sub.i is a negative candidate, the loss function L(d.sub.i) may include a (focal) binary cross-entropy loss—pulling an objectness score to zero. If d.sub.i is a positive candidate, the loss function L(g.sub.i+d.sub.i) may capture classification and regression losses.").
Park does not expliclty disclose but Tarrant teaches wherein the training of the vulnerability assessment model comprises training, based on a second CNN, the vulnerability assessment model ([0038] For example, the review engine 108 may be implemented to include a process for an output review (block 600) and a process for a vulnerability review and a model update (block 601). Through these processes, the review engine 108 may review the vulnerabilities 127 that the system 101 determined as being exploitable, and may use such vulnerabilities 127 to retrain the model 141 for future usage. This review may be transmitted back into the model 141 in order to further train the model 141.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park’s known disclosure of image recognition, with Tarrant’s known teaching of remedying security vulnerabilities and vulnerability modeling, in order to yield predictable results of intentionally adding a slight alteration of an image—such as adding noise, rotating, translating, or changing brightness in order to evaluate, test, or improve the robustness of machine learning models.
Claim(s) 5, 14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Park (US 20240020953 A1), in view of Tarrant (US 20220100868 A1) and further in view of Barbalho (US 20230132330 A1).
Regarding claims 5 and 15, Park implicitly discloses wherein the vulnerability score indicates an object recognition accuracy of at least one object associated with the image data obtained by the sensor part ([0052] If d.sub.i is a negative candidate, the loss function L(d.sub.i) may include a (focal) binary cross-entropy loss—pulling an objectness score to zero. If d.sub.i is a positive candidate, the loss function L(g.sub.i+d.sub.i) may capture classification and regression losses.).
Park and Tarrant do not explicitly disclose, but in a similar field of endeavor of adversarial image generator, Barbalho teaches wherein the vulnerability score indicates an object recognition accuracy of at least one object associated with the image data obtained by the sensor part ([0016] Advantageously, an embodiment may help to reduce or eliminate the vulnerability of an image segmentation ML model to attacks involving image perturbations. Advantageously, an embodiment may help to reduce or eliminate the likelihood that the operation of an AV (autonomous vehicle) will be compromised by the introduction of image perturbations. Various other advantageous aspects of some example embodiments will be apparent from this disclosure.
[0059] All the images in the test set were re-scaled to 512×1024 format. Note that the complexity of the attack increases with the size of the image. The experiments were run on a Dell PowerEdge C4140 server, using Ubuntu and a Nvidia Tesla V100 GPU. Three metrics were used to evaluate the model and its robustness against attacks: (i) accuracy, before and after the attack; (ii) targeted class accuracy, to measure the effectiveness of targeted attacks; (iii) and, image dissimilarity to measure the number of pixels changed as a result of the attack.
[0061] The experimental results show that the model accuracy decreases to a large extent when traditional FGSM is used. The same does not happen when the optimization process, according to some example embodiments, is applied to the FGSM framework. Rather, Opt-FGSM shows a slight decrease in the model accuracy, from 0.762579 (no attack) to 0.738837 (Opt-FGSM) which means that the attack seems to affect only the targeted class, vehicles in this example. The following metrics, see the table in FIG. 6, confirm this. For example, the accuracy of the target class shows that all attacks are effective, reducing the accuracy of cars from 0.271 to 0.023525 and 0.01395.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park and Tarrant’s known disclosure of image recognition remedying security vulnerabilities and vulnerability modeling, with Barbalho’s teaching of an object recognition vulnerability score, in order to determine whether the image received from the autonomous vehicle is adversarial (abstract of Barbalho).
Regarding claim 17, Park and Tarrant do not disclose but Barbalho teaches based on a quantity of pieces of logging data stored in a learning database (DB) cloud satisfying a quantity threshold, retraining the image recognition model and the vulnerability assessment model ([0035] For the targeted version of FGSM attack, the goal is to fool the model to misclassify the image as a specific class (target class). FIG. 2 shows how the gradient is used in this case. Given the DNN classification f(x;θ), the target classification y.sub.t, and a loss function L, the gradient of the loss is computed with respect to the input x. This gradient indicates in which direction each pixel of the image should be changed to maximize the loss, but here the goal is to minimize the loss, that is, to make the classification f(x;θ) equal to the target class y.sub.t. Thus, the optimization takes the opposite direction of the gradient. To avoid a large perturbation, such as may be perceptible by a human, only the sign of the gradient is taken into consideration and is multiplied by a maximum perturbance degree e.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to combine Park and Tarrant’s known disclosure of image recognition remedying security vulnerabilities and vulnerability modeling, with Barbalho’s teaching of an object recognition vulnerability score, in order to determine whether the image received from the autonomous vehicle is adversarial (abstract of Barbalho).
Allowable Subject Matter
Claims 6 and 15 (and dependent claims 7 and 16) objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Prior art does not teach or disclose the unique combinations of derive a probability density function with the vulnerability score as an x-axis for each object recognition result which is a false positive (FP), a false negative (FN), or a true positive (TP) for the extracted feature map; and
determine, based on the derived probability density function, the threshold as a logging threshold.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230115927 A1 to claim 5: [0275] In some embodiments, the computing device 300 may determine a vulnerability probability value of the plaque by inputting feature quantification information corresponding to the plurality of images into a probability determination model, and determine the detection result of plaque stability based on the vulnerability probability value.
US 20230252157 A1 to claim 8 and 17:
[0046] FIG. 2 depicts a simplified flowchart 200 depicting processing performed by a vulnerability assessment system (VAS) according to certain embodiments. The processing depicted in FIG. 2 may be implemented in software (e.g., code, instructions, program) executed by one or more processing units (e.g., processors, cores) of the respective systems, in hardware, or combinations thereof. The software may be stored on a non-transitory computer-readable storage medium (e.g., on a memory device).
[0133] In certain use cases, VAS 110 (or some other downstream consumer of vulnerability scores for container images) may determine whether or not to perform or initiate an action by comparing the vulnerability score computed for a container image in 210 to a preconfigured threshold value. VAS 110 may decide to perform/initiate an action only if the computed vulnerability score for the container image exceeds the threshold value.
US 20170205405 A1 to claims 6 and 7: [0146] This is a graph of the true positive (or sensitivity) rate on the y-axis and the true negative (specificity 1) rate on the x-axis. The true positive rate (TPR) is the probability that a case of disease is correctly classified and the true negative rate (TNR) is the probability that a normal true case is correctly classified. The ROC curve can also be used to compare the performance of two or more diagnostic tests.sup.7,8. An alternative to the ROC curve is the detection error tradeoff (DET) graph, representing the false negative rate (detections that are missed) with respect to the false positive rate (false alarms) on the logarithmic x and y axes. This alternative takes up a larger graph area in the region of interest, i.e., the region with the minimum false rate. The DET graph is made by superimposing a normal distribution (probability density function) determined by the experimentally obtained mean value and standard deviation.
[0146] The sensitivity and specificity of a diagnostic test are a function of a chosen threshold value. Changing the threshold value such that sensitivity increases will decrease specificity, and vice versa. The receiver operating characteristic (ROC) curve is a graph of all the sensitivity/specificity pairs resulting from continuously changing the decision threshold with respect to the complex range of results observed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675