DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 30th, 2026, has been entered.
Response to Amendment
Claims 1-12 in the claim set filed November 17th, 2025, were pending for examination in the Application No. 18/144,919 filed May 9th, 2023. In the remarks and amendments received on June 30th, 2026, no claims are amended. Accordingly, claims 1-12 are currently pending for examination in the application.
Response to Arguments
Applicant’s arguments filed June 30th, 2026, regarding the rejection(s) of the independent claims have been fully considered but are not persuasive.
The examiner respectfully disagrees with Applicant’s assertion that Padfield does not disclose the present invention because Padfield “teaches ignoring corresponding to ‘unknown category’ and particularly states that ‘If the label value for a given class is set to the unknown value, the indicator function identifies this and the loss function 260 will not use that information to supervise learning or otherwise impact the loss function’” (pg. 5 of Applicant’s Remarks).
A limitation of claim 1 recites that “the first model comprises a loss function which penalizes prediction of additional labeled defects in the negative images and does not penalize prediction of additional labeled defects in the positive images” (emphasis added). The statement of Padfield as remarked by Applicant above teaches not penalizing the prediction of additional labeled defects in positive images as recited in the claim 1 limitation above. As detailed in the previous rejection and in the currently set forth rejection below, lines 17-47 of col. 3 of Padfield disclose that ‘unknown’ labels is a “new value for defects with an unknown ground truth value”, where the “defects with an unknown ground truth” are additional labeled defects predicted by the training model that were initially “unnoticed or unmarked” in the ground truth data. Therefore, the loss function of “model 200A” which does not penalize ‘unknown’ labels on images from training data as remarked in lines 24-28 of col. 8 of Padfield discloses the aspect of the present invention of “not penaliz[ing] prediction of additional labeled defects in the positive images” as recited in claim 1 of Applicant’s claims (see the rejection of claim 1 limitation “…wherein the first model comprises a loss function which penalizes…” below).
The examiner respectfully disagrees with Applicant’s assertion that Padfield dose not disclose the claim 1 limitation of “…wherein the first model comprises a loss function which penalizes prediction of additional labeled defects in the negative images” because “even if we correlate the aspect of detection of false ‘unknown labels’ of Padfield et. al., with the aspect of ‘prediction of additional labelled defects in the negative images’ as disclosed in the present invention[,] [t]he loss function in Padfield et. al., as described in the model 200B above is simply imposing penalty on every prediction of ‘unknown’ label (or additional labelled defects) irrespective of the fact whether they are predicted in a positive image or a negative image” (pg. 5 of Applicant’s Remarks).
The examiner appreciates Applicant’s remark that “model 200B” of Padfield “impos[es] penalty on every prediction of ‘unknown’ label (or additional labelled defects) irrespective of the fact whether they are predicted in a positive image or a negative image”; however, the examiner relies on the embodiment of “model 200A” of Padfield and not “model 200B” of Padfield as remarked by Applicant to disclose the above claim 1 limitation. As detailed in the previous rejection and the currently set forth rejection below, the embodiment of “model 200A” and its loss function are considered “the first model compris[ing] a loss function” recited in the claim 1 limitation above.
Further, although Padfield includes teachings for using “model 200B”, Padfield does not discourage the use of “model 200A”. For example, the examiner would like to bring to Applicant’s attention that the “model 200B” of Padfield (which includes penalties for predictions of labeled defects as remarked by Applicant) is only one of the multiple models disclosed and used in Applicant's disclosure. The model “200A” (which does not penalize prediction of additional labeled defects in positive images) is the model being relied upon in the rejection of claim 1 of Applicant’s claims. As recited in lines 47-56 of col. 13 of Padfield, any of models 200A, 200B can be trained using the training method set forth in Padfield's disclosure. Padfield further discloses their machine learning system can consist of any of the models 200A, 200B in their machine learning system as disclosed in lines 34-52 of col. 14. Furthermore, Padfield claims said model not comprising the small penalty within their claims, such as claim 5 of Padfield recites “the loss function prevents the machine learning model from being penalized for making an erroneous prediction for a defect type having an unknown label”. Therefore, although Padfield discloses an alternative “model 200B” comprising a small penalty for "unknown" labels, Padfield does not discourage the use of a “model 200A” not comprising the small penalty as it is one of the model options in the machine learning system of Padfield.
The examiner respectfully disagrees with Applicant that “none of Padfield, Bak[h]shmand or Ranson teach the aspects of” the claim 1 limitations of (A) “outputting, by the processor (142), the training images with the predicted one or more of the labeled defects and the one or more additional defects to the client device for human review”; (B) “receiving, by the processor (142), inputs on the one or more additional defects from the client device based on the human review”; and (C) “using, by the processor (142), the training images with the predicted one or more of the labeled defects and with the inputs on the one or more additional defects to train the supervised learning algorithm to create a second model for object detection” (pg. 5 of Applicant’s Remarks).
As detailed in the current rejection below, lines 47-56 of col. 13 and lines 22-27 of col. 17 of Padfield teaches the claim 1 limitation (A) of outputting training images (e.g., “training data set”) with predicted one or more labeled defects (e.g., “pre-selected identified defects”) and one or more additional defects (e.g., “non-selected defect[s]”) for human review (e.g., “user feedback”); where lines 12-28 of col. 17 of Padfield further teaches outputting the predictions to the client device for human review (see the rejection of claim 1 limitation (A) in the rejection of claim 1 below).
As detailed in the current rejection below, lines 29-34 of col. 17 of Padfield teaches the claim 1 limitation (B) of receiving inputs (e.g., “changes made by the user”) on the one or more additional defects (e.g., “non-selected defect”) from the client device based on the human review (e.g., “user feedback”) (see the rejection of claim 1 limitation (B) in the rejection of claim 1 below).
As detailed in the current rejection below, lines 47-56 of col. 13 of Padfield discloses the claim 1 limitation (C) of using the training images (e.g., “training data set”) with the predicted one or more of the labeled defects (e.g., “predictions and the labels”) and with the inputs received (e.g., “user feedback”) to train the supervised learning algorithm (e.g., “training” using “annotations provided by the human reviewer”) to create a second model for object detection (e.g., an “updated” model “200A”); wherein para(s). [0263-0265] of Bakhshmand teaches in the same field of endeavor of training a defect detection model predicting both one or more labeled defects and one or more additional defects in training images that the received inputs used to train the defect detection model include inputs on one or more additional defects (e.g., “new defect type”) based on the human review (e.g., “label or tag” the “new defect type” by the “user”) (see the rejection of claim 1 limitation (C) in the rejection of claim 1 below).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-7, and 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Padfield et al. (Padfield; US 11,250,552 B1) in view of Bakhshmand et al. (Bakhshmand; US 2024/0160194 A1).
Regarding claim 1, Padfield discloses a method for training in object detection (lines 40-44 of col. 5, recite(s)
[lines 40-44 of col. 5] “Using this partial information, the disclosed techniques can build a model that can predict information corresponding to all the defects. Specifically, the disclosed techniques can train a network based on a collection of annotated images…”
, where “defects” are objects) comprising:
receiving, by a processor (142), from a client device (lines 64-67 of col. 13 to lines 1-7 of col. 14, lines 19-26 of col. 14, and lines 53-56 of col. 14, recite(s)
[lines 64-67 of col. 13 to lines 1-7 of col. 14] “The interactive computing system 306 can communicate over network 304 with user devices 302. The network 304 can include any appropriate network, including an intranet, the Internet, a cellular network, a local area network or any other such network or combination thereof. User devices 302 can include any network-equipped computing device, for example desktop computers, laptops, smartphones, tablets, e-readers, gaming consoles, and the like. Users can access the interactive computing system 306 and interact with items therein via the network 304 and can be provided with recommendations via the network 304.”
[lines 19-26 of col. 14] “The interactive computing system 306 may include at least one memory 310 and one or more processing units (or processor(s)) 320. The memory 310 may include more than one memory and may be distributed throughout the interactive computing system 306. The memory 310 may store program instructions that are loadable and executable on the processor(s) 320 as well as data generated during the execution of these programs…”
[lines 53-56 of col. 14] “The processor 320 may include one or more general purpose computers, dedicated microprocessors, graphics processors, or other processing devices capable of communicating electronic information…”
, where “user devices” are client devices), training images comprising positive images with defects and negative images with no defects (lines 62-67 of col. 4 to lines 1-9 of col. 5, lines 10-21 of col. 9, and lines 4-8 of col. 16, recite(s)
[lines 62-67 of col. 4 to lines 1-9 of col. 5] “…The disclosed techniques can train a deep neural network that can generate information about the presence and/or absence of all the defects based on the images and partially labelled information from human annotators. With respect to this labelled information, one implementation of the review process that generates the labels is as follows. Formally, for the jth image Ij, human annotators can ‘accept’ the image (da j=1) or mark the presence of one or more image defects dl j∈{0,1}, Σldj l≤m, for a total of m defects. An image marked ‘accept’ indicates the absence of all defects. …”
[lines 10-21 of col. 9] “Furthermore, this framework can weight individual defect losses using wl and positive and negative cases using wp and wn respectively. Weighting the positive and negative losses helps in imbalanced cases where a training data set may have very few examples of a particular defect compared to the total number of acceptable images. Equation 3 above may be unchanged by images the label d=−1 and using the indicator function in Equation 2. By using the indicator function to identify unknown ground truth labels, the loss function can ignore the predictions corresponding to ‘unknown’ category and generates individual defect classification using a weighted cross-entropy loss. …”
[lines 4-8 of col. 16] “At block 405, the machine learning system 200 accesses training data including specific images, and either (1) any rejections annotated for a given image by a human reviewer, or (2) an indication that a human reviewer accepted the image. …”
, where “training data” are training images comprising positive images with defects (e.g., “positive” cases which are images labeled or marked with “the presence of one or more image defects”) and negative images with no defects (e.g., “negative” cases which are images that are “accepted” indicating “the absence of all defects”));
using, by the processor (142), the training images to train a supervised learning algorithm to create a first model for object detection (lines 62-67 of col. 4 to lines 1-9 of col. 5 and lines 10-21 of col. 9—see citations immediately above—, where lines 23-28 of col. 8 further recite(s):
[lines 23-28 of col. 8] “The model 200A can be trained using a learning framework that minimizes a loss function 260 to reduce the discrepancy between the output of the neural network 210 and the information from the human associate decision while not penalizing the ‘unknown’ labels on images from training data. …”
, where the training the “model 200A” by minimizing a loss function based on the labeled images (e.g., “positive and negative cases”) is training a supervised learning algorithm to create a first model (e.g., “model 200A”));
inputting, by the processor (142), the training images without labeled defects to the first model, which predicts in the training images one or more of the labeled defects and one or more additional defects (lines 23-28 of col. 8—see citation above—, where lines 47-56 of col. 13 and lines 14-26 of col. 16 further recite(s):
[lines 47-56 of col. 13] “In some embodiments, the images in the training data set can be forward passed through any of the models 200A, 200B, after training. The predictions output by the model can be compared to the annotations provided by the human reviewer. If there are discrepancies between the predictions and the labels, this can be considered as an error and another reviewer can determine the correct labeling (e.g., the correct ground truth labeling). If the image was incorrectly labeled, the training phase may then be updated using the correct ground truth labeling for this image (or a number of images).”
[lines 14-26 of col. 16] “At block [4]10, the machine learning system 200 applies at least three label values to each class in the task set for classification. For example, as described in conjunction with FIG. 1B, a positive ground truth (e.g., a defect is present, accept this image) can be labeled with the value “1.” A negative ground truth (e.g., no defects are present in an accepted image, no acceptance of an image with defect(s)) can be labeled with the value “0.” An unknown ground truth (e.g., no specification either way regarding a particular defect) can be labeled with the value “−1.” These are exemplary values used in the disclosed implementation, and other implementations may use other values while achieving a similar effect.”
, where “forward pass[ing]” the “training data set” through at least the object detection model (e.g., “model 200A”) is inputting the training images without the labeled defects to the first model, which predicts (e.g., “predictions output by the model”) one or more labeled defects (e.g., the “positive ground truth” label of value “1”) and one or more additional defects (e.g., “unknown ground truth” label of value “-1”)), wherein the first model comprises a loss function which penalizes prediction of additional labeled defects in the negative images and does not penalize prediction of additional labeled defects in the positive images (lines 17-47 of col. 3, lines 24-28 of col. 8, and lines 37-53 of col. 16, recite(s)
[lines 17-47 of col. 3] “The disclosed training techniques overcome the challenge of partial information by labeling known (user-identified) defects as positive cases, not labeling other defects (e.g., those for which no user feedback has been received) as negative cases, and instead using a new label value for defects with an unknown ground truth. This avoids mislabeling a class as not present, when in fact the class may be present but was just unnoticed or unmarked by the user. In addition, the disclosed training techniques use a novel loss function that accounts for this unknown nature of some classes when it determines how to update the machine learning model's parameters. The loss function measures how different a prediction made by the machine learning model is from its expected value, and then this difference is used to tune the model parameters such that it can make a more accurate prediction. However, in the scenario of partial information, the true expected value is not known. The disclosed loss function checks to determine whether a certain class has an unknown ground truth label. If so, the loss function may not use any information relating to that class to supervise the learning process. As such, the machine learning model is not penalized during training for making “mistakes” with respect to classes having an unknown ground truth. Because the overall training set spans many images with different defects, it should include information regarding each of the image classes in question. By training across a large data set and accounting for unknown ground truth labels, the machine learning model is thus able to use many images with partial information to learn to make “complete” classifications across the entire class set with respect to new images.”
[lines 24-28 of col. 8] “The model 200A can be trained using a learning framework that minimizes a loss function 260 to reduce the discrepancy between the output of the neural network 210 and the information from the human associate decision while not penalizing the ‘unknown’ labels on images from training data. …”
[lines 37-53 of col. 16] “At block 425, the machine learning system 200 can determine updates to the weight of the neural network 210 in order to make more accurate predictions. These updates can be based on the values of the predictions compared to the corresponding values of the labels for known positive and negative ground truth labels (e.g., the predicted likelihood for “logo” compared to the ground truth label for “logo”, etc. for each task). As described above, the loss function for training the network includes an indicator function that checks for label values indicating an unknown ground truth. The deviations between the prediction and these labels may not contribute to the weight update (models 200A, 200B) or may contribute little to the weight update (model 200B). Further, as described with respect to the model 200B, some implementations of block 425 can involve regularization to impose sparsity on the predictions of the model 200B.”
, where the “loss function” includes comparing the predictions to “corresponding values of labels for known positive and negative ground truth labels” is at least penalizing prediction of additional labeled defects in the negative images; and the “loss function” includes “not penalizing the ‘unknown’ labels on images from training data” is at least not penalizing prediction of additional labeled defects in the positive images);
outputting, by the processor (142), the training images with the predicted one or more of the labeled defects and the one or more additional defects(lines 47-56 of col. 13 and lines 22-27 of col. 17, recite(s)
[lines 47-56 of col. 13] “…the images in the training data set can be forward passed through any of the models 200A, 200B, after training. The predictions output by the model can be compared to the annotations provided by the human reviewer. If there are discrepancies between the predictions and the labels, this can be considered as an error and another reviewer can determine the correct labeling (e.g., the correct ground truth labeling). If the image was incorrectly labeled, the training phase may then be updated using the correct ground truth labeling for this image (or a number of images).”
[lines 22-27 of col. 17] “At optional block 455, the machine learning system 200 can receive user feedback regarding the classes depicted in the image. This can involve, for example, the user interface manager 318 logging any changes made by the user to pre-selected identified defects, non-selected defect, or any entries made by the user with respect to identified defects.”
, where the “predictions output by the model” can be sent to “another reviewer” is outputting the training images with the predictions for human review; and “pre-selected identified defects” are at least one or more labeled defects and “non-selected defect[s]” are at least one or more additional defects);
receiving, by the processor (142), inputs(lines 47-56 of col. 13 and lines 22-27 of col. 17—see preceding citation above—, where the “correct ground truth labeling” is a received input on the one or more additional defects based on the human review); and
using, by the processor (142), the training images with the predicted one or more of the labeled defects and with the inputs received(lines 47-56 of col. 13 and lines 22-27 of col. 17—see citation above—, where “updat[ing]” the “training phase” using the training images with the predictions and the inputs received is training the supervised learning algorithm to create a second model for object detection (e.g., an updated first model—i.e., model 200A—is a second model)).
Where Padfield does not specifically disclose
outputting, by the processor (142), the …images with the predicted one or more of the labeled defects and the one or more additional defects to the client device for human review; and
receiving, by the processor (142), inputs on the one or more additional defects from the client device based on the human review;
Padfield further teaches in a further embodiment
outputting, by the processor (142), …images with the predicted one or more of the labeled defects and the one or more additional defects to the client device for human review (lines 12-28 of col. 17, recite(s)
[lines 12-28 of col. 17] “At optional block 450, the user interface manager 318 can pre-populate a user interface with an indication of the predicted depicted classes. This user interface may be shown to a human reviewer during the image review process described herein, and one example is depicted in FIG. 6. Beneficially, pre-populating the user interface with this prediction of depicted defects can increase the speed with which human reviewers can successfully review and removed effective images, can increase the accuracy of the review process by revealing potential defects that the user may not have noticed (see example 510 in FIG. 5), and also provides the user opportunity to correct any erroneous predictions made by the model by deselecting the corresponding defect. Pre-populating the user interface can include pre-selecting identified defects, while leaving non-identified defects unselected, or listing a textual description of the identified defects”
, where the “user interface” is a client device; and the outputted images with “predicted depicted classes” are images with the predicted one or more of the labeled defects and the one or more additional defects); and
receiving, by the processor (142), inputs on the one or more additional defects from the client device based on the human review (lines 29-34 of col. 17, recite(s)
[lines 29-34 of col. 17] “At optional block 455, the machine learning system 200 can receive user feedback regarding the classes depicted in the image. This can involve, for example, the user interface manager 318 logging any changes made by the user to pre-selected identified defects, non-selected defect, or any entries made by the user with respect to identified defects.”
, where the “user feedback” are received inputs including inputs on the one or more additional defects from the client device (e.g., “potential defects that the user may not have noticed” as recited in para(s). above)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Padfield to incorporate outputting the training images with the predicted one or more labeled defects and the one or more additional defects for human review and receiving inputs on the one or more additional defects from the client device based on the human review to improve updating the training phase of the first model to train the supervised learning algorithm to create a second model for object detection by correcting any erroneous predictions made by the model or add potential defects that were not previously labeled in the training images.
Where Padfield does not specifically disclose
using …the training images with the predicted one or more of the labeled defects and with the inputs received on the one or more additional defects to train the supervised learning algorithm to create a second model for object detection;
Bakhshmand teaches in the same field of endeavor of a defect detection model predicting both one or more labeled defects and one or more additional defects in training images
using …the training images with the predicted one or more of the labeled defects and with the inputs received on the one or more additional defects to train the supervised learning algorithm to create a second model for object detection (para(s). [0263-0265], recite(s)
[0263] “Differences between the golden sample and object detection outputs may be flagged by the anomaly detection module 540 and presented to a user (e.g. expert) for review. This may include presentation of differences via an annotated inspection image including the differences in a user interface displayed on a user device. Items that are present in the golden sample output and not in the object detection output may be either a new defect type (i.e. a defect type not currently detected by the object detector model 508) or an anomaly (a non-defect variation or artifact).”
[0264] “Items detected by the golden sample module 526 but not by the object detection module 504 (that is, differences between the two outputs) may be presented in a user interface. For example, the user interface may present an annotated version of the inspection image 502, or a portion thereof, which includes details on differences and request a user input to identify the new item (i.e. difference) as a new defect type or an anomaly. The user can view the annotated inspection image in the user interface and provide input data identifying the item as an anomaly of new defect. In the case of a new defect type, the user interface may be configured present a user interface element for receiving user input data comprising a defect class label indicating the defect type.”
[0265] “The anomaly detection module 540 may then label or tag the new detected item as an anomaly or new defect type based on the input data received from the user and store this information in the system. This may include a unique anomaly identifier for an anomaly or a new defect identifier for a new defect. Annotated inspection images that contain new defect types may also be identified or tagged as training samples (which may include label information such as a new defect class). The training samples can be used as training data for further training of the object detection model 508 and/or defect classification model 522 such that the models 508, 522 can be updated to detect and/or classify the new defect type. Such updating of models 508, 522 may occur regularly. The anomaly detection module 540 is configured to send inspection images which have been identified as including a new defect type to a training database for use in subsequent training.”
, where the “object detection outputs” are training images with predicted one or more labeled defects and the “input data identifying the item as an anomaly of new defect” is inputs received on the one or more additional defects (e.g., “new” defects) to train a supervised learning algorithm to create a second model for object detection (e.g., an updated model: “further training of the object detection model 508 and/or defect classification model 522 such that the models 508, 522 can be updated to detect and/or classify the new defect type”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Padfield to incorporate using the training images with the predicted one or more of the labeled defects and with the inputs received on the one or more additional defects to train the supervised learning model to create a second model for object detection to update the first model to be able to detect the new defect type as taught by Bakhshmand above (para(s). [0265]—see citation above).
Regarding claim 3, Padfield in view of Bakhshmand discloses the method as claimed in claim 1, wherein Padfield further discloses indicators to accept, reject, or modify the one or more additional defects are output to the client device (lines 12-28 of col. 17—see citation in claim 1 limitation “outputting, by the processor (142), …images with…” above—, where the user interface depicts “pre-select[ed] identified defects” and “unselected” identified defects” to which a user “correct[s] any erroneous predictions made by the model” such as by “deselecting the corresponding defect” is providing indicators to accept, reject, or modify the one or more additional defects output to the client device).
Regarding claim 4, Padfield in view of Bakhshmand discloses the method as claimed in claim 3, wherein Padfield further discloses the received inputs comprise one or more actions of accepting, rejecting, or modifying the one or more additional defects from the client device (lines 12-28 of col. 17—see citation in claim 1 limitation “outputting, by the processor (142), …images with…” above—, where “correct[ing] any erroneous predictions made by the model” includes “deselecting the corresponding defect” is at least rejecting or modifying the one or more additional defects output to the client device).
Regarding claim 5, Padfield in view of Bakhshmand discloses the method as claimed in claim 1, wherein Padfield further discloses the first model is a classifier which predicts in the training images, one or more of the labeled defects and one or more additional defects (lines 14-26 of col. 16—see citation in claim 1 limitation “inputting, by the processor (142), the training images without…” above—, where the first model predicts at least “three label values” for classification is the first model being a classifier which predicts in the training images one or more of the labeled defects (e.g., “positive ground truth”) and one or more additional defects (e.g., “unknown ground truth”)).
Regarding claim 6, Padfield in view of Bakhshmand discloses the method as claimed in claim 1, wherein Padfield further discloses the first model is a convolutional neural network (lines 52-55 of col. 6, recite(s)
[lines 52-55 of col. 6] “For example, the artificial neural network 210 may be a convolutional neural network (“CNN”). A CNN is a type of artificial neural network that is commonly used for image analysis…”
).
Regarding claim 7, the claim differs from claim 1 in that the claim is in the form of a system comprising: a memory (144), and at least one processor (142) communicatively coupled to the memory (144), wherein the at least processor (142) is configured to execute instructions stored in the memory to: perform the method of claim 1. Padfield discloses said memory (144) and at least one processor (142) (lines 19-26 of col. 14—see citation in claim 1 limitation “receiving, by a processor (142), from a client device” above). Therefore, claim 7 recites similar limitations to claim 1 and is rejected for similar rationale and reasoning (see the analysis for claim 1 above).
Regarding claim 9, the claim recites similar limitations to claim 3 and is rejected for similar rationale and reasoning (see the analysis for claim 3 above).
Regarding claim 10, the claim recites similar limitations to claim 4 and is rejected for similar rationale and reasoning (see the analysis for claim 4 above).
Regarding claim 11, the claim recites similar limitations to claim 5 and is rejected for similar rationale and reasoning (see the analysis for claim 5 above).
Regarding claim 12, the claim recites similar limitations to claim 6 and is rejected for similar rationale and reasoning (see the analysis for claim 6 above).
Claims 2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Padfield in view of Bakhshmand as applied to claim(s) 1 and 7 above, and further in view of Ranson et al. (Ranson; US 2024/0119581 A1).
Regarding claim 2, Padfield in view of Bakhshmand discloses the method as claimed in claim 1, wherein Ranson teaches in the same field of endeavor of defect detection and human review using a client device the defects in the positive images are labeled as bounding boxes by a human reviewer at the client device (para(s). [0048], [0070], and [0073], recite(s)
[0048] “The defect detection algorithm used by the model can include object detection architecture that accepts bitmap images as input and attempts to identify the presence of a certain set of predefined objects within the image. In the implementation of the system, the objects identified can include specific installation defects. The algorithm can consist of a network (e.g., a neural network) of mathematical operations performed on the pixel values of the input image that are defined by adjustable parameters (e.g., weights). These operations can produce a set of numbers that define information related to object(s) detected within the image, including, but not limited to: (i) the position(s) of a bounding box describing the location(s) of the detected object(s), (ii) the probability that a given bounding box contains an object, (iii) and the probabilities of a detected object belonging to any of the possible predefined object classes (e.g., defect types).”
[0070] “…The model can be trained by consuming images and annotated bounding boxes of defects in the images to learn to predict bounding boxes and the associated class of defects on new, previously unseen images. The model can be updated once enough newly validated data from the reviewer becomes available. The model can be continuously updated based on newly input images/media and/or feedback from a reviewer when the confidence level is below a threshold value. Feedback from the end user can also be used by the system 470 to update the model, thereby continuously improving operation of the machine learning and/or AI based model.”
[0073] “At step 480, if additional review of the analyzed image is needed based on a low confidence level value, the detection system passes an image highlighting or otherwise labeling portions where additional review is needed. The reviewer can either confirm or reject the suggested low confidence predicted defects. Images that require additional review can be transmitted to a user interface that allows a reviewer to electronically adjust the location/size of predicted bounding boxes generated by the system, add or remove bounding boxes, adjust the category of defect for each bounding box, and/or then return the validated image to the database for further use. The system can combine the reviewer and model based defect detection results and transmits the results back to the end user. Validated images can be used to both retrain the defect detection model and to provide feedback to end users. Validated images can be used by the system to retrain the model and continuously improve performance.”
, where the “user interface” is a client device and the “input images” with defects labeled as “bounding boxes” is labeling defects in positive images with bounding boxes).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Padfield in view of Bakhshmand to incorporate labeling the defects in the positive images as bounding boxes by a human reviewer at a client device to improve identification and labeling of defects in positive images by further including position and size of the defects in the images.
Regarding claim 8, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA Z YAO whose telephone number is (571)272-2870. The examiner can normally be reached Monday - Friday (8:30AM - 5PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.Z.Y./Examiner, Art Unit 2666
/MING Y HON/Primary Examiner, Art Unit 2666
