DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, 6, 8, 11, 12, 15, 16, 18 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gilbertson et al. (PGPUB Document No. US 2022/0327689) in view of Gershtein et al (PGPUB Document No. US 2015/0302255) in view of Lewus et al. (“Validation & Compliance: Using Risk Analysis in Process Validation”, BioPharm International-02-01-2007, Volume 20, Issue 2).
Regarding claim 1, Gilbertson teaches a computer-implemented method comprising:
Generating, by one or more processors, a transformed image from a production line image corresponding to a primary orientation (“generates a plurality of transformed training production line images” (Gilbertson: 0088, 0157). Note, the Examiner construes the transformed training production line images to be of a primary orientation. The original orientation of transformed training production line images prior to applying further transformations (cropping, rotation) correspond to having a primary orientation);
Generating, by the one or more processors, a plurality of derivative transformed images for the production line image, each corresponding to a plurality of derivative orientations from the primary orientation (“If a training set has a low number of images (e.g., less than 1000 images), data augmentation may be done to train the one or more CNN models. An image may be cropped to remove the vial, and a sliding window may be used to obtain different views of the image. Each different view can further be translated or rotated to generate more images.” (Gilbertson: 0028, 0103));
Generating, by the one or more processors and using a categorical validation machine learning model, a plurality of validation predictions for the production line image based on the transformed image and the a plurality of derivative transformed images (generates the categorical validation machine learning model based at least in part on the plurality of transformed training production line images (Gilbertson: 0104) and the images that have gone through further transformation (Gilbertson: 0103));
Generating, by the one or more processors, a validation prediction based on the validation predictions (“determining an initial validation score based at least in part on one or more outputs of the categorical validation machine learning model; determining an adjusted validation score based at least in part on adjusting the initial validation score using a false positive penalty weight and a false negative penalty weight; and determining the validation prediction based at least in part on the adjusted validation score.” (Gilbertson: 0042));
And initiating, by the one or more processors, performance of a prediction-based action based on the validation prediction (“the predictive data analysis computing entity 106 performs one or more prediction-based actions based at least in part on the validation prediction” (Gilbertson: 0119)).
As stated in the rejection above, Gilbertson teaches determining scores to the validation prediction with regards to whether a production line image (transformed (Gilbertson: 0157) and further modified images (Gilbertson: 0103)) corresponds to a validation category. Gershtein teaches a similar classification method for identifying pills using a classification algorithm. Wherein said process corresponds to aggregating the validation predictions, as the process involves examining scores of multiple images to determine the most likely pill category/type (Gershtein: 0028).
The teachings of both Gilbertson and Gershtein were known in the art to applying image analysis to for the purpose of accurately classifying target objects.
The validation prediction steps of Gilbertson could have been substituted with the classification method of Gershtein.
The results would have been predictable and resulted in equally classifying target objects within images. Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Further, the combined teachings above do not expressly teach but Lewus teaches a prediction-based action that is selected based on the aggregate validation prediction and a plurality of action thresholds (Lewus teaches the concept of utilizing a plurality of action thresholds (Lewus: THE INITIATION PHASE). A hierarchical validation scheme is implemented. Operations above the threshold falls under the scope of the process validation protocol, and operations that fall below the threshold undergo a secondary evaluation based on regulatory expectations or historical commitments (Lewus: RISK EVALUATION).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the risk assessment concept taught by Lewus, because this enables the use of a multi-factor validation process for effective risk assessment in the production line of the combined teachings above.
Regarding claim 2, the combined teachings teach the computer-implemented method of claim 1, wherein the production line image is reflective of a production line item associated with a target validation category (“production line image is an image of a production line item associated with a corresponding target validation category” (Gilbertson: 0104)) and the categorical validation machine learning model corresponds to the target validation category (“a categorical validation machine learning model is a machine learning model that is configured to determine whether an input production line image corresponds to a target validation category” (Gilbertson: 0105)).
Regarding claim 5, the combined teachings teach the computer-implemented method of claim 1, wherein the plurality of validation predictions for the production line image comprises (i) a first validation prediction corresponding to the transformed image (“generating a validation prediction for an input production line image” (Gilbertson: 0017)) and (ii) a second validation prediction corresponding to a first derivative transformed image of the one or more derivative transformed images (validation predictions applied to production line images that were applied one or more translation/rotation operations (Gilbertson: 0102)).
Regarding claim 6, the combined teachings teach the computer-implemented method of claim 5, wherein generating the plurality of validation predictions comprises: generating, using the categorical validation machine learning model, the first validation prediction based on the transformed image (categorical validation machine learning model based transformed training production line images (Gilbertson: 0104)); and generating, using the categorical validation machine learning model, the second validation prediction based on the first derivative transformed image (categorical validation machine learning model based transformed training production line images (Gilbertson: 0104) for trained production line images that were applied one or more translation/rotation operations (Gilbertson: 0102) ).
Regarding claim 8, the combined teachings teach the computer-implemented method of claim 1, wherein the prediction-based action comprises one of one or more production line routing actions (“As another example, the validation prediction threshold may recommend that, when a validation prediction for a production lime image with respect to a corresponding validation category falls below a particular numeric value, the one or more production line control actions for the corresponding validation category should be performed. An exemplary process control action may be triggering an exception that causes a production line item associated with a production line image to be diverted to a human inspector.” (Gilbertson: 0043)).
Claim(s) 11, 12, 15 and 16 are corresponding computing system claim(s) of claim(s) 1, 2, 5 and 6. The limitations of claim(s) 11, 12, 15 and 16 are substantially similar to the limitations of claim(s) 1, 2, 5 and 6. Therefore, it has been analyzed and rejected substantially similar to claim(s) 11, 12, 15 and 16. Note, the combined teachings above teach a computing system comprising memory (Gilbertson: 0059) and one or more processors (Gilbertson: 0057).
Claim(s) 18 and 19 are corresponding computer-readable storage media claim(s) of claim(s) 1 and 8. The limitations of claim(s) 18 and 19 are substantially similar to the limitations of claim(s) 1 and 8. Therefore, it has been analyzed and rejected substantially similar to claim(s) 18 and 19. Note, the combined teachings teach a computer-readable storage medium (Gilbertson: 0046).
Allowable Subject Matter
Claims 3, 4, 7, 9, 10, 13, 14, 17 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights