Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 9/24/2025 has been entered and made of record. Claims 1-6, 8-18, and 20-21 remain pending in this application.
Response to Arguments
Applicant’s arguments with respect to the rejection of claims 1-6, 8-18, and 20-21 under 35 U.S.C. § 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of Iwami (US 2021/0019886 A1) as included in the IDS filed 9/24/2025. Further discussions are addressed in the rejection as set forth below. Because the reference being cited was provided in the IDS by the applicant, the action is made FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-8, 13-14, and 17-21 are rejected under 35 U.S.C. 103 as being unpatentable over Goodwin et al (US 2024/0095983 A1) in view of Iwami (US 2021/0019886 A1), as provided by the applicant in the IDS filed 9/24/25.
Regarding claim 1, Goodwin teaches a method comprising:
receiving an image of a drug product package that contains one or more drug products therein (Goodwin Par. [0071-0073], sampled images depict syringes, vials, or cartridges containing drug products);
detecting, using an artificial intelligence engine that is trained on a plurality of drug product packaging system (Goodwin Par. [0135], [0137], [0141], deep-learning convolution model generates realistic synthetic images by learning features that match the appearances of the original images; see also [0058] and [0064], a plurality of types of drug container images and features thereof are sampled and used to train the neural networks for generating modified images), characteristics of the image that are associated with a first drug product packaging system of the plurality of drug product packaging systems (Goodwin Par. [0081-0082], pixel intensity areas associated with the drug container in the image are detected and analyzed; see also [0059-0063], a plurality of types of drug container images and features thereof are sampled); and
generating a modified image of the drug product package based on the characteristics of the first drug product packaging system (Goodwin Par. [0095], realistic synthetic drug container images are generated using the algorithm of Figure 4A; algorithm places a defect on an original drug container image by detecting pixel intensity values around the desired placement location).
However, Goodwin does not specifically teach where Iwami teaches detecting specific characteristics of the image that are associated with a first drug product packaging system and distinguish the first drug product packaging system from other ones of the plurality of drug product packaging systems (Iwami Par. [0157] and Fig. 9, identifying characteristics of an image, such as identifying information on the medicine that distinguishes one packaged medicine from another).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Goodwin using Iwami’s teachings by incorporating detecting specific characteristics of the image distinguishing one item from another in order to improve the processing of different images based on the identified drug packaging system.
Regarding claim 2, Goodwin, as modified by Iwami, further teaches the method of Claim 1, wherein the characteristics of the first drug product packaging system comprise one or more image capture light source characteristics, one or more image capture surface characteristics, one or more packaging material characteristics, and/or one or more camera characteristics (Goodwin Par [0081-0082], areas are detected that correspond to glass/fluid portions, or reflective portions).
Regarding claim 5, Goodwin, as modified by Iwami, further teaches the method of Claim 2, wherein the one or more packaging material characteristics comprise packaging material transparency, packaging material shadow, labeling color, packaging material color, and/or packaging material hot spot (Goodwin Par. [0081-0082], areas are detected that correspond to glass/fluid portions, or reflective portions).
Regarding claim 6, Goodwin, as modified by Iwami, further teaches the method of Claim 2, wherein the one or more camera characteristics comprise camera number, camera position, camera resolution, and/or camera image type (Goodwin Par. [0077], matrix used for generating the modified image depends on the resolution of the image taken by the camera).
Regarding claim 8, Goodwin, as modified by Iwami, further teaches the method of Claim 1, wherein the image is a first image and the modified image is a first modified image (Goodwin, Par. [0071-0073], the method for generating modified images can be processed for a plurality of drug product packages types, i.e., for a first and second image), the method further comprising:
receiving a second image of a second drug product package that contains one or more drug products therein (Goodwin Par. [0071-0073], sampled images depict syringes, vials, or cartridges containing drug products);
detecting, using the artificial intelligence engine, characteristics of the second image that are associated with a second drug product packaging system of the plurality of drug product packaging systems (Goodwin Par. [0081-0082], pixel intensity areas associated with the drug container in the image are detected and analyzed; see also [0059], [0063], and [0093], a plurality of types of drug container images and features thereof are sampled, thus different image characteristics are associated with different drug product packaging systems) and distinguish the second drug product packaging system from the other ones of the plurality of drug product packaging systems (Iwami Par. [0157] and [0161], and Fig. 9, identifying characteristics of an image in each of the packaging bags, such as identifying information on the medicine that distinguishes one packaged medicine from another); and
generating a second modified image of the second drug product package based on the characteristics of the second drug product packaging system (Goodwin Par. [0095], realistic synthetic drug container images are generated using the algorithm of Figure 4A; algorithm places a defect on an original drug container image by detecting pixel intensity values around the desired placement location).
Regarding claims 13 and 17-21, the rationale provided in the rejection of claims 1-2 and 5-8 is incorporated herein. Further, the method of claims 1-2 and 5-8 corresponds to the system of claims 13 and 17-20 (Goodwin Par. [0052]), as well as the computer program product of claim 21 (Goodwin Claim 20, non-transitory computer-readable media), and performs the steps disclosed herein.
Claims 3 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Goodwin and Iwami, as applied to claims 2 and 14, and further in view of Brentjens et al (KR 102009721 B1, provided by Applicant’s Information Disclosure Statement – IDS, the attached English language translation is used hereinafter as the Official English language translation of this KR document).
Regarding claim 3, the combination of Goodwin and Iwami together does not disclose where Brentjens teaches the method of Claim 2, wherein the one or more light source characteristics comprise strength of an image capture light source, intensity of the image capture light source, and/or location of the image capture light source (Brentjens Par. [0058-0060], the shooting conditions of a drug image can be identified, including the brightness, contrast, and tone).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Goodwin and Iwami using Brenthens’s teachings by incorporating light source characteristics to the image characteristics in order to account for lighting differences that occur between the capturing of different drug product images (see Brentjens Par. [0007]).
Regarding claim 15, the rationale provided in the rejection of claim 3 is incorporated herein. Further, the method of claim 3 corresponds to the system of claim 15 (Goodwin Par. [0052]: computer system), and performs the steps disclosed herein.
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Goodwin and Iwami as applied to claims 2 and 14, and further in view of Amano et al (US 2014/0033644 A1).
Regarding claim 4, the combination of Goodwin and Iwami together does not disclose where Amano teaches the method of Claim 2, wherein the one or more image capture surface characteristics comprise background location and/or background color (Amano Par. [0340], when inspecting a medicine image, darker colors in the image are assumed to be a part of the background and are filtered out).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Goodwin and Iwami using Amano’s teachings by incorporating the background color as one of the image characteristics in order to exclude regions that do not help accurately inspect the imaged drug product.
Regarding claim 16, the rationale provided in the rejection of claim 4 is incorporated herein. Further, the method of claim 4 corresponds to the system of claim 16 (Goodwin Par. [0052], computer system), and performs the steps disclosed herein.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Goodwin and Iwami, in view of Byoung et al (KR 20130091422 A, the attached English language translation is used hereinafter as the Official English language translation of this KR document).
Regarding claim 9, the combination of Goodwin and Iwami together does not disclose where Byoung teaches the method of Claim 1, further comprising:
detecting labeling content on a surface of the drug product package (Byoung Par. [0042] and Figure 7(a) and Figure 8, label information is extracted from a drug package image);
wherein generating the modified image of the drug product package comprises generating a modified image of the drug product package that has the labeling content removed from the surface thereof (Byoung Par. [0042] and Figure 8, the drug package label information is extracted and filtered from the drug package image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Goodwin and Iwami using Byoung’s teachings by incorporating removing labeling content when generating the modified images in order to reduce noise when recognizing drug products in images (see Byoung Par. [0006]).
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Goodwin, Iwami, and Byoung as applied to claim 9, and further in view of Amano et al (US 2014/0033644 A1).
Regarding claim 10, the combination of Goodwin, Iwami, and Byoung together discloses teaches the method of Claim 9, wherein the artificial intelligence engine is a first artificial intelligence engine (Goodwin Par. [0135], [0137], [0141], deep-learning convolution model generates realistic synthetic images by learning features that match the appearances of the original images;) and the modified image is a first modified image (Goodwin Par. [0095], realistic synthetic drug container images are generated using the algorithm of Figure 4A;), but together fails to teach where Amano discloses the method further comprising:
receiving order information for the one or more drug products and an identifier for the drug product package (Amano Par. [0261], acquire prescription information of the drug product being imaged, including an identifier number);
detecting, using a second artificial intelligence engine, individual ones of the one or more drug products in the first modified image (Amano Par. [0259-0260], detect the individual types of medicine detected in the medicine image); and
generating a second modified image of the drug product package that includes indicia that distinguish between the individual ones of the one or more drug products and associate the one or more drug products with the order information and the identifier for the drug product package (Amano Par. [0271], a display image is created that provides information of the identified names of the drugs, and prescription information including the identifier number).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Goodwin, Iwami, and Byoung using Amano’s teachings by incorporating means for detecting and identifying the drug product in the first modified image in order to ensure that the contents of the drug product package match the acquired order information for the drug product.
Regarding claim 11, the combination of Goodwin, Iwami, Byoung, and Amano, further teaches the method of Claim 10, wherein the order information comprises names for the one or more drug products in the drug product package (Amano Par. [0261], prescription information includes the type and quantity of the medicine), the method further comprising:
identifying, using a third artificial intelligence engine, at least some of the one or more drug products in the second modified image based on the names for the one or more drug products (Amano Par. [0351], medicine types are detected by matching medicine from the prescription information with attributes associated with that medicine in the image to be inspected);
wherein the names are associated with drug product attributes in a reference database (Amano Par. [0351], medicine attributes are registered in a database for each medicine type).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Goodwin, Iwami, and Byoung, as applied to claim 9, and further in view of Wang et al (CN 111627025 A, the attached English language translation is used hereinafter as the Official English language translation of this CN document).
Regarding claim 12, the combination of Goodwin, Iwami, and Byoung together discloses the method of Claim 9, further comprising:
wherein detecting the labeling content comprises:
detecting the labeling content on the surface of the image of the drug product package (Byoung Par. [0042], Figure 7(a) and Figure 8, label information is extracted from a drug package image).
However, Goodwin, Iwami, and Byoung together does not specifically teach where Wang teaches wherein the image of the drug product package is foreground-background separated, the process comprising:
performing gamma correction on the image of the drug product package responsive to receiving the image of the drug product package to generate a gamma corrected image of the drug product package (Wang Par. [0010-0011], gamma correction is applied to an image of a bottled liquid medicine [0004]);
performing gaussian blur denoising on the gamma corrected image of the drug product package to generate a reduced noise image of the drug product package (Wang Par. [0011], gaussian filtering is also applied to denoise the image of the liquid medicine bottle); and
performing automatic image thresholding on the reduced noise image of the drug product package to generate a foreground-background separated image of the drug product package (Wang Par. [0013], iterative threshold segmentation separates an image target and background).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Goodwin, Iwami, and Byoung using Wang’s teachings by incorporating noise reduction and foreground thresholding techniques before detecting the labeling content in order to separate target noise that requires attention from the rest of the image (see Wang Par. [0030]).
Conclusion
Applicant's submission of an information disclosure statement under 37 CFR 1.97(c) with the timing fee set forth in 37 CFR 1.17(p) on 9/24/2025 prompted the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 609.04(b). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Rudolph whose telephone number is (571)272-8243. The examiner can normally be reached M-F 7:30 AM - 3:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINCENT RUDOLPH/ Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights