Detailed Action
1. Claims 1-15 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
3. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-13 and 15 are rejected under 35 U.S.C 101.
Regarding claim 1,
Step 1: Statutory CategoryThe claim is directed toward a machine or device.
Step 2A, Prong 1: Judicial ExceptionThe claim recites the following limitations: detecting a foreign object within a liquid and identifying the type of foreign object from a plurality of images obtained by continuously capturing a container; and outputting the detected foreign object type to a display device. These steps—visually observing a container for contaminants and evaluating the observation—are representative of mental processes and abstract ideas, which can be performed mentally by a person using a standard camera or a generic computer equipped with a camera.
Step 2A, Prong 2: Practical ApplicationThe recited judicial exception is not integrated into a practical application. Specifically, the claim lacks additional elements or meaningful limitations that would meaningfully tie the abstract idea to a specific, real-world technological context.
As for Step 2B,
As discussed step 2A prong 2 above claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Accordingly, claim 1 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 8
Step 1: Statutory Category The claim is directed toward a method . .
Claim 8 is rejected the same as claim 1 except claim 8 is directed to method claim . All the limitations of claim 8 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 8.
Regarding claim 15
Step 1: Statutory Category
The claim is directed toward a computer program claim.
Claim 15 is rejected the same as claim 1 except claim 15 is directed to program claim . All the limitations of claim 15 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 15.
Dependent claims
Regarding claim 2:
Step 1: Statutory CategoryThe claim is directed toward a machine or device
Step 2A, prong 1: The claim is a part of the judicial exception as noted above in claim 1.
Step 2A, prong 2: The claim further recites “output a moving locus image of the foreign object in a still image, to the display device.”. However, the additional elements amount to mere insignificant pre-extra solution steps of data gathering and therefore do not integrate the judicial exception into a practical application.
Accordingly, claim 2 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 9
Step 1: Statutory Category The claim is directed toward a method . .
Claim 9 is rejected the same as claim 2 except claim 9 is directed to method claim . All the limitations of claim 9 are addressed in claim 2. Thus, argument analogous to that presented above for claim 2 is applicable to claim 9..
Regarding claim 3:
Step 1: Statutory CategoryThe claim is directed toward a machine or device
Step 2A, prong 1: The claim is a part of the judicial exception as noted above in claim 1.
Step 2A, prong 2: The claim further recites “output an index indicating a part serving as a basis for detecting the foreign object in the moving locus image.”. However, the additional elements amount to mere insignificant pre-extra solution steps of data gathering and therefore do not integrate the judicial exception into a practical application.
To identify particulate foreign object (e.g., metals, fibers, or glass fragments) within a bottle, agitate the container to induce particle movement. Position the bottle in front of a camera to capture a sequence of images. By analyzing the object's trajectory on a display screen, the user can plot its motion and classify the debris type.
Therefore, a person would be able to perform the above recited forming mentally, using a camera and using generic computer which as a camera
Accordingly, claim 3 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 10
Step 1: Statutory Category The claim is directed toward a method . .
Claim 10 is rejected the same as claim 3 except claim 10 is directed to method claim . All the limitations of claim 10 are addressed in claim 3. Thus, argument analogous to that presented above for claim 3 is applicable to claim 10..
Regarding claim 4:
Step 1: Statutory CategoryThe claim is directed toward a machine or device
Step 2A, prong 1: The claim is a part of the judicial exception as noted above in claim 1.
Step 2A, prong 2: The claim further recites “the index is designated, output the moving locus image of the foreign object in a moving image to the display device.”. However, the additional elements amount to mere insignificant pre-extra solution steps of data gathering and therefore do not integrate the judicial exception into a practical application.
To identify particulate foreign object (e.g., metals, fibers, or glass fragments) within a bottle, agitate the container to induce particle movement. Position the bottle in front of a camera to capture a sequence of images. By analyzing the object's trajectory on a display screen, the user can plot its motion and classify the debris type.
Therefore, a person would be able to perform the above recited forming mentally, using a camera and using generic computer which as a camera
Accordingly, claim 4 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 11
Step 1: Statutory Category The claim is directed toward a method . .
Claim 11 is rejected the same as claim 4 except claim 11 is directed to method claim . All the limitations of claim 11 are addressed in claim 4. Thus, argument analogous to that presented above for claim 4 is applicable to claim 11..
Regarding claim 5:
Step 1: Statutory CategoryThe claim is directed toward a machine or device
Step 2A, prong 1: The claim is a part of the judicial exception as noted above in claim 1.
Step 2A, prong 2: The claim further recites “output information related to a factor of contamination of the foreign object, to the display device.”. However, the additional elements amount to mere insignificant pre-extra solution steps of data gathering and therefore do not integrate the judicial exception into a practical application.
Accordingly, claim 5 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 12
Step 1: Statutory Category The claim is directed toward a method . .
Claim 12 is rejected the same as claim 5 except claim 12 is directed to method claim . All the limitations of claim 12 are addressed in claim 5. Thus, argument analogous to that presented above for claim 5 is applicable to claim 15..
Regarding claim 6:
Step 1: Statutory CategoryThe claim is directed toward a machine or device
Step 2A, prong 1: The claim is a part of the judicial exception as noted above in claim 1.
Step 2A, prong 2: The claim further recites “output a type of a foreign object floating in the liquid in the container, to the display device.” However, the additional elements amount to mere insignificant pre-extra solution steps of data gathering and therefore do not integrate the judicial exception into a practical application.
Accordingly, claim 6 is directed to non-eligible patent subject matter and is therefore rejected.
Regarding claim 13
Step 1: Statutory Category The claim is directed toward a method . .
Claim 13 is rejected the same as claim 6 except claim 13 is directed to method claim . All the limitations of claim 13 are addressed in claim 6. Thus, argument analogous to that presented above for claim 6 is applicable to claim 13..
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
4. Claims 1-6, 8-13 and 15 rejected under 35 U.S.C. 102(a)(1) as being anticipated by Milne; et al., ( hereafter Milne ), US 9418416 B2, pub. 16/08/2016
As to claim 1, Milne teaches foreign object inspection device comprising: (Fig. 30, abstract, col. 1 lines 45-55, an apparatus for nondestructive detection of a particle (i.e., an undissolved particle) in a vessel that is at least partially filled with a fluid, such as an aqueous fluid, an emulsion, an oil, an organic solvent )
a memory containing program instructions; and a processor coupled to the memory, wherein the processor is configured to execute the program instructions (col.35 lines 7-43 , A memory operably coupled to the imager stores the time-series data, and a processor operably coupled to the memory detects and/or identifies the particle) to:
detect a foreign object existing in liquid and a type of the foreign object (col. 9, lines 60- 65, a robot 180, and a vial tray 172, which holds uninspected and/or inspected containers 10 in individual container wells. Upon instructions from a user or automatic controller (not shown), the robot 180 moves a container 10 from the vial tray 172 to an inspection module 160, which captures and records time-series data of particles moving in the container 10. ) from a plurality of images obtained by continuously capturing a container in which the liquid is contained; and (col. 10, lines 52- 55, col. 11, lines 1- 3, After
the imager 110 has acquired enough time-series data, the processor 130 subtracts background data, which may represent dirt and/or scratches on one or more of the surfaces of the container. It may also filter noise from the time-series data as understood by those of skill in the art and perform intensity thresholding as described below. ) output the type of the detected foreign object to a display device (fig. 26a, 26b, col. 32, lines 24, the particle tracking process produces a time-dependent spreadsheet, such as the one shown in FIG. 26B, that contains details of all relevant parameters, including position, velocity of movement, direction of movement, acceleration, size (e.g., two-dimensional area), size (maximum Feret diameter), elongation, sphericity, contrast, and brightness. These parameters provide a signature that can be used to classify a particle as a particular species).
Claim 8 is rejected the same as claim 1 except claim 8 is directed to method claim . All the limitations of claim 8 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 8.
As to claim 15, Milne teaches A non-transitory computer-readable medium storing thereon a program comprising instructions for causing a computer to execute processing (col.35 lines 7-43);
regarding remaining limitations of claim 15, all the remaining limitation are rejected the same as claim 1 except claim 15 is directed to a computer program claim . All the remaining limitations of claim 15 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 8.
As to claim 2, Milne teaches wherein the processor is further configured to execute the instructions to output a moving locus image of the foreign object in a still image, to the display device(Fig.26A, col 31 line53- col 32 line10, FIG. 26A shows successive frames of time-series data with both the particles and their trajectories. The roughly planar tracks represent trajectories of 100-micron polymer microspheres that mimic protein aggregates. These particles, which are almost neutrally buoyant move with the fluid and do not noticeably sink or rise. The vertically descending tracks represent the trajectories of 100-micron glass beads, which rotated with the fluid initially but sank as the sequence progressed. Rising tracks represent the trajectories of air bubbles and particles with positive buoyancy).
As to claim 3, Milne teaches wherein the processor is further configured to execute the instructions to output an index indicating a part serving as a basis for detecting the foreign object in the moving locus image(col. 10, lines 62- 64, the processor 130 also selects start and end points of the data to be analyzed as described below. The processor 130 may perform background subtraction, noise filtering, intensity thresholding, time-series data reversal, and start/end point determination in any order.) The processor 130 tracks particles moving in or with the fluid in step 212, then sizes, counts, and/or otherwise characterizes the particles based on their trajectories in step 214.)
As to claim 4, Milne teaches wherein the processor is further configured to execute the instructions to, when the index is designated, output the moving locus image of the foreign object in a moving image to the display device(Fig.26A, col 31 line53- col 32 line10, FIG. 26A shows successive frames of time-series data with both the particles and their trajectories. The roughly planar tracks represent trajectories of 100-micron polymer microspheres that mimic protein aggregates. These particles, which are almost neutrally buoyant move with the fluid and do not noticeably sink or rise. The vertically descending tracks represent the trajectories of 100-micron glass beads, which rotated with the fluid initially but sank as the sequence progressed. Rising tracks represent the trajectories of air bubbles and particles with positive buoyancy.)
As to claim 5, Milne teaches wherein the processor is further configured to execute the instructions to output information related to a factor of contamination of the foreign object, to the display device (col. 23, lines 45-59, the processor scores the MIP image by counting the number of pixels in the MIP image whose value exceeds a predetermined threshold. If the score exceeds a historical value representing the number of lamellae in a similar vessel, the processor determines that the vessel is statistically likely to contain glass lamellae. The processor may also determine the severity of lamellae contamination by estimating the number, average size, and/or size distribution of the glass lamellae from the MIP image).
As to claim 6, Milne teaches wherein the processor is further configured to execute the instructions to output a type of a foreign object floating in the liquid in the container, to the display device(Fig. 26 col23 lines 15-30, In order to create a system for differentiation based on this principle, the imager can be aligned with a vial in a typical fashion and oriented the incident lighting through the bottom of the container (orthogonal to the camera axis). This yields very little signal from particles that scatter (e.g., proteins), and a large signal from particles that reflect (e.g., glass lamellae). In other words, as the lamellae float through the vessel, they appear to flash intermittently. This technique has shown to be highly specific in differentiating lamellae particles from protein aggregates).
Claim 9 is rejected the same as claim 2 except claim 9 is directed to method claim . All the limitations of claim 9 are addressed in claim 2. Thus, argument analogous to that presented above for claim 2 is applicable to claim 9.
Claim 10 is rejected the same as claim 3 except claim10 is directed to method claim . All the limitations of claim 10 are addressed in claim 3. Thus, argument analogous to that presented above for claim 3 is applicable to claim 10.
Claim 11 is rejected the same as claim 4 except claim11 is directed to method claim . All the limitations of claim 11 are addressed in claim 4. Thus, argument analogous to that presented above for claim 4 is applicable to claim 11.
Claim 12 is rejected the same as claim 5 except claim12 is directed to method claim . All the limitations of claim 12 are addressed in claim 5. Thus, argument analogous to that presented above for claim 5 is applicable to claim 12.
Claim 13 is rejected the same as claim 6 except claim13 is directed to method claim . All the limitations of claim 13 are addressed in claim 6. Thus, argument analogous to that presented above for claim 6 is applicable to claim 13.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 7 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Milne , US 9418416 B2, ,in view of Khosla et al., (hereafter Khosla ), US20180300553, pub 10/18/2018.
As to claim 7, Milne teaches the liquid is a medical supply, (col. 6, lines 22- 26) the type of the foreign object is determined in such a manner as to input moving locus information of a floating object to col. 32, lines 14- 24)
acquire an identification class and an identification score of the floating object
the display control means further outputs a type of a foreign object existing on a bottom surface of the container, to the display device, so that the foreign object inspection device is optimized for fining out contamination of a foreign object. (col. 11, last line- col. 12, first lines: It is implied that the metal flakes are contaminating the drug. The contamination detection is disclosed in col. 13, lines 64- 65, col. col. 23, lines 55- 56)
It is noted that Milne does not specifically teach “learning model that learned by machine learning; and from the learning model determine an identification class having the highest identification score to be an identification class of the
On the other hand Khosla teaches learning model that learned by machine learning; and from the learning model determine an identification class having the highest identification score to be an identification class of the (A system for visual activity recognition comprise a method detecting a set of objects of interest in video data and determining an object classification for each object in the set of objects of interest. For each object of interest and using a feature extractor, determining a corresponding feature in the video data by performing feature extraction based on the corresponding activity track, the feature extractor comprising a convolutional neural network; and for each object of interest, based on the output of the feature extractor, determining a corresponding activity classification for each object of interest )
It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to incorporate a widely known and most robust Convolutional Neural Network (CNNs)that provide most accurate classification of objects taught by Khosla into Milne
The motivation for doing the CNNs enable users Mine to perform automatic feature extraction while simultaneously preserving critical spatial relationships within the image data.
Claim 14 is rejected the same as claim 7 except claim14 is directed to method claim . All the limitations of claim 14 are addressed in claim 7. Thus, argument analogous to that presented above for claim 7 is applicable to claim 14.
Prior art not used in rejections but pertinent to the claims or disclosure
“On-line Detection of Foreign Substances in Glass Bottles Filled with Transfusion Solution through Computer Vision”, Proceedings of the 2008 IEEE, to Juan Lu et al., disclosed:
“ A real time computer vision system was developed to detect a glass bottle filled with transfusion solution on a packing line. The theory, the mechanical structure, and the detection algorithm are described. Foreign substances mixed in transfusion solution are distinguished from bottle side wall defects by a relative movement between a resting bottle and the inside moving solution. Foreign objects moving down in the bottle are first detected with background subtracting, where a normalized cross correlation is employed to identify two frames between which no moving objects overlap and the static back-ground is obtained through an inter-frame difference. In the second stage, falling objects are recognized through an adapted mean-shift tracker, in which an adaptive searching window is used to only find falling objects. The system was tested on 250-ml transfusion bottles. The result shows that it can effectively detect the small visible foreign substances fulfilling the requirements of the application.”, See abstract .
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday-Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR. Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights