DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 5, 11, 12, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Son et al (Son hereinafter US 20180216155 A1) in view of Nakagaki et al (Nakagaki hereinafter US 20050051725 A1)
As per claim 1
Son teaches A motility analysis system (Figure 4, Figure 5) comprising: a memory storing processor-executable instructions; and one or more processors communicatively coupled to the memory and configured to execute the processor-executable instructions from the memory, the one or more processors configured (Figure 1, Paragraph [007] “The images are processed by computer to determine trajectory data for the at least one individual microorganism. The motility kinematics data are processed by computer for the at least one individual microorganism or a population of the individual microorganisms” Paragraph [0075] “A computer stores the videos acquired by the camera in a temporary data storage unit 40 (e.g., a RAM). Typically, the same software installed on the computer controlling the microscope and the camera can be used to control the storage of the videos.”) when executing the instructions, to: receive video data of a fluid sample containing bacteria captured through a microscope (Paragraph [0037] “in some examples, the bacterial suspension is loaded into two or more (e.g., many) wells of a cartridge or other device to hold samples, in particular devices designed for high-throughput video microscopy,” Paragraph [0062] “In some cases the microscope can have multiple imaging systems (e.g., multiple objectives and cameras) to acquire multiple videos of either the same well or of different wells, either simultaneously or non-simultaneously” Paragraph [0068] “Connected to the microscope is a camera 30 used to acquire videos (i.e., sequences of images) of the sample in each well.” ) generate a two-dimensional (2D) time-projection of motile tracks of the bacteria based on a pixel-by-pixel analysis of the video data (Figure 9, Figure 10, Paragraph [0077] “Some examples of image analysis techniques for this purpose use a minimum-intensity projection as shown in FIG. 9 (e.g., suitable for images with a white or otherwise light background); some examples use a maximum-intensity projection (e.g., suitable for images with a black or otherwise dark background). In these approaches, a minimum (or maximum) intensity projection image is created in which each pixel is given the minimum (or maximum) intensity that that pixel has across all images in the video. In this manner, bacterial trajectories appear as streaks 50, 52 that often make motility or changes of motility immediately apparent. The ASK software determines the presence of motility in a given sample by identifying these streaks in a minimum (or maximum) intensity projection image. “) process the 2D time-projection of the motile tracks with a quantitative analysis model (Figure 4, Figure 5 Paragraph [0078] “As shown in FIG. 4, the ASK software includes three main components: a swimming kinematics analyzer 56, an ASK database management system 58 that includes the motility database among other things, and an ASK tuner 60. If motile bacteria are present in the sample as determined by the pre-screening step (as discussed with respect to FIG. 9), the swimming kinematics analyzer (see also FIG. 5) in the ASK software performs image and data analysis routines on image data (“image frames”) “ )
In regard to “identify a match between the 2D time-projection of the motile tracks and a plurality of predetermined 2D time-projections” Son states in paragraph [0078] that “Selected image frames 62 (e.g., subset of a video or the whole video) can be also stored in the ASK database management system 58.” And states that “ For example, for a given bacterium detected in a given image frame, the currently detected bacterium is assigned to a previously identified trajectory within a search radius by one or more trajectory identifiers based on proximity 90 of positions…The search radius can be defined by the user or adaptively set by the software (e.g., the ASK tuner 60) based on previous tracking information stored in the database of the ASK database management system…scoring function that utilizes one or more of the criteria described above is then used to determine the best match 100 among a certain number of candidate bacteria based on a scoring function 98 and thus link bacterial positions in consecutive frames into the trajectory 102 of a given bacterium.” Son without further inquiry generates the 2D time projection. Every pixel gets a min and max intensity across all frames in the video creating a projection in which “bacterial trajectories appear as streaks” but the matching step is unclear. The identification step compares swimming kinematics with known swimming data from a motility database. Claim 1 seems to require the matching step to be the actual 2D projections against the plurality of other 2D projections.
For the reasoning above, Son does not teach identify a match between the 2D time-projection of the motile tracks and a plurality of predetermined 2D time-projections; and output data indicative of an identity of the bacteria in response to identifying the match.
Son in view of Nakagaki teaches identify a match between the 2D time-projection of the motile tracks and a plurality of predetermined 2D time-projections; (Nakagaki states that in paragraph [0031] “a processing unit 112 for processing image data; a storage unit 116 for storing processing results, etc.; a database 117 for providing reference data used to identify an imaged microscopic thing” as well as “When the captured image data has been stored in the storage unit 116, the overall control unit 115 sends the captured image stored in the storage unit 116 and the image data of the reference microorganism also stored in the storage unit 116 to the processing unit 112 and instructs the processing unit 112 to perform comparison and determination processing.” In paragraph [0046]. Furthermore, Nakagaki states “FIG. 6 is a flowchart illustrating the comparison and determination processing in the processing unit 112 (for calculating a degree of similarity). The processing unit 112 receives the captured image data at step 601 and the image data of the reference microorganism” in paragraph [0048] and “Thus, in the above-described comparison and determination processing at step 209, the processing unit 112 compares each reference microorganism image with each possible microorganism image within each image “ in paragraph [0058]. Nakagaki also does pixel by pixel comparison “Then, the correlation coefficient between the image in the local window at each position within the captured image 501 and the template image 506 may be calculated. Of the images defined by the local window 505 at each position, those having a correlation value larger than a predetermined value may be regarded as images of the reference microorganism.” Paragraph [0050]. The time projection limitation designated to the predetermined images comes from the modification of references itself.
In regard to “output data indicative of an identity of the bacteria in response to identifying the match” both Son and Nakagaki output the bacteria identification in response to finding a match. (Nakagaki: Figure 7, Figure 9). Son: Figure 7, Paragraph [0103] “The ASK software also determines the statistics of the comparison (i.e., whether statistically significant and at what level of significance), providing metrics of confidence in the identity of the pathogens or other microorganisms”
In a combined teaching, Son generates and processes a 2D time projection where the motile tracks of bacterium appear as streaks. Nakagaki identifies a match between a subject microorganism image and plurality of predetermined reference microorganism images based on quantitative pixel by pixel image similarity. The combined system would process Son’s 2D time projections of the motile tracks and compare the resulting 2D projections against a plurality of predetermined 2D time projections within the database that are correlated with known microorganisms, the motile behaviors and or their trajectories to identify a match.
A person of ordinary skill in the art, at the time this invention was effectively filed would have found it obvious to modify Son in view of Nakagaki. Son already generates and processes a 2D projection containing characteristics of bacteria motility tracks and uses bacteria motility characteristics for microorganism identification. Nakagaki identifies microorganisms by comparing a captured microorganism image with a plurality of predetermined reference images and determines a similarity between the two. Applying this known comparison technique to Son’s 2D motility projection pipeline would have been a predictable use of a known microorganism image matching technique to compare the captured motility pattern with known patterns within a database. This observed and reference image matching technique gives Son another avenue for identifying the microorganisms based on motility information. It allows Son’s system to directly compare captured image projections with predetermined projections. It gives a direct, pixel by pixel, image comparison for bacterial motility patterns leading to identification.
As per claim 2
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Son teaches wherein the one or more processors are configured, when executing the instructions, to generate the 2D time-projection (Figure 9) by determining a pixel characteristic based on a time-varying intensity of each pixel in the video data over a duration of the video data reshaping the pixel characteristic of each pixel to an original image size of the video data to generate the 2D time-projection (Paragraph [0077] “examples of image analysis techniques for this purpose use a minimum-intensity projection as shown in FIG. 9 (e.g., suitable for images with a white or otherwise light background); some examples use a maximum-intensity projection (e.g., suitable for images with a black or otherwise dark background). In these approaches, a minimum (or maximum) intensity projection image is created in which each pixel is given the minimum (or maximum) intensity that that pixel has across all images in the video. In this manner, bacterial trajectories appear as streaks 50, 52 that often make motility or changes of motility immediately apparent.” Son attaches a characteristic for the pixel (min or max pixel intensity) across video frames. That time based pixel parameter are arranged spatially according to the video data dimension to form the projection. This teaches “reshaping” the pixel characteristic (intensity) to the original image size to get the time projection we see in figure 9)
As per claim 4
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Son teaches wherein the one or more processors are further configured, when executing the instructions, to: receive additional video data of the fluid sample captured after application of an antibacterial compound to the fluid sample (Figure 21, Paragraph [0037] “In some examples, the bacterial suspension is loaded into two or more (e.g., many) wells of a cartridge or other device to hold samples, in particular devices designed for high-throughput video microscopy, to test the susceptibility of the microorganism to one or more (e.g., many) antibiotic conditions (types of antibiotic, concentration of antibiotic, antibiotic exposure time, combination of antibiotics, or additional substances that supplement the antibiotics such as one or more enzyme inhibitors…The image sequences can be acquired at a rate that captures a continuous movie of the motion of the bacteria (e.g., 15 frames per second; typically less for twitching motility). We refer to the image sequences as videos regardless of the frame capture rate” Paragraph [0044] “Typical heights can be, for example, from 10 μm to 10 mm. The use of many wells allows high-throughput in AST (enabling many antibiotics, or many antibiotic conditions, or many samples, or combinations of these, to be tested in respective wells), Paragraph [0047] “or pathogen AST purposes, typically a minimum of two wells is used if the user intends to compare the motility kinematics between two fluid samples (e.g., one with and one without an antibiotic or other antimicrobial) at approximately or exactly the same time” “ Paragraph [0062] “in some cases the microscope can have multiple imaging systems (e.g., multiple objectives and cameras) to acquire multiple videos of either the same well or of different wells, either simultaneously or non-simultaneously. Other combinations are possible.” Paragraph [0082]-[0083] “he identified swimming trajectories can be used to obtain a count of the number of motile bacteria in the sample. The identified Brownian motion trajectories can be used to obtain a count of the number of non-motile bacteria in the sample. Because the imaging volume is known, this count can be converted into a concentration of motile bacteria or non-motile bacteria or both… Information derived from the tracking process can include the presence, absence, count, percentage or concentration (or combinations of those categories) of motile bacteria in the sample …The change over time in the number of motile bacteria or the change over time in the fraction of motile versus non-motile bacteria can provide information on the susceptibility of bacteria to a given antibiotic” Paragraph [0104] “The swimming kinematics change upon exposure to antibiotics, providing a biomarker for AST. AST can be performed in one of two ways, or a combination of them: without or with use of a reference database.” Paragraph [0116] “FIG. 16 show a motility-plate assays for PSA 71/36 assessed after overnight incubation, for the antibiotic-free control (FIG. 16a) and 4 μg/ml ciprofloxacin (FIG. 16b). Note the outward expansion in both cases, denoting no impairment of motility by ciprofloxacin for this resistant strain, as obtained with the ASK software in less than 30 min (FIG. 15a).” generate an additional 2D time-projection of motile tracks of the bacteria (Figure 21, Figure 9, Figure 3., Paragraph [0007] “ The images are processed by computer to determine trajectory data for the at least one individual microorganism. …The imaging device is used to capture at successive times one, two, or more digital images of a sample containing the at least one individual microorganism, the sample containing one or more antibiotics or other antimicrobials, “ Son teaches applying its motility trajectory processing to the additional video data obtained under antibiotic condition.) output data indicative of a resistance of the bacteria to the antibacterial compound based on a comparison between the 2D time-projection and the additional 2D time-projection. (Paragraph [0031] “susceptibility of a microorganism to an antibiotic or other antimicrobial can be determined rapidly by comparing the motility kinematic properties of the microorganism at one time or in one sample with motility kinematic properties of the microorganism at another time or in another sample derived from another set of images. “ Paragraph [0116] “s shown in FIGS. 14 and 15, the ASK software can discriminate, for a given antibiotic, susceptible strains and resistant strains, as shown here with P. aeruginosa exposed to ciprofloxacin (FIGS. 14 and 15). FIG. 14 shows that ciprofloxacin impairs motility of a susceptible clinical isolate of P. aeruginosa (PSA 9/53, MIC<0.125 μg/ml) and FIG. 15 shows that ciprofloxacin does not impair the motility of a resistant clinical isolate of P. aeruginosa (PSA 71/36, MIC=8 μg/ml). In FIG. 14a and FIG. 14c, two replicate experiments for PSA 9/53 for 4 μg/ml ciprofloxacin normalized to the antibiotic-free control (straight line at a value of 1) are shown (CLSI resistant breakpoint: ≥4 μg/ml)….Note the outward expansion in both cases, denoting no impairment of motility by ciprofloxacin for this resistant strain, as obtained with the ASK software in less than 30 min (FIG. 15a).” Son shows comparison of bacterial motility with and without antibiotics against their respective motility. Keeping in mind the projection technique disclosed in claim 1, the comparison is motility information represented by the original and additional 2D projections with retained motility indicating antibacterial resistance.
As per claim 5
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Son teaches wherein the one or more processors are further configured, when executing the instructions, to: query a database of a plurality of bacterial treatments based on the identity of the bacteria; and output data indicative of a selected bacterial treatment based on the query. (Paragraph [0005] “Detecting bacterial presence and counts, identification (ID), and antibiotic susceptibility testing (AST) are used to determine the best treatment option for a bacterial infection.”
As per claim 11
Claim 11 is the method claim that parallels system claim 1. Claim 11 will be rejected under the same premise.
As per claim 12
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Claim 12 is the method claim that parallels system claim 2. Claim 12 will be rejected under the same premise.
As per claim 14
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Claim 14 is the parallel method claim of system claim 4 and will be rejected under the same premise.
As per claim 15
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Claim 15 is the parallel method claim of system claim 5 and will be rejected under the same premise.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Son et al ( Son hereinafter US 20180216155 A1) in view of Nakagaki et al (Nakagaki hereinafter US 20050051725 A1) in further view of Riekeles et al (Riekeles hereinafter “Machine Learning Algorithms Applied to Identify Microbial Species by Their Motility”)
As per claim 3
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Son in view of Nakagaki teach wherein the plurality of predetermined 2D time-projections are labeled with a corresponding plurality of bacteria samples (Son states “The known identity of a species or strain of one or more microorganisms is stored in a database that is part of a database management system” in paragraph [0007] and that “ The available motility kinematics data of the one or more microorganisms are stored in a motility database that is part of a database management system.” In the same paragraph. Son also has a goal of distinguishing motile and non motile bacterium stating “ratio of motile and non-motile bacteria,” in paragraph [0025] and also states that “The identified swimming trajectories can be used to obtain a count of the number of motile bacteria in the sample. The identified Brownian motion trajectories can be used to obtain a count of the number of non-motile bacteria in the sample” in paragraph [0082] Nakagaki states that “Reference numerals 901(A), 901(B), and 901(C) denote captured images for the reference microorganisms A, B, and C, respectively; reference numerals 902(A), 902(B), and 902(C) denote the template images of the reference microorganisms A, B, and C, respectively, as retrieved from the database;” in paragraph [0063]. Figure 9 shows the labeling. Therefore Son in view of Nakagaki teach a predetermined reference representation corresponding to known microorganism identities. Son supplies the 2D projections as well as the relationship between the image and the known strain of bacteria through the AST/ASK system. Nakagaki supplies the reference images and labels that is compared to the captured imagery. This combination allows the 2D projections to correspond to labeled bacterium identity from the reference images found in the database.
Although Son states that “ the ASK software makes use of motility information together with changes in any one of these other key phenotypes by using a combination of criteria to further enhance the reliability of AST.” and that “Prior data (i.e., historical data) stored in the database is utilized by the ASK tuner module to continuously upgrade the swimming kinematics analyzer and enhance the accuracy of ID or AST (or both) routines in subsequent tests.” Theres no true indication that this is necessarily machine learning as those skilled in the art would assume.
Son and Nakagaki do not teach wherein the quantitative analysis model is trained on the plurality of predetermined 2D time-projections to distinguish and identify motile bacteria via machine learning.
Son in view of Nakagaki in view of Riekeles teaches wherein the quantitative analysis model is trained on the plurality of predetermined 2D time-projections to distinguish and identify motile bacteria via machine learning. (Riekeles states that “Supervised machine learning algorithms were also used to specifically identify these species based on their characteristic motility behavior.” And that “It is crucial to select features that contribute most to the predicted output. Irrelevant features in the data can decrease the accuracy of the classification. Feature selection means less redundant data and, therefore, less impact from noise, more accurate models, and faster training of algorithms… 10-fold cross-validation for all algorithms, during which the dataset was shuffled randomly and split into ten groups. Each group was used once as a test group, with the remaining groups being used as training data. A model fitting the training set was evaluated in the test set” within the Classes and Features of Data section.
In a combined teaching Son in view of Nakagaki in further view of Riekeles teaches using supervised machine learning with information pertaining to characteristics of motility that reflects bacterial identity. Son/Nakagaki provide the predetermined 2D projection associated with the bacteria’s identity. Riekeles teaches using known species motility behavioral information with supervised machine learning to identify the strain. In combination the predetermined 2D time projections and their reflected bacterial identities would constitute the labeled information used to train a machine learning algorithm in conjunction to the quantitative analysis provided by Son’s AST/ASK system.
Accordingly a person of ordinary skill in the art, at the time this invention was effectively filed would have found it obvious to continue to modify the Son/Nakagaki pipeline with Riekeles concept of using a machine learning algorithm to train the quantitative analysis model (Son’s AST/ASK system) with the predetermined 2D time projections to identify motile bacterium. Son already stores microorganism identities and their motility kinematics. Son also uses this to characterize the bacterium. Nakagaki allows predetermined reference images for comparison with the bacterium in question. Riekeles shows that supervised machine learning algorithms can be used to identify microbial species based on their motility behavior. A person of ordinary skill in the art would have reason and find it obvious to train the quantitative analysis modeling of the Son/Nakagaki system with the predetermined 2D time projection/known motility characteristics and the bacterial labels they represent. The modified system already provided the dataset, Riekeles allows the pipeline to train by that dataset in a predictable automated fashion for identification of bacteria from motility patterns. A person of ordinary skill in the art recognizes this modification allows the system to automatically learn and distinguish motility patterns associated with certain bacteria strains and use those relationships to identify observed bacteria in question.
As per claim 13
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Claim 13 is the parallel method claim of system claim 3 and will be rejected under the same premise.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Son et al ( Son hereinafter US 20180216155 A1) in view of Nakagaki et al (Nakagaki hereinafter US 20050051725 A1) in further view of Hu et al (Hu hereinafter CN 113205085 A “An Image Recognition Method And Device”)
As per claim 6
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Son and Nakagaki do not teach wherein the one or more processors are further configured, when executing the instructions, to: receive user input indicative of an accuracy of the identity of the bacteria; and train the quantitative analysis model based on the 2D time-projection and the accuracy of the identity of the bacteria.
Hu teaches the one or more processors are further configured, when executing the instructions, to: receive user input indicative of an accuracy of the identity of the bacteria (Contents of invention: “in the large classification of the microorganism, combining the position of the microorganism, the second classification unit 13 to identify the object to be identified for secondary identification of fine granularity…if the user is not satisfied with some video frame detection effect, manually marking the position information and classification, then according to the system prediction result and manual correction result… capable of automatically performing target detection to the input file, fine granularity identification, finally outputting the microorganism type and number. If the detection result is bad, the user can also manually mark and modify it, finally detecting the data, detecting result for storing, for later checking. thirdly, the detection result is more accurate; using two-step classification method, firstly locating and classifying the microorganism coarse particle size, then performing more fine and accurate identification to a certain large class, realizing more accurate identification…The expert judges whether the second-level identification result is accurate. if so, generating the label file; if not, manually correcting and labelling the data, and then storing in the label file.”)
Son in view of Nakagaki in further view of Hu teach train the quantitative analysis model based on the 2D time-projection and the accuracy of the identity of the bacteria. (Hu: Content of invention: “ if the user is not satisfied with some video frame detection effect, manually marking the position information and classification, then according to the system prediction result and manual correction result, generating the label file for each frame image, and expanding the data set of the model training. data new increment reaches a certain value (such as 500), the system will use the current data set to automatically perform secondary training and optimization to the model, and the original model parameter is replaced with the latest model parameter, forming a learning, detecting, and then learning closed loop, convenient for continuously depositing the microscopic examination knowledge, accumulating working experience…improving the recognition rate and robustness of the algorithm. establishing the user self-labeling, updating the data mechanism, forming learning, detecting, and then learning closed loop, convenient for continuously depositing the microscopic examination knowledge, accumulating working experience.” Hu discloses repeatedly refining a microorganism identification model using newly obtained image information and user corrected classification. Son in view of Nakagaki in view of Hu teach the limitations of claim 6. Son/Nakagaki supplies the 2D time projection from which the bacteria is identified. Hu supplies the technique of incorporating the corresponding microorganism image information and user feedback concerning the accuracy of the result into a training and optimization setting of the model. The closed loop of learning and detection Hu shows supports the claimed continuous training rather than being seen as a single training operation.
In a combined teaching Son in view of Nakagaki in further view of Hu teaches generating and processing a 2D time projection of bacterial motility to determine bacterial identity as well as receiving user input indicating the accuracy of the result. The combined system also uses the corresponding image information and user correction to continuously train and optimize the identification model in a closed loop of learning detection and subsequent learning.
Accordingly a person of ordinary skill in the art, at the time this invention was effectively filed would have found it obvious to further modify the Son/Nakagaki pipeline to include Hu’s concepts of continuous learning based off of user input and correction accuracy. A person of ordinary skill in the art understands that Hu addresses the same problem of computerized microorganism identification from microscopic images and or video data. A person of ordinary skill in the art recognizes that system generated microorganism classification may require manual correction to keep results accurate. Hu teaches incorporating the system prediction results and manual correction result into future model improvement and automatically performs secondary training and optimization on the model. A skilled person would have had reason to apply Hu’s feedback and retraining technique to the bacterial identification model of Son/Nakagaki so that the user confirmed or corrected bacterial identities and their corresponding 2D projections could be used to refine later identifications.
As per claim 16
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Claim 16 is the parallel method claim of system claim 6 and will be rejected under the same premise.
Claims 7, 9, 10, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Son et al ( Son hereinafter US 20180216155 A1) in view of Nakagaki et al (Nakagaki hereinafter US 20050051725 A1) in further view of Fan et al (Fan hereinafter US 20170160534 A1)
As per claim 7
Son and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection
Son teaches a sample retaining device configured to receive an observation retainer in view of the camera and the microscope lens (Figure 1, Figure 20)
Son and Nakagaki do not teach comprising: a microscope attachment configured to couple to a computing device having a camera to capture the video data, wherein the microscope attachment comprises: a microscope lens to magnify images captured through the camera; a light source to illuminate the fluid sample;
Fan teaches a microscope attachment configured to couple to a computing device having a camera to capture the video data (Figure 2A, Figure 3, Figure 4 Paragraph [0018] “microscope attachment 200 (e.g., microscope attachment 100) attached to personal electronic device 206. Attachment mechanism 205 is disposed to connect microscope attachment 200 to personal electronic device 206.”) wherein the microscope attachment comprises: a microscope lens to magnify images captured through the camera (Figure 3, Figure 4, Paragraph [0015] lens apparatus 101 includes one or more lenses which may be configured to provide variable states of magnification and/or focus a microscope lens to magnify images captured through the camera…Lens apparatus 101 is disposed to enlarge an optical area in sample holder 103, which may correspond to the area in sample holder”) a light source to illuminate the fluid sample; (Figure 3, Figure 4, Paragraph [0019] “microscope attachment 300 includes: lens apparatus 301 (including one or more lenses), sample holder 303, spacer 305 and, light source 307. Specifically, microscope attachment 300 is used with a cellular phone (personal electronic device 306)…Sample holder 303 is disposed to receive light from light source 307, and sample holder 303 is optically aligned with lens apparatus 301 such that sample light travels from a sample, through lens apparatus 301 and into image sensor 343.”). Fan also teaches a sample retaining device configured to receive an observation retainer in view of the camera and the microscope lens. (Figure 3, Figure 4, Paragraph [0017] “FIG. 1C depicts loading sample carrier chip 111 into microscope attachment 100. In the depicted example, a sample has been placed into a recessed portion of sample carrier chip 111, and sample carrier chip 111 was loaded into sample holder 103 to be imaged. As illustrated, sample carrier chip 111 may be transparent to permit light to travel from light source 107, through sample carrier chip 111, and into lens apparatus 101” Paragraph [0019] “sample holder 303 is optically aligned with lens apparatus 301 such that sample light travels from a sample, through lens apparatus 301 and into image sensor 343.”)
Accordingly a person of ordinary skill in the art, at the time this invention was effectively filed would have found it obvious to further modify the Son/Nakagaki pipeline to further include Fan’s microscope attachment architecture. Son already teaches acquisition of sequences of microscopic images/videos of bacterial suspension for motility analysis. Fan allows for a known portable architecture to perform microscopic sample imagining using the camera of a computing device. Fan specifically teaches attaching its microscope assembly to a smartphone and optically aligning its lens and sample holder with the smartphone’s camera. A person of ordinary skill in the art would have had a reason to implement Son’s bacterial image acquisition arrangement using the known microscope attachment as taught by Fan to enable the video acquisition of Son to be conveniently acquired using a camera equipped computing device like a smartphone. This gives the advantage of a compact and portable implementation of sons bacterial imagining arrangement in which the lens illumination source and sample holder are integrated into an attachment that uses the image sensor of a computing device. This allows the Son/Nakagaki pipeline to have a video acquisition without requiring a separate conventional microscope/camera arrangement.
As per claim 9
Son, Nakagaki and Fan teach all claim limitations previously rejected in claim 7’s 103 rejection. See claim 7’s 103 rejection
Son teaches wherein the observation retainer comprises a microscope glass slide (Paragraph [0043] “many wells fit on a small cartridge (for example, the size of a microscope slide or a 96-well plate).”)
Fan (Paragraph [0017] “a sample has been placed into a recessed portion of sample carrier chip 111, and sample carrier chip 111 was loaded into sample holder 103 to be imaged…sample carrier chip 111 may be a commercially available glass microscope slide.”)
As per claim 10
Son, and Nakagaki teach all claim limitations previously rejected in claim 1’s 103 rejection. See claim 1’s 103 rejection.
Fan teaches wherein the memory, the one or more processors, and a camera to capture the video are operatively coupled together within a smartphone or a tablet. (Paragraph [0018] “Although in the depicted example, personal electronic device 206 is a smart-phone, in other examples personal electronic device 206 may include smart-watches, smart-glasses, tablets, or the like.” Paragraph [0019] “Lens apparatus 301 is configured to be removably attached to a surface of the cellular phone and optically align lens apparatus 301 with image sensor 343, where image sensor 343 is included in the cellular phone (e.g., a cell phone camera)” Paragraph [0033] “Process block 507 depicts sending the image data to a processor. The processor is coupled to the personal electronic device, and the processor analyzes the image data. “)
As per claim 17
Son and Nakagaki teach all claim limitations previously rejected in claim 11’s 103 rejection. See claim 11’s 103 rejection.
Fan teaches coupling a microscope attachment to a computing device having a camera to capture the video data (Figure 2A) wherein the microscope attachment contains a sample retaining device configured to receive an observation retainer. (Figure 3, Figure 4)
As per claim 20
Son, Nakagaki and Fan teach all claim limitations previously rejected in claim 17’s 103 rejection. See claim 17’s 103 rejection
Fan teaches wherein the observation retainer comprises a microscope glass slide. (Figure 3, Figure 4, Paragraph [0017] “a sample has been placed into a recessed portion of sample carrier chip 111, and sample carrier chip 111 was loaded into sample holder 103 to be imaged…sample carrier chip 111 may be a commercially available glass microscope slide.”))
Allowable Subject Matter
Claim 8, 18 and 19 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANE WRENSFORD CODRINGTON whose telephone number is (571)272-8130. The examiner can normally be reached 8:00am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHANE WRENSFORD CODRINGTON/Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights