DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 8 is objected to as indefinite. Claim 8 recites "the provided model." There is insufficient antecedent basis: claim 1 recites "a model" and "transferring the model," but never "the provided model." Amending claim 8 to recite "the model" would resolve the objection.
Claim 10 is objected to under 37 CFR 1.75 as containing a typographical informality: there should only be one comma after “contrast.”
Claim 14 is objected to under 37 CFR 1.75 as containing a typographical informality:
"the set of two or more image devices" should read "the set of two or more imaging devices."
Appropriate correction is required. (This claim is additionally rejected under 35 U.S.C. 112(b)
below for the antecedent issue discussed therein.)
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6, 7, 9, 12, 14, and 15 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6, 7, 14, and 15 recites the limitation "the set of two or more imaging devices" (claim 14: "the set of two or more image devices"). There is insufficient antecedent basis for this limitation in the claim. Independent claim 1 recites "one or more imaging devices," not a set of two or more. It is therefore unclear whether the claims require two or more devices or are consistent with the "one or more" of claim 1. Amending claim 1 to recite "a set of two or more imaging devices," or amending each dependent claim to recite "the one or more imaging devices," would resolve the rejection as to these four claims.
Claim 9 is rejected under 35 U.S.C. 112(b) as indefinite. Claim 9 recites "a given one of the set of imaging devices." There is insufficient antecedent basis: claim 1 recites "one or more imaging devices," and never "the set of imaging devices."
Claim 12 is rejected under 35 U.S.C. 112(b) as indefinite. Claim 12 recites "the deep learning neural network." There is insufficient antecedent basis: claim 1 recites "a neural network," and parent claim 9 (augmentation) does not recite a neural network. It is further unclear whether claim 12 is intended to depend from claim 9 (augmentation) or from a claim reciting the network architecture. Clarification and a corrected antecedent are required.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 10 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 10 depends from claim 7 and recites "wherein augmentation is based on … ," but claim 7 does not recite an augmentation step; the augmentation step is first introduced in claim 9. Claim 10 therefore fails to further limit the subject matter of the claim from which it depends and lacks antecedent basis for "augmentation." Amending claim 10 to depend from claim 9 would resolve the rejection. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) mathematical concepts and mental processes together as a single abstract idea. This judicial exception is not integrated into a practical application because the neural network and the imaging devices are recited generically and do no more than carry out the idea on standard equipment. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because The additional elements, taken alone and together, are no more than well-understood, routine, and conventional activity, as the specification itself concedes (SPEC: "In a well-known manner, each analyzer ... receives a set of one or more plates"(¶ 14); "As is known, ML algorithms iteratively learn from the data ... without being explicitly programmed" (¶ 30); "As is well-known, cloud computing ... "(¶ 37).). Claim 1 is taken as representative. Claims 16 and 17 stand or fall with it, and the dependent claims are addressed afterward.
Step 1: Claim 1 is a process and falls within a statutory category.
Step 2A, Prong One: Claim 1 recites an abstract idea. The step of "training a model ... wherein the model is a neural network" is a mathematical concept, because a neural network is trained through calculation and optimization. See the 2024 AI SME Update, Example 47. The steps of "determining whether the given image data should be processed … by discriminating objects …
in confluent association ... or that are obscured" and "generating the training data … comprising a set of labels" describe what a person does by eye and judgment when screening an image and
then labeling it, which is a mental process. See MPEP § 2106.04(a)(2). The two are treated
together as a single abstract idea.
Step 2A, Prong Two: The rest of the claim does not put that idea to a practical application. Receiving the image data from the imaging devices is ordinary data gathering. Transferring the
trained model "to one or more entities for detection of the objects of interest" is ordinary output followed by a statement of what the model is for. The claim never actually performs any
detection, and the specification confirms that the model is simply handed off once trained
(SPEC: "Following training, the model is provided for detection of the objects of interest"
(¶ 8).) The neural network and the imaging devices are recited generically and do no more than
carry out the idea on standard equipment. See MPEP § 2106.05(f), (h). The specification credits
the invention with giving consistent counts from one machine to the next, but claim 1 does not
say how that is done; it claims only training and hand-off. Aiming a generic neural network at a
new kind of data is not a technical improvement. See Recentive Analytics v. Fox Corp. 134 F.4th 1205 (Fed. Cir. 2025). This is the same shortfall as Example 48, claim 1, which stopped at the model output and was ineligible. Unlike claims 2 and 3 of that example, which recited how the output was used, there is no particular machine, no transformation, and no detection or control step.
Step 2B: The additional elements, taken alone and together, are no more than well-understood,
routine, and conventional activity, as the specification itself concedes (SPEC: "In a well-known
manner, each analyzer ... receives a set of one or more plates"(¶ 14); "As is known, ML
algorithms iteratively learn from the data ... without being explicitly programmed" (¶ 30);
"As is well-known, cloud computing ... "(¶ 37).) Gathering and sending data are conventional computer functions . See MPEP § 2106.05(d). The claim adds nothing significantly more, and claim 1 is ineligible.
Claims 16 and 17.
Claims 16 and 17 recite the same steps as an apparatus and as a computer program product. The added elements, namely "one or more hardware processors," "computer memory," and a "non-transitory computer-readable medium holding computer program code", are generic computer parts that do no more than carry out the abstract idea on a computer. See MPEP § 2106.05(f). They are ineligible for the same reasons as claim 1.
Claims 2-15.
The dependent claims each carry the same abstract idea and add nothing that makes them eligible. Claims 2 and 3 name the assay and the imaging device, which is a field of use and generic hardware. Claims 4 through 7 add more data gathering. Claim 8 calls the model entity independent or entity-specific without saying how it works. Claims 9 and 10 add ordinary data
augmentation ahead of the same generic training step, and the specification lists the
augmentation techniques as known at ¶21 . Claim 11 comes the closest, since it brings in a
second model to handle the crowded or hidden objects, but it still recites only a result, not an
architecture or any account of how the second model improves detection, so it does not change the outcome (compare Example 48, claim 2, which recites the mechanism). Claim 12 names a generic encoder-decoder architecture. Claim 13 recites a software-as-a-service business arrangement. Claims 14 and 15 add only generic devices. Each dependent claim is ineligible.
Eligibility note: The most promising route to allowance is claim 11 . A claim that actually recites
detecting or segmenting the objects of interest, together with the second-model mechanism
described at ¶26 for handling objects in confluent association or obscured by other structures,
rather than only "transferring the model ... for detection," would have a real argument for a
practical application under Example 48, claims 2 and 3.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, 8, 11, and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Goldberg et al., US 2022/0261990 A1 (hereinafter "GOLDBERG") in view of Smedman et al., US 2020/0116636 A1 (hereinafter "SMEDMAN").
Claims 1, 16, and 17.
GOLDBERG in view of SMEDMAN discloses
An apparatus, comprising: one or more hardware processors; and computer memory holding computer program code executed by the one or more hardware processors to … the computer program code configured to (GOLDBERG: claim 1) …
Acomputer program product comprising a non-transitory computer-readable medium
holding computer program code executable by a hardware processor to provide (GOLDBERG: ¶ 171) …
A method for machine learning-assisted detection of objects of interest in live cell-based assays, where the objects of interest are larger than cells used in the assay (GOLDBERG: "A plaque assay is one of the most important assays in
virology" (¶ 4), in which there are "localized clusters of dead cells defining each plaque" (¶
46); SMEDMAN: "secretory footprints (or spots) of the secreted cytokines are captured by
way of imaging" (¶ 3). A plaque (a cluster of dead cells), and a secreted-analyte spot, is an
object larger than the individual cells in the assay.), comprising:
receiving image data that has been captured from one or more imaging devices (GOLDBERG: "The raw image data and associated metadata can be ... transferred from a user's storage device/database via the internet into the storage device ... for cloud image processing" (¶ 64).);
for given image data, determining whether the given image data should be processed to generate training data by discriminating objects in the given image data that are situated in (GOLDBERG teaches the gating decision: a '"prefilter' network is used to eliminate tiles" unsuitable for training so as to "discard tiles" before training (¶ 106); claim 8: "the tiles are prefiltered to reject tiles" unsuitable for use. SMEDMAN supplies the confluent/obscured discrimination that is the object of that gating: "Often, the cells are closely located to each other, thereby, leading to overlap of the spots in the captured images", requiring "distinguishing individual spots from overlapped spots" (¶ 4). Together: gating image data on whether objects are in confluent association or obscured.);
at least in part using the given image data that has been determined should be processed, generating the training data, the training data comprising a set of labels for the image data (GOLDBERG: "The objects tagged or labeled in the image can be recognized by machine learning with annotations (type of object, location) added as part of the metadata" (¶ 70).); training a model using the set of labels for the image data, wherein the model is a neural network (GOLDBERG: training "CNN artificial intelligence models" (¶ 147); "Convolutional Neural Networks (CNNs)" (¶113)); and
following training, transferring the model to one or more entities for detection of the objects of interest (GOLDBERG: "the developed models are shareable worldwide using a web-based platform ... sharing of models merely requires a web-browser" (¶ 87).).
GOLDBERG teaches the machine-learning detection pipeline (prefilter gated training data generation, labeling, neural-network training, and worldwide model transfer) applied to a live cell-based plaque assay in which the objects of interest are larger than the cells. GOLDBERG
does not expressly teach the discrimination of confluent, overlapping analyte spots. However, SMEDMAN, in the same field of automated immunoassay image analysis, teaches the confluent
spot problem (SMEDMAN: "Often, the cells are closely located to each other, thereby, leading to overlap of the spots in the captured images ... distinguishing individual spots from overlapped spots" (¶¶ 4, 88).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to apply GOLDBERG's deep learning detection and transfer method to
SMEDMAN's spot images. SMEDMAN itself identifies its art as ready for improvement (SMEDMAN: "Currently existing analytic techniques are not sufficiently developed to accurately determine secretion intensities of individual cells, and distinguish individual spots" (¶ 4).) and GOLDBERG teaches that a trained neural network detects, in this assay setting, what is otherwise (GOLDBERG claim 1: "undeterminable by human eyesight.") Applying a known technique (GOLDBERG's neural network detection and prefilter gated training) to a known assay expressly acknowledged to need better spot discrimination (SMEDMAN's FLUOROSPOT) yields the predictable result of consistent, automated spot counts that resolve overlapping/confluent spots. See KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398 (2007); MPEP § 2143(C), (D).
Claim 2.
GOLDBERG in view of SMEDMAN teaches the method as described in claim 1 wherein the image data is derived from one of: an enzyme-linked immune absorbent spot (ELISPOT) assay, a FLUOROSPOT assay, a viral plaque neutralization assay, a CRISPR-based DNA assay, and a cellular colony counting assay, the cellular colony being one of: bacterial, yeast and stem cells (SMEDMAN: "Enzyme-Linked ImmunoSpot (ELISPOT) assay, fluorospot assay" (¶ 2); GOLDBERG: "viral plaque assay " (¶4).).
Claim 3.
GOLDBERG in view of SMEDMAN teaches the method as described in claim 1 wherein the imaging device is one of: a microscope with digital camera, a digital camera with micro or macro zoom or fixed lens, a flatbed scanner, a smartphone or tablet, an ELISPOT analyzer, and a FLUOROSPOT analyzer (SMEDMAN: a system "for analysing fluorospot assays" that images the wells (112-3).).
Claim 4.
GOLDBERG in view of SMEDMAN teaches the method as described in claim 1 wherein the image data is received from one or more entities (GOLDBERG: image data "transferred from a user's storage device/database via the internet" (¶ 64).).
Claim 5.
GOLDBERG in view of SMEDMAN teaches the method as described in claim 4 wherein the one or more entities include a first entity, and further including receiving from the first entity a set of first entity labels for the image data associated with the first entity, and using the first entity labels in the model training (GOLDBERG: user "annotations (type of object, location)" are used by the machine learning (¶¶ 68, 70).).
Claim 8.
The method as described in claim 1 wherein the provided model is one of: an entity-independent model, and an entity-specific model (GOLDBERG: the "developed models are shareable worldwide" – an entity-independent model (¶ 87).).
Claim 11.
GOLDBERG in view of SMEDMAN teaches the method as described in claim 1, further including using a second model to discriminate the objects in the given image data that are situated in confluent association with objects of interest or that are obscured by other structures (GOLDBERG: a separate "background model Al" performs the prefilter/gating step (¶ 106); SMEDMAN: the confluent/overlapping spots are resolved by "clustering a plurality of co-positioned fluorospots as a multiple secretion fluorospot" (¶ 88). The combination provides a second model directed to the confluent/obscured objects.).
Claims 6, 7, 9, 10, 12, 13, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over GOLDBERG in view of SMEDMAN, and further in view of Polit et al., -US 2014/0029052 A1 (hereinafter "POLIT").
Claim 6.
GOLDBERG in view of SMEDMAN discloses the method as described in claim 1 except for explicitly teaching wherein the set of two or more imaging devices include at least a first imaging device of a first entity, and a second imaging device of a second entity, the first and second entities being a same entity, or distinct from one another (POLIT: matching multiple image acquisition devices so they "provide consistent image data regardless of differences in the devices" (¶30).).
It would have been obvious to one of ordinary skill in the art before the effective filing date to combine POLIT with GOLDBERG and SMEDMAN. The motivation would have been to
deliver the trained detection model as a network service to multiple laboratory sites and to use each device’s historical profiles to compensate for inter-instrument variation (e.g., brightness, gamma, focus), thereby producing consistent detection results across different analyzers – the same instrument variation problem the present specification identifies. This is a combination of known elements (GOLDBERG's model training/transfer, POLIT's subscription image services and per-device profiling) according to known methods to yield predictable results. See KSR, 550 U.S. 398.
Claim 7.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 1 wherein the set of two or more imaging devices include at least a first imaging device of a first entity, and a second imaging device of the first entity, the first and second imaging devices differing from one another in at least one operating characteristic (POLIT: matching multiple image acquisition devices “include color space data (e.g., RGB, CMYK, YCbCr, etc.)” so they "provide consistent image data regardless of differences in the devices" (¶30).).
Claim 9.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 1 wherein the training further includes augmenting at least some of the image data based on historical data associated with a given one of the set of imaging devices to generate a set of augmented image data, and using the set of augmented image data in the training (POLIT: "processor ... maintains a history of acquisition profiles for each device" (¶ 48); correction algorithms including "gamma curve corrections" (¶ 50). GOLDBERG: train CNN artificial intelligence models … use[ing] … data augmentation (¶147).).
Claim 10.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 7 wherein augmentation is based on one of: information about at least one operating parameter of the given imaging device, and image properties that are one of: orientation, position, brightness, contrast, gamma, zoom factor, focus precision and color (POLIT: "processor ... maintains a history of acquisition profiles for each device" (¶ 48); correction algorithms including "gamma curve corrections" (¶ 50).).
Claim 12.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 9 wherein the deep learning neural network is an encoder-decoder based neural network (The examiner takes Official Notice that encoder-decoder (e.g., U-Net) neural-network architectures were well-known and conventional for biomedical image segmentation before the effective filing date. It would have been obvious to implement GOLDBERG's neural network as an encoder-decoder network to obtain the well-known pixel-level segmentation benefits of that architecture.).
Claim 13.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 4 wherein the method is provided as software-as-a-service by an operating entity that is distinct from the one or more entities (POLIT: “a subscription fee for access via a network to a remote image system management service” (¶ 10)).
Claim 14.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 1 wherein the set of two or more image devices includes imaging devices of a same type (POLIT: “This matching of performance characteristics of image acquisition devices 16 may be useful when, for example, a user desired multiple image acquisition devices 16 in image system 12 to provide consistent image data regardless of differences in the devices” (¶ 30). “by manufacturer and/or model number” (¶ 24).).
Claim 15.
GOLDBERG in view of SMEDMAN in view of POLIT teaches the method as described in claim 1 wherein the set of two or more imaging devices includes first and second imaging devices of a same type and with distinct configurations (POLIT: provide consistent data “regardless of differences in the devices,” (¶ 30) each with its own stored “configuration information” (¶ 24).).
Conclusion
The prior art made of record but not relied, yet considered pertinent to the applicant’s disclosure, is listed on the PTO-892 form.
US 6,410,252 B1 (Lehmann et al., Case Western Reserve University) discloses automated ELISPOT spot counting (US 6,410,252: "the present invention employs a computer coupled to a camera and a microscope to locate and categorize spots in multiple wells of a typical assay plate" (Method of Automated Image Analysis). This is conventional parametric spot counting of the type referenced at specification ¶5.) This reference names Alexey Karulin, who is also a named inventor of the present application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights