DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
As detailed on the Filing Receipt filed 5/3/2023, the instant application claims priority to as early as 4/19/2022. At this point in prosecution, all claims are accorded the earliest claimed priority date.
Inventorship
35 USC § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
35 USC § 115(a) reads as follows (in part):
An application for patent that is filed under section 111(a) or commences the national stage under section 371 shall include, or be amended to include, the name of the inventor for any invention claimed in the application.
The present application sets forth the incorrect inventorship because a published prior art reference has similar content to Applicant’s disclosure, and listed provisional patent application (63/632,646), but indicates a different inventive entity from that listed in the present application. Specifically, the reference is: Opadeji, Victor T, ‘Accurate Sickle Cell Detection Using Deep Transfer Learning Based Feature Extraction, Classification, and Retrieval of Blood Smear Images’ [thesis], Morgan State University, published May 2021.
This reference is the Master’s thesis of Opadeji, Victor Toluwanimi (hereafter, “Opadeji”), who is listed as its author. Some inventive connection between the Applicant and the reference is clear, as the concerned thesis work was carried out at Morgan State University and Rahman, MD Mahmudur is listed as the thesis chair.
The reference nonetheless indicates authorship and submission by Opadeji in partial fulfillment of the degree of Master of Science (pg. iii). The reference further indicates that the embodied thesis was approved (pg. iv), while the cover styling of Opadeji as ‘Master of Science in Bioinformatics’ (pg. i) indicates that Opadeji was in fact conferred said degree based in part upon the work therein. Review of the thesis demonstrates significant subject matter correspondence between the thesis and the present application (see ‘Claim Rejections – 35 USC § 102’ section below for full details).
The nature and content of the reference may thus indicate joint inventorship by Opadeji of the subject matter of the present application based on the preponderance of the evidence standard. However, Opadeji is not listed as an inventor in the present application.
Claims 1-17 are rejected under 35 USC §§ 101 and 115(a) for failing to set forth the correct inventorship for the reasons stated above.
Additional Claim Rejections - 35 USC § 101
A quotation of 35 USC § 101, which forms the basis for the rejections under this section made in this Office action, can be found in the ‘Inventorship’ section above.
Claims 1-17 are further rejected under 35 USC § 101 because the claimed invention is directed to an abstract idea and a natural phenomenon without significantly more (i.e., non-statutory subject matter).
"Claims directed to nothing more than abstract ideas, natural phenomena, and laws of nature are not eligible for patent protection" (MPEP 2106.04 § I).
Abstract ideas include mathematical concepts (including formulas, equations and calculations), and procedures for evaluating, analyzing or organizing information, which are a type of mental process (MPEP 2106.04(a)(2)).
Natural phenomena and laws of nature include principles, relations, and products that are naturally occurring or do not have markedly different characteristics compared to what occurs in nature (MPEP 2106.04(b)).
The claims as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea and a natural phenomenon.
Step 1: The Four Categories of Statutory Subject Matter (MPEP 2106.03)
The claims are directed to a method (claims 1-7), system (claims 8-14) and non-transitory computer-readable medium (claims 15-17), which fall under categories of statutory subject matter.
Step 2A, Prong One: Whether the Claims Set Forth or Describe a Judicial Exception (MPEP 2106.04 § II.A.1)
‘Mathematical concepts’ are relationships between variables and numbers, numerical formulas or equations, or acts of calculation, which need not be expressed in mathematical symbols (MPEP 2106.04(a)(2) § I). The claims recite elements which encompass mathematical concepts, at least under their broadest reasonable interpretation, including:
applying a deep feature extraction to a query image to generate a feature vector (claims 3 and 10), i.e., calculating an output vector, via a machine learning algorithm, based on an input array of pixel values, further comprising:
using a plurality of pretrained convolutional neural networks feature vectors to generate a combined feature vector (claims 4 and 11), and
using both logistical regression and support vector classifier processes (claims 6 and 13); and
steps are performed based on a distance measure between feature vectors (claims 7, 14 and 17).
The recited acts of calculation constitute mathematical concepts.
‘Mental processes’ are processes that can be performed in the human mind at least with use of a physical aid, e.g., a slide rule or pen and paper (MPEP 2106.04(a)(2) § III). The claims recite elements that encompass processes that are practicably performable in the human mind, at least under their broadest reasonable interpretation, including:
comparing a query image to a plurality of images (claims 1, 8 and 15);
selecting a plurality of images that have a designated similarity to the query image (claims 1, 8 and 15), i.e., selecting images based on an associated metric; and
applying a classification to the feature vector (claims 5 and 12), i.e., associating an attribute with a vector.
The recited steps of evaluating information, which are practicably performable in the human mind, constitute mental processes.
Mathematical concepts and mental processes are enumerated groupings of abstract ideas (MPEP 2106.04(a)(2) §§ I and III). Hence, the claims recite elements that, individually and in combination, constitute an abstract idea.
The claims further recite the following claim elements, which require that analyzed data embodies particular natural phenomena and/or laws of nature:
the method operates upon captured images of blood smears and blood smear images corresponding to pathologically confirmed types of sickle cell disease (claims 1, 8 and 15);
displayed probabilities regard the query image blood smear having a pathology matching each of a plurality of sickle cell disease types (claims 1, 8 and 15); and
classifications correspond to types of sickle cell disease (claims 5 and 12).
The above elements specify that analyzed data represents naturally occurring user attributes, i.e., natural phenomena, having naturally occurring relationships with user creatinine levels and health risk, i.e., laws of nature, that the claimed invention allows a user of the claimed method, system and/or computer-readable media to observe.
The claims must therefore be examined further to determine whether they integrate these judicial exceptions into a practical application (MPEP 2106.04(d)).
Step 2A, Prong Two: Whether the Claims Contain Additional Elements that Integrate the Judicial Exception(s) into a Practical Application (MPEP 2106.04 § II.A.2)
The claims recite additional elements that gather data necessary for performance of claimed method steps, including:
receiving a query image from a data capture device (claims 1, 8 and 15).
Necessary data gathering is considered to be insignificant pre-solution activity, and as such insufficient to integrate an abstract idea into a practical application (MPEP 2106.05(g)).
The claims further recite additional elements that output results of claimed method steps, including:
displaying, to a user, probabilities (claims 1, 8 and 15); and
displaying, to a user, said plurality of images (claims 2, 9 and 16).
Data output is also considered to be insignificant pre-solution activity, and as such insufficient to integrate an abstract idea into a practical application (MPEP 2106.05(g)).
The claims further recite additional elements that require performance of claimed functions on a computer and/or constitute computer hardware for performing claimed functions, including:
steps are performed at a computer processor and/or cause a processor to perform activity (claims 1, 8 and 15);
steps operate upon digital images (claims 1, 8 and 15):
a system comprising a memory and a processor in data communication with said memory, the memory having computer executable instructions stored thereon configured to be executed by the processor to cause the system to perform claimed functions (claim 8); and
a non-transitory computer-readable medium having stored thereon one or more code sections each comprising a plurality of instructions executable by one or more processors, the instructions configured to cause the one or more processors to perform claimed functions (claim 15).
The claims do not describe any specific computational steps by which a computer performs or carries out functions drawn to the judicial exceptions, nor do they provide any details of how specific structures of a computer are used to implement these functions. The claims state nothing more than that a generic computer performs functions drawn to the judicial exceptions, and are therefore mere instructions to apply the judicial exceptions using a computer. As such, the claims do not integrate the judicial exceptions into a practical application (see MPEP 2106.04(d) § I and 2106.05(f)).
No further additional elements are recited.
When the claims are considered as a whole: they do not improve the functioning of a computer, other technology, or technical field (MPEP 2106.04(d)(1) and 2106.05(a)); they do not apply the judicial exceptions to effect a particular treatment or prophylaxis for a disease or medical condition (MPEP 2106.04(d)(2)); they do not implement the judicial exceptions with, or in conjunction with, a particular machine (MPEP 2106.05(b)); they do not effect a transformation or reduction of a particular article to a different state or thing (MPEP 2106.05(c)); and they do not apply or use the judicial exceptions in some other meaningful way beyond linking the use of the judicial exceptions to a particular technological environment and/or field of use (e.g., diagnosis of sickle cell disease; MPEP 2106.05(e) and 2106.05(h)).
Hence, the recited judicial exceptions are not integrated into a practical application. See MPEP 2106.04(d) § I.
Because the claims recite an abstract idea and a natural phenomenon, and do not integrate those judicial exceptions into a practical application, the claims are directed to those judicial exceptions. Claims that are directed to judicial exceptions must be examined further to determine whether the additional elements besides the judicial exceptions render the claims significantly more than the judicial exceptions. Additional elements besides the judicial exceptions may constitute inventive concepts that are sufficient to render the claims significantly more (MPEP 2106.05).
Step 2B: Whether the Claims Contain Additional Elements that Amount to an Inventive Concept (MPEP 2106.05)
As noted above, several recited additional elements amount to insignificant extra-solution activity. Mere addition of insignificant extra-solution activity does not amount to an inventive concept that would render the claims significantly more than the recited judicial exceptions, particularly when the activities are well-understood or conventional (MPEP 2106.05(g)). The conventionality of recited additional elements that amount to insignificant extra-solution activity must be further considered.
Recited additional elements amounting to insignificant extra-solution activity encompass the following, which are indicated as activity that may be performed with general purpose (i.e., commercially-available) products by the instant specification (see MPEP 2106.07(a) § III):
receiving blood smear images (para. 0022 indicates that blood smear images can be obtained via the internet from Google Images, i.e., via the internet); and
computer implementation (para. 0044 indicates that the computer system may be a general-purpose desktop computer, while para. 0049 indicates that the computer-readable medium may be any type of random access memory, hard drive, floppy disk and so on).
Additionally, recited additional elements amounting to insignificant extra-solution activity encompass the following computer-implemented functions, which the courts have held as coextensive with a general-purpose computer and/or well-understood, routine and conventional:
Receiving, storing, and processing data (In re Katz Interactive Call Processing Patent Litigation, 639 F.3d 1303, 1316 (Fed. Cir. 2011); EON Corp. IP Holdings LLC v. AT&T Mobility LLC, 785 F.3d 616, 622 (Fed. Cir. 2015));
Receiving or transmitting data over a network (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355 (Fed. Cir. 2014); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015));
Storing and retrieving information (OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015); Versata Dev. Group, Inc. v. SAP America, Inc., 793 F.3d 1306, 1334 (Fed. Cir. 2015));
Selecting information for display (Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55 (Fed. Cir. 2016));
Displaying the result of data analysis (TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 612-13 (Fed. Cir. 2016));
Displaying electronic data (Interval Licensing LLC v. AOL, Inc., 896 F.3d 1335, 1344-45 (Fed. Cir. 2018));
Hence, the encompassed extra-solution activity is considered well-understood, routine and conventional. Well-understood, routine and conventional activity is insufficient to constitute an inventive concept that would render the claims significantly more than judicial exceptions (MPEP 2106.05(d)).
Mere instructions to implement judicial exceptions using a computer are, when considered individually, similarly insufficient to constitute an inventive concept that would render the claims significantly more than said judicial exceptions (see MPEP 2106.05(f)).
When the claims are considered as a whole, they do not integrate the judicial exceptions into a practical application; they do not confine the use of the judicial exceptions to a particular technology; they do not solve a problem rooted in or arising from the use of a
particular technology; they do not improve a technology by allowing the technology to
perform a function that it previously was not capable of performing; and they do not
provide any limitations beyond generally linking the use of the judicial exceptions to a particular technological environment and/or field of use (e.g., diagnosis of sickle cell disease; MPEP 2106.05(e) and 2106.05(h)).
Hence, the claims do not include additional elements that are sufficient to amount to significantly more than the recited judicial exceptions. See MPEP 2106.05.
Conclusion: Claims are Directed to Non-statutory Subject Matter
For these reasons, the claims, when the limitations are considered individually and as a whole, are directed to judicial exceptions and lack an inventive concept. Hence, the claimed invention does not constitute significantly more than the judicial exceptions, so the claims are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 USC §§ 102 and 103 is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraph of 35 USC § 102 that forms the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7-12 and 14-17 are rejected under 35 USC § 102(a)(1) based upon a public use or sale or other public availability of the invention, as indicated by Opadeji (‘Accurate Sickle Cell Detection Using Deep Transfer Learning Based Feature Extraction, Classification, and Retrieval of Blood Smear Images’ [thesis], Morgan State University; published May 2021).
Claim 1 is directed to an automated method for diagnosing sickle cell disease type from a blood smear image, comprising: receiving at a processor of a diagnosing system computer a digital query image of a blood smear from a data capture device; comparing at said processor said digital query image to a plurality of digital images in a database, wherein said database comprises digital blood smear images of pathologically confirmed types of sickle cell disease; selecting at said processor a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and causing said processor to display to a user, probabilities that said digital query image displays a blood smear having a pathology matching each of a plurality of sickle cell disease types.
With respect to claim 1, Opadeji discusses the development of an automated system for sickle cell disease detection by performing classification and retrieval of blood smear images (pg. ii, para. 1). Opadeji contextualizes the presented work by discussing prior implementation of blood cell image classification via computer technology, and describes image classification as a process comprising computer recognition of pixels within an input image (pg. 4, paras. 1-2). Given this discussion, one of ordinary skill in the art would understand that the disclosed techniques are necessarily implemented at a processor of a computer.
Opadeji describes image retrieval as a process wherein a query image is input to a machine learning algorithm by a user, and the expected outcome is an array of images with the same features as the query (pg. 4, para. 3). In other words, given the computing environment, a process comprising receiving, at a processor of a computer, a digital query image. Opadeji also discusses capture of blood sample images using a smartphone-based microscope (pg. 18, para. 4 – pg. 19, para. 1), i.e., a capture device.
Opadeji discusses provision by the disclosed techniques of a computerized library with a set of pathologically confirmed images of past cases (pg. ii, para. 1). Opadeji further discusses extracting features from a set of training images, having abnormal cell shapes annotated with a plurality of sickle cell shape labels (e.g., Ss and Sc), and saving them in a database (pg. 29, paras. 1-3). One of ordinary skill in the art would understand the disclosed database as equivalent to the claimed database.
Opadeji discusses comparing query image features with database image features according to similarity, and returning a given number (N) of the top most similar images (pg. 29, para. 1).
Opadeji discloses classification output including predicted label and prediction probability (pg. 15, para. 3), i.e., the probability that the query matches a given label. Opadeji discusses classifying images according to a plurality of sickle-cell disease types, e.g., SS and SC (pg. 8, para. 3; pg. 11, paras. 2-4; pg. 22, para. 4 – pg. 23, para. 1), wherein image-level classification depends on constituent sickle cell shapes (pg. 26, paras. 2-3).
With respect to claim 2, Opadeji hypothesizes that providing a computerized library with a set of pathologically confirmed images of past cases can aid a pathologist in diagnosis via concrete visualizations (pg. ii, para. 1). In this way, Opadeji is considered to disclose displaying a set of pathologically confirmed digital images to a user.
With respect to claim 3, Opadeji discloses calculating feature vectors of query and reference images (pg. 31, para. 2).
With respect to claim 4, Opadeji discloses extraction of features using a deep transfer learning model (the ‘newly built model’) consisting of a plurality of pre-trained convolutional neural networks (ResNet50, Inception V3 and VGG-16) and fully-connected layers, wherein features are passed from the penultimate layer of each pre-trained model through the fully-connected layers and output as a combined feature vector (pg. 6, para. 1 – pg. 8, para. 1; pg. 27, para. 3; pg. 29, Fig. 11, description of feature vector as ‘from the fully connected layers’).
With respect to claim 5, Opadeji discusses classifying images according to multiple types of sickle-cell disease, e.g., SS and SC (pg. 8, para. 3; pg. 11, paras. 2-4; pg. 22, para. 4 – pg. 23, para. 1).
With respect to claim 7, Opadeji discloses assessment of similarity between query and reference images by calculating Euclidean distance measures between corresponding feature vectors (pg. 31, para. 2).
Claim 8 is directed to a system for the automated diagnosing of sickle cell disease type from a blood smear image, comprising a memory and a processor in data communication with said memory, the memory having computer executable instructions stored thereon configured to be executed by the processor to cause the system to perform recited functions bearing substantive similarity to the recited steps of the method of claim 1.
With respect to claim 8, Opadeji discusses the development of an automated system for sickle cell disease detection by performing classification and retrieval of blood smear images (pg. ii, para. 1). Opadeji contextualizes the presented work by discussing prior implementation of blood cell image classification via computer technology, and describes image classification as a process comprising computer recognition of pixels within an input image (pg. 4, paras. 1-2). Given this discussion, one of ordinary skill in the art would understand that the disclosed techniques are necessarily implemented using a computer. A processor and memory are generic hardware components of a computer.
The disclosure of Opadeji is considered to read on the functional limitations of the claim in the same manner as outlined above with respect to the process limitations of claim 1.
With respect to claim 9, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 2.
With respect to claim 10, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 3.
With respect to claim 11, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 4.
With respect to claim 12, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 5.
With respect to claim 14, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 7.
Claim 15 is directed to a non-transitory computer-readable medium having stored thereon one or more code sections each comprising a plurality of instructions executable by one or more processors, the instructions configured to cause the one or more processors to perform the actions of an automated method for diagnosing a sickle cell disease type, the actions of the method comprising recited steps bearing substantive similarity to the recited steps of the method of claim 1.
With respect to claim 15, Opadeji discloses implementation of the discussed techniques using Python (pg. ii, para. 1; pg. 22, para. 3). Python is a computer programming language, and Python implementation would take the form of one or more processor-executable code sections stored on a computer readable medium as claimed.
The disclosure of Opadeji is considered to read on the functional limitations of the claim in the same manner as outlined above with respect to the process limitations of claim 1.
With respect to claim 16, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 2.
With respect to claim 17, the disclosure of Opadeji is considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 7.
In this way, the disclosure of Opadeji anticipates the limitations of claims 1-5, 7-12 and 14-17. Thus, the claimed invention is anticipated.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 USC § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 USC § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-17 are rejected under 35 USC § 103 as being obvious over Layode (Proceedings of the 2019 IEEE Applied Imagery Pattern Recognition Workshop, 7 pages; conference held 10/15-17/2019, available online via IEEE Xplore 8/24/2020), in view of Cumming (US 2021/0248419; effectively filed 3/7/2018).
Claim 1 is directed to an automated method for diagnosing sickle cell disease type from a blood smear image, comprising: receiving at a processor of a diagnosing system computer a digital query image of a blood smear from a data capture device; comparing at said processor said digital query image to a plurality of digital images in a database, wherein said database comprises digital blood smear images of pathologically confirmed types of sickle cell disease; selecting at said processor a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and causing said processor to display to a user, probabilities that said digital query image displays a blood smear having a pathology matching each of a plurality of sickle cell disease types.
With respect to claim 1, Layode discloses an integrated classification and retrieval based decision support system for skin cancer detection (pg. 1, Abstract) and discusses operation wherein: an unknown query image is submitted to the system (pg. 2, r. column), i.e., the system receives a query image; and similarity matching is performed between the query image and each reference image in a database to find the k most similar reference regions-of-interest to the queried region-of-interest (pg. 4, r. column), i.e., comparing said query image to a plurality of images in a database and selecting a plurality of images from said database that have a designated similarity to said query image.
Layode characterizes the operation of the system, in retrieving images along with a tagged proven pathological diagnosis, as providing a set of pathologically-confirmed cases as computer output that could be utilized to guide a dermatologist to a precise diagnosis (pg. 2, l. column). In other words, the queried reference database comprises images of pathologically confirmed types of skin cancer.
Layode further discloses that the trained network returns the probabilities for each class label, given an input data point (pg. 4, r. column), and depicts a graphical user interface implementation that displays results including class probabilities (pg. 6, l. column and Fig. 6).
The subject matter of Layode bears significantly structural similarity to the claimed invention. However, Layode particularly discloses application of the discussed deep learning-based classification and image retrieval system to process images of skin lesions for diagnosis of skin cancer types, while the method recited by claim 1 processes images of blood smears for diagnosis of sickle cell disease types.
Cumming discloses methods and systems for the identification and quantitation of objects of biological origin which are typically the subject of microscopic analysis, such as cells, via computer-based image recognition (Abstract). These include steps of: accessing a test image of a sample obtained by microscopy (para. 0032), wherein images may be captured via microscope or modified mobile device and output to a processor-enabled device, as an electronic image file, and the processor-enabled device is configured to perform further disclosed method steps (paras. 0047-50 and 0189); and comparing a test image against each labeled image stored in a reference database to identify a target biological material (para. 0175), wherein the test and reference image are compared based on their Euclidean distance and the probability that a particular set of features indicates the presence of an object is computed (paras. 0182-84).
Cumming discusses typical applications of microscopic analysis including for the diagnosis of medical diseases, e.g., to identify a particular type of host cell among other host cells (para. 0002), and discloses embodiments wherein the target biological material is a discrete cell and the location detection method identifies a plurality of discrete objects (paras. 0042-44). Of particular relevance, Cumming further discloses applications of the technique to identify blood cells (para. 0067) and to differentiate blood cell types including sickle cell (para. 0198).
With respect to claim 2, Layode depicts a graphical user interface implementation that displays a plurality of similar reference images (pg. 6, l. column and Fig. 6).
With respect to claim 3, Layode discloses performance of deep feature extraction to generate feature vectors from images, and discusses computation of similarity between the query image and the database based on calculating differences between the feature vector of the query image and the feature vectors of reference images (pg. 3, l. column – pg. 4, r. column).
With respect to claim 4, Layode discloses feature extraction using a plurality of pre-trained convolutional neural networks and generation of a combined feature vector therefrom (pg. 4, l. column and Fig. 3).
With respect to claim 5, Layode characterizes the function of the system as, based on an image-based visual query, classifying the image category as different types of skin cancer (pg. 1, Abstract). Layode also discloses extracting feature vectors from query images, and training classifiers on extracted feature vectors (pg. 3, l. column – pg. 4, r. column). In other words, the system applies a classification to the query feature vector as one of multiple types of skin cancer.
Cumming discusses typical applications of microscopic analysis including for the diagnosis of medical diseases, e.g., to identify a particular type of host cell among other host cells (para. 0002), and further discloses application of their technique to identify blood cells (para. 0067) and differentiate blood cell types including sickle cell (para. 0198).
With respect to claim 6, Layode discloses classification of images via an ensemble method comprising logistic regression and support vector classifier models (pg. 4; r. column and Fig. 3).
With respect to claim 7, Layode discloses similarity matching wherein, for a given query image, a feature-based search is made on the images from the dataset and the similarity between the query image and the database is computed based on calculating different distance measures between the feature vector of the query image and the feature vectors of reference images (pg. 4, r. column).
Claim 8 is directed to a system for the automated diagnosing of sickle cell disease type from a blood smear image, comprising a memory and a processor in data communication with said memory, the memory having computer executable instructions stored thereon configured to be executed by the processor to cause the system to perform recited functions bearing substantive similarity to the recited steps of the method of claim 1.
With respect to claim 8, Layode discusses an integrated classification and retrieval based decision support system (pg. 1, Abstract). Layode contextualizes the presented work by discussing prior computer-aided diagnosis systems of digital images (pg. 1, r. column), and characterizes operation of the system as providing computer output (pg. 2, l. column). Given this discussion, one of ordinary skill in the art would understand that the disclosed techniques are necessarily implemented using a computer. A processor and memory are generic hardware components of a computer.
Additionally, Cumming states that their methods and systems may be deployed through one or more processors that execute computer software, program codes and/or instructions, and may access an associated storage medium, e.g., memory for storing methods, codes, program instructions or other types of instructions capable of being executed by the processing device (paras. 0208-10).
The combined teachings of Layode and Cumming are considered to read on the functional limitations of the claim in the same manner as outlined above with respect to the process limitations of claim 1.
With respect to claim 9, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 2.
With respect to claim 10, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 3.
With respect to claim 11, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 4.
With respect to claim 12, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 5.
With respect to claim 13, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 6.
With respect to claim 14, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 7.
Claim 15 is directed to a non-transitory computer-readable medium having stored thereon one or more code sections each comprising a plurality of instructions executable by one or more processors, the instructions configured to cause the one or more processors to perform the actions of an automated method for diagnosing a sickle cell disease type, the actions of the method comprising recited steps bearing substantive similarity to the recited steps of the method of claim 1.
With respect to claim 15, Layode discloses implementation of the system as a graphical user interface via PyQt (pg. 6, l. column and Fig. 6), thus indicating that the disclosed techniques were implemented in the computer environment using Python. Python is a computer programming language, and Python implementation would take the form of one or more processor-executable code sections stored on a computer readable medium as claimed.
Additionally, Cumming states that their methods and systems may be deployed through one or more processors that may access an associated storage medium such as a hard disk, floppy disk or CD-ROM (i.e., a non-transitory computer readable storage medium) for storing methods, codes, program instructions or other types of instructions capable of being executed by the processing device (paras. 0208-10).
The combined teachings of Layode and Cumming are considered to read on the functional limitations of the claim in the same manner as outlined above with respect to the process limitations of claim 1.
With respect to claim 16, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 2.
With respect to claim 17, the combined teachings of Layode and Cumming are considered to read on the unique limitations of the claim in the same manner as outlined above with respect to the unique limitations of claim 7.
An invention would have been obvious to one of ordinary skill in the art if simple substitution of one known element for another, to yield predictable results, would have led one of ordinary skill in the art to arrive at the claimed invention. Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to apply the image classification techniques of Layode to process blood smear images for classification of sickle cell disease types, as Cumming indicates that image classification techniques are applicable to differential identification of sickle cells among blood cells (see paras. 0067 and 0198).
In this way the disclosure of Layode, in view of Cumming, makes obvious the limitations of claims 1-17. Thus, the claimed invention is prima facie obvious.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Instant claims 1-17 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7, 9-15 and 17-19 of U.S. Patent No. 11,538,577 (hereafter, “‘577”), in view of Cumming (cited above). ‘577 shares an inventor (Md Mahmadur Rahman) and common assignee (Morgan State University) with the instant application. Although the claims at issue are not identical, they are not patentably distinct from each other because: Instant claim 1 is directed to an automated method for diagnosing sickle cell disease type from a blood smear image, comprising: receiving at a processor of a diagnosing system computer a digital query image of a blood smear from a data capture device; comparing at said processor said digital query image to a plurality of digital images in a database, wherein said database comprises digital blood smear images of pathologically confirmed types of sickle cell disease; selecting at said processor a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and causing said processor to display to a user, probabilities that said digital query image displays a blood smear having a pathology matching each of a plurality of sickle cell disease types.
With respect to instant claim 1, ‘577 claims an automated method for diagnosing a skin cancer type from a dermoscopic image, comprising: receiving at a processor of a diagnosing system computer a digital query image of a skin lesion from a data capture device; comparing at said processor said digital query image to a plurality of digital images in a database, wherein said database comprises digital images of pathologically confirmed types of skin lesions; selecting at said processor a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and causing said processor to display to a user probabilities that said digital query image displays a skin lesion having a pathology matching each of a plurality of skin cancer types (claim 1).
The limitations are significantly similar between the two, however, the method recited by claim 1 of ‘577 processes images of skin lesions for diagnosis of skin cancer types, while the method recited by instant claim 1 processes images of blood smears for diagnosis of sickle cell disease types.
Cumming discloses methods and systems for the identification and quantitation of objects of biological origin which are typically the subject of microscopic analysis, such as cells, via computer-based image recognition (Abstract). These include steps of: accessing a test image of a sample obtained by microscopy (para. 0032), wherein images may be captured via microscope or modified mobile device and output to a processor-enabled device, as an electronic image file, and the processor-enabled device is configured to perform further disclosed method steps (paras. 0047-50 and 0189); and comparing a test image against each labeled image stored in a reference database to identify a target biological material (para. 0175), wherein the test and reference image are compared based on their Euclidean distance and the probability that a particular set of features indicates the presence of an object is computed (paras. 0182-84).
Cumming discusses typical applications of microscopic analysis including for the diagnosis of medical diseases, e.g., to identify a particular type of host cell among other host cells (para. 0002), and discloses embodiments wherein the target biological material is a discrete cell and the location detection method identifies a plurality of discrete objects (paras. 0042-44). Of particular relevance, Cumming further discloses applications of the technique to identify blood cells (para. 0067) and to differentiate blood cell types including sickle cell (para. 0198).
Although the techniques disclosed by Cumming are not fully equivalent to those of ‘577, they are significantly similar. For example, the techniques of Cumming and those of the reference patent both involve identifying constituent cells within and extracting feature vectors from test and reference biomedical images, calculating distance measures between test and reference feature vectors, and classifying images based thereon.
With respect to instant claim 2, ‘577 claims the method of claim 1 further comprising the step of causing said processor to display said plurality of pathologically confirmed digital images to said user (claim 2).
With respect to instant claim 3, ‘577 claims the method of claim 1, wherein said comparing step further comprises applying at said processor a deep feature extraction to said digital query image to generate a feature vector quantifying contents of the digital query image (claim 3).
Additionally, Cumming discloses method steps of: applying to the test image a computer-implemented location detection method configured to identify the location of a potential target biological material (e.g., sickle cell) in the test image, applying a computer-implemented feature extraction method to the location identified to provide one or a set of extracted features, and matching the extracted feature(s) with one or a set of extracted features stored on a database having one or a set of feature(s) in association with biological material identity information (para. 0032).
Cumming also discusses embodiments wherein the step of feature extraction comprises extracting potentially identification-relevant feature vectors (para. 0087), and discloses comparison of test and reference images, wherein features are extracted from the test image file and the reference image file, and the test and reference image are compared based on Euclidean distance of their feature vectors, and the probability that a particular set of features indicates the presence of an object is computed (paras. 0182-84).
With respect to instant claim 4, ‘577 claims the method of claim 3 wherein said step of applying a deep feature extraction to said digital query image further comprises using at said processor a plurality of pretrained Convolutional Neural Networks feature vectors to generate a combined feature vector (claim 4).
Additionally, Cumming discloses feature extraction wherein feature vectors are generated from the second-to-last layer of a convolutional neural network (para. 0111).
With respect to instant claim 5, ‘577 claims the method of claim 3, wherein said comparing step further comprises applying at said processor a classification to said feature vector as one of multiple types of skin cancer (claim 5).
Cumming discusses typical applications of microscopic analysis including for the diagnosis of medical diseases, e.g., to identify a particular type of host cell among other host cells (para. 0002), and further discloses application of their technique to identify blood cells (para. 0067) and differentiate blood cell types including sickle cell (para. 0198).
With respect to instant claim 6, ‘577 claims the method of claim 5, wherein applying a classification to said feature vector further comprising using both Logistical Regression and Support Vector Classifier processes (claim 6).
Additionally, Cumming discloses embodiments wherein feature vectors are passed to a support vector machine for classification (para. 0111).
With respect to instant claim 7, ‘577 claims the method of claim 1, further comprising the step of causing said processor to select said plurality of said pathologically confirmed digital images based on a distance measure between a feature vector of said digital query image and said plurality of pathologically confirmed digital images (claim 7).
Additionally, Cumming discloses comparison of test and reference images wherein features are extracted from the test image file and the reference image file, and the test and reference image are compared based on Euclidean distance between their feature vectors (paras. 0182-84).
Instant claim 8 is directed to a system for the automated diagnosing of sickle cell disease type from a blood smear image, comprising a memory and a processor in data communication with said memory, the memory having computer executable instructions stored thereon configured to be executed by the processor to cause the system to perform recited functions bearing substantive similarity to the recited steps of the method of instant claim 1.
With respect to instant claim 8, ‘577 claims a system for the automated diagnosing of a skin cancer type from a dermoscopic image, comprising a memory and a processor in data communication with said memory, the memory having computer executable instructions stored thereon configured to be executed by the processor to cause the system to: receive a digital query image of a skin lesion from an image capture device; compare at said processor said digital query image to a plurality of digital images in a database, wherein said database comprises digital images of pathologically confirmed types of skin lesions; select a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and display to a user probabilities that said digital query image displays a skin lesion having a pathology matching each of a plurality of skin cancer types (claim 9).
The combination of ‘577 and Cumming is considered to further read on the instant claim in the same manner as outlined above with respect to instant claim 1.
With respect to instant claim 9, ‘577 claims the system of claim 9, wherein said computer executable instructions are further configured to cause said processor to display said plurality of pathologically confirmed digital images to said user (claim 10).
With respect to instant claim 10, ‘577 claims the system of claim 9, wherein said computer executable instructions configured to compare said digital query image to the plurality of digital images are further configured to apply a deep feature extraction to said digital query image to generate a feature vector quantifying contents of the digital query image (claim 11).
With respect to instant claim 11, ‘577 claims the system of claim 11, wherein said computer executable instructions configured to apply a deep feature extraction to said digital query image are further configured to use a plurality of pretrained Convolutional Neural Networks feature vectors to generate a combined feature vector (claim 12).
With respect to instant claim 12, ‘577 claims the system of claim 11, wherein said computer executable instructions configured to compare said digital query image to the plurality of digital images are further configured to apply a classification to said feature vector as one of multiple types of skin cancer (claim 13).
With respect to instant claim 13, ‘577 claims the system of claim 13, wherein said computer executable instructions configured to apply a classification to said feature vector are further configured to use both Logistical Regression and Support Vector Classifier processes (claim 14).
With respect to instant claim 14, ‘577 claims the system of claim 9, wherein said computer executable instructions are further configured to select said plurality of said pathologically confirmed digital images based on a distance measure between a feature vector of said digital query image and said plurality of pathologically confirmed digital images (claim 15).
Instant claim 15 is directed to a non-transitory computer-readable medium having stored thereon one or more code sections each comprising a plurality of instructions executable by one or more processors, the instructions configured to cause the one or more processors to perform the actions of an automated method for diagnosing a sickle cell disease type, the actions of the method comprising recited steps bearing substantive similarity to the recited steps of the method of instant claim 1.
With respect to instant claim 15, ‘577 claims a non-transitory computer-readable medium having stored thereon one or more code sections each comprising a plurality of instructions executable by one or more processors to perform the actions of an automated method for diagnosing a skin cancer type, the actions of the method comprising the steps of: receiving a digital query image of a skin lesion from an image capture device; comparing said digital query image to a plurality of digital images in a database, wherein said database comprises digital images of pathologically confirmed types of skin lesions; selecting a plurality of said pathologically confirmed digital images from said database that have a designated similarity to said digital query image; and displaying. to a user probabilities that said digital query image displays a skin lesion having a pathology matching each of a plurality of skin cancer types (claim 17).
The combination of ‘577 and Cumming is considered to further read on the instant claim in the same manner as outlined above with respect to instant claim 1.
With respect to instant claim 16, ‘577 claims the non-transitory computer-readable medium of claim 17, the method further comprising the step of causing said processor to display said plurality of pathologically confirmed digital images to said user (claim 18).
With respect to instant claim 17, ‘577 claims the non-transitory computer-readable medium of claim 17, the method further comprising the step of selecting said plurality of said pathologically confirmed digital images based on a distance measure between a feature vector of said digital query image and said plurality of pathologically confirmed digital images (claim 19). An invention would have been obvious to one of ordinary skill in the art if simple substitution of one known element for another, to yield predictable results, would have led one of ordinary skill in the art to arrive at the claimed invention. Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to apply the image classification techniques of ‘577 to process blood smear images for classification of sickle cell disease types, as Cumming indicates that image classification techniques are applicable to differential identification of sickle cells among blood cells (see paras. 0067 and 0198).
In this way instant claims 1-17 are not patentably distinct from claims of ‘577, in view of Cumming.
Conclusion
At this point in prosecution, no claim is allowed.
The following prior art, made of record and not relied upon, is considered pertinent to applicant's disclosure:
Alzubaidi (Electronics 9: 427, 18 pages; published 3/4/2020) discusses classification of microscopy images based on constituent red blood cell shapes, via a transfer learning model, for diagnosis of sickle cell disease (pg. 1, Abstract);
Rahman (CEUR Workshop Proceedings 2696(202), 10 pages; published September 2020) discusses automated multi-label concept detection in medical images (pg. 1, Abstract);
Sanghavi (Proc 2013 Nat’l Conf on Innov. Paradigms in Eng & Tech, pp. 11-15; published 2013) discusses comparison of blood sample image features to a database and according classification, for automatic diagnosis of diseases including sickle cell anemia, via a content-based image retrieval framework (pg. 11, Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Theodore C. Striegel whose telephone number is (571)272-1860. The examiner can normally be reached Mon-Fri 12pm-8pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Olivia M. Wise can be reached at (571)272-2249. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.C.S./Examiner, Art Unit 1685
/JESSE P FRUMKIN/Primary Examiner, Art Unit 1685 September 4, 2026
Prosecution Insights