DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1, 2, 4, 7 and 13-18 have been amended.
Claims 19 and 20 are newly presented.
Claims 1-20 as presented June 25, 2026 are currently pending and considered below.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. This is a new matter rejection.
As to claim 1, 7 and 13, the claims recite the “generating one or more transformed features…the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata”. The amended limitation “natural language description” constitutes new matter not supported by the specification. A review of the specification reveals the phrase “natural language” is utilized exclusively to describe an incoming query provided by the user to the system ([0027], [0072], [0081]). The specification never uses the term “natural language” to characterize the generated output. Instead, the specification defines the generated output features as being stored as “plain text” ([0064], [0068]) or “unstructured text” ([0068]). Thus, the specification fails to provide written description support for using the phrase “natural language description” to describe the generated transformed features or the system’s output. For the purposes of compact prosecution, the claim will be interpreted in a manner as best understood by the examiner and in light of the specification as “generating one or more transformed features…the one or more transformed features comprises formatting the extracted features into plain text or unstructured text identifying i) a corresponding physical feature, ii) metadata”.
Claims 2-6, 19 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph due to their dependence on claim 1.
Claims 8-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph due to their dependence on claim 7.
Claims 14-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph due to their dependence on claim 13.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Claims 1-6, 19 and 20 recite a method for automatically generating a structured medical record from endoscopy data, which is within the statutory category of a process. Claims 7-12 recite a data processing system configured for automatically generating a structured medical record from endoscopy data, which is within the statutory category of a machine. Claims 13-18 recite one or more non-transitory computer readable media storing instructions for automatically generating a structured medical record from endoscopy data, which is within the statutory category of an article of manufacture.
Step 2A - Prong One:
Regarding Prong One of Step 2A, the claim limitations are to be analyzed to determine whether, under their broadest reasonable interpretation, they "recite" a judicial exception or in other words whether a judicial exception is "set forth" or "described" in the claims. An "abstract idea" judicial exception is subject matter that falls within at least one of the following groupings: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. Representative independent claim 1 includes limitations that recite at least one abstract idea.
Specifically, independent claim 1 recites: A method for automatically generating a structured medical record from endoscopy data, the method comprising:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store;
obtaining image data comprising a sequence of video frames of a gastrointestinal (GI) tract of a patient;
identifying one or more symptoms of a disease by signature matching a region of a video frame of the sequence of video frames to a known signature representing a symptom;
tagging the identified symptoms to exact anatomical landmarks within the GI tract;
based on the tagging, extracting one or more features each representing a physical feature of the GI tract;
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification, the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof; and
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.
The underlined limitations constitute methods of organizing human activity and concepts performed in the human mind. The claim recites steps of generating one or more transformed features corresponding to one or more data entries, receiving [a request] and providing the data entries. These steps, under its broadest reasonable interpretation, are categorized as methods of organizing human activity, specifically associated with managing personal behavior or relationships or interactions between people (e.g. creating a medical record). The claim encompasses a person following rules or instructions to receive and process data in the manner described in the abstract idea. If the claim limitation, under its broadest reasonable interpretation, covers managing personal behavior or interactions between people but for the recitation of generic computer components, then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. See MPEP § 2106.04(a). The Examiner further notes that “Certain Methods of Organizing Human Activity” includes a person's interaction with a computer (see October 2019 Update: Subject Matter Eligibility at Pg. 5). Additionally, the claim encompasses a mental process of identifying one or more symptoms of a disease by signature matching, tagging the identified symptoms, extracting one or more features, and a natural language description of a feature or metadata. The identified abstract idea, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind except for the recitation of generic computer components. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind except for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. The abstract idea for Claims 7 and 13 are identical as the abstract idea for Claim 1, because the only difference between Claim 1 and 7 is that Claim 1 recites a method, whereas Claim 7 recites a system, and because the only difference between Claims 1 and 13 is that Claim 1 recites a method, whereas Claim 13 recites one or more non-transitory computer readable media. Any limitations not identified above as part of methods of organizing human activity or mental process, are deemed “additional elements” and will be discussed further in detail below. Accordingly, claims 1, 7 and 13 recite at least one abstract idea.
Similarly, dependent claims 2-6, 8-12 and 14-18 further narrow the abstract idea described in the independent claims. Claims 2, 3, 8, 9, 14, 15 and 20 describe applying a model to identify and classify the malignancy. Claims 4, 5, 10, 11, 16 and 17 describe comparing current and historical medical records to determine physiological changes over time. Claims 6, 12 and 18 describe processing the query. Claims 2, 4, 6, 8, 10, 12, 14 and 16 partially narrow the abstract idea as described above, and also introduce additional element(s) which will be discussed in Step 2A Prong 2 and Step 2B. These limitations only serve to further limit the abstract idea and hence, are directed toward fundamentally the same abstract ideas as independent claims 1, 7 and 13, even when considered individually and as an ordered combination.
Step 2A - Prong Two:
Regarding Prong Two of Step 2A, it must be determined whether the claim as a whole integrates the abstract idea into a practical application. It must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a "practical application."
In the present case, claims 1-20 as a whole do not integrate the abstract idea into a practical application because they do not impose meaningful limits on practicing the abstract idea. The additional elements or combination of additional elements, beyond the above-noted at least one abstract idea will be described as follows (where the bolded portions are the “additional limitations” while the underlined portions continue to represent the “abstract idea(s)”).
Specifically, independent claim 1 recites: A method for automatically generating a structured medical record from endoscopy data, the method comprising:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store;
obtaining image data comprising a sequence of video frames of a gastrointestinal (GI) tract of a patient;
identifying one or more symptoms of a disease by signature matching a region of a video frame of the sequence of video frames to a known signature representing a symptom;
tagging the identified symptoms to exact anatomical landmarks within the GI tract;
based on the tagging, extracting one or more features each representing a physical feature of the GI tract;
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification, the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof; and
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.
The independent claims recite the additional elements of a data store, data processing system, endoscopy data, image data, video frames of a GI tract, structured queries, one or more non-transitory computer readable media, processor, and memory that implement the identified abstract idea. The data store, data processing system, structured queries, one or more non-transitory computer readable media, processor, and memory are not described by the applicant and are recited at a high-level of generality such that they amounts to no more than mere instructions to apply the exception using a generic computer component (i.e., merely invoking the computer structure as a tool used to execute the limitations, MPEP 2106.05(f)). The endoscopy data, image data, and video frames of a GI tract are recited at a high-level of generality such that they are generally linking the use of a judicial exception to a particular technological environment or field of use, and thus, do not integrate a judicial exception into a practical application.
Claim 1 further recites the additional elements of obtaining a data specification, obtaining image data and storing data entries. Claims 2, 4, 8, 10, 14 and 16 recite outputting feature data and accessing data entries. Obtaining a data specification, obtaining image data, storing data entries, outputting feature data and accessing data entries are forms of extra-solution activity. MPEP 2106.5(g) indicates the term "extra-solution activity" can be understood as activities incidental to the primary process or product that are merely a nominal or tangential addition to the claim. Therefore, even in combination, these additional elements do not integrate the abstract idea into a practical application.
The dependent claims 2, 6, 8, 12, 14, 18 and 19 recite additional element(s) beyond those already recited in the independent claims that implement the identified abstract idea. Claims 2, 8, and 14 recite a machine learning model. Claims 6, 12 and 18 recites a trained language model. Claim 19 recites API functions. However, these additional elements do not integrate the abstract idea into a practical application because, as stated above, they represent mere instructions to apply the abstract idea on a computer (i.e., merely invoking the additional element as a tool used to execute the limitations).
Accordingly, the claims as a whole do not integrate the abstract idea into a practical application as they do not impose any meaningful limits on practicing the abstract idea.
Step 2B
Regarding Step 2B, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application.
When viewed as a whole, claims 1-20 do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims recite processes that are abstract and simply implementing the process on a computer(s) is not enough to qualify as "significantly more."
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a data store, data processing system, structured queries, one or more non-transitory computer readable media, processor, and memory to perform the noted steps amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept (“significantly more”). In addition, the additional elements of endoscopy data, image data and video frames of a GI tract generally link the use of a judicial exception to a particular technological environment or field of use, and thus, do not amount to significantly more than the judicial exception.
Also, as discussed above with respect to integration of the abstract idea into a practical application, the additional elements of obtaining a data specification, obtaining image data, storing data entries, outputting feature data and accessing data entries were considered extra-solution activity. This has been re-evaluated under the “significantly more” analysis and determined to be well-understood, routine, conventional activity in the field. MPEP 2106.05(d)(II) indicates storing and retrieving data from memory is recognized as well- understood, routine, and conventional activity of a general-purpose computing device (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)). MPEP indicates that displaying results of a rudimentary analysis (TLI Communications, 823 F.3d at 612-13,118 USPQ2d at 1747-48) (MPEP 2106.05(a)) and/or mere data gathering (MPEP 2106.05(g)) has been held by the courts to be insignificant extra-solution activity. Well-understood, routine, conventional activity cannot provide an inventive concept (“significantly more”). As such, the claims also do not recite significantly more than the abstract idea and are not patent eligible.
The dependent claims 2, 6, 8, 12, 14, 18 and 19 recite additional element(s) beyond those already recited in the independent claims that implement the identified abstract idea. Claims 2, 8, and 14 recite a machine learning model. Claims 6, 12 and 18 recites a trained language model. Claim 19 recites API functions. However, these functions are not deemed significantly more than the abstract idea because, as stated above, they represent mere instructions to apply the abstract idea on a computer (i.e., merely invoking the computer structure as a tool used to execute the limitations).
Therefore, claims 1-20 are rejected under 35 USC §101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Mohapatra (US 2022/0398458 A1) in further view of Nygaard Espeland (US 2022/0296081 A1), Swisher (US 2021/0241884 A1) and Orosco (US 2018/0350452 A1).
Regarding claim 1, Mohapatra teaches: A method for automatically generating a structured medical record from endoscopy data (e.g. see [0001]-[0002]), the method comprising:
obtaining image data comprising a sequence of video frames of a gastrointestinal (GI) tract of a patient; (“capture a video of a colonoscopy procedure…extract frames from the video or images associated with the colonoscopy procedure”, e.g. see [0005])
identifying one or more symptoms of a disease by signature matching a region of a video frame of the sequence of video frames to a known signature representing a symptom; (obtaining a neural network model for “classifying objects that appear in the frames or images” and determining a “classification of a polyp that appears in at least one of the frames or images based at least in part on applying the frames or images to an input layer of the model. The classification of the polyp can include an adenoma polyp, a hyperplastic polyp, or a serrated polyp.”, e.g. see [0005]; matching regions of the image to known signatures by extracting “image-based features…a fractal dimension of a shape of the polyp, an intensity and a size of dark internal pixel clusters in the polyp, an intensity and a size of bright internal pixel clusters in the polyp”, e.g. see [0063])
based on the tagging, extracting one or more features each representing a physical feature of the GI tract; (“extract frames 727 from the video, or images of the videos and/or images 724 associated with the colonoscopy procedure”, e.g. see [0068]; extraction of specific physical parameters from these images, such as color, vascular patterns, surface patterns and 3D DNA shape, e.g. see [0038]; “Image-based features of the videos and/or images 724” can include “size”, “shape” and “intensity”, e.g. see [0063]; determining image-based features including “color, vessels, and surface pattern”, e.g. see [0025])
Mohapatra does not teach:
tagging the identified symptoms to exact anatomical landmarks within the GI tract;
However, Nygaard Espeland in the analogous art of detection of “anatomical structures and/or anatomical landmarks in endoscopic videos” (e.g. see [0001]) teaches:
tagging the identified symptoms to exact anatomical landmarks within the GI tract; (real-time processing of an endoscopic video to detect “anatomical structures” simultaneously with “anatomical landmarks”; using these anatomical landmarks to localize the specific segment of the GI tract where the symptom is occurring, e.g. see [0015], [0025]-[0026])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra to include tagging the identified symptoms to exact anatomical landmarks within the GI tract as taught by Nygaard Espeland, so that “the practitioner will not only receive information that an inflammation is present but also the localization of said inflammation, i.e. in this case whether the inflammation occurs in the small intestine or the large intestine” (Nygaard Espeland [0026e]).
Mohapatra and Nygaard Espeland do not teach:
[…] the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof;
However, Swisher in the analogous art of automating the interpretation of medical images (e.g. see [0001]) teaches:
[…] the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof; (generating textual, “natural language description” of an “identified abnormality” (i.e. physical feature) using “extracted features” and a trained “long short-term memory (LSTM) network”, e.g. see [0006]-[0007], [0034]; applying this to endoscopy to generate descriptive text such as “"mucosal erosions seen at multiple locations of upper GI."”, e.g. see [0048])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra and Nygaard Espeland to include the one or more transformed features comprising a natural language description of a corresponding physical feature, metadata describing the sequence of video frames in relation to the transformed feature or a combination thereof as taught by Swisher, for the purposes of “summarizing the information contained therein in a text format is an intensely time-consuming task that often represents a bottleneck”, and automating this text generation “would significantly improve diagnosis and/or treatment workflow, thereby lowering costs and improving patient care”(Swisher [0002]-[0003]).
Mohapatra, Nygaard Espeland and Swisher do not teach:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store;
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.
However, Orosco in the analogous art of converting clinical information into a unified to format to generate and update structured electronic medical records (e.g. see [0012]-[0014]) teaches:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store; (a middleware application that utilizes a “unified data model” (i.e. data specification) and “a unified data format” (i.e. data requirements) to define the required “schema” for data entries of an EMR database (i.e. data store), e.g. see [0024]-[0025], claim 1)
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification (converting source data into formatted “JSON objects” or “XML file” (i.e. transformed features) that corresponds to the data elements of a structured medical record and satisfy the data schema constraints of the “unified data model”, e.g. see [0024]-[0026])
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries. (transmitting the converted JSON/XML results via an electronic network to store them in the target EMR system, e.g. see [0031], claim 1; the EMR system is configured to receive structured queries, e.g. HTTP GET/POST API requests specifying patient ID’s and clinical categories, to retrieve and provide the stored patient data records, e.g. see [0018], [0023], Table 1, [0028]-[0029])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland and Swisher to include obtaining a data specification indicating a set of data requirements for data entries of records of a data store, generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification and storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries as taught by Orosco, for the purposes of overcoming issues with data entry and retrieval with incompatible databases and being able to “process data elements retrieved from different EMR systems 105 in the same way” (Orosco [0003], [0024]).
Regarding claim 7, Mohapatra teaches: A data processing system configured for automatically generating a structured medical record from endoscopy data (e.g. see [0001]-[0002]), the data processing system comprising:
at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: (e.g. see [0059])
obtaining image data comprising a sequence of video frames of a gastrointestinal (GI) tract of a patient; (“capture a video of a colonoscopy procedure…extract frames from the video or images associated with the colonoscopy procedure”, e.g. see [0005])
identifying one or more symptoms of a disease by signature matching a region of a video frame of the sequence of video frames to a known signature representing a symptom; (obtaining a neural network model for “classifying objects that appear in the frames or images” and determining a “classification of a polyp that appears in at least one of the frames or images based at least in part on applying the frames or images to an input layer of the model. The classification of the polyp can include an adenoma polyp, a hyperplastic polyp, or a serrated polyp.”, e.g. see [0005]; matching regions of the image to known signatures by extracting “image-based features…a fractal dimension of a shape of the polyp, an intensity and a size of dark internal pixel clusters in the polyp, an intensity and a size of bright internal pixel clusters in the polyp”, e.g. see [0063])
based on the tagging, extracting one or more features each representing a physical feature of the GI tract; (“extract frames 727 from the video, or images of the videos and/or images 724 associated with the colonoscopy procedure”, e.g. see [0068]; extraction of specific physical parameters from these images, such as color, vascular patterns, surface patterns and 3D DNA shape, e.g. see [0038]; “Image-based features of the videos and/or images 724” can include “size”, “shape” and “intensity”, e.g. see [0063]; determining image-based features including “color, vessels, and surface pattern”, e.g. see [0025])
Mohapatra does not teach:
tagging the identified symptoms to exact anatomical landmarks within the GI tract;
However, Nygaard Espeland in the analogous art teaches:
tagging the identified symptoms to exact anatomical landmarks within the GI tract; (real-time processing of an endoscopic video to detect “anatomical structures” simultaneously with “anatomical landmarks”; using these anatomical landmarks to localize the specific segment of the GI tract where the symptom is occurring, e.g. see [0015], [0025]-[0026])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra to include tagging the identified symptoms to exact anatomical landmarks within the GI tract as taught by Nygaard Espeland, so that “the practitioner will not only receive information that an inflammation is present but also the localization of said inflammation, i.e. in this case whether the inflammation occurs in the small intestine or the large intestine” (Nygaard Espeland [0026e]).
Mohapatra and Nygaard Espeland do not teach:
[…] the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof;
However, Swisher in the analogous art teaches:
[…] the one or more transformed features comprising a natural language description of i) a corresponding physical feature, ii) metadata describing the sequence of video frames in relation to the transformed feature; or iii) a combination thereof; (generating textual, “natural language description” of an “identified abnormality” (i.e. physical feature) using “extracted features” and a trained “long short-term memory (LSTM) network”, e.g. see [0006]-[0007], [0034]; applying this to endoscopy to generate descriptive text such as “"mucosal erosions seen at multiple locations of upper GI."”, e.g. see [0048])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra and Nygaard Espeland to include the one or more transformed features comprising a natural language description of a corresponding physical feature, metadata describing the sequence of video frames in relation to the transformed feature or a combination thereof as taught by Swisher, for the purposes of “summarizing the information contained therein in a text format is an intensely time-consuming task that often represents a bottleneck”, and automating this text generation “would significantly improve diagnosis and/or treatment workflow, thereby lowering costs and improving patient care”(Swisher [0002]-[0003]).
Mohapatra, Nygaard Espeland and Swisher do not teach:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store;
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.
However, Orosco in the analogous art teaches:
obtaining a data specification indicating a set of data requirements for data entries of records of a data store; (a middleware application that utilizes a “unified data model” (i.e. data specification) and “a unified data format” (i.e. data requirements) to define the required “schema” for data entries of an EMR database (i.e. data store), e.g. see [0024]-[0025], claim 1)
generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification (converting source data into formatted “JSON objects” or “XML file” (i.e. transformed features) that corresponds to the data elements of a structured medical record and satisfy the data schema constraints of the “unified data model”, e.g. see [0024]-[0026])
storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries. (transmitting the converted JSON/XML results via an electronic network to store them in the target EMR system, e.g. see [0031], claim 1; the EMR system is configured to receive structured queries, e.g. HTTP GET/POST API requests specifying patient ID’s and clinical categories, to retrieve and provide the stored patient data records, e.g. see [0018], [0023], Table 1, [0028]-[0029])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland and Swisher to include obtaining a data specification indicating a set of data requirements for data entries of records of a data store, generating one or more transformed features corresponding to one or more data entries of fields of a structured medical record that satisfies the set of data requirements of the data specification and storing, in a data store, the one or more data entries including the transformed features, wherein the data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries as taught by Orosco, for the purposes of overcoming issues with data entry and retrieval with incompatible databases and being able to “process data elements retrieved from different EMR systems 105 in the same way” (Orosco [0003], [0024]).
Regarding claim 19, Mohapatra, Nygaard Espeland, Swisher and Orosco teach the method of claim 1 as described above.
Mohapatra does not teach:
wherein the structured query specifies one or more application programming interface (API) functions in an API call, the API functions for querying the structured data record including one or more data entries including the transformed features, the one or more API functions corresponding to the transformed features
However, Orosco in the analogous art teaches:
wherein the structured query specifies one or more application programming interface (API) functions in an API call, the API functions for querying the structured data record including one or more data entries including the transformed features, the one or more API functions corresponding to the transformed features (querying the EMR data store using API functions in an API call, e.g. see [0023], Table 1; utilizing HTTP GET API calls specifying URIs and parameters that correspond directly to the transformed features, e.g. see [0029], Table 1)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland and Swisher to include the structured query specifies one or more application programming interface (API) functions in an API call, the API functions for querying the structured data record including one or more data entries including the transformed features, the one or more API functions corresponding to the transformed features as taught by Orosco, for the purposes of allowing real-time retrieval of clinical features without having to rely on stale temporary databases (Orosco [0018]).
Claim 13 recites substantially similar limitations as those already addressed in claim 7, and, as such is rejected for similar reasons as given above.
Claims 2, 3, 5, 8, 9, 11, 14, 15, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mohapatra, Nygaard Espeland, Swisher and Orosco in further view of Li (WO 2020/233254).
Regarding claim 2, Mohapatra, Nygaard Espeland, Swisher and Orosco teach the method of claim 1 as described above.
Mohapatra further teaches:
applying a machine learning model to the image data, the machine learning model being trained with images of GI tracts to classify image signatures for the signature mapping, wherein the signature matching is based on one or more thresholds set or adjusted by the machine learning module; (applying SVM or neural networks trained on colonoscopy images to classify polyps using image signatures, e.g. see [0005], [0045]; the classification/signature matching is based on thresholds, e.g. intensity and clustering thresholds, defined by the machine learning module to determine physical features, and outputting those physical features as data, e.g. see [0063], [0085])
identifying, from the applying, a malignancy in the GI tract; (identifying and classifying polyps into “malignant”, e.g. categories such as “adenoma”, e.g. see [0071], [0067])
determining one or more physical features of the malignancy; (determining image-based features, including numerical features, of the polyps such as “color, vessels, and surface pattern”, e.g. see [0025], [0063]).
Mohapatra does not teach:
outputting the one or more physical features as feature data
However, Li in the analogous art of medical image analysis (e.g. see [0005]) teaches:
outputting the one or more physical features as feature data (calculating discrete data points, e.g. Si, (x, y), Wi, Hi, Pi, and outputting them into a “structured data table”, e.g. see [0039])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra to include outputting the one or more physical features as feature data as taught by Li, for the purposes of “structur[ing] and transform[ing]” key image information so it can be fused with other patient data to form a system with “high efficiency and high utilization” (Li [0027]).
Regarding claim 3, Mohapatra, Nygaard Espeland, Swisher, Orosco and Li teach the method of claim 2 as described above.
Mohapatra further teaches:
wherein the one or more physical features comprise a color of the malignancy, an outer shape of the malignancy, an area of the malignancy, a location of the malignancy, or an orientation of the malignancy (analyzing the color of the adenoma polyp and its location withing the frame, e.g. see [0044]; “Naming color” as a specific parameter for classification of the malignancy, e.g. see [0038]; assessing the structural form of the lesion including “surface patterns”, e.g. oval, tubular, or branched structures, and “3D DNA Shape”, e.g. see [0025], [0038]).
Regarding claim 5, Mohapatra, Nygaard Espeland, Swisher and Orosco teach the method of claim 1 as described above.
Mohapatra does not teach:
wherein the structured medical record comprises a set of instances of records each associated with a common identifier, and wherein the method comprises:
comparing data of a first field of a first instance of the instances of the records with a second field of a second instance of the instances of the records for generating comparative feature data
However, Li in the analogous art teaches:
wherein the structured medical record comprises a set of instances of records each associated with a common identifier, and wherein the method comprises: (forming an “image information structured data table” (i.e. structured medical record) that stores multiple different “CT image analysis results” linked to the “same patient id” (i.e. a common identifier), e.g. see [0039], [0067])
comparing data of a first field of a first instance of the instances of the records with a second field of a second instance of the instances of the records for generating comparative feature data (analyzing “different CT image analysis results of the same patient id”; the system extracts the area S1 of the feature from the first image (i.e. a first field of a first instance) and the area S2 from the corresponding feature of the second image (i.e. a second field of a second instance) and compares these features by evaluating them to find the “largest Si” (i.e. comparative features data), which is used to form the “structured information data”, e.g. see [0067], [0061]-[0063])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra to include the structured medical record comprises a set of instances of records each associated with a common identifier, and comparing data of a first field of a first instance of the instances of the records with a second field of a second instance of the instances of the records for generating comparative feature data as taught by Li, for the purposes of attaining “rapid specificity analysis processing capability” required of medical image data (Li [0002]).
Claims 8 and 14 recite substantially similar limitations as those already addressed in claim 2, and, as such are rejected for similar reasons as given above.
Claims 9 and 15 recite substantially similar limitations as those already addressed in claim 3, and, as such are rejected for similar reasons as given above.
Claims 11 and 17 recite substantially similar limitations as those already addressed in claim 5, and, as such are rejected for similar reasons as given above.
Regarding claim 20, Mohapatra, Nygaard Espeland, Swisher, Orosco and Li teach the method of claim 2 as described above.
Mohapatra does not teach:
wherein signature matching the region of the video frame of the sequence of video frames to the known signature representing the symptom comprises pixel thresholding based on the one or more thresholds
However, Li in the analogous art teaches:
wherein signature matching the region of the video frame of the sequence of video frames to the known signature representing the symptom comprises pixel thresholding based on the one or more thresholds (extracting features from unstructured medical image data using pixel thresholding; converting the image segmentation map such that “a pixel point value of the target segmentation region is 1, and the remaining pixel point values are 0”, e.g. see [0007], [0037)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra to include signature matching the region of the video frame of the sequence of video frames to the known signature representing the symptom comprises pixel thresholding based on the one or more thresholds as taught by Li, for the purposes of enabling the precise calculation of physical features of a targeted region (Li [0007]).
Claims 4, 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mohapatra, Nygaard Espeland, Swisher and Orosco in further view of Paik (WO 2022/212771).
Regarding claim 4, Mohapatra, Nygaard Espeland, Swisher and Orosco teach the method of claim 1 as described above.
Mohapatra does not teach:
accessing data entries in the data store that are associated with a consistent key value that represents a patient;
However, Orosco in the analogous art teaches:
accessing data entries in the data store that are associated with a consistent key value that represents a patient; (“commands sent to the middleware 102 by third-party application…may include a patient ID” e.g. see [0028]; the JSON response payload lists “PatientId” as the consistent key value associating the data entries together, e.g. see [0024])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland and Swisher to include accessing data entries in the data store that are associated with a consistent key value that represents a patient as taught by Orosco, for the purposes of overcoming issues with data entry and retrieval with incompatible databases and being able to “process data elements retrieved from different EMR systems 105 in the same way” (Orosco [0003], [0024]).
Mohapatra, Nygaard Espeland, Swisher and Orosco do not teach:
comparing transformed features among the data entries by comparing the one or more physical features representing a malignancy in a patient to corresponding one or more additional features extracted from a prior instance of image data associated with the patient;
generating, based on the comparing, comparative features data representing a growth or shrink rate of the malignancy in the patient.
However, Paik in the analogous art of utilizing AI and computer vision to analyze medical images (e.g. see [0021]) teaches:
comparing transformed features among the data entries by comparing the one or more physical features representing a malignancy in a patient to corresponding one or more additional features extracted from a prior instance of image data associated with the patient; (the “local database” or “cloud database” stores medical images and metadata linked to a “specific subject”, e.g. see [0253]; accessing a patient’s historical image records by determining “all patient prior imaging studies” relevant to the current study, e.g. see [0234]; extracting and comparing the physical feature of a tumor across multiple points in time, the extraction of features from a “time-series of 3D images showing progression of a tumor…include x/y/z 3-D coordinates and also the time (t) dimension”, e.g. see [0252])
generating, based on the comparing, comparative features data representing a growth or shrink rate of the malignancy in the patient. (based on the comparison of the 3-D coordinates of the tumor, the system generates metrics representing the “progression of a tumor”, e.g. see [0252]; “Quantitative image features such as tumor sizing measures can be used for treatment response tracking.”, e.g. see [0201])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland, Swisher and Orosco to include comparing transformed features among the data entries by comparing the one or more physical features representing a malignancy in a patient to corresponding one or more additional features extracted from a prior instance of image data associated with the patient and generating, based on the comparing, comparative features data representing a growth or shrink rate of the malignancy in the patient as taught by Paik, for the purposes of “treatment response tracking” (Paik [0201]).
Claims 10 and 16 recite substantially similar limitations as those already addressed in claim 4, and, as such are rejected for similar reasons as given above.
Claims 6, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mohapatra, Nygaard Espeland, Swisher and Orosco in further view of Ullaskrishnan (US 2024/0170112 A1).
Regarding claim 6, Mohapatra, Nygaard Espeland, Swisher and Orosco teach the method of claim 1 as described above.
Mohapatra, Nygaard Espeland, Swisher and Orosco do not teach:
wherein the structured queries comprise a natural language query, and
wherein the method comprises processing the natural language query using a language model trained based on a library of terms associated with the fields of the structured medical record
However, Ullaskrishnan in the analogous art of “field of data retrieval based on a query…from an electronic medical record database” (e.g. see [0002]) teaches:
wherein the structured queries comprise a natural language query, and (“the query may comprise natural language…receiving the query may comprise obtaining an utterance from a user… receiving the query may comprise obtaining a text input by a user”, e.g. see [0027])
wherein the method comprises processing the natural language query using a language model trained based on a library of terms associated with the fields of the structured medical record (“providing a natural language processing algorithm” which “comprises a transformer network”, e.g. see [0068], [0076]; “the natural language processing algorithm may comprise a large language model”, e.g. see [0083]; “the fine-tuning may comprise further training the transformer network with medical texts with…medical ontologies such as RadLex and/or SNOMED” (i.e. library of terms); “SNOMED…is a systematically organized computer-processable collection of medical terms”, e.g. see [0078], [0046]; “the medical ontology…encodes references of codes of the medical ontology to data identifiers of the electronic medical record database”, e.g. see [0152]; “sub-unit 23 may be configured to match the codes…with data identifiers of the electronic medical record database EMR and query the EMR on that basis”, e.g. see [0188])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Mohapatra, Nygaard Espeland, Swisher and Orosco to include the structured queries comprise a natural language query and processing the natural language query using a language model trained based on a library of terms associated with the fields of the structured medical record as taught by Ullaskrishnan, for the purposes of “allowing for an efficient provision of information from an electronic medical record database based on a query containing natural language” (Ullaskrishnan [0007]).
Claims 12 and 18 recite substantially similar limitations as those already addressed in claim 6, and, as such are rejected for similar reasons as given above.
Response to Arguments
Regarding the rejection under 35 U.S.C. § 101 of Claims 1-20, the Examiner has considered the Applicant’s arguments; however the arguments are not persuasive.
Applicant argues the claims recite a “specific technical process of extracting physical parameters directly from endoscopic image data and transforming those features into a structured format”.
The Examiner respectfully disagrees. Applicant alleges the claims involve a technical process using computer vision techniques such as “object detection” and “pixel thresholding”. However, it is noted that the features upon which Applicant relies (i.e. “object detection”, “pixel thresholding” or specific computer vision algorithms) are not recited in the claim 1. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Instead, claim 1 broadly recites the steps of “identifying…by signature matching”, “tagging” and “extracting”. These steps describe observations, evaluations and classifications that can be practically performed in the human mind. The machine learning generally applies the abstract idea without placing any limits on how the machine learning functions. These limitations only recite the outcome and do not include any details on how the outcome is accomplished (see MPEP 2106.05(f)).
Applicant argues the claims provide a specific technical improvement to EMR systems by reducing physician variability, standardizing documentation and enabling database systems to use “consistent key values” to prevent data conflicts.
The Examiner respectfully disagrees. MPEP 2106.04(d)(1) and MPEP 2106.05(a) indicates that a practical application may be present where the claimed invention provides a technical solution to a technical problem. See, e.g. DDR Holdings, LLC. v. Hotels.com, L.P., 773 F.3d 1245, 1259 (Fed. Cir. 2014) (finding that claiming a website that retained the “look and feel” of a host webpage provided a technological solution to the problem of retention of website visitors by utilizing a website descriptor that emulated the “look and feel” of a host webpage, where the problem arose out of the internet and was thus a technical problem). Here, the Applicant’s argued problem is not a technological problem caused by the computer (the technological environment to which the claims are confined). The problem of documentation by physicians working from memory or reviewing video data as time-consuming, lacking uniform detail, and resulting in lost data is not a problem by the computer, is it a problem that existed and/or exists regardless of whether a computer is involved the process. At best, Applicant’s identified problem is a healthcare management problem. Because, no technological problem is present, the claims do not provide a practical application.
In addition, automating a task previously performed by physicians from memory to achieve greater speed, standardization or accuracy is not recognized technological improvement under MPEP 2106.05(a). Using a generic computer to merely automate a human process more efficiently lacks an inventive concept. Second, the purported database improvements relied upon by the Applicant, such as utilizing a “valid key rule” that “does not conflict with one or more other key values”, are nowhere to be found in the language of claim 1.
Regarding the rejection under 35 U.S.C. § 103 of Claims 1-20, the Examiner has considered the Applicant’s arguments; however, these arguments are moot given the new grounds of rejection as necessitated by amendment.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaisha Abdullah whose telephone number is (571)272-5668. The examiner can normally be reached on Monday through Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter H Choi can be reached on (469) 295-9171. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.A./Examiner, Art Unit 3686
/PETER H CHOI/Supervisory Patent Examiner, Art Unit 3681
