Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to response to
This action is made Final.
Claims 8-20 are pending in the case. Claims 8 and 17 are independent claims.
Response to Arguments
Applicant's arguments filed 5/5/2026 have been fully considered but they are not persuasive. Applicant remarks that Phister in view of Voznyuk does not teach every feature of claims 8-12 and 14-20. More specifically, Applicant remarks that the cited references fail to teach or suggest “constructing a prompt containing the point-of-interest and the context set of points and generating text data via an LLM” (pages 2-3). The Examiner disagrees. Phister discusses listing all modules that require a node labeled ROI when a user hovers over an anomaly in a medical image, which the Examiner maintains is equivalent to the claimed “constructing a prompt containing the point of interest and the context set of points”. Phister also discusses a radiologist selecting one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” being added to the graph. The Examiner maintains that this is equivalent to the claimed “generating text data via an LLM”. Regarding the claim limitation “generating a context set of points nearest to the point-of-interest”, the Examiner maintains that this feature is taught by Voznyuk where Voznyuk discusses a pre-treated image being processed by a convolutional neural network (CNN) followed by a fully connected regression neural network configured to provide as output five continuous values. A tanh activation operation layer (from −1 to 1, or from 0 to 1) and a scaling operator (for values 1 and 3—from 0 to the image width, for values 2 and 4—from 0 to the image height, for value 5—from 0 to 89 degrees) are configured to use the result of the regression algorithm to provide the coordinates and orientation of the bounding box defining the global ROI comprising all landmarks present in the image. Voznyuk further discusses evaluating whether the dimension or quality (for example the signal-to-noise ratio) of the defined ROI are acceptable, with the identified landmarks being compared to the landmarks in the landmarks database. The Examiner therefore maintains that the cited references teach every feature of the claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8-12 and 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phister (USPUB 20230386629 A1) in view of Voznyuk (USPUB 20240358354 A1).
Claim 8:
Phister discloses A method of determining text data associated with a point-of-interest, comprising: receiving a user selection of the point-of-interest in a medical image (0102 and 0114: The module may display the image within a viewer (e.g., in the first portion of the display unit 108) and query a click from the user. The coordinates of the click may be appended as properties of a node labeled “ROI” to the node labeled “CT“via an edge labeled”:SHOWS”... if a user draws a rectangle in a CT image to define a region (e.g., ROI), an algorithmic module may be triggered that searches for anomalies in the specific region. A simple example for a trigger via a user interaction may be a simple button that is clicked by a user); constructing a prompt containing the point-of-interest and the context set of points; and generating, by a large language model, text data associated with the point-of- interest in response to the prompt (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Phister, by itself, does not seem to completely teach generating a context set of points nearest to the point-of-interest.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches generating a context set of points nearest to the point-of-interest (0186-189).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 9:
Phister, by itself, does not seem to completely teach determining normalized coordinates of the point-of-interest.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches determining normalized coordinates of the point-of-interest (0186).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 10:
Phister, by itself, does not seem to completely teach determining the normalized coordinates of the point-of-interest comprises using a trained regression neural network.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches determining the normalized coordinates of the point-of-interest comprises using a trained regression neural network (0186).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 11:
Phister, by itself, does not seem to completely teach generating the context set of points comprise identifying the context set of points from a database of landmarks.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches generating the context set of points comprise identifying the context set of points from a database of landmarks (0186-189).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 12:
Phister, by itself, does not seem to completely teach the database of landmarks are defined in a normalized coordinate system.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches the database of landmarks are defined in a normalized coordinate system (0186-189).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 14:
Phister teaches constructing the prompt comprises constructing the prompt that requests for a description of an anatomical feature at the point-of-interest (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Claim 15:
Phister teaches the prompt requests for the description of the anatomical feature oat the point-of-interest given normalized coordinates of the point-of-interest and the context set of points (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Claim 16:
Phister teaches the large language model is trained with medical health record data (0023).
Claim 17:
Phister teaches One or more non-transitory computer-readable media comprising computer-readable instructions, that when executed by a processor device, cause the processor device to perform steps comprising: receiving a user selection of a point-of-interest in a medical image (0102 and 0114: The module may display the image within a viewer (e.g., in the first portion of the display unit 108) and query a click from the user. The coordinates of the click may be appended as properties of a node labeled “ROI” to the node labeled “CT“via an edge labeled”:SHOWS”... if a user draws a rectangle in a CT image to define a region (e.g., ROI), an algorithmic module may be triggered that searches for anomalies in the specific region. A simple example for a trigger via a user interaction may be a simple button that is clicked by a user); constructing a prompt containing the point-of-interest and the context set of points; and generating, by a large language model, text data associated with the point-of- interest in response to the prompt (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Phister, by itself, does not seem to completely teach determining normalized coordinates of the point-of-interest; generating a context set of points nearest to the point-of-interest, wherein the context set of points are identified from a database of landmarks defined in a normalized coordinate system.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches determining normalized coordinates of the point-of-interest; generating a context set of points nearest to the point-of-interest, wherein the context set of points are identified from a database of landmarks defined in a normalized coordinate system (0186-189: the image 310 is provided as input to a first stage configured to perform a pre-treatment of the image comprising at least one of the following operations of cropping, scaling and padding, detecting the image type (i.e., simple image, doppler image, two different images on the same frame), improving qualitatively and quantitatively the image, and/or normalizing the image. Then the pre-treated image is processed by a convolutional neural network (CNN) followed by a fully connected regression neural network configured to provide as output five continuous values. A tanh activation operation layer (from −1 to 1, or from 0 to 1) and a scaling operator (for values 1 and 3—from 0 to the image width, for values 2 and 4—from 0 to the image height, for value 5—from 0 to 89 degrees) are than configured to use the result of the regression algorithm to provide the coordinates and orientation of the bounding box defining the global ROI comprising all landmarks present in the image ...The at least one processor is further configured to evaluate whether the dimension or quality (for example the signal-to-noise ratio) of the defined ROI are acceptable 330 to be able to perform a correct further processing of the image 310. An area of the ROI may be compared to a predefined threshold, and so that the processor proceeds to the localization of the landmarks only if the area of the ROI exceeds the predefined threshold. Otherwise, the processor will trigger the reception of at least one additional image 310. The area of the ROI may be for example evaluated as the number of pixels comprised in the ROI or any other measure known by the person skilled in the art... the step of detection and identification of the landmarks 340 is performed only in the ROI. When the step of detection and identification also comprises localization (FIG. 16), these operations may all be performed on the ROI).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Claim 18:
Phister teaches constructing the prompt comprises constructing the prompt that requests for a description of an anatomical feature at the point-of-interest (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Claim 19:
Phister teaches the prompt requests for the description of the anatomical feature oat the point-of-interest given normalized coordinates of the point-of-interest and the context set of points (0115-116: if a user hovers over a highlighted anomaly that may display the inputs of a node labeled “ROI”, a module database query might be performed. All modules that require an input node labeled “ROI” may be listed and displayed to a user. Examples of such modules may include: “COMPUTE_SIZE”, a module that may be configured to compute a size of the ROI; “SPECIFY_FINDING”, a GUI module that may requests the user to specify different types of diseases connected to this finding via a drop-down menu; and “COMPUTE_LOCATION”, an algorithmic module that may be configured to compute the location of the finding by atlas matching. In other words, the selection of the at least one program module may be performed by a user, possible based on a list of program modules having input requirements matching a region specified by the user... For each of the one or more ROIs, their anatomical location, and a patient history node labeled “HISTORY”, an algorithmic module may compute a list of possible diagnoses and append a node labeled “DIAGNOSIS_LIST” comprising the list as a property. This node may be connected to the respective node(s) labeled “ROI”, “LOC” and “HISTORY”. A UI module may in response to the generation of the node labeled “DIAGNOSIS_LIST” display a textual representation of the node labeled “DIAGNOSIS_LIST” via a GUI, e.g., by displaying each of the possible diagnoses in the list in text format in the second portion of the display 108. Subsequently, a radiologist may select one of the textual representations to confirm a diagnosis, and a corresponding node labeled “DIAGNOSIS” may be added to the graph).
Claim 20:
Phister, by itself, does not seem to completely teach determining the normalized coordinates of the point-of-interest comprises using a trained regression neural network.
The Examiner maintains that these features were previously well-known as taught by Voznyuk.
Voznyuk teaches determining the normalized coordinates of the point-of-interest comprises using a trained regression neural network (0186).
Phister and Voznyuk are analogous art because they are from the same problem-solving area, analyzing images to determine information associated with the images.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Phister and Voznyuk before him or her, to combine the teachings of Phister and Voznyuk. The rationale for doing so would have been to obtain the benefit of accurate object identification.
Therefore, it would have been obvious to combine Phister and Voznyuk to obtain the invention as specified in the instant claim(s).
Allowable Subject Matter
Claim 13 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Note
The Examiner cites particular columns, line numbers and/or paragraph numbers in the references as applied to the claims below for the convenience of the Applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the Applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2123.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMED H ZUBERI whose telephone number is (571)270-7761. The examiner can normally be reached Mon – Th 10AM-8PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached on (571) 272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMMED H ZUBERI/Primary Examiner, Art Unit 2178
Prosecution Insights