DETAILED ACTION
Claims 1-20 are currently pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/6/26 has been entered.
Response to Arguments
Applicant's arguments filed 4/6/26 have been fully considered but they are not persuasive.
The Applicant argues on page 10 of the response in essence that: In the present case, Applicants respectfully submit that "skipping at least one voxel between two sampled voxels" does not "necessarily" flow from uniformly distributed pixels as described in Tewfik. For instance, if the area for sampling is very small and there are many pixels to be sampled (i.e., dense sampling distribution), pixels next to each other may be sampled even in a uniform sampling distribution. Applicants respectfully submit that "skipping at least one voxel between two sampled voxels" is not inherently necessary for a uniform distribution of pixels, and as such, Tewfik does not inherently disclose this feature to the satisfaction of the MPEP.
Tewfik discloses that the global sampling strategy uses single surface pixels uniformly distributed around the organ as shown in FIG. 3A (paragraph 78). FIGS. 3A, 3B, and 3C each show that there are no sampled pixels next to each other.
The Applicant argues on pages 10 and 11 of the response in essence that: Tewfik also fails to teach or suggest constructing a descriptor comprising intensity values of the sampled voxels and identifying the type of organ at the single point of interest by applying a trained classifier to the descriptor. Instead, Tewfik describes surface representations broadly, with no mention whatsoever of constructing a descriptor comprising intensity values of the sampled voxels. See Tewfik at, e.g., paragraph [0047].
Tewfik discloses at 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system (paragraph 183). Generating an image involves generating a plurality of intensity values which make up the pixels within the image.
The Applicant argues on page 17 of the response in essence that: The Examiner acknowledges that Tewfik and He fails to disclose comparing the identified type of organ with the received type of the at least one organ; and modifying the classifier depending on the comparison of step e) to obtain a trained classifier, and relies on Do to compensate for their defects. However, Do at most describes evaluating whether the network's ability to identify a feature in an image exceeds a defined threshold of speed, classification accuracy, reproducibility, efficacy, or other performance metric, not comparing the identified type of organ with the received type of the at least one organ. See Do at, e.g., paragraph [0046]. Additionally, Do describes refining the labeled data in response to not achieving a desired level of performance, not modifying the classifier depending on the comparison. See Do at, e.g., paragraph [0047].
Tewfik discloses training an untrained classifier to identify the type of organ, but does not disclose expressly the training using supervised learning. Do discloses using supervised learning by evaluating the classification accuracy to classify unlabeled data (paragraph 46). The classification accuracy describes the percentage of the time or the frequency with which an image feature or region is identified correctly (paragraph 47). The classifier will be modified if the confidence value does not exceed the threshold (paragraph 47).
Applicant’s remaining arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-9, 11-15, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521 (hereafter “Tewfik”) and El-Zehiry et al. US Publication 2021/0279884 (hereafter ”El-Zehiry”).
Referring to claims 1 and 17, Tewfik discloses a computer-implemented method for identifying a type of organ in a volumetric medical image, comprising:
a) receiving the volumetric medical image, the volumetric medical image comprising at least one organ or portion thereof (paragraph 183, At 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system);
c) sampling voxels from the volumetric medical image (paragraph 183, At 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system), wherein at least one voxel is skipped between two sampled voxels (paragraph 78, The global sampling strategy uses single surface pixel uniformly distributed around the organ as shown in FIG. 3A); and
d) identifying the type of organ at the single point of interest by applying a trained classifier to the sampled voxels (paragraph 47, an example of the present subject matter identifies a specific, structured, sparse representation of the 3D organ surface that matches the very limited observed data and is suitable for a naturally shaped or a deformed organ).
While Tewfik discloses the volumetric medical image, Tewfik does not disclose expressly receiving a single point of interest within the volumetric medical image.
El-Zehiry discloses b) receiving a single point of interest within the volumetric medical image (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted); and
c) sampling, from a sampling model centered on the single point of interest, voxels from the volumetric medical image (paragraph 56, In act E, the 3D volume M and any information identifying the area of interest are then fed to the model 1 which has been trained as explained above and illustrated in FIG. 2).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to receive a single point of interest within the volumetric medical image. The motivation for doing so would have been to increase efficiency by allowing the user to select the area of the image to be analyzed. Therefore, it would have been obvious to combine El-Zehiry with Tewfik to obtain the invention as specified in claims 1 and 17.
Referring to claim 5, Tewfik discloses wherein the sampled voxels are less than 1% of a total number of voxels in the volumetric medical image (paragraph 73, Each brain mesh can include 40962 points and spherical harmonics up to degree 80 can be used for approximation. Based on the training deformations, deformation subspaces can be identified using the ISI approach. The brain surface can be reconstructed by monitoring 29 sample positions).
Referring to claim 6, Tewfik discloses wherein the sampled voxels are less than 0.1 % of a total number of voxels in the volumetric medical image (paragraph 73, Each brain mesh can include 40962 points and spherical harmonics up to degree 80 can be used for approximation. Based on the training deformations, deformation subspaces can be identified using the ISI approach. The brain surface can be reconstructed by monitoring 29 sample positions).
Referring to claim 7, Tewfik discloses wherein the sampled voxels are less than 0.01 % of a total number of voxels in the volumetric medical image (paragraph 73, Each brain mesh can include 40962 points and spherical harmonics up to degree 80 can be used for approximation. Based on the training deformations, deformation subspaces can be identified using the ISI approach. The brain surface can be reconstructed by monitoring 29 sample positions).
Referring to claim 8, Tewfik discloses the trained classifier, but does not disclose expressly wherein the trained classifier is a neural network.
El-Zehiry discloses wherein the trained classifier is a neural network (paragraph 10, In one example, the convolutional neural network is U-Net, but any other semantic segmentation architecture (such as a residual neural network, SegNet, DeepLab etc.) may be implemented).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to use a neural network classifier. The motivation for doing so would have been to increase the efficiency and accuracy of classifying medical images. Therefore, it would have been obvious to combine El-Zehiry with Tewfik to obtain the invention as specified in claim 8.
Referring to claim 9, El-Zehiry discloses wherein the neural network is a multilayer perceptron, a convolutional neural network, a Siamese network or a triplet network (paragraph 10, In one example, the convolutional neural network is U-Net, but any other semantic segmentation architecture (such as a residual neural network, SegNet, DeepLab etc.) may be implemented).
Referring to claim 11, El-Zehiry discloses wherein the single point of interest is selected by a user (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted).
Referring to claim 12, El-Zehiry discloses wherein the single point of interest is selected by pausing a cursor operated by a user on the volumetric medical image or a part thereof displayed on a graphical user interface (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted).
Referring to claim 13, Tewfik discloses wherein a user takes a measurement with respect to the volumetric medical image or a part thereof (paragraph 184, Sampling unit 720 can include an endoscope, a needlescope, a camera, or other instrument to collect sample measurements of a surface of an object (or organ)), and the identified type of organ is saved in a database along with the measurement (paragraph 9, Training data can be generated and used to construct dictionaries and identify subspaces).
Referring to claim 14, Tewfik discloses receiving a number of untrained classifiers for identifying organ specific abnormalities (paragraph 48, Training 110 can include constructing dictionaries and identifying subspaces using MRI, CT scans, ultrasound, or modeling);
selecting one or more of the untrained classifiers from the number of classifiers depending on the identified organ (paragraph 98, Information about the tumor shape and coarse consistency can be determined from the pre-operative scans to select the proper subspaces); and
training the one or more untrained classifiers using the volumetric medical image (paragraph 63, At 215, a subspace is learned from the training data).
Referring to claim 15, Tewfik discloses performing or repeating steps a) to d) for each of N-1 single points of interest within the volumetric medical image, wherein N is less than or equal to a total number of voxels of the volumetric medical image (paragraph 68, Scanning data for approximately 10% of an organ surface is sufficient for reconstruction).
Referring to claim 20, Tewfik discloses wherein step c) comprises sampling the voxels in a sparse or random manner (paragraph 42, Sparse surface and internal structure representations can be used in reconstruction methods based on limited view data from the surface of the organ).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521 and El-Zehiry et al. US Publication 2021/0279884 as applied to claims 1 and 17 above, and further in view of Yang et al. US Publication 2005/0285858 (hereafter “Yang.”).
Referring to claim 2, Tewfik discloses the sampling model and El-Zehiry discloses wherein a cube is centered on the single point of interest (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted).
Tewfik and El-Zehiry do not disclose expressly wherein the sampling model comprises multiple cubes nested within each other.
Yang discloses wherein the sampling model comprises multiple cubes nested within each other (paragraph 29, The top level node of an octree is referred to as the root node, which usually contains information about the entire dataset volume. Each intermediate layer node contains information about the nodes at the next lower level).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to provide multiple cubes nested within each other. The motivation for doing so would have been to increase the efficiency and speed of rendering volumetric images. Therefore, it would have been obvious to combine Yang with Tewfik and El-Zehiry to obtain the invention as specified in claim 2.
Claims 3 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521 and El-Zehiry et al. US Publication 2021/0279884 as applied to claims 1 and 17 above, and further in view of Brosch et al. US Publication 2020/0410691 (hereafter “Brosch”).
Referring to claims 3 and 18, Tewfik discloses wherein step c) comprises sampling the voxels with a sampling rate per unit length, area or volume (paragraph 183, At 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system), but does not disclose expressly wherein the sampling decreases with a distance of a respective voxel from the single point of interest.
Brosch discloses wherein step c) comprises sampling the voxels with a sampling rate per unit length, area or volume which decreases with a distance of a respective voxel from the single point of interest (paragraph 25, the sampling rate can be reduced with increasing distance to the center of the respective surface element, wherein the resulting reduced overall sampling rate can lead to reduced computational efforts needed for segmenting the object in the image).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to decrease sampling with a distance from a single point of interest. The motivation for doing so would have been to increase efficiency by reducing computational efforts needed to segment the object in the image. Therefore, it would have been obvious to combine Brosch with Tewfik to obtain the invention as specified in claims 3 and 18.
Claims 4 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521, El-Zehiry et al. US Publication 2021/0279884 and Brosch et al. US Publication 2020/0410691 as applied to claims 3 and 18 above, and further in view of Xu et al. Publication 2009/0087122 (hereafter “Xu”).
Referring to claims 4 and 19, Tewfik discloses sampling the voxels (paragraph 183, At 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system).
Brosch discloses wherein the sampling rate decreases (paragraph 25, the sampling rate can be reduced with increasing distance to the center of the respective surface element, wherein the resulting reduced overall sampling rate can lead to reduced computational efforts needed for segmenting the object in the image).
Tewfik and Brosch do not disclose expressly wherein the sampling rate decreases at a non-linear rate.
Xu discloses wherein the sampling rate decreases at a non-linear rate (paragraph 42, The original video data may undergo a step of compression such as reducing the number of pixels in the frames by a non-linear spatial sub-sampling operation).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to decrease the sampling rate at a non-linear rate. The motivation for doing so would have been to reduce data processing to improve efficiency without sacrificing image analysis. Therefore, it would have been obvious to combine Xu with Tewfik and Brosch to obtain the invention as specified in claims 4 and 19.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521 and El-Zehiry et al. US Publication 2021/0279884 as applied to claim 1 above, and further in view of He et al. US Publication 2021/0327563 (hereafter “He”).
Referring to claim 10, Tewfik discloses wherein the volumetric medical image or a part thereof comprising the single point of interest is displayed on a graphical user interface, wherein a semantic description of the identified type of organ is generated and displayed (paragraph 91, A living 3D reconstructed image will move on the monitor in real time as the organ itself or adipose tissue moves and may assist the surgeon in keeping track of a tumors location in relation to the organs surface and the location of blood vessels in adipose tissue, while manipulating and exposing the organ during an operation).
El-Zehiry discloses wherein the volumetric medical image or a part thereof comprising the single point of interest is displayed on a graphical user interface (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted).
Tewfik and El-Zehiry do not disclose expressly wherein a semantic description of the identified type of organ is generated and displayed at or adjacent to the single point of interest.
He discloses wherein the volumetric medical image or a part thereof comprising the single point of interest is displayed on a graphical user interface, wherein a semantic description of the identified type of organ is generated and displayed at or adjacent to the single point of interest (paragraph 120, Subsequent to such receipt, autonomously or via clinician activation, image viewer 51 proceeds to display the feature assessment data 31a in a textual format or a graphical format, and salient image generator 53 process salient image data 36a to generate salient image(s) 33 for display by image viewer 51).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to display a semantic description of the identified type of organ. The motivation for doing so would have been to inform the user of relevant information related to the medical image. Therefore, it would have been obvious to combine He with Tewfik and El-Zehiry to obtain the invention as specified in claim 10.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Tewfik et al. US Publication 2011/0044521, El-Zehiry et al. US Publication 2021/0279884 and Do et al. US Publication 2020/0285906 (hereafter “Do”).
Referring to claim 16, Tewfik discloses a computer-implemented method of training a classifier for identifying a type of organ in a volumetric medical image, comprising:
a) receiving the volumetric medical image, the volumetric medical image comprising at least one organ or portion thereof (paragraph 66, At 235, data is captured. The data capture can be accomplished using model data, ultrasound data, computerized tomography (CT), or magnetic resonance imaging (MRI) data. At 240), and receiving the type of the at least one organ (paragraph 64, An iterative subspace identification (ISI) method is used to learn the subspaces);
c) sampling voxels from the volumetric medical image (paragraph 183, At 610, the organ surface is sampled. Sampling can include generating data using an MRI modality, a CT scan, an ultrasound, a video camera or other system), wherein at least one voxel is skipped between two sampled voxels (paragraph 78, The global sampling strategy uses single surface pixel uniformly distributed around the organ as shown in FIG. 3A);
d) identifying the type of organ by applying an untrained classifier to the sampled voxels (paragraph 63, At 210, data from organ 205 is transformed using spherical harmonics. At 215, a subspace is learned from the training data);
While Tewfik discloses the volumetric medical image, Tewfik does not disclose expressly receiving a single point of interest within the volumetric medical image.
El-Zehiry discloses b) receiving a single point of interest within the volumetric medical image (paragraph 56, In act D, the clinician identifies an area of interest, for example, a chamber or a blood vessel of the heart. It may be sufficient for the clinician to position a cursor over the area of interest, (e.g., “inside” the cavity shown on the monitor), so that the coordinates of the cursor relative to the image may be noted); and
c) sampling, from a sampling model centered on the single point of interest, voxels from the volumetric medical image (paragraph 56, In act E, the 3D volume M and any information identifying the area of interest are then fed to the model 1 which has been trained as explained above and illustrated in FIG. 2).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to receive a single point of interest within the volumetric medical image. The motivation for doing so would have been to increase efficiency by allowing the user to select the area of the image to be analyzed.
While Tewfik discloses applying an untrained classifier to the sampled voxels, Tewfik does not disclose expressly the training including e) comparing the identified type of organ with the received type of the at least one organ; and f) modifying the classifier depending on the comparison of step e) to obtain a trained classifier.
Do discloses e) comparing the identified type of organ with the received type of the at least one organ (paragraph 46, at step 350, the system may evaluate whether the network's ability to identify a feature in an image exceeds a defined threshold of speed, classification accuracy, reproducibility, efficacy, or other performance metric); and
f) modifying the classifier depending on the comparison of step e) to obtain a trained classifier (paragraph 47, If a desired level of performance is not achieved, then the labeled data may be refined at step 360).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to perform supervised training. The motivation for doing so have been to improve the results of the classifier by modifying incorrect classifications from the machine learning model. Therefore, it would have been obvious to combine El-Zehiry and Do with Tewfik to obtain the invention as specified in claim 16.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights