DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(u)(1) because the view numbers are preceded by “Figure” instead of “FIG.”
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The abstract of the disclosure is objected to because it is not concise. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b). The following is a suggested amendment to overcome the objection:
A method and apparatus for generating cutting trajectories of medical image targets[[.]], [[The]] the method including obtaining a target image to be processed, which includes chest X-ray images, cardiac MRI images, and dermatoscope detection images[[;]], selecting an initial point of the target image and using it as the starting point for a navigation agent[[;]] and guiding the navigation agent to generate trajectory points until a cutting trajectory containing the target to be segmented is generated. Based on the generated trajectory points and the sampling areas corresponding to each sampling operation, real-time calculation is used to determine the deviation of each sampling. The method also includes correcting the sampling direction, generating cutting trajectories on the target image, and optimizing the generated cutting trajectories to obtain a target image containing the segmented target.
Claim Objections
Claim 3 is objected to because of the following informalities: “an image being processed” should be changed to “[[an]] the image being processed” to clarify that it is the same “image to be processed” from claim 1.
Claim 9 is objected to because of the following informalities: “the medical image” should be changed to “the
Claim 10 is objected to because of the following informalities: “an sampling operation module” should be changed to “[[an]] a sampling operation module” to improve clarity.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a creation module, configured to acquire an input image, which includes chest X-ray images, cardiac MRI images and dermatoscope detection images and contains target structures for medical dissection” in claims 10-11.
“an sampling operation module, configured to: select an initial point of the input image; use the initial point as a starting point ... perform sampling operations ... input the obtained sampling areas ... and generate ... continuous trajectory points” in claims 10-11.
“a correction module, configured to: calculate, at real-time, a deviation of each sampling operation ... and correct a direction ... to optimize the generated cutting trajectory while the cutting trajectory is generated on the input image” in claims 10-11.
“a cutting module, configured to segment the input image based on the optimized cutting trajectory to obtain a target image containing the target structures for medical dissection” in claims 10-11.
“a selection module, configured to select the initial point of the input image and perform recursive sampling ... for the predetermined number of times ... and adjust a sampling direction ... to obtain a corrected sampling direction of the initial sampling area” in claim 11.
“a recursive sampling module, configured to perform the recursive sampling for each trajectory point to be formed on the input image for the predetermined number of times to obtain the displacement corresponding to each corrected trajectory point” in claim 11.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed functions, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed functions); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed functions so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-11 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 1 recites “wherein the image to be processed comprises chest X-ray images, cardiac MRI images and dermatoscope detection images”. Claim 10 recites similar language. Taking this language at face value, it means “the image” is made of multiple chest X-ray images, multiple cardiac MRI images and multiple dermatoscope detection images, which is confusing because it is unclear how all of the images could be a singular “image”. For purposes of applying prior art, this language is interpreted to mean that the “input image” is an X-ray image, a cardiac MRI image or a dermatoscope image. Dependent claims 2-9 and 11 are rejected for inheriting and not curing the deficiencies of claims 1 and 10.
Claim 1 recites “the image to be processed contains the target to be cut with relevant medical anatomical structures”. The phrase “to be cut with relevant medical anatomical structures” is confusing because the descriptive term “relevant” is subjective and the claims do not provide sufficient context for understanding the distinction between “relevant” and “not relevant” structures. Despite reciting the terms “cut”, “cutting” and the like, the broadly-recited claims could be interpreted as referring to actual cutting as in a surgical procedure, or could be referring to the image processing operation of cutting or extracting a region from an image, e.g., segmentation. Thus, the scope of the “medical anatomical structures” is ambiguous. Although claim 10 does not have the same issue as claim 1 with respect to the term “relevant”, as claim 10 instead recites “target structures”, claim 10 also includes several variations of “cut”, “cutting trajectory”, and the like, and is therefore indefinite for the same reasons as claim 1. For purposes of applying prior art, “cut”, “cutting”, “cutting trajectory” and variations thereof are interpreted to exclude the physical, surgical, or “dissecting” type of interpretation and instead be limited to the image processing type of interpretation, such as image segmentation, or image object contouring to extract an object boundary. Dependent claims 2-9 and 11 are rejected for inheriting and not curing the deficiencies of claims 1 and 10.
Claim 7 recites “processing the current trajectory point iteratively through the inputting and the adjusting”. The antecedent basis of each of “the inputting” and “the adjusting” is unclear. Claim 1 recites “inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to a next trajectory point”, claim 2 recites “adjusting the sampling direction of an initial sampling area” and claim 7 further recites “inputting the extracted image block corresponding to the sampling area into a pre-trained deep learning model to output a temporary sampling direction; adjusting the sampling direction of the current trajectory point's sampling area to match the temporary sampling direction”. Since there are two instances of adjusting and two instances of inputting, it is unclear which ones correspond to “the inputting and the adjusting” in claim 7. For purposes of applying prior art, it is assumed that “the inputting and the adjusting” refers to the inputting and adjusting recited in claim 7.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8 and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by The Deep Poincaré Map: A Novel Approach for Left Ventricle Segmentation to Mo et al. (hereinafter “Mo”).
Regarding claim 1, Mo teaches a method for generating a cutting trajectory (Contour path) of a medical image for a target to be cut (image structure to be segmented), comprising:
obtaining an image to be processed (Mo, FIG. 1, “256x256 Image”), wherein the image to be processed comprises cardiac MRI images (Mo, Abstract, “cardiac MRI images”), and the image to be processed contains the target to be cut with relevant medical anatomical structures (Mo, Abstract, “left ventricle (LV)”);
selecting an initial point of the image to be processed (Mo, section 2.2, “At the inference stage, before the first time step t = 0, we determine an initial, rough, starting point using a basic LV detection module and a random sampling direction.”);
using the initial point as a starting point of a navigation agent (Mo, section 2.2, “At each step, given an position pt and a sampling direction st of the agent (which is unknown and is thus inferred as the difference between the current sampling direction and the last), a local patch is extracted and used as the input to the CNN-based policy model.”);
guiding the navigation agent to generate trajectory points until the cutting trajectory containing the target to be cut is generated, comprising (Mo, section 2.2, “The policy model then predicts the displacement for the agent to move, which in turn leads to the next local patch sample. This process iterates until the limit cycle is reached as illustrated earlier (Fig 1).”):
performing sampling operations for each trajectory point (Mo, pg. 2, “In each time step, the DPM extracts a locally oriented patch from the original image”) to obtain a sampling area (Mo, section 2.2, “next local patch sample”);
inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to a next trajectory point (Mo, pg. 2, “The extracted patch will be fed into a CNN to predict the next step displacement for the agent.”; pg. 3, “The DPM uses a CNN-based policy model, trained on locally oriented patches from manually segmented data, to navigate an agent over a cardiac MRI image (256x256) using a locally oriented square patch (64x64) as its input. The agent creates a trajectory over the image tracing the boundary of the LV– no matter where the agent starts on the image. A crucial prerequisite of this methodology is the creation of a vector field whose limit cycle is equal to the boundary surrounding the ROI.”); and
allowing the navigation agent to generate continuous trajectory points to include the cutting trajectory of the target to be cut (Mo, pg. 2, “After a finite number of iterations, A trajectory will be created by the agent.”);
calculating, at real-time (i.e., calculating, generally, as part of a single operation and/or without interruption), a deviation (Mo, section 2.2, “difference between the current sampling direction and the last”) for each sampling operation based on the generated trajectory points and a corresponding sampling area (Mo, section 2.2, “We define a sampling direction which is equal to the velocity vector of the associated point.”);
correcting a sampling direction to optimize the cutting trajectory generated (Mo, section 2.2, “The policy model then predicts the displacement for the agent to move, which in turn leads to the next local patch sample. This process iterates until the limit cycle is reached as illustrated earlier (Fig 1).”), while generating the cutting trajectory on the image to be processed (Mo, pg. 6, “After t iterations, the agent moves slowly toward the boundary of the object. Due to the underlying customized vector field, the DPM is able to guarantee that using different starting points we converge to the same unique periodic orbit (limit cycle).”); and
cutting the image to be processed according to the optimized cutting trajectory to obtain a target image containing the target to be cut (Mo, section 2.3, “Instead of identifying the periodic orbit (the limit cycle) from the trajectory itself, we introduce the Poincaré section [7] which is a hyperplane, ∑, transversal to the trajectory. This cuts through the trajectory of the vector field, as seen in Fig 5a. The stability of a periodic orbit in the image can be reflected by the procession of corresponding points of intersection in ∑ (a lower dimensional space). The Poincaré map is the function which maps successive intersection points with the previous point, and thus, when the mapping reaches a small enough value we may say that the procession of the agent in the image has converged to the boundary (the limit cycle). The convergence of customized dynamic has been studied using the Poincaré-Bendixson theorem [8], however the details are beyond the scope of this paper.”).
Regarding claim 8, Mo teaches the method according to claim 1, further comprising:
based on convergence criteria (Mo, pg. 6, “Due to the underlying customized vector field, the DPM is able to guarantee that using different starting points we converge to the same unique periodic orbit (limit cycle).”), determining whether a process of generating the cutting trajectory containing the target to be cut has been completed (Mo, pg. 6, “DPM is able to guarantee that using different starting points we converge to the same unique periodic orbit (limit cycle).”), comprising:
determining detection lines for the convergence criteria (A Poincaré section (hyperplane) is constructed and each point of the trajectory that intersects the hyperplane is recorded as the agent orbits or laps around the boundary. See Mo at Fig. 5(a). The points are in the 2D image plane. The hyperplane is a line in the 2D image plane connecting intersecting points on the trajectory. Fig. 5(a) is a simplified/instructive example of the theory. For an image of the size used in Fig. 5(b), more than 3 intersection points are used. With four points for example, the hyperplane forms a line between all four points, thereby forming a first detection line between two points and another between the other two points. The same concept applies to three points, except the “detection lines” would partially overlap.);
in the process of generating the cutting trajectory containing the target to be cut, forming interval lines based on intersection points between generated trajectory lines and the detection lines (The hyperplane line segments connecting pairs of points are “interval lines”. See Mo at Figure 5.); and
comparing the interval lines with a preset distance to determine whether the process of generating the cutting trajectory containing the target to be cut has been completed (The distance between successive intersection points on the Poincaré section is the magnitude of the Poincaré map. See Mo at pg. 2, “The magnitude of the Poincaré map is used to determine the final periodic orbit which is coincident with the boundary around the ROI.”).
Claim 10 substantially corresponds to claim 1 by reciting a medical image target segmentation trajectory generation apparatus comprising a creation module (A CNN-based image segmentation algorithm that processes a set of digital images implicitly discloses a general-purpose computer having a generic storage or memory storing instructions executed by a generic processor, or equivalent thereof. See Mo at section 3, “The DPM was trained on the given training subset.”), a sampling operation module (See Mo at section 3), a correction module (See Mo at section 3), and a cutting module (See Mo at section 3) configured to perform the method of claim 1.
Allowable Subject Matter
Claims 2-7, 9 and 11 would be allowable if rewritten to overcome any applicable rejection under 35 U.S.C. 112(b) and any applicable objection for minor informalities set forth in this Office action, and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Image Segmentation Based on the Poincaré Map Method to Zeng et al. is pertinent because it provides relevant background information and additional details related to the Poincaré Map used by Mo.
U.S. Pat. Appl. Pub. No. 20110182489 to Chang et al. discloses methods for segmenting tumors from medical images by sampling areas in a similar manner to the “navigation agent” recited in the instant independent claims. See, e.g., Chang et al. at Figures 3 and 4.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN P POTTS/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights