DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 - 13 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “a trained neural network, generating from said received ultrasound images, without reliance on external position or motion sensors, a spatial mapping which encodes one or more adjustments in probe position and rotation required to achieve a target spatial pose associated with the canonical ultrasound view …”
The specification discloses that “the ground-truth corrective orientations were determined by from inertia measurement unit (IMU) data” ([0015], [0041], [0045], inter alia, as published; instant claim 8).
Applicant has therefore not adequately disclosed a trained neural network that generates a spatial mapping “without reliance on external position or motion sensors” because the training relies on external position or motion sensors (i.e., the IMUs).
There is thus insufficient written description of the claimed invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 - 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 is indefinite because it is unclear what is meant by the method comprising “a trained neural network, generating …” The claim appears to be missing text. Examiner suggests amending the limitation to recite a verb as the first word.
Claim 1 is indefinite because it is unclear what the scope of “external position or motion sensors” is. It is unclear what the prohibited position/motion sensors are external to.
Claim 5 is indefinite because it is unclear if “the graphical user interface provides…” attempts to modify the graphical user interface or to set forth a method step of ‘providing, via the graphical user interface’. Moreover, it is unclear how the “instruction’ relates to the visual guidance elements. It is unclear if these are two different elements or not.
Claim 7 is indefinite because there is insufficient antecedent basis for “each ground-truth corrective orientation”. No “corrective” orientation has been recited. It is unclear what is being referred to.
Claim 8 is indefinite because it is unclear how the “the ground-truth corrective orientations were determined by from inertia measurement unit (IMU) data received from an IMU mounted on a training probe during a training session”, as claim 1 prohibits reliance on external position or motion sensors. Claim 8 thus appears mutually exclusive with claim 1.
Examiner note: dependent claims that are listed in the rejection header above as being rejected while not having been specifically addressed are rejected by virtue of dependency.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1 - 13 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 - 24 of U.S. Patent No. 11,593,638. Although the claims at issue are not identical, they are not patentably distinct from each other because both sets of claims are directed to devices and methods involving receiving ultrasound images; generating, using a trained orientation neural network, a transformation/spatial mapping which encodes one or more adjustments in probe position and rotation required to achieve a target spatial pose associated with a canonical ultrasound view; transforming the spatial mapping into visual guidance data to achieve the required adjustment; generating and displaying, on a graphical user interface, one or more visual guidance elements derived from the visual guidance data to instruct a user how to manipulate the probe; and updating the visual guidance elements until the canonical ultrasound view is achieved. Specifically:
Claim 1 is suggested by reference claim 1 and/or 13.
Claims 2 - 6 are suggested by reference claim 1 and/or 13. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1 and/or 13 to include the variously recited display features, in order to provide the orientation instructions of the reference claims with easy-to-understand visual forms.
Claims 7 - 8 are suggested by reference claim 4.
Claim 9 is suggested by reference claim 3 and/or 15.
Claim 10 is suggested by reference claim 5.
Claim 11 is suggested by reference claim 5/6 and/or 17/18.
Claim 12 is suggested by reference claim 7 and/or 19.
Claim 13 is suggested by reference claim 8 and/or 20.
Claims 1 - 13 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 - 14 of U.S. Patent No. 12,450,483. Although the claims at issue are not identical, they are not patentably distinct from each other because both sets of claims are directed to devices and methods involving receiving ultrasound images; generating, using a trained orientation neural network, a transformation/spatial mapping which encodes one or more adjustments in probe position and rotation required to achieve a target spatial pose associated with a canonical ultrasound view; transforming the spatial mapping into visual guidance data to achieve the required adjustment; generating and displaying, on a graphical user interface, one or more visual guidance elements derived from the visual guidance data to instruct a user how to manipulate the probe; and updating the visual guidance elements until the canonical ultrasound view is achieved. Specifically:
Claim 1 is suggested by reference claim 1, 4, 8, and/or 11.
Claims 2 - 6 are suggested by reference claim 1, 4, 8, and/or 11. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1, 4, 8, and/or 11 to include the variously recited display features, in order to provide the orientation instructions of the reference claims with easy-to-understand visual forms.
Claim 7 is suggested by reference claim 1, 4, 8, and/or 11.
Claim 8 is suggested by reference claim 2 and/or 9.
Claims 9 - 10 is suggested by reference claim 1, 4, 8, and/or 11. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1, 4, 8, and/or 11 to have the loss function have an additional term, in order to constrain the loss function, as is conventional in the art.
Claim 11 is suggested by reference claim 1, 4, 8, and/or 11.
Claim 12 is suggested by reference claim 3 and/or 10.
Claim 13 is suggested by reference claim 1, 4, 8, and/or 11. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1, 4, 8, and/or 11 to have the computing device be one of: a smartphone, a tablet, a laptop, a personal computer, or a smart appliance, in order to use well-known devices as the computing device, as is conventional in the art.
Claims 1 - 13 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 - 14 of copending Application No. 19/386,344 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because both sets of claims are directed to devices and methods involving receiving ultrasound images; generating, using a trained orientation neural network, a transformation/spatial mapping which encodes one or more adjustments in probe position and rotation required to achieve a target spatial pose associated with a canonical ultrasound view; transforming the spatial mapping into visual guidance data to achieve the required adjustment; generating and displaying, on a graphical user interface, one or more visual guidance elements derived from the visual guidance data to instruct a user how to manipulate the probe; and updating the visual guidance elements until the canonical ultrasound view is achieved. Specifically:
Claim 1 is suggested by reference claim 1 and/or 8.
Claims 2 - 6 are suggested by any of reference claim 2 - 7 and/or 9 - 14. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 2 - 7 and/or 9 - 14 to include the variously recited display features, in order to provide the orientation instructions of the reference claims with easy-to-understand visual forms.
Claim 7 and 9 - 10 are suggested by reference claim 1 and/or 8.
Claim 8 is suggested by reference claim 1 and/or 8. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1 and/or 8 to have the ground-truth corrective orientations be determined from inertia measurement unit (IMU) data received from an IMU mounted on a training probe during a training session, in order to provide accurate probe movement data for the training.
Claim 11 is suggested by reference claim 1 and/or 8.
Claim 12 is suggested by reference claim 1 and/or 8. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1 and/or 8 to make a diagnosis as recited, in order to use the ultrasound images data for its intended function.
Claim 13 is suggested by reference claim 1 and/or 8. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of reference claim 1, 4, 8, and/or 11 to have the computing device be one of: a smartphone, a tablet, a laptop, a personal computer, or a smart appliance, in order to use well-known devices as the computing device, as is conventional in the art.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 - 8, 11, and 13 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Cadieu et al. (US 2018/0153505, of record).
Regarding claim 1, Cadieu shows a computer-implemented method for providing real-time visual guidance for orienting an ultrasound probe toward a canonical ultrasound view, the method comprising:
receiving successive ultrasound images acquired by an ultrasound probe during a scanning procedure unit (“image 130 of the target organ is presented input to an estimator 140, such as a neural network”, [0016] and fig. 1), each ultrasound image reflecting a current probe position and rotation relative to an anatomical region (implicit - the image data necessarily reflect the probe position and rotation);
a trained neural network (neural network, [0016]), generating from said received ultrasound images, without reliance on external position or motion sensors (block 340 is optional, [0033]; “case where an inertial measurement unit is not present”, [0034]), a spatial mapping (deviation 190, [0016]) which encodes one or more adjustments in probe position and rotation required to achieve a target spatial pose associated with the canonical ultrasound view (“input of the contemporaneously acquired image 130 to the estimator 140 produces an output of a deviation 190 of the contemporaneous pose of the ultrasound probe 120 from an optimal pose of the ultrasound probe 120”, [0016] and fig. 1; “neural network providing decisioning of a deviation from an optimal probe pose relative to an input image of a target organ of a human form”, [0022]);
transforming the spatial mapping into visual guidance data to achieve the required adjustment probe (“logic 160 then processes the pose deviation 190 of the ultrasound probe 120 and emits feedback 170 in the form of visual feedback”, [0018] and fig. 1);
generating and displaying, on a graphical user interface, one or more visual guidance elements derived from the visual guidance data to instruct a user how to manipulate the probe (“visual feedback such as a three-dimensionally rendered scene with the two probe models showing current and optimal probe poses with suggested maneuver; the amount of agreement between the current and optimal poses; or red, green or yellow colors indicating how large an adjustment of the orientation of the ultrasound probe 120 is required to approach the optimal pose”, [0018]); and
updating the visual guidance elements until the canonical ultrasound view is achieved (see iterative process in fig. 3).
Regarding claim 2, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the one or more visual guidance elements comprise a set of arrows, and wherein the type of rotation visually communicated is one of a pitch or a yaw (“visual feedback such as a three-dimensionally rendered scene with the two probe models showing current and optimal probe poses with suggested maneuver; the amount of agreement between the current and optimal poses; or red, green or yellow colors indicating how large an adjustment of the orientation of the ultrasound probe 120 is required to approach the optimal pose”, [0018]).
Regarding claim 3, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the one or more visual guidance elements comprise colored rotation markings (“visual feedback such as a three-dimensionally rendered scene with the two probe models showing current and optimal probe poses with suggested maneuver; the amount of agreement between the current and optimal poses; or red, green or yellow colors indicating how large an adjustment of the orientation of the ultrasound probe 120 is required to approach the optimal pose”, [0018]).
Regarding claim 4, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the updating is performed until the current probe orientation is within a predefined error range of the target probe orientation ([0034] and iterative process in fig. 3).
Regarding claim 5, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the graphical user interface provides an instruction to the user to either move the ultrasound probe ([0018]).
Regarding claim 6, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the updating step generates updated visual guidance for each new subsequent ultrasound image captured by the probe ([0034] and iterative process in fig. 3).
Regarding claim 7, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the trained orientation neural network was trained using a training dataset comprising at least one canonical view, a plurality of non-canonical images captured around the at least one canonical view, and corresponding ground-truth transformations, each ground-truth corrective orientation representing a required adjustment in position and rotation between a non-canonical image and the canonical view (“estimator 140 is trained based upon a set of training images 150 of one or more different target organs, each with a known probe pose deviation from the optimal probe pose”, [0016] and fig. 1; [0022]).
Regarding claim 8, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the ground-truth corrective orientations were determined by from inertia measurement unit (IMU) data received from an IMU mounted on a training probe during a training session ([0024] - [0029]).
Regarding claim 11, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the canonical view is one of a plurality of available canonical views for a body part (implicit: plural canonical views for a body part are necessarily “available” in some way, as a matter of logic).
Regarding claim 13, Cadieu discloses the claimed invention substantially as noted above. Cadieu further shows the computing device is a personal computer ([0038]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 9 - 10 are rejected under 35 U.S.C. 103 as being unpatentable over Cadieu et al. (US 2018/0153505 , of record) in view of Petkov et al. (US 2018/0260997, of record).
Regarding claim 9, Cadieu discloses the claimed invention substantially as noted above.
While those of ordinary skill in the art would understand that the neural network of Cadieu must be trained using a loss function, Cadieu is silent as to the function’s presence.
Petkov discloses medical imaging techniques. Petkov teaches utilizing a loss function to update a neural network in training, wherein the loss function is configured to reduce a distance between an answer/output of the neural network and a ground truth/desired answer (“…cross-entropy loss function is used to train. Other functions measuring the difference of a current output from the ground truth and optimizing the neural network to minimize the difference may be used”, [0050]). The loss function further includes an additional term based on a probability measure to constrain the solutions, as any term in the loss function meets this limitation.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the training of Cadieu’s neural network to include utilizing a loss function to update the neural network, wherein the loss function is configured to reduce a distance between an answer/output of the neural network and a ground truth/desired answer, wherein the loss function further includes an additional term based on a probability measure to constrain the solutions, as taught by Petkov, in order to optimize the neural network, as explained by Petkov ([0050]), especially since it is understood in the art that neural networks require such loss functions to be implemented during training in order to be properly trained to produce desired output. In the combined invention of Cadieu and Petkov, the ‘answer/output of the neural network’ is the calculated corrective orientation, and the ‘ground truth/desired answer’ is the ground truth corrective orientation.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Cadieu et al. (US 2018/0153505 , of record) in view of Baek (US 2013/0070993, of record).
Regarding claim 12, Cadieu discloses the claimed invention substantially as noted above.
Cadieu does not explicitly discuss making a diagnosis from the current ultrasound image upon the current probe orientation reaching the target probe orientation.
Baek discloses ultrasound diagnosis systems and methods. Baek teaches making a diagnosis from the current ultrasound image upon the current probe orientation reaching the target probe orientation (“standard image generating unit 120 generates an ultrasound standard image, a standardized diagnosis image is used, thereby improving the accuracy of diagnosis of the object,” [0039]; [0099]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Cadieu to include making a diagnosis from the current ultrasound image upon the current probe orientation reaching the target probe orientation, as taught by Baek, in order ascertain the patient’s need, as diagnosis is a notoriously well-understood need in the medical arts, while improving the accuracy of diagnosis, as suggested by Baek ([0009]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMELIE R DAVIS whose telephone number is (571)270-7240. The examiner can normally be reached Monday-Friday, 9:30 - 6:00 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571)272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMELIE R DAVIS/Primary Examiner, Art Unit 3798
