Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Interpretation
The scope of the pending claims is interpreted as follows:
Claims 1-19, 30 are given their broadest reasonable interpretation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-3, 5-8, 15-18, 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schreckenberg (US 2022/0130046) in view of Barak (US 2024/0041535)
Claim 1
Schreckenberg discloses a system comprising:
a memory storing instructions (Fig. 10); and
one or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising (Fig. 10):
generating, based on imagery of a scene, a deformable three-dimensional (3D) model of the scene (Schreckenberg, ¶ 78: “For a better visualisation of left ventricular activity, a dynamic surface model 14, represented in FIG. 2, may be used. This surface model 14 represents a simplified model of the blood-tissue boundary of the left ventricle 4. It essentially has the shape of a bag 18 with two openings 16, 20 at the top: The opening 16 represents the mitral annulus, whereas the opening 20 represents the left ventricular outflow track (LVOT), in which the aortic valve is located. The dynamic surface model 14 may be derived from a sequence of three-dimensional ultrasound images of the heart, in particular by tracking anatomical features or by tracking speckle from one image to the next. Thereby, a surface model consisting of a set of vertices 22 spanning a wire mesh 24 may be generated for each three-dimensional image. Such dynamic surface model 14 may be viewed in cine mode, i.e. like a movie, so that the viewer gains an impression of the movement of the left ventricle with heartbeat”);
identifying a first point on an anatomical object located in the scene (Schreckenberg, ¶ 43: “For example, the user may selects two points on the volume rendering, and the distance between such points will automatically be calculated.”); and
determining, based on the deformable 3D model, a dynamic measurement value representative of a contour physical distance between the first point and a second point in the scene (Schreckenberg, ¶ 43: “For example, the user may selects two points on the volume rendering, and the distance between such points will automatically be calculated.”),
wherein the dynamic (Schreckenberg, ¶ 37: “These points may be used as the basis for defining the VOI, which accordingly moves with the heartbeat, thereby preventing the mitral valve from moving out of the volume of interest. In a useful embodiment, the user may measure e.g. the diameter of the mitral valve and accordingly select the best fitting valve from a library.”).
Schreckenberg does not explicitly disclose, but Barak discloses dynamic measurement value dynamically updates with movement of one or more objects within the scene (Barak, ¶¶ 130-132, 147: “This type of representation provides potential advantages for realistic modeling of dynamically changing lung geometries, and/or for reducing the number of computations involved in updating a model lung geometry to match measurements of the modeled lung itself. In some embodiments, the updating occurs in real time, allowing the model to be used in dynamic tracking of the lung shape, e.g., during a navigational bronchoscopy procedure… An aspect of some embodiments of the present disclosure relates to the dynamic display of 3-D image data surfaces transformed to correspond to current positions of anatomical features they represent, based on real-time updating measurements indicative of shape of the represented anatomical features.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to dynamically update measurements as claimed.
One of ordinary skill in the art would have motivation to compensate for movement, deformation, and the like. One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 2
Schreckenberg discloses wherein at least one of the first point or the second point is on an outer surface of the anatomical object (¶¶ 13, 15, 78:
PNG
media_image1.png
285
279
media_image1.png
Greyscale
“Visualizing a dynamical anatomical structure… dynamic model, e.g. a shape/surface model…left ventricular activity, a dynamic surface model 14, represented in FIG. 2, may be used. This surface model 14 represents a simplified model of the blood-tissue boundary of the left ventricle 4. It essentially has the shape of a bag 18 with two openings 16, 20 at the top: The opening 16 represents the mitral annulus, whereas the opening 20 represents the left ventricular outflow track (LVOT), in which the aortic valve is located.”).
Claim 3
Schreckenberg discloses wherein at least one of the first point or the second point moves as the anatomical object is deformed (Schreckenberg, ¶ 37: “These points may be used as the basis for defining the VOI, which accordingly moves with the heartbeat, thereby preventing the mitral valve from moving out of the volume of interest”)
Claim 5
Schreckenberg discloses wherein the identifying the first point includes detecting a user input designating the first point within a select one or both of the imagery of the scene or the deformable 3D model (Schreckenberg, ¶¶ 42-44: “an input tool is provided together with the three-dimensional visualisation environment… the input tool allows a user to select points and to take measurements on the anatomical structure”).
Claim 6
Schreckenberg does not disclose, but Barak discloses wherein the identifying the first point includes using a machine learning algorithm to designate the first point on the anatomical object located in the scene (Barak, ¶ 308: “trained machine-learning models are used to predict the propagation (or extrapolation) of the deformation from known control points to other lung tissue”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to use machine learning as claimed.
One of ordinary skill in the art would have motivation to “pro[vide] better predictions for the deformation of the lungs in between control points.” (¶ 308). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 7
Schreckenberg discloses further comprising identifying the second point in the scene, the identifying the second point comprising detecting a user input designating the second point within a select one or both of the imagery of the scene or the deformable 3D model (Schreckenberg, ¶¶ 42-44: “an input tool is provided together with the three-dimensional visualisation environment… the input tool allows a user to select points and to take measurements on the anatomical structure”).
Claim 8
Schreckenberg does not disclose, but Barak discloses further comprising identifying the second point in the scene by using a machine learning algorithm to designate the second point in the scene (Barak, ¶ 308: “trained machine-learning models are used to predict the propagation (or extrapolation) of the deformation from known control points to other lung tissue”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to use machine learning as claimed.
One of ordinary skill in the art would have motivation to “pro[vide] better predictions for the deformation of the lungs in between control points.” (¶ 308). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 15
Schreckenberg discloses wherein the process further comprises performing an operation based on the dynamic measurement value (Schreckenberg, ¶ 18: “Therefore, such simplified models are useful in providing an orientation to the user, for example when planning an intervention or making measurements on a particular part of the anatomy.”).
Claim 16
The same teachings and rationales in claim 1 are applicable to claim 16.
Claim 17
The same teachings and rationales in claim 2 are applicable to claim 17.
Claim 18
The same teachings and rationales in claim 3 are applicable to claim 18.
Claim 30
Examiner’s Interpretation:
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Claim 30 as drafted recites A non-transitory computer-readable medium.
Because non-transitory without additional definition excludes signals and the like, the broadest reasonable interpretation of the claimed medium in view of Applicant’s specification covers only eligible subject matter.
Claim Mapping:
The same teachings and rationales in claim 1 are appliable to claim 30.
Claim(s) 10-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schreckenberg (US 2022/0130046) in view of Barak (US 2024/0041535) and Shelton (US 2021/0212794)
Claim 10
Schreckenberg does not disclose, but Shelton discloses wherein the first point is positioned in a first field of view of an imaging device that captures the imagery and the second point is positioned in a second field of view of the imaging device that is different than the first field of view (Shelton, ¶¶ 321-323: “The visualization display 6620 can be configured to display the video feed and transition to the 3D representations when the user causes the visualization display 6620 to display a POV that differs from the real-time video feed POV provided by the camera(s) 6520. In this particular example, the visualization display 6620 displays 3D representations of a lung 6508, a tumor 6506, and various structures 6610 (e.g., vessels). The 3D representations can be generated from non-real-time image sources and real-time images 6600 captured via the camera 6520, for example. The 3D representations can include portions that lie at least partially outside of the FOV of the camera 6520 and that can be displayed on the visualization display 6620 when desired by the user, as shown in FIG. 32.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider different views.
One of ordinary skill in the art would have motivation to “provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV.” (¶ 321). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 11
Schreckenberg does not disclose, but Shelton discloses wherein the process further comprises instructing one or more display devices to display the imagery depicting the scene in a first display view and the deformable 3D model in a second display view (Shelton, ¶¶ 321-323: “The visualization display 6620 can be configured to display the video feed and transition to the 3D representations when the user causes the visualization display 6620 to display a POV that differs from the real-time video feed POV provided by the camera(s) 6520. In this particular example, the visualization display 6620 displays 3D representations of a lung 6508, a tumor 6506, and various structures 6610 (e.g., vessels). The 3D representations can be generated from non-real-time image sources and real-time images 6600 captured via the camera 6520, for example. The 3D representations can include portions that lie at least partially outside of the FOV of the camera 6520 and that can be displayed on the visualization display 6620 when desired by the user, as shown in FIG. 32.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider different views.
One of ordinary skill in the art would have motivation to “provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV.” (¶ 321). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 12
Schreckenberg does not disclose, but Shelton discloses wherein the first display view is coupled with the second display view such that the imagery depicting the scene in the first display view moves simultaneously with the deformable 3D model in the second display view (¶ 323; “n various aspects, the imaging system 142 can display the 3D representations generated from the fused images and provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV. As one example, FIG. 32 is a visualization display 6620 and GUI 6622 of the surgical procedure of FIG. 27 provided by an imaging system 142. The visualization display 6620 and GUI 6622 can be displayed on a display 146 (FIG. 2), for example. The visualization display 6620 can include a real-time video feed from the camera(s) 6520 supplemented with the generated 3D representation of the surgical site, for example. The video feed provided by the camera(s) 6520 can be overlaid on or otherwise displayed in conjunction with the generated 3D representations on the visualization display 6620”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider different views.
One of ordinary skill in the art would have motivation to “provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV.” (¶ 321). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 13
Schreckenberg does not disclose, but Shelton discloses wherein the first display view is uncoupled from the second display view such that the deformable 3D model in the second display view moves independently from the imagery depicting the scene in the first display view (¶ 323: “The visualization display 6620 can be configured to display the video feed and transition to the 3D representations when the user causes the visualization display 6620 to display a POV that differs from the real-time video feed POV provided by the camera(s) 6520. In this particular example, the visualization display 6620 displays 3D representations of a lung 6508, a tumor 6506, and various structures 6610 (e.g., vessels). The 3D representations can be generated from non-real-time image sources and real-time images 6600 captured via the camera 6520, for example. The 3D representations can include portions that lie at least partially outside of the FOV of the camera 6520 and that can be displayed on the visualization display 6620 when desired by the user, as shown in FIG. 32.”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider different views.
One of ordinary skill in the art would have motivation to “provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV.” (¶ 321). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 14
Schreckenberg does not disclose, but Shelton discloses wherein the process further comprises instructing the display device to display a 3D contour extending between the first point and the second point along a surface of the one or more objects within the scene, wherein the 3D contour is representative of the contour physical distance (e.g. d1 d2; Fig. 32; ¶ 327: “In another aspect, the GUI 6622 can include a distance GUI element 6627 indicating the calculated distances between the various tissues, structures, and/or objects being visualized, i.e. between the corresponding elements in column A and column B. The aforementioned measurements are provided for illustrative purposes and the surgical system can calculate and display a variety of other measurements or parameters.”)
PNG
media_image2.png
464
610
media_image2.png
Greyscale
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider different UI views.
One of ordinary skill in the art would have motivation to “provide users with controls (e.g., a GUI) for manipulating the displayed visualization POV.” (¶ 321). One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim(s) 4, 9, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schreckenberg (US 2022/0130046) in view of Barak (US 2024/0041535) and LaMarca, DefSlam
Claim 4
Schreckenberg as modified by Barak does not explicitly disclose, but Lamarca discloses wherein the generating the deformable 3D model includes using a simultaneous localization and mapping heuristic (Lamarca, abstract: “mapping (SLAM) algorithms perform robustly when observing rigid scenes…close-up sequences of deforming scenes, both in a laboratory-controlled experiment and in medical endoscopy sequences, producing accurate 3-D models of the scene with respect to the moving camera”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to use SLAM as claimed.
One of ordinary skill in the art would have motivation to compensate for movement, deformation, and the like. One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 9
Schreckenberg as modified by Barak does not explicitly disclose, but Lamarca discloses wherein the imagery of the scene comprises stereoscopic images captured by an imaging device (Section II: “The deformable SLAM methods in the literature rely on sensors providing depth information, i.e., RGB-D or stereo sensors”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to use SLAM as claimed.
One of ordinary skill in the art would have motivation to compensate for movement, deformation, and the like. One of ordinary skill in the art would have had a reasonable expectation of success because Schreckenberg also considers that the object is deformable.
Claim 19
The same teachings and rationales in claim 4 are applicable to claim 19.
Additional Prior Art
Additional prior art relevant to Applicant’s disclosure but not relied upon:
Kolesov (US 2019/0028637) discloses an analogous deformable object:
PNG
media_image3.png
361
466
media_image3.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights