Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-17, 19-24 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Cvetko (Patent No. US 20230038130 A1).
Regarding claim 1, Cvetko teaches A method for aligning 3D image data with an anatomical structure of a person using a head mounted display, comprising: (Cvetko, [0006] In one embodiment, a method for augmenting real-time views of a patient with three-dimensional (3D) data may include various acts. For example, the method may include identifying 3D data for a patient with the 3D data including an outer layer of the patient and multiple inner layers of the patient. The method may also include determining virtual morphometric measurements of the outer layer of the patient from the 3D data. The method may further include registering a real-time position of the outer layer of the patient in a 3D space. The method may also include determining real-time morphometric measurements of the outer layer of the patient. The method may further include automatically registering the position of the outer layer of the patient from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements. The method may also include displaying, in an augmented reality headset, one of the inner layers of the patient from the 3D data projected onto real-time views of the outer layer of the patient.)
displaying a slice of the 3D image data with respect to an axis; (Cvetko, “[0048] As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient.”)
and receiving adjustments to the slice until an edge of the anatomical structure in the slice aligns with an edge of the anatomical structure of the person. (Cvetko, “[0033] (3) adjust the characteristics of the 3D data that is projected onto the patient 106, such as the brightness and color of the projected 3D data, [0034] (4) adjust the alignment of the 3D data with the patient 106,”)
Regarding claim 2, Cvetko teaches The method as in claim 1, further comprising limiting movement of the slice with respect to the axis. (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”, The examiner interpret that through the different counter-clockwise perspective of the figures, the place of the slice is locked.)
PNG
media_image1.png
684
600
media_image1.png
Greyscale
Regarding claim 3, Cvetko teaches The method as in claim 1, further comprising displaying the slice of the 3D image data orthogonal to the axis. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 4, Cvetko teaches The method as in claim 1, wherein the axis is identified from a plurality of pre-defined axes of the 3D image data. (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space. As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 5, Cvetko teaches The method as in claim 1, further comprising receiving adjustments to the slice that are orthogonal to the axis, until the edge of the anatomical structure in the slice aligns with the edge of the anatomical structure of the person, as detected from a perspective of an AR headset. (Cvetko, “[0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data, [0037] (7) drag the 3D data in a direction of the user 104, such as in the repositioning of a slice of the 3D data, [0038] (8) display different slices of the 3D data including, but not limited to, axial slices, coronal slices, sagittal slices, and oblique slices, and [0039] (9) perform other advanced features of the AR headset 108.”)
Regarding claim 6, Cvetko teaches The method as in claim 1, further comprising applying the adjustments to the slice to the 3D image data. (Cvetko, “[0030] In some embodiments, the virtual user interface 114 may be any virtual user interface generated by the AR headset 108 that includes options for altering the display of the projected inner layer(s) of the patient 106 from the 3D data of the patient 106. For example, the options included in the virtual user interface 114 may include, but are not limited to, options that cause the AR headset 108 to: [0031] (1) quit viewing the augmented view of the patient 106, [0032] (2) display a demo of the capabilities of the AR headset 108, [0033] (3) adjust the characteristics of the 3D data that is projected onto the patient 106, such as the brightness and color of the projected 3D data, [0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data, [0037] (7) drag the 3D data in a direction of the user 104, such as in the repositioning of a slice of the 3D data, [0038] (8) display different slices of the 3D data including, but not limited to, axial slices, coronal slices, sagittal slices, and oblique slices, and [0039] (9) perform other advanced features of the AR headset 108.”)
Regarding claim 7, Cvetko teaches The method as in claim 3, further comprising repeating the displaying the slice, limiting movement of the slice (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”)
and receiving adjustments to the 3D image data for two additional axes. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 8, Cvetko teaches The method as in claim 1, wherein an alignment between the edge of the anatomical structure in the slice and the edge of the anatomical structure of the person is detected by using a sensor of a head mounted display and an edge detection process. (Cvetko, “[0044] Further, in some embodiments, an attachment may be added to the object 118, such as to a handle of the object 118, to enable more accurate tracking of the object 118 by the AR headset 108. For example, the attachment may include any of the enhancements noted above to make the attachment more easily sensed by sensors of the AR headset 108 and, by virtue of the attachment being attached to the object 118, thereby also making the object 118 more easily sensed by sensors of the AR headset 108. Further, the attachment may be designed to attach to the portion of the object 118 that is intended to be inserted into the patient 106, such as the tip of the object 118, so that sensors of the AR headset 108 can actually sense the attachment inside the patient 106. For example, a small magnetic-field-emitting attachment may be attached to a tip of the object 118, and a magnetic sensor of the AR headset I 08 may then be able to sense the exact location of the attachment within the patient I 06, thereby helping to improve the accuracy of the virtual inserted portion 118a displayed to the user 104.”)
Regarding claim 9, Cvetko teaches The method as in claim 4, wherein the axis may be selected from at least one of: sagittal, axial, and coronal axes of a person's anatomy in the 3D image data. (Cvetko, [0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 10, Cvetko teaches A method for aligning 3D image data with an anatomical structure of a person, comprising: (Cvetko, “[0006] In one embodiment, a method for augmenting real-time views of a patient with three-dimensional (3D) data may include various acts. For example, the method may include identifying 3D data for a patient with the 3D data including an outer layer of the patient and multiple inner layers of the patient. The method may also include determining virtual morphometric measurements of the outer layer of the patient from the 3D data. The method may further include registering a real-time position of the outer layer of the patient in a 3D space. The method may also include determining real-time morphometric measurements of the outer layer of the patient. The method may further include automatically registering the position of the outer layer of the patient from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements. The method may also include displaying, in an augmented reality headset, one of the inner layers of the patient from the 3D data projected onto real-time views of the outer layer of the patient.”)
identifying the 3D image data, having a plurality of pre-defined axes, in proximity of the anatomical structure of the person; (Cvetko, “[0041] In some embodiments, the virtual spatial difference box 116 may be generated by the AR headset 108 to confine within a volume of the virtual spatial difference box 116 the projected inner layer of the patient 106 from the 3D data. For example, the projected bones 106b of the patient 106 may be confined within the virtual spatial difference box 116 in FIG. 1. In some embodiments, the virtual spatial difference box 116 may also assist the user when navigating the projected 3D data by providing a frame of reference for the user 104. For example, this frame of reference may assist the user when moving axial slices, coronal slices, sagittal slices, or oblique slices of the 3D data within the virtual spatial difference box 116. Slices may be two-dimensional (2D) slices and/or 3D slices. 3D slices may include curved slices, such as curved slices that follow the natural curve of an anatomical feature, or slices that have a depth as well as a height and width. The user 104 may move these slices using hand gestures that require the user 104 to generally move his hand in the directions of the lines of the virtual spatial difference box 116, so the display of the virtual spatial difference box 116 may make these hand movements easier for the user 104.”)
displaying a slice of the 3D image data orthogonal to an axis identified from the pre-defined axes; (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
locking the slice to stop movement of the slice with respect to the axis; (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”)
receiving adjustments to the slice of the 3D image data that are orthogonal to the axis until an edge of the anatomical structure in the slice aligns with an edge of the anatomical structure of the person as detected from a perspective of an AR headset, in order to align the 3D image data with the person; (Cvetko, “[0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data, [0037] (7) drag the 3D data in a direction of the user 104, such as in the repositioning of a slice of the 3D data, [0038] (8) display different slices of the 3D data including, but not limited to, axial slices, coronal slices, sagittal slices, and oblique slices”)
and applying the adjustments to the 3D image data. (Cvetko, “[0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data”)
Regarding claim 11, Cvetko teaches The method as in claim 10, further comprising repeating the displaying the slice, locking the slice (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”)
and receiving adjustments to the 3D image data in two additional axes. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 12, Cvetko teaches The method as in claim 10, wherein alignment between the edge of the anatomical structure in the slice aligns with the edge of the anatomical structure of the person is detected by using a sensor of the AR headset and an alignment process. (Cvetko, “[0044] Further, in some embodiments, an attachment may be added to the object 118, such as to a handle of the object 118, to enable more accurate tracking of the object 118 by the AR headset 108. For example, the attachment may include any of the enhancements noted above to make the attachment more easily sensed by sensors of the AR headset 108 and, by virtue of the attachment being attached to the object 118, thereby also making the object 118 more easily sensed by sensors of the AR headset 108. Further, the attachment may be designed to attach to the portion of the object 118 that is intended to be inserted into the patient 106, such as the tip of the object 118, so that sensors of the AR headset 108 can actually sense the attachment inside the patient 106. For example, a small magnetic-field-emitting attachment may be attached to a tip of the object 118, and a magnetic sensor of the AR headset I 08 may then be able to sense the exact location of the attachment within the patient I 06, thereby helping to improve the accuracy of the virtual inserted portion 118a displayed to the user 104.”)
Regarding claim 13, Cvetko teaches The method as in claim 10, wherein alignment between the edge of the anatomical structure in the slice with a border of the anatomical structure of the person is determined by receiving input from a user of the AR headset. (Cvetko, “[0041] The user 104 may move these slices using hand gestures that require the user 104 to generally move his hand in the directions of the lines of the virtual spatial difference box 116, so the display of the virtual spatial difference box 116 may make these hand movements easier for the user 104.”)
Regarding claim 14, Cvetko teaches The method as in claim 10, wherein the adjustments to the slice take place in a plane and include translation and rotation adjustments in the plane. (Cvetko, “[0048] As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset.”)
PNG
media_image2.png
597
423
media_image2.png
Greyscale
Regarding claim 15, Cvetko teaches The method as in claim 10, further comprising identifying the slice of the 3D image data that has a probability of matching the anatomical structure of the person. (Cvetko, “[0018] For example, where a surgeon needs to remove a tumor from a patient's brain, the surgeon may view an image of the patient's brain on a computer display. After viewing the location of the tumor on the computer display, the surgeon may then shift his view from the computer display to the actual patient on an operating table and attempt to identify the approximate location on the actual patient of the tumor inside the patient's brain. This method of identifying the approximate location of the tumor can be difficult and error-prone. For example, the surgeon may accidentally identify the left side of the brain in the image as having the tumor when in reality the tumor is in the right side of the brain. This error may lead to the surgeon erroneously making an unnecessary incision on the left side of the patient's skull. / [0062] At block 602, an AR headset may identify 3D data for a patient. For example, the AR headset 108 of FIG. 1 may identify, at block 602, 3D data for the patient 106.”)
Regarding claim 16, Cvetko teaches The method as in claim 10, further comprising matching the slice of the 3D image data to anatomical structure of the person using pattern matching or machine learning. (Cvetko, “[0069] At block 610, an AR headset may automatically register the position of the outer layer of the patient from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space. For example, the AR headset 108 of FIG. 1 may automatically register, at block 610, the position of the outer layer of the patient 106 from the 3D data to align with the registered real-time position of the skin 106a and clothing 107 of the patient 106 in the 3D space 102. In some embodiments, this automatic registration may include automatically aligning the point cloud generated at block 604 with the point cloud generated at block 608 using point set registration. The automatic registration at block 610 may be less time consuming, less cumbersome, and less error-prone than manual methods of aligning an outer layer of a patient from 3D data with a real-time outer layer of the patient. Further, in some embodiments, the block 610 may be performed without using any non-anatomical fiducial, which may avoid the time consuming, cumbersome, and inaccurate placement of non-anatomical fiducials.”)
Regarding claim 17, Cvetko teaches The method as in claim 10, further comprising sending a notification when the slice of the 3D image data matches the edge of the anatomical structure of the person. (Cvetko, “[0071] At block 614, an AR headset may generate a confidence score that the automatic registration is correct and, at block 616, an AR headset may present the confidence score to a user. For example, the AR headset 108 of FIG. 1 may generate, at block 614, a confidence score that the automatic registration that was performed at block 612 is correct and present the confidence score to the user 104 visually, audibly, or in some other presentation format. In some embodiments, the AR headset 108 may generate this confidence score as part of the point set registration performed at block 612. For example, the confidence score may be a score between 0% and 100% indicating the level of confidence that the outer layer of the 3D data matches the outer layer of the patient 106. Where the confidence score is relatively high, the user 104 may proceed with training, research, diagnosis, or treatment on the patient 106 using the AR headset 108 with confidence that the 3D data being projected onto the patient 106 does, in reality, correspond to the patient 106. On the other hand, where the confidence score is relatively low, the user 104 may halt any training, research, diagnosis, or treatment on the patient 106 using the AR headset 108 because there may be legitimate doubts as to whether the 3D data being projected onto the patient 106 does, in reality, correspond to the patient 106.”)
Regarding claim 19, Cvetko teaches The method as in claim 10, further comprising receiving a slice selection from a user. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 20, Cvetko teaches The method as in claim 19, further comprising: enabling navigation through a plurality of slice views that are orthogonal to the axis by the user; and receiving a selection of a slice view by the user. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 21, Cvetko teaches The method as in claim 10, further comprising aligning the 3D image data to the anatomical structure using a silhouette of the anatomical structure of the person in the slice as aligned to a silhouette of the anatomical structure detectable by the AR headset. (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. / [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data, [0037] (7) drag the 3D data in a direction of the user 104, such as in the repositioning of a slice of the 3D data, [0038] (8) display different slices of the 3D data including, but not limited to, axial slices, coronal slices, sagittal slices, and oblique slices, and [0039] (9) perform other advanced features of the AR headset 108.”)
PNG
media_image1.png
684
600
media_image1.png
Greyscale
Regarding claim 22, Cvetko teaches A system for aligning a 3D image data with an anatomical structure of a person, the system comprising: at least one processor; at least one memory device including a data store to store a plurality of data and instructions that, when executed, cause the system to: (Cvetko, “[0054] Generally, the processor 502 may include any suitable special-purpose or general-purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media. For example, the processor 502 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data, or any combination thereof. In some embodiments, the processor 502 may interpret and/or execute program instructions and/or process data stored in the memory 504 and/or the file system 506. In some embodiments, the processor 502 may fetch program instructions from the file system 506 and load the program instructions into the memory 504. After the program instructions are loaded into the memory 504, the processor 502 may execute the program instructions. In some embodiments, the instructions may include the processor 502 performing one or more blocks of the method 600 of FIGS. 6A-6E.”)
identifying the 3D image data, having a plurality of pre-defined axes, in proximity of the anatomical structure of the person; (Cvetko, “[0041] In some embodiments, the virtual spatial difference box 116 may be generated by the AR headset 108 to confine within a volume of the virtual spatial difference box 116 the projected inner layer of the patient 106 from the 3D data. For example, the projected bones 106b of the patient 106 may be confined within the virtual spatial difference box 116 in FIG. 1. In some embodiments, the virtual spatial difference box 116 may also assist the user when navigating the projected 3D data by providing a frame of reference for the user 104. For example, this frame of reference may assist the user when moving axial slices, coronal slices, sagittal slices, or oblique slices of the 3D data within the virtual spatial difference box 116. Slices may be two-dimensional (2D) slices and/or 3D slices. 3D slices may include curved slices, such as curved slices that follow the natural curve of an anatomical feature, or slices that have a depth as well as a height and width. The user 104 may move these slices using hand gestures that require the user 104 to generally move his hand in the directions of the lines of the virtual spatial difference box 116, so the display of the virtual spatial difference box 116 may make these hand movements easier for the user 104.”)
displaying a slice of the 3D image data orthogonal to an axis identified from the pre-defined axes; (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
locking the slice to stop movement of the slice with respect to the axis; (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”)
receiving adjustments to the slice of the 3D image data that are orthogonal to the axis until an edge of the anatomical structure in the slice aligns with an edge of the anatomical structure of the person as detected from a perspective of an AR headset, in order to align the 3D image data with the person; (Cvetko, “[0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data, [0037] (7) drag the 3D data in a direction of the user 104, such as in the repositioning of a slice of the 3D data, [0038] (8) display different slices of the 3D data including, but not limited to, axial slices, coronal slices, sagittal slices, and oblique slices”)
and applying the adjustments to the 3D image data. (Cvetko, “[0034] (4) adjust the alignment of the 3D data with the patient 106, [0035] (5) display the virtual spatial difference box 116, [0036] (6) display a slice of the 3D data instead of a volume of the 3D data”)
Regarding claim 23, Cvetko teaches The system as in claim 22, further comprising repeating the displaying the slice, locking the slice (Cvetko, “[0048] FIGS. 2A-2F are photographs of the AR environment 100 of FIG. 1 with a first patient. As disclosed in FIG. 2A, a user may wear an AR headset that augments a real-time view of the first patient with 3D data of the first patient. Also disclosed in FIG. 2A, the user may employ hand gestures in order to manipulate virtual controls of the AR headset. FIGS. 2B-2F disclose actual views from the perspective of the user wearing an AR headset. As disclosed in FIG. 2B, a user may employ a hand gesture in order to have an axial slice of 3D data representing a CT image of the patient's brain projected onto a real-time view of the first patient. As disclosed in FIG. 2C, a virtual spatial difference box may be employed by the user to view the same axial slice of the 3D data representing the CT image of the first patient's brain. As disclosed in FIG. 2D, as the user wearing the AR headset walks counter-clockwise around the first patient, the same axial slice of the 3D data representing the CT image of the first patient's brain may be viewed from different angles in the 3D space.”)
and receiving adjustments to the 3D image data in two additional axes. (Cvetko, “[0048] As disclosed in FIGS. 2E and 2F, the user may manipulate one or more options of a virtual user interface to change from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a coronal slice, respectively.”)
Regarding claim 24, Cvetko teaches The system as in claim 22, wherein alignment between the edge of the anatomical structure in the slice aligns with the edge of the anatomical structure of the person is detected by using a sensor of the AR headset and an alignment process. (Cvetko, “[0044] Further, in some embodiments, an attachment may be added to the object 118, such as to a handle of the object 118, to enable more accurate tracking of the object 118 by the AR headset 108. For example, the attachment may include any of the enhancements noted above to make the attachment more easily sensed by sensors of the AR headset 108 and, by virtue of the attachment being attached to the object 118, thereby also making the object 118 more easily sensed by sensors of the AR headset 108. Further, the attachment may be designed to attach to the portion of the object 118 that is intended to be inserted into the patient 106, such as the tip of the object 118, so that sensors of the AR headset 108 can actually sense the attachment inside the patient 106. For example, a small magnetic-field-emitting attachment may be attached to a tip of the object 118, and a magnetic sensor of the AR headset I 08 may then be able to sense the exact location of the attachment within the patient I 06, thereby helping to improve the accuracy of the virtual inserted portion 118a displayed to the user 104.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cvetko (Patent No. US 20230038130 A1), in view of Sofka (Patent No. US 20200341095 A1.
Regarding claim 18, Cvetko is silent about The method as in claim 10, further comprising identifying the slice of the 3D image data that includes anatomical structure with a widest diameter of the anatomical structure orthogonal to the axis.
Sofka teaches The method as in claim 10, further comprising identifying the slice of the 3D image data that includes anatomical structure with a widest diameter of the anatomical structure orthogonal to the axis. (Sofka, “[0179] For example, the deep learning techniques developed by the inventors may be used to segment (e.g., identify the outlines of) hemorrhages in MRI images, identify points that specify major axes of a 2D or 3D bounding region (e.g., box), identify a maximum diameter of the hemorrhage and a maximum orthogonal diameter of the hemorrhage that is orthogonal to the maximum diameter, and/or perform any other processing in furtherance of identifying the size of the hemorrhage.”)
Therefore, it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Cvetko’s art by including The method as in claim 10, further comprising identifying the slice of the 3D image data that includes anatomical structure with a widest diameter of the anatomical structure orthogonal to the axis as taught by Sofka and use that with Cvetko’s Augmenting Real-Time Views Of A Patient With Three-Dimensional Data.
The motivation of the combination is to measure and identify the size of the slices and assign the slices properly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to CHAK FUNG A LAM whose telephone number is (571)272-9823. The examiner can
normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using
A USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to
use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
Said Broome can be reached at 5712722931. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center
andhttps://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/C.A.L./Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
