DETAILED ACTION
This office action is in response to the communication received on 05/23/2025 concerning application no. 19/132,762 filed on 05/23/2025.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 11 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 11 recites “further comprising predicting positions of each of the third plurality of virtual fiducial markers using at least one machine learning model trained to output the positions based on an input comprising the at least one image and the three-dimensional virtual model”. While paragraphs 0018, 0058, 0122, and 0146 mention the use of machine learning, the specification fails to disclose the manner of operation or function of the model such that it performs the outputting. For example, type of machine learning {e.g. supervised machine learning, unsupervised machine learning, self-supervised machine learning, reinforcement learning, semi-supervised learning, etc.} and/or architecture {e.g. layers, weights, feedback, feedforward, kernel, etc.} of the machine learning in sufficient details to inform one of ordinary skill the intended function of the system. The specification does not describe how the data is input or out, how it is analyzed or manipulated, what elements of the data are considered, what weighting considerations are given, or what parameters are being used in the algorithm. In conclusion, the specification provides no written support on what the algorithm is, how it is utilized, or how it functions with the information fed into it. Due to this, the claim contains subject matter that is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor or joint inventor had possession of the claim invention at the time of filing.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 is indefinite for the following reasons:
Recites “in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generating a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what the correction of the second plurality of virtual fiducial markers is given that the preceding claim element establishes that the marking is done such that there are “a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers”.
Applicant is encouraged to provide consistent and clear language.
Recites “a body part”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “body part” is the same as the “physical body part” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Claim 13 is indefinite for the following reasons:
Recites “in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generate a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what the correction of the second plurality of virtual fiducial markers is given that the preceding claim element establishes that the marking is done such that there are “a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers”.
Applicant is encouraged to provide consistent and clear language.
Recites “a body part”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “body part” is the same as the “physical body part” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Claim 20 is indefinite for the following reasons:
Recites “in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generating a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what the correction of the second plurality of virtual fiducial markers is given that the preceding claim element establishes that the marking is done such that there are “a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers”.
Applicant is encouraged to provide consistent and clear language.
Recites “a body part”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “body part” is the same as the “physical body part” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(b) are also rejected because they inherit the indefiniteness of the claims they respectively depend upon.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-6, 8-13, 17-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Porat et al. (PGPUB No. US 2022/0301268) in view of Poltaretskyi et al. (PGPUB No. US 2019/0380792).
Regarding claim 1, Porat teaches a method for placement of a virtual medical overlay on a physical body part using virtual fiducial markers, the method comprising:
retrieving, from memory by an extended reality device, a three-dimensional virtual model of a body part, wherein the three-dimensional virtual model is tagged with a first plurality of virtual fiducial markers each indicative of a respective anatomical landmark on the body part (Paragraph 0032 teaches registration of AR device implemented 3D imaging anatomical models. The spatial content such as internal structures may be registered correctly with respect to the patient's anatomical features can be done automatically or manually. Paragraphs 0059-60 teaches the use of an HMD for AR and modeling. Paragraph 0031 teaches registration of the AR system with the image data that can be ultrasound, MRI, or CT. Paragraph 0052 teaches that the modifiable templates may be used with the patient characteristics such as size, age, anatomical features, etc., and by modified to fit a specific patient upon demand. Paragraph 0060 teaches the use of the template);
capturing, by the extended reality device, at least one image of the body part in a physical world (Paragraphs 0032-34 teaches that the HMD has a camera that captures the patient);
generating a request to mark, using the at least one image, a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers (Paragraph 0052 teaches that the templates are able to be modified for the specific patient and can be provided together with the device template(s) as instructions for adjustment of the template(s) to specific situation. Paragraph 0060 teaches that the template and adjustment can be carried out with the device templates. The markers may be used to designate the templates. The templates are having differing sizes and proportions of the parts);
in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generating a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers (Paragraph 0067 teaches the use of Boolean operations for the correction in the context of CAD for the correction and adjustment of the template to be to the exact patient features as derived from a scanning of the patient and/or imaging model. Paragraphs 0031-34 teach that the verification may be manual or automatic and the registration and adjustment is done accordingly).
While Parot teaches generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers (Paragraphs 0032, 0037, 0060-62, and 0067), Parot is silent regarding is silent regarding a method, generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding image registration and modeling, Poltaretskyi teaches a method, generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers (Paragraph 0280 teaches the alignment and orientation of the virtual and observed tissue. The virtual landmark is placed and the location is used in the registration transformation for the virtual and observed tissue. The alignment achieved between the virtual and observed bone using the virtual landmark can be further adjusted by the user using voice commands, hand gestures, virtual interface buttons, and/or by positioning additional virtual markers at various locations on the bone surface. Paragraphs 0284-86 teaches the registration methods for the anatomy and the use of the camera system and an HMD system. Paragraph 0362 teaches the overlay of the virtual information on the actual object of the patient. Paragraph 0973 teaches that the operator may used anchorage points).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Parot with Poltaretskyi’s teaching of overlay of the multiple markers. This modified method would allow the user to improve surgical outcomes by customizing a surgical plan for each patient (Paragraph 0003 of Poltaretskyi). Furthermore, the modification may help to automate the process to aid the nurse and help reduce errors in the surgery (Paragraph 0547 of Poltaretskyi).
Regarding claim 5, modified Parot teaches the method in claim 1, as discussed above.
Parot further teaches a method, further comprising: generating the three-dimensional virtual model of the body part by performing segmentation on one or more of: X-ray scan, MRI scan, CT scan, an ultrasound scan (Paragraph 0031 teaches registration of the AR system with the image data that can be ultrasound, MRI, or CT).
Regarding claim 6, modified Parot teaches the method in claim 5, as discussed above.
Parot further teaches a method, further comprising: tagging the generated three-dimensional virtual model with the first plurality of virtual fiducial markers (Paragraphs 0059-67 teach the generation of a 3D model according to the template for registration purposes. The patient anatomy model is generated).
Regarding claim 8, modified Parot teaches the method in claim 1, as discussed above.
Parot further teaches a method, further comprising:
placing a virtual fiducial marker on a physical optical code placed on the body part; and updating positions of the third plurality of virtual fiducial markers in response to detecting a change in a position of the virtual fiducial marker, wherein the virtual fiducial marker is a reference point with a trackable position (Paragraph 0032 teaches the registration can be done according to the anatomical features. The registration may be done with respect to markings, patterns and/or codes (e.g., a QR code) on physical device. Paragraphs 0059-67 teach the generation of a 3D model according to the template for registration purposes. The patient anatomy model is generated with respect to the template and the features of the anatomy and the markers).
Regarding claim 9, modified Parot teaches the method in claim 1, as discussed above.
Parot further teaches a method, wherein the extended reality device is worn by a user, and wherein the extended reality device comprises at least one camera and a display configured to generate virtual reality visuals, augmented reality visuals, or mixed reality visuals (Paragraph 0028 teaches AR visualization. Paragraph 0034 teaches the use of an HMD with a camera and the display of the information).
Regarding claim 10, modified Parot teaches the method in claim 9, as discussed above.
Parot further teaches a method, wherein the selection is received by any combination of: (1) tracking a gaze of the user by the at least one camera; (2) tracking a hand or finger position of the user by the at least one camera; (3) tracking a stylus position by the at least one camera; (4) detecting a voice command by the user; and (5) detecting a physical input on the extended reality device (Paragraphs 0069-75 teaches that the gestures can be tracked. Patient movements and position adjustments can be tracked. A a mouse, a keyboard, a touch screen or pad or any suitable input device may be used).
Regarding claim 11, modified Parot teaches the method in claim 1, as discussed above.
Parot further teaches a method, further comprising predicting positions of each of the third plurality of virtual fiducial markers using at least one machine learning model trained to output the positions based on an input comprising the at least one image and the three-dimensional virtual model (Paragraph 0059 teaches the use of AI or deep learning to extract the features, import the registration and apply the morphological adjustments and aligned the model and export the model).
Regarding claim 12, modified Parot teaches the method in claim 1, as discussed above.
Parot further teaches a method, further comprising:
detecting at least one marker outlier based on a comparison between the first plurality of virtual fiducial markers and the second plurality of virtual fiducial markers; and determining a corrected position solely for the at least one marker outlier for inclusion in the third plurality of virtual fiducial markers (Paragraph 0049 teaches the assessment of the deviation and alerting to prevent registration inaccuracies. Paragraph 0067 teaches the use of Boolean operations for the correction in the context of CAD for the correction and adjustment of the template to be to the exact patient features as derived from a scanning of the patient and/or imaging model. Paragraphs 0031-34 teach that the verification may be manual or automatic and the registration and adjustment is done accordingly).
Regarding claim 13, Parot teaches a system for placement of a virtual medical overlay on a physical body part using virtual fiducial markers, comprising:
at least one memory; and
at least one hardware processor of an extended reality device, wherein the at least one hardware processor is coupled with the at least one memory and configured, individually or in combination, to:
retrieve, from the at least one memory, a three-dimensional virtual model of a body part, wherein the three-dimensional virtual model is tagged with a first plurality of virtual fiducial markers each indicative of a respective anatomical landmark on the body part (Paragraph 0032 teaches registration of AR device implemented 3D imaging anatomical models. The spatial content such as internal structures may be registered correctly with respect to the patient's anatomical features can be done automatically or manually. Paragraphs 0059-60 teaches the use of an HMD for AR and modeling. Paragraph 0031 teaches registration of the AR system with the image data that can be ultrasound, MRI, or CT. Paragraph 0052 teaches that the modifiable templates may be used with the patient characteristics such as size, age, anatomical features, etc., and by modified to fit a specific patient upon demand. Paragraph 0060 teaches the use of the template);
capture at least one image of the body part in a physical world (Paragraphs 0032-34 teaches that the HMD has a camera that captures the patient);
generate a request to mark, using the at least one image, a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers (Paragraph 0052 teaches that the templates are able to be modified for the specific patient and can be provided together with the device template(s) as instructions for adjustment of the template(s) to specific situation. Paragraph 0060 teaches that the template and adjustment can be carried out with the device templates. The markers may be used to designate the templates. The templates are having differing sizes and proportions of the parts);
in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generate a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers (Paragraph 0067 teaches the use of Boolean operations for the correction in the context of CAD for the correction and adjustment of the template to be to the exact patient features as derived from a scanning of the patient and/or imaging model. Paragraphs 0031-34 teach that the verification may be manual or automatic and the registration and adjustment is done accordingly).
While Parot teaches generate a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers (Paragraphs 0032, 0037, 0060-62, and 0067), Parot is silent regarding is silent regarding a system, generate a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding image registration and modeling, Poltaretskyi teaches a system, generate a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers (Paragraph 0280 teaches the alignment and orientation of the virtual and observed tissue. The virtual landmark is placed and the location is used in the registration transformation for the virtual and observed tissue. The alignment achieved between the virtual and observed bone using the virtual landmark can be further adjusted by the user using voice commands, hand gestures, virtual interface buttons, and/or by positioning additional virtual markers at various locations on the bone surface. Paragraphs 0284-86 teaches the registration methods for the anatomy and the use of the camera system and an HMD system. Paragraph 0362 teaches the overlay of the virtual information on the actual object of the patient. Paragraph 0973 teaches that the operator may used anchorage points).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Parot with Poltaretskyi’s teaching of overlay of the multiple markers. This modified apparatus would allow the user to improve surgical outcomes by customizing a surgical plan for each patient (Paragraph 0003 of Poltaretskyi). Furthermore, the modification may help to automate the process to aid the nurse and help reduce errors in the surgery (Paragraph 0547 of Poltaretskyi).
Regarding claim 17, modified Parot teaches the system in claim 13, as discussed above.
Parot further teaches a system, wherein the at least one hardware processor is configured to: generate the three-dimensional virtual model of the body part by performing segmentation on one or more of: an X-ray scan, an MRI scan, an ultrasound scan, and/or a CT scan (Paragraph 0031 teaches registration of the AR system with the image data that can be ultrasound, MRI, or CT).
Regarding claim 18, modified Parot teaches the system in claim 17, as discussed above.
Parot further teaches a system, wherein the at least one hardware processor is configured to: tag the generated three-dimensional virtual model with the first plurality of virtual fiducial markers (Paragraphs 0059-67 teach the generation of a 3D model according to the template for registration purposes. The patient anatomy model is generated).
Regarding claim 20, Parot teaches a non-transitory computer readable medium storing thereon computer executable instructions for placement of a virtual medical overlay on a physical body part using virtual fiducial markers, including instructions for:
retrieving, from memory by an extended reality device, a three-dimensional virtual model of a body part, wherein the three-dimensional virtual model is tagged with a first plurality of virtual fiducial markers each indicative of a respective anatomical landmark on the body part (Paragraph 0032 teaches registration of AR device implemented 3D imaging anatomical models. The spatial content such as internal structures may be registered correctly with respect to the patient's anatomical features can be done automatically or manually. Paragraphs 0059-60 teaches the use of an HMD for AR and modeling. Paragraph 0031 teaches registration of the AR system with the image data that can be ultrasound, MRI, or CT. Paragraph 0052 teaches that the modifiable templates may be used with the patient characteristics such as size, age, anatomical features, etc., and by modified to fit a specific patient upon demand. Paragraph 0060 teaches the use of the template);
capturing, by the extended reality device, at least one image of the body part in a physical world (Paragraphs 0032-34 teaches that the HMD has a camera that captures the patient);
generating a request to mark, using the at least one image, a second plurality of virtual fiducial markers corresponding to anatomical landmarks indicated by the first plurality of virtual fiducial markers (Paragraph 0052 teaches that the templates are able to be modified for the specific patient and can be provided together with the device template(s) as instructions for adjustment of the template(s) to specific situation. Paragraph 0060 teaches that the template and adjustment can be carried out with the device templates. The markers may be used to designate the templates. The templates are having differing sizes and proportions of the parts);
in response to receiving a selection of the second plurality of virtual fiducial markers based on the request, generating a third plurality of virtual fiducial markers with corrected positions of the second plurality of virtual fiducial markers (Paragraph 0067 teaches the use of Boolean operations for the correction in the context of CAD for the correction and adjustment of the template to be to the exact patient features as derived from a scanning of the patient and/or imaging model. Paragraphs 0031-34 teach that the verification may be manual or automatic and the registration and adjustment is done accordingly).
While Parot teaches generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers (Paragraphs 0032, 0037, 0060-62, and 0067), Parot is silent regarding is silent regarding a non-transitory computer-readable storage medium, generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding image registration and modeling, Poltaretskyi teaches a non-transitory computer-readable storage medium, generating a virtual overlay of the three-dimensional virtual model on the body part captured by the extended reality device in real-time by performing registration on the first plurality of virtual fiducial markers and the third plurality of virtual fiducial markers (Paragraph 0280 teaches the alignment and orientation of the virtual and observed tissue. The virtual landmark is placed and the location is used in the registration transformation for the virtual and observed tissue. The alignment achieved between the virtual and observed bone using the virtual landmark can be further adjusted by the user using voice commands, hand gestures, virtual interface buttons, and/or by positioning additional virtual markers at various locations on the bone surface. Paragraphs 0284-86 teaches the registration methods for the anatomy and the use of the camera system and an HMD system. Paragraph 0362 teaches the overlay of the virtual information on the actual object of the patient. Paragraph 0973 teaches that the operator may used anchorage points).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Parot with Poltaretskyi’s teaching of overlay of the multiple markers. This modified apparatus would allow the user to improve surgical outcomes by customizing a surgical plan for each patient (Paragraph 0003 of Poltaretskyi). Furthermore, the modification may help to automate the process to aid the nurse and help reduce errors in the surgery (Paragraph 0547 of Poltaretskyi).
Claims 2-4 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Porat et al. (PGPUB No. US 2022/0301268) in view of Poltaretskyi et al. (PGPUB No. US 2019/0380792) further in view of Pavlovskaia et al. (PGPUB No. US 2019/0388123).
Regarding claim 2, modified Porat teaches the method in claim 1, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a method, wherein generating the third plurality of virtual fiducial markers comprises, for each respective marker of the second plurality of virtual fiducial markers:
identifying, within the at least one image, a region associated with the respective marker, wherein the respective marker is represented by coordinates and the region is bounded by a threshold distance from the coordinates (Paragraphs 0031 and 0059 teach the initial alimewnt making. Paragraphs 0053-60 teach the adjustment of regions around the markers to corresponding template markers in proximity. adjusting placement until a required accuracy/coincidence (threshold difference) is reached and using the placements to carry out the third marker placements and templates).
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a method, fitting, within the region, at least one polynomial curve on a visual feature captured in the at least one image; retrieving, from the memory, polynomial constraints associated with an anatomical landmark of the respective marker and a relative position of a corrected marker based on the polynomial constraints; and aligning the respective marker with the corrected marker by comparing the at least one polynomial curve against the polynomial constraints, wherein the aligned respective marker is included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of fitting with polynomial equations with respect to the markers. This modified method would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Regarding claim 3, modified Porat teaches the method in claim 1, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a method, wherein generating the third plurality of virtual fiducial markers comprises: identifying a first anchor marker in the first plurality of virtual fiducial markers; generating a first plurality of vectors originating from the first anchor marker to remaining markers in the first plurality of virtual fiducial markers; identifying a second anchor marker in the second plurality of virtual fiducial markers; generating a second plurality of vectors originating from the second anchor marker to remaining markers in the second plurality of virtual fiducial markers; and repositioning each of the second plurality of virtual fiducial markers such that a difference between the first plurality of vectors and the second plurality of vectors is less than a threshold difference, wherein repositioned markers in the second plurality of virtual fiducial markers are included in the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a method, wherein generating the third plurality of virtual fiducial markers comprises: identifying a first anchor marker in the first plurality of virtual fiducial markers; generating a first plurality of vectors originating from the first anchor marker to remaining markers in the first plurality of virtual fiducial markers; identifying a second anchor marker in the second plurality of virtual fiducial markers; generating a second plurality of vectors originating from the second anchor marker to remaining markers in the second plurality of virtual fiducial markers; and repositioning each of the second plurality of virtual fiducial markers such that a difference between the first plurality of vectors and the second plurality of vectors is less than a threshold difference, wherein repositioned markers in the second plurality of virtual fiducial markers are included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of setting and control with respect to anchor points. This modified method would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Regarding claim 4, modified Porat teaches the method in claim 1, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a method, wherein generating the third plurality of virtual fiducial markers comprises:
generating a virtual mesh of body part using the at least one image and sensors of the extended reality device, wherein the virtual mesh comprises depth information of the body part; and for each respective marker of the second plurality of virtual fiducial markers: determining a terminus point of a virtual ray used to select the respective marker, wherein the terminus point is an intersection of the virtual ray and the virtual mesh; and repositioning the respective marker along a depth axis based on the terminus point, wherein the respective marker repositioned is included in the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a method, wherein generating the third plurality of virtual fiducial markers comprises:
generating a virtual mesh of body part using the at least one image and sensors of the extended reality device, wherein the virtual mesh comprises depth information of the body part; and for each respective marker of the second plurality of virtual fiducial markers: determining a terminus point of a virtual ray used to select the respective marker, wherein the terminus point is an intersection of the virtual ray and the virtual mesh; and repositioning the respective marker along a depth axis based on the terminus point, wherein the respective marker repositioned is included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of the use of a mesh in registration. This modified m would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Regarding claim 14, modified Porat teaches the system in claim 13, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a system, wherein the at least one hardware processor is configured to generate the third plurality of virtual fiducial markers by, for each respective marker of the second plurality of virtual fiducial markers: identifying, within the at least one image, a region associated with the respective marker, wherein the respective marker is represented by coordinates and the region is bounded by a threshold distance from the coordinates (Paragraphs 0031 and 0059 teach the initial alimewnt making. Paragraphs 0053-60 teach the adjustment of regions around the markers to corresponding template markers in proximity. adjusting placement until a required accuracy/coincidence (threshold difference) is reached and using the placements to carry out the third marker placements and templates).
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a system, fitting, within the region, at least one polynomial curve on a visual feature captured in the at least one image; retrieving, from the memory, polynomial constraints associated with an anatomical landmark of the respective marker and a relative position of a corrected marker based on the polynomial constraints; and aligning the respective marker with the corrected marker by comparing the at least one polynomial curve against the polynomial constraints, wherein the aligned respective marker is included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of fitting with polynomial equations with respect to the markers. This modified system would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Regarding claim 15, modified Porat teaches the system in claim 13, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a system, wherein the at least one hardware processor is configured to generate the third plurality of virtual fiducial markers by: identifying a first anchor marker in the first plurality of virtual fiducial markers; generating a first plurality of vectors originating from the first anchor marker to remaining markers in the first plurality of virtual fiducial markers; identifying a second anchor marker in the second plurality of virtual fiducial markers; generating a second plurality of vectors originating from the second anchor marker to remaining markers in the second plurality of virtual fiducial markers; and repositioning each of the second plurality of virtual fiducial markers such that a difference between the first plurality of vectors and the second plurality of vectors is less than a threshold difference, wherein repositioned markers in the second plurality of virtual fiducial markers are included in the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a system, wherein the at least one hardware processor is configured to generate the third plurality of virtual fiducial markers by: identifying a first anchor marker in the first plurality of virtual fiducial markers; generating a first plurality of vectors originating from the first anchor marker to remaining markers in the first plurality of virtual fiducial markers; identifying a second anchor marker in the second plurality of virtual fiducial markers; generating a second plurality of vectors originating from the second anchor marker to remaining markers in the second plurality of virtual fiducial markers; and repositioning each of the second plurality of virtual fiducial markers such that a difference between the first plurality of vectors and the second plurality of vectors is less than a threshold difference, wherein repositioned markers in the second plurality of virtual fiducial markers are included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of setting and control with respect to anchor points. This modified system would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Regarding claim 16, modified Porat teaches the system in claim 13, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a system, wherein the at least one hardware processor is configured to generate the third plurality of virtual fiducial markers by: generating a virtual mesh of body part using the at least one image and sensors of the extended reality device, wherein the virtual mesh comprises depth information of the body part; and for each respective marker of the second plurality of virtual fiducial markers: determining a terminus point of a virtual ray used to select the respective marker, wherein the terminus point is an intersection of the virtual ray and the virtual mesh; and repositioning the respective marker along a depth axis based on the terminus point, wherein the respective marker repositioned is included in the third plurality of virtual fiducial markers.
In an analogous imaging field of endeavor, regarding augmented reality modeling, Pavlovskaia teaches a system, wherein the at least one hardware processor is configured to generate the third plurality of virtual fiducial markers by: generating a virtual mesh of body part using the at least one image and sensors of the extended reality device, wherein the virtual mesh comprises depth information of the body part; and for each respective marker of the second plurality of virtual fiducial markers: determining a terminus point of a virtual ray used to select the respective marker, wherein the terminus point is an intersection of the virtual ray and the virtual mesh; and repositioning the respective marker along a depth axis based on the terminus point, wherein the respective marker repositioned is included in the third plurality of virtual fiducial markers (Paragraph 0580 teaches the registration process maximizes the mesh and the landmarks in the target image with adjustments according to the model and thereby adjusting the location of the reference image coordinates in the image. Paragraph 0625-44 teaches that the polynomial is used for the rendering and defining of the model. The surface is applied to multiple image slices with mesh and control points. The equation considers the matrices of vertex weights on each point and blending of polynomial functions of a set degree. This provides the basis of surface modeling and the position of a point p is with respect to the coordinate space. A Bernstein polynomial is used).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with Pavlovskaia’s teaching of the use of a mesh in registration. This modified system would allow the user to improve the success rate of medical procedures and improve accuracy (Paragraph 0012 of Pavlovskaia). Furthermore, the modification ensures accurate alignment (Paragraph 0011 of Pavlovskaia).
Claims 7 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Porat et al. (PGPUB No. US 2022/0301268) in view of Poltaretskyi et al. (PGPUB No. US 2019/0380792) further in view of V et al. (PGPUB No. US 2024/0189041).
Regarding claim 7, modified Porat teaches the method in claim 1, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a method, wherein the second plurality of virtual fiducial markers comprises at least four markers and at most ten markers.
In an analogous imaging field of endeavor, regarding use of virtual fiducial markers, V teaches a method, wherein the second plurality of virtual fiducial markers comprises at least four markers and at most ten markers (Paragraph 0107 teaches the use of virtual fiducial markers. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with V’s teaching of virtual fiducial markers between 4 and 10. This modified apparatus would allow the user to improve tracking reliability (Paragraph 0004 of V). Furthermore, the modification provides improved means of subject tracking during the acquisition of medical imaging data by a medical imaging system (Paragraph 0006 of V).
Regarding claim 19, modified Porat teaches the system in claim 13, as discussed above.
However, the combination of Porat and Poltaretskyi is silent regarding a system, wherein the second plurality of virtual fiducial markers comprises at least four markers and at most ten markers.
In an analogous imaging field of endeavor, regarding use of virtual fiducial markers, V teaches a system, wherein the second plurality of virtual fiducial markers comprises at least four markers and at most ten markers (Paragraph 0107 teaches the use of virtual fiducial markers. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Porat and Poltaretskyi with V’s teaching of virtual fiducial markers between 4 and 10. This modified apparatus would allow the user to improve tracking reliability (Paragraph 0004 of V). Furthermore, the modification provides improved means of subject tracking during the acquisition of medical imaging data by a medical imaging system (Paragraph 0006 of V).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Cali et al. (PGPUB No. US 2018/0140362): Teaches generating a three dimensional reconstruction of an image stack representing a target area of a patient, and superimposing, by a head-mounted display, a projection of the three dimensional reconstruction onto a field of view of a user.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADIL PARTAP S VIRK whose telephone number is (571)272-8569. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached on 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADIL PARTAP S VIRK/Primary Examiner, Art Unit 3798
Prosecution Insights