Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims 1, 14 and 22 in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 5-10, 13-22, 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang (US 20220079675) in view of Kohli (US 20200138402).
Regarding claim 1 Lang teaches a head mounted display (HMD) device ([0004] Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display) comprising:
imaging means generating pixel data representative of a scene in use ([0056]);
display means outputting a graphical user interface in use ([0056]);
power means ([0437] battery power in the HMD), data storage means ([0649] ) storing geometrical datasets and data processing means operably interfaced with the imaging means and the display means, wherein the data processing means ([0005]) comprises-
at least one parallel processing module ([0154] Exemplary optical head mounted displays include…containing the CPU and GPU indicative of parallel processing) configured with a plurality of pixel- respective data processing threads, adapted to-
filter pixel data with a predetermined value to segregate first pixel data from second pixel data ([0228] In some embodiments, virtual data of a patient can be superimposed onto live data seen through the optical head mounted display. The virtual data can be raw data in unprocessed form, e.g. preoperative images of a patient, or they can be processed data, e.g. filtered data or segmented data, i.e. segregate first pixel data from second pixel data also see [0229] When images of the patient are superimposed onto live data seen through the optical head mounted display), wherein the first pixel data is representative of an optical contrasting agent applied to at least one target in the scene ([0461] In some embodiments, pre-operative imaging is performed to acquire 3D data of the patient. The pre-operative imaging can, for example, entail ultrasound, CT or MRI, any of the foregoing, optionally with administration of a contrast agent), and
Lang is silent on compute two-dimensional (2D) pixel coordinate data from segregated first pixel data; and
wherein the data processing means is further adapted to- transform the computed 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the device, into three-dimensional (3D) coordinate data representative of the or each target relative to the device, generate 3D guidance data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective target in the scene , and output the 3D guidance data to the graphical user interface.
However, Kohli teaches compute two-dimensional (2D) pixel coordinate data from segregated first pixel data; and
wherein the data processing means is further adapted to- transform the computed 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the device, into three-dimensional (3D) coordinate data representative of the or each target relative to the device, generate 3D guidance data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective target in the scene ([0024] [0059] [0060]), and output the 3D guidance data to the graphical user interface ([0036] also see fig. 1A).
Therefore, it would have been obvious to one of the ordinary skilled in the art to combine Land in light of Kohli teaching so that it may include on compute two-dimensional (2D) pixel coordinate data from segregated first pixel data; and
wherein the data processing means is further adapted to- transform the computed 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the device, into three-dimensional (3D) coordinate data representative of the or each target relative to the device, generate 3D guidance data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective target in the scene , and output the 3D guidance data to the graphical user interface.
The motivation is to provide aid a healthcare provider to guide a medical device in a patient or to provide user viewing an object with additional information such that it can be used for treatment of tumors, fibroids or cysts, with bipolar radiofrequency medical device ablation, multiple microwave medical devices, electroporation, and/or electrochemotherapy systems, etc.
Regarding claim 3 Lang teaches wherein an origin of the at least one coordinate system originating at the device is selected from an aperture of the imaging means ([0217]), a display unit of the display means and one of the HMD wearer's eyes ([0051]).
Regarding claim 5 Lang teaches wherein the data processing means is further adapted to transform the computed 2D pixel coordinate data by solving for rotation and translation ([0016]) based on the 2D pixel coordinate data ([0215]).
Regarding claim 6 Lang in view of Kohli teaches wherein at least one target in the scene is a tool in use by or proximate the HMD wearer, and at least one amongst the one or more further geometrical datasets comprises a three-dimensional model representative of the tool (Kohli: fig. 1B,);
and/or
wherein at least one target in the scene is a marker defining a location in the scene, and at least one amongst the one or more further geometrical datasets comprises a three-dimensional model representative of the marker.
Regarding claim 7 Lang in view of Kohli teaches wherein the scene comprises at least two targets and wherein the data processing means is further programmed to generate display data representative of a pathway between the two targets in the scene when generating the 3D guidance display data (Kohli: fig. 1C [0068]).
Regarding claim 8 Lang teaches further comprising a switchable source of illumination operably connected to the power means for supply, configured to excite the optical contrasting agent in the scene ([0215] [0440] [0228]).
Regarding claim 9 Lang in view of Kohli teaches wherein the data processing means further comprises a graphical processing unit (‘GPU’) programmed to generate 3D guidance display data according to the 3D guidance data ([0154] Exemplary optical head mounted displays include…containing the CPU and GPU), by reference to the one or more further geometrical datasets (fig. 4C); and wherein the data processing means is further adapted to output the 3D guidance display data to the graphical user interface (Kohli :[0036] also see fig. 1A).
Regarding claim 10 Lang teaches wherein the data processing means is further adapted to determine a mismatch between the generated 3D guidance display data and the HMD wearer eye based on a distance measurement and a position of the wearer's eye, and adjust a position of the generated guidance display data in the graphical user interface according to the determined mismatch; and optionally wherein the distance measurement is performed based on stereoscopic image data or performed with an optional distance sensor of the HMD device ([0784] [0785]).
Regarding claim 13 Lang teaches wherein the parallel processing module is selected from the group comprising field programmable gate arrays ('FPGA'), graphical processing units ('GPU'), video processing units ('VPU'), application specific integrated circuits ('ASIC'), image signal processor ('ISP'), digital signal processors ('DSP'); alternatively wherein the data processing means is selected from the group comprising hybrid programmable parallel-central processing units and configurable processors ([0154] Exemplary optical head mounted displays include…containing the CPU and GPU indicative of parallel processing).
Regarding claim 14 Lang teaches an image-based guidance system ([0004] Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display), comprising:
at least one detectable target ([0114]), one or more portions of which is configured with an optical contrasting agent ([0461] In some embodiments, pre-operative imaging is performed to acquire 3D data of the patient. The pre-operative imaging can, for example, entail ultrasound, CT or MRI, any of the foregoing, optionally with administration of a contrast agent);
The other limitations are significantly similar to the limitations of claim 1 so rejected same way.
Regarding claim 15 Lang teaches wherein the optical contrasting agent is an active agent emitting a light wave ([0459]).
Regarding claim 16 Lang teaches wherein the optical contrasting agent is a passive agent, the system further comprising a source of illumination configured to excite the optical contrasting agent ([0459] [0204]).
Regarding claim 17 Lang teaches wherein the HMD device comprises the source of illumination ([0407] [0215] [0440] [0228]).
Regarding claim 18 Lang teaches wherein each of the one or more portions of the detectable target is a marker having a predetermined, relative geometric relationship therewith ([0167]).
Regarding claim 19 the limitations are similar to claim 6 so rejected same way.
Regarding claim 20 Lang teaches wherein the marker is a matrix barcode, one or more portions of which is configured with the optical contrasting agent ([0421] [0459] [0204]).
Regarding claim 21 the limitations are similar to claim 7 so rejected same way.
Regarding claim 22 Lang teaches a method of guiding a detectable target with a head mounted display (HMD) device ([0004] Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display), comprising:
the steps of-
wherein the detectable target is in the scene ([0233] Once the data have been superimposed, the surgeon can optionally register the virtual data with the live data. This registration can be as simple as described here, e.g. a visual confirmation from the surgeon that virtual and live data are substantially matching or substantially superimposed. At this time, the surgeon can optionally reference the virtual data and/or the coordinate system of the virtual data in 2, 3 or more dimensions with the live data and/or the coordinate system of the live data. Once the data are registered, the surgeon can move his/her head into any desired position or orientation, for example for viewing the patient's brain or a lesion and adjacent, e.g. sensitive, anatomy from different view angles. The IMU of the OHMD will register the head movement, the direction of the head movement, the new head position and head orientation. The change in location and orientation of the surgeon's head can be simultaneously or, if desired, non-simultaneously applied to the virtual data which can now be superimposed with the resultant new position and orientation in relationship to the live data. In addition, when the surgeon moves his/her head or body further away from the target anatomy, the change in position and the increase in distance from the target anatomy can be measured by the IMU), with at least one parallel processing module of the HMD device ([0154] Exemplary optical head mounted displays include…containing the CPU and GPU indicative of parallel processing),
wherein the module is configured with a plurality of pixel-respective data processing threads, filtering pixel data with a predetermined value to segregate first pixel data from second pixel data ([0228] In some embodiments, virtual data of a patient can be superimposed onto live data seen through the optical head mounted display. The virtual data can be raw data in unprocessed form, e.g. preoperative images of a patient, or they can be processed data, e.g. filtered data or segmented data, i.e. segregate first pixel data from second pixel data also see [0229] When images of the patient are superimposed onto live data seen through the optical head mounted display),
wherein the first pixel data is representative of one or more portions of the detectable target configured with an optical contrasting agent ([0461] In some embodiments, pre-operative imaging is performed to acquire 3D data of the patient. The pre-operative imaging can, for example, entail ultrasound, CT or MRI, any of the foregoing, optionally with administration of a contrast agent),
Lang is silent on generating pixel data of a scene with imaging sensors of the HMD device,
computing two-dimensional (2D) pixel coordinate data from segregated first pixel data; and with at least one further processing unit of the IMD device, transform 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the HMD device,into three- dimensional (3D) coordinate data representative of the one or more portions of the detectable target, and generating 3D guidance display data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective detectable target in the scene ; and outputting the 3D guidance display data to a graphical user interface on at least one display of the IMD device.
However, Kohli teaches generating pixel data of a scene with imaging sensors of the HMD device (fig. 1B, [0094] ),
computing two-dimensional (2D) pixel coordinate data from segregated first pixel data; and with at least one further processing unit of the IMD device, transform 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the HMD device, into three- dimensional (3D) coordinate data representative of the one or more portions of the detectable target, and generating 3D guidance display data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective detectable target in the scene ([0024] [0059] [0060]); and
outputting the 3D guidance display data to a graphical user interface on at least one display of the IMD device ([0036] also see fig. 1A).
Therefore, it would have been obvious to one of the ordinary skilled in the art to combine Land in light of Kohli teaching so that it may include generating pixel data of a scene with imaging sensors of the HMD device,
computing two-dimensional (2D) pixel coordinate data from segregated first pixel data; and with at least one further processing unit of the IMD device, transform 2D pixel coordinate data by reference to a first geometrical dataset representative of at least one coordinate system originating at the HMD device,into three- dimensional (3D) coordinate data representative of the one or more portions of the detectable target, and generating 3D guidance display data according to the 3D coordinate data, by reference to one or more further geometrical datasets, each representative of a respective detectable target in the scene ; and outputting the 3D guidance display data to a graphical user interface on at least one display of the IMD device.
The motivation is to provide aid a healthcare provider to guide a medical device in a patient or to provide user viewing an object with additional information such that it can be used for treatment of tumors, fibroids or cysts, with bipolar radiofrequency medical device ablation, multiple microwave medical devices, electroporation, and/or electrochemotherapy systems, etc.
Regarding claim 24 the limitations are similar to claim 5 so rejected same way.
Claim(s) 2, 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang (US 20220079675) in view of Kohli (US 20200138402) as applied to claim 1 above, and further in view of Fujimaki (US 20170287222).
Regarding claim 2 Lang in view of Kohli teaches herein the data processing means is further adapted to transform the computed 2D pixel coordinate data by reference to the first geometrical dataset ([0024] [0059] [0060]).
But silent on computed 2D pixel coordinate data by triangulating the 2D pixel coordinate.
However, Fujimaki teaches c omputed 2D pixel coordinate data by triangulating the 2D pixel coordinate ([0132] [0118]).
Therefore, it would have been obvious to one of the ordinary skilled in the art to combine Land in light of Fujimaki teaching so that it may include computed 2D pixel coordinate data by triangulating the 2D pixel coordinate.
The motivation is to provide head mounted display capable of displaying an virtual object in a position and at an inclination intended by a user.
Regarding claim 23 the limitations are similar to claim 2 so rejected same way.
Allowable Subject Matter
Claim 4, 11, 12 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20110046483 is related prior art as it teaches some limitations of claim 1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TOWFIQ ELAHI whose telephone number is (571)270-1687. The examiner can normally be reached M-F: 10AM-3PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at (571)272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TOWFIQ ELAHI/Primary Examiner, Art Unit 2625
Prosecution Insights