DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Federal Republic of Germany on 01/12/2024. It is noted, however, that applicant has not filed a certified copy of the DE10 2024 100 940.3 application as required by 37 CFR 1.55.
Claim Interpretations - 35 USC § 101
The limitations “displaying or not displaying the vital information on the basis of the field of view parameter”, are considered a practical application of providing additional information to a surgeon regarding vital information not in their current field of view to monitor patient critical vitals during a procedure.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: field of view detection unit (camera in conjunction with processor and functions), patient monitoring unit (vital monitoring medical device), display device (external display showing vitals), alarm device (acoustic or vibratory alarm, see paragraphs 0073-0075 of the specification as filed) in claims 1, 3, 4, 6-8, 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 6, 8-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Robaina et al. (US 2019/0011703)(Hereinafter referred to as Robaina).
Regarding claim 1, Robaina teaches a medical system (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]) , comprising:
a patient monitoring unit configured to record a vital sign of a patient (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]);
a portable display, which can be worn on the head of a user and which, when worn, provides the user with an at least substantially unobstructed view of an environment within a field of vision of the user, wherein the portable display is designed to display vital information within the field of vision of the user when worn, which relates to the vital sign (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]); and
a field of view detection unit configured to determine at least one field of view parameter relating to a current field of view of the user (vital item inside or outside field of view) (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]);
wherein the portable display is configured to either display or not display the vital information on the basis of the field of view parameter (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 2, Robaina teaches The medical system according to claim 1, wherein the field of view parameter comprises information relating to the contents of the field of view of the user (vital item inside or outside field of view) (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 3, Robaina teaches The medical system as set forth in claim 2, further including a display device configured to display a vital information image that relates to the vital sign, and where in the portable display is configured not to display the vital information if the field of view parameter indicates that the display device is located within the field of view of the user (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 6, Robaina teaches The medical system as set forth in claim 1, wherein the field of view detection unit is arranged on the portable display (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 8, Robaina teaches the medical system as set forth in claim 1, wherein the field of view detection unit is configured to determine the viewing direction of the user; and wherein the field of view parameter relates to the determined viewing direction (Sensors and/or cameras such as described herein ( e.g., head pose sensors such as accelerometers, gyros, IMU's, and/or cameras 24, 28, 30, 32, 34) may be used to determine the user's head pose, such as the direction the head is positioned ( e.g., straight or tilted with respect to the horizon). By using one or more of such information (e.g., information on eye tracking, facial indicators, and/or head pose) in conjunction with information regarding user environment from outward-facing cameras 34, local processing and data module 70 (and/or remote) processing module 72 from FIG. 2) may be configured to determine a region of interest in the user's peripheral field of view. See paragraph [1565]).
Regarding claim 9, Robaina teaches the medical system as set forth in claim 1, wherein the portable display can be integrated into optical corrective glasses (FIG. 2 illustrates an example of wearable display system 80. The display system 80 includes a display 62, and various mechanical and electronic modules and systems to support the functioning of that display 62. The display 62 may be coupled to a frame 64, which is wearable by a display system user or viewer 60 and which is configured to position the display 62 in front of the eyes of the user 60. The display 62 may be considered eyewear in some embodiments. See paragraph [1464]).
Regarding claim 10, Robaina teaches The medical system as set forth in claim 1, wherein the portable display comprises an alarm device configured to alert the user if at least one of the vital signs is within a critical range (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 11, Robaina teaches The medical system as set forth in claim 1, wherein the portable display is configured to display the vital information independently of the field of view parameter if at least one of the vital signs is within a critical range (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Regarding claim 12, Robaina teaches a method for operating the medical system as set (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]), comprising the steps of:
recording a vital sign of a patient (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]);
displaying the vital information, which relates to the vital sign, within the field of view of the user by means of the portable display when the display is being worn (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]);
determining a field of view parameter relating to a current field of view of the user using the field of view detection unit (vital item inside or outside field of view) (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]); and
displaying or not displaying the vital information on the basis of the field of view parameter (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robaina et al. (US 2019/0011703)(Hereinafter referred to as Robaina) in view of Jarc et al. (US 2016/0314710)(Hereinafter referred to as Jarc).
Regarding claim 4, Robaina teaches the medical system as set forth in claim 1, further including: and wherein the portable display is configured not to display the vital information if the field of view parameter indicates that the display device is located within the field of view of the user (As described herein, the display device can be configured to highlight images of objects in the environment around the user. Such objects may appear in a variety of sensing regions of the user, such as within the user's field of view. In such instances, the object may be in the user's central field of view or in the peripheral field of view. Alternatively, the objects may be within the user's field of regard but outside the user's field of view field of view. For example, in some embodiments, bringing an image of an object that is outside the user's field of view, into the user's field of view may be useful, to alert the user. In the case of surgery, a screen that is not within the surgeon's field of view when focusing attention on the surgical site may include vital signs of the patient. When such vital signs reach a certain level, the surgeon may be alerted by providing an image of the screen on the head mounted display device display. This screen may be imaged by a video camera placed on the head mounted display device. This camera (or cameras) may be a forward directed camera or possibly even a side directed camera or rear directed camera. As another example, the alert may include an icon (e.g., a copy of the image) or an audio to indicate the potential situation to the surgeon. As yet another example, an icon (e.g., a heart or a thermometer) may appear on the display to remind the surgeon to check certain vital signs (e.g., heart rate or temperature) of the patient. A small icon may provide less distraction. The surgeon may then choose either to move his or head to view the actual object or to view the object as an AR image. See paragraph [1583]), but is silent to a medical imaging instrument configured to generate image data and transmit the image data; a display device configured to receive the image data and generate an image for a user;
Jarc teaches an endoscopic video system displayed on an external side surgeon control display and an augmented reality display (The simulation system can display a virtual environment (e.g., ignore endoscope feed and instruments if installed), combined or augmented environment (e.g., endoscope feed with generated graphical visual overlays or virtual environment objects), or entirely visual imaging (e.g., endoscopic) feed. For example, some implementations can display the virtual environment and/or endoscopic video feed from the patient side cart on the surgeon console display screen and on one or more external display screens (e.g., in 2D or 3D). In some examples, virtual or augmented images can be output through display systems (such as using TilePro from Intuitive Surgical, Inc.) on the surgeon console during any training exercise to provide instruction or performance metrics. For example, virtual reality (VR) or other generated images, and/or augmented reality (AR) ghost images overlaid on camera images, can be displayed on system display devices to indicate or highlight system areas of concern or interest, such as patient cart setup joints having incorrect positions highlighted, reachability limits of instruments displayed, and/or internal/external collisions. This can reduce the burden on training assistants to catch mistakes during training procedures. See paragraph [0182]).
Robaina and Jarc teach of presenting virtual information and utilizing augmented reality and Jarc teaches that endoscopic information can be presented on a side medical display device, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Robaina with the side display techniques of Jarc such that the augmented reality system could be integrated into existing medical equipment available to the surgeon.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robaina et al. (US 2019/0011703)(Hereinafter referred to as Robaina) in view of Vilsmeier et al. (US 2021/0113269)(Hereinafter referred to as Vilsmeier).
Regarding claim 5, Robaina teaches the medical system as set forth in claim 1, but is silent to wherein the portable display is configured to display the vital information if the field of view parameter indicates that the patient is within the field of view of the user.
Vilsmeier teaches registering medical data with a patient in the field of view of the user (In a further embodiment, in a radiotherapy patient positioning use case, the augmented reality device displays 2D x-ray images perpendicularly aligned to the actual x-ray path of the radiotherapy positioning system. The x-ray images are visible dependent on the angle of the view direction in comparison to the virtual position of the x-ray image. Further, after aligning the augmented reality device view direction with the actual 2D x-ray path, a DRR rendering of the patient's CT images may be displayed overlaid over and registered to the patient's anatomy. This allows a comparison of the 2D x-ray image with the 3D DRR image and thereby enables a correction of the patient positioning. This correction could also be performed with the augmented reality device, e.g. by detecting gestures moving the virtual 3D DRR image to match better with the 2D x-ray images. See paragraph [0057]).
Robaina and Vilsmeier teach of presenting medical information in augmented reality and Vilsmeier teaches that the medical data can be registered to a patient, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Robaina with the registration techniques of Vlismeier such that the medical professional can visualize previous medical scan data or other medical data registered to the patient.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robaina et al. (US 2019/0011703)(Hereinafter referred to as Robaina) in view of Cheung et al. (“Eye Gaze Tracking with a Web Camera in a Desktop Environment” 2015.)(Hereinafter referred to as Cheung).
Regarding claim 7, Robaina teaches the medical system as set forth in claim 1, but is silent to wherein the field of view detection unit is arranged separately from the portable display.
Cheung teaches utilizing an external camera to perform eye gaze tracking (See abstract).
Robaina and Cheung teach of eye tracking and Cheung teaches that by utilizing an external web camera a lower cost option can be made available, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Robaina with the gaze tracking techniques of Cheung such that the system could integrate more readily available low cost hardware to reduce the overall system cost.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pace et al. (US 2020/0125165)(Hereinafter referred to as Pace), generally describes displaying vitals in the field of view of a medical professional utilizing a near eye display (See paragraph [0002]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights