Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 19 describes “At least one computer readable storage medium”. Applicant has failed to define or limit the claimed “computer readable storage medium” to a non-transitory statutory medium. Therefore, it would be reasonable to interpret the claimed “computer readable storage medium” to comprise a signal or a carrier wave; neither of which falls into one of the four statutory categories invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 13-15 and, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169).
Regarding claim 1, Stawiaski et al. (hereinafter Stawiaski) discloses an apparatus for registering a patient with a medical device (Stawiaski, [0032], “a surgical system 10 is illustrated for performing surgery on a patient”. In addition, in paragraph [0079], “method for calibrating or registering objects, including surgical tools and patient anatomy, for use with computer navigated surgical operations”), the apparatus comprising:
at least one computer hardware processor (Stawiaski, [0038], “central processing unit (CPU) and/or other processors”); and
at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to (Stawiaski, [0038], “Navigation computer 26 has the displays 28, 29, central processing unit (CPU) and/or other processors, memory (not shown), and storage (not shown). The navigation computer 26 is loaded with software as described below”) perform:
obtaining a series of images of a scene, the scene containing the medical device and a tracking device, the series of images including at least the tracking device (Stawiaski, [0011], “capturing one or more first exposure images of the target space containing the physical object and the tracker at a first exposure with a first of at least one camera”. In addition, in paragraph [0017], “the physical object is a surgical tool”);
generating, a visualization of representation of a patient and the medical device (Stawiaski, [0003], “These trackers allow a surgeon to see the position and/or orientation of the surgical tool overlaid on a monitor in conjunction with a preoperative or an intraoperative image of the patient. The preoperative images may be generated by MRI, CT scans, or other well-known medical imaging technologies, prior to beginning the surgical operation”);
Stawiaski does not expressly disclose “accessing relative pose information indicative of a relative spatial relationship between the tracking device and the medical device”;
Magaraggia et al. (hereinafter Magaraggia) discloses accessing relative pose information indicative of a relative spatial relationship between the tracking device and the medical device (Magaraggia, [0039], “tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device…Corresponding markers able to be acquired by the tracking system may be arranged on the robot, in particular, on the instrument arm. In this way, the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”. Determining a relative spatial relationship between a tracking marker and an imaging system in order to register coordinate systems is considered accessing relative pose information indicative of a relative spatial relationship between the tracking device and the medical device);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use the tracking method of Magaraggia which performs registration between the coordinate systems of a robot and an imaging device to determine relative pose information indicative of a relative spatial relationship between tracker and medical tool of Stawiaski. the motivation for doing so would have been enabling accurate tracking, registering and visualizing the device’s position within surgical environment.
Stawiaski as modified by Magaraggia does not expressly disclose “registering a 3D representation of a patient”;
Nagao et al. (hereinafter Nagao) discloses registering a 3D representation of a patient with a medical device using images and relative pose information (Nagao, [0160], “creates a 3D organ from the CT image…aligns a patient during surgery with the CT image…defines the virtual wall on the basis of a coordinate system of the patient…aligns the patient with an arm…obtains a transform matrix from a coordinate system of the arm to the patient coordinate system…a space of the virtual wall in the arm coordinate system using the transform matrix and uses the described space to control the arm”. Creating a 3D organ model, aligning the patient, defining a virtual wall, and mapping it into arm coordinates using a transformation matrix reads on registering a 3D representation of the patient with a medical device using images and relative pose information).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to track the objects in Stawiaski using transformation matrix taught by Nagao in order to register 3D organ of a patient with a medical device. The motivation for doing so would have been improving navigation accuracy.
Stawiaski as modified by Magaraggia and Nagao does not expressly disclose “generating, based on results of the registering, a mixed reality visualization”;
Gibby et al. (hereinafter Gibby) discloses generating, based on registering, a mixed reality visualization of image data of a patient and actual views of the patient (Gibby, [0057], “as disclosed in FIG. 5E, the AR headset may display in real-time, in the AR headset and based on the registering, the view shown in the photograph of FIG. 5E, which is the inner layer of the patient (e.g., the bones of the patient) from the image data (e.g., the CT images of FIGS. 5C-5D) projected onto actual views of the patient”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to display navigation information disclosed in Stawiaski using the concept of mixed reality visualization taught by Gibby. The motivation for doing so would have been providing intuitive navigation by overlaying virtual information onto the real-world view.
Regarding claim 2, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 disclose a 3D representation (Nagao, [0160], “creates a 3D organ from the CT image”).
Stawiaski as modified by Magaraggia, Nagao and Gibby with the same motivation from claim 1 discloses render a representation of the generated mixed reality visualization (Gibby, [0057], “as disclosed in FIG. 5E, the AR headset may display in real-time, in the AR headset and based on the registering, the view shown in the photograph of FIG. 5E, which is the inner layer of the patient (e.g., the bones of the patient) from the image data (e.g., the CT images of FIGS. 5C-5D) projected onto actual views of the patient”).
Regarding claim 3, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses mapping a pose of the 3D representation to a machine coordinate space (Nagao, [0160], “creates a 3D organ from the CT image…aligns a patient during surgery with the CT image…defines the virtual wall on the basis of a coordinate system of the patient…aligns the patient with an arm…obtains a transform matrix from a coordinate system of the arm to the patient coordinate system…a space of the virtual wall in the arm coordinate system using the transform matrix and uses the described space to control the arm”).
Regarding claim 4, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses (a) the tracking device to the medical device, (b) the medical device to the tracking device, or some combination thereof (Magaraggia, [0039], “tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device…Corresponding markers able to be acquired by the tracking system may be arranged on the robot, in particular, on the instrument arm. In this way, the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”).
Regarding claim 5, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses a transformation, a mapping, or some combination thereof (Magaraggia, [0039], “tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device…Corresponding markers able to be acquired by the tracking system may be arranged on the robot, in particular, on the instrument arm. In this way, the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”).
Regarding claim 7, Stawiaski the tracking device being mounted to the medical device (Stawiaski, [0040], “An instrument tracker 48 is coupled to the surgical instrument 22”).
Regarding claim 13, Stawiaski as modified by Magaraggia, Nagao and Gibby with the same motivation from claim 1 discloses discloses access patient data comprising an image, medical data, or some combination thereof (Gibby, [0057], “when image data (e.g., CT images) of the patient is captured using the non-optical imaging modality (e.g., the CT modality)”);
generate an updated mixed reality visualization with a visual representation of the patient data (Gibby, [0080], “displaying the newly-captured image data from the repeated performance(s) of action 406 along with, or overlaid on, the originally-captured image data from the original performance of action 406”).
Regarding claim 14, Stawiaski discloses an image-based optical tracker an image-based optical tracker (Stawiaski, [0047], “the objects to be located/tracked can be identified using an interface in which one of the participants outlines or selects the objects to be tracked on one or more of the displays 28, 29. For instance, images taken by the optical sensors 40, or video camera 41, of the surgical site may be displayed on one or more of the displays 28, 29 (and/or other displays)”).
Regarding claim 15, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses 3D representation of the patient (Nagao, [0160], “creates a 3D organ from the CT image”);
Gibby discloses generate a mixed reality visualization of the representation of the patient, the patient, and a visual indication indicative of an alignment of the patient with the 3D representation (Gibby, [0081], “alignment of image data of the patient 106 with actual views of the patient 106 using an optical code 200 affixed to the patient 106. Further, this alignment may enable a medical professional to view a virtual interior of the patient 106 while looking at the actual patient 106 through the AR headset 108”).
Regarding claim 18, Stawiaski discloses a computerized method for registering a patient with a medical device for treating the patient (Stawiaski, [0038], “The navigation computer 26 can be a personal computer or laptop computer.”. In addition, in paragraph [0079], “method for calibrating or registering objects, including surgical tools and patient anatomy, for use with computer navigated surgical operations”).
The remaining limitations recite in claim 18 are similar in scope to the functions recited in claim 1 and therefore are rejected under the same rationale.
Regarding claim 19, Stawiaski discloses at least one computer readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to perform (Stawiaski, [0038], “Navigation computer 26 has the displays 28, 29, central processing unit (CPU) and/or other processors, memory (not shown), and storage (not shown). The navigation computer 26 is loaded with software as described below”).
The remaining limitations recite in claim 19 are similar in scope to the functions recited in claim 1 and therefore are rejected under the same rationale.
Regarding claim 20, Stawiaski discloses a method of registering a patient with a medical device for treating the patient (Stawiaski, [0079], “method for calibrating or registering objects, including surgical tools and patient anatomy, for use with computer navigated surgical operations”), the method comprising:
Stawiaski as modified by Magaraggia with the same motivation from claim 1 disclose indicative of a relative spatial relationship between the tracking device and the medical device (Magaraggia, [0039], “For acquiring and tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device, (e.g., a tracking system)…the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”. The registration between the arm coordinate system to the tracking coordinate system provides arm pose information that indicates relative spatial relationship between the arm and the tracking device).
The remaining limitations recite in claim 20 are similar in scope to the functions recited in claim 1 and therefore are rejected under the same rationale.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 1, in further view of Maltz (US 2021/0330992).
Regarding claim 6, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses the 3D representation (Nagao, [0160], “creates a 3D organ from the CT image”);
Stawiaski as modified by Magaraggia, Nagao and Gibby with the same motivation from claim 1 discloses generating the mixed reality visualization with the representation at a pose such that the representation is on a couch (Gibby, [0057], “as disclosed in FIG. 5E, the AR headset may display in real-time, in the AR headset and based on the registering, the view shown in the photograph of FIG. 5E, which is the inner layer of the patient (e.g., the bones of the patient) from the image data (e.g., the CT images of FIGS. 5C-5D) projected onto actual views of the patient”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “a radiotherapy device”;
Maltz discloses a radiotherapy device (Maltz, [0003], “a radiotherapy device for treating a subject”);
a couch of the radiotherapy device (Maltz, [0060], “The table 175 may be configured to support the subject during the radiation treatment”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Stawiaski as modified by Magaraggia’s x-ray device using the radiotherapy device of Maltz. The motivation for doing so would have been allowing image guided radiotherapy for improved targeting accuracy.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 1, in further view of Weiss (US 2016/0310219).
Regarding claim 8, Stawiaski as modified by Magaraggia with the same motivation from claim 1 discloses alignment of the tracking device to the medical device (Magaraggia, [0039], “tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device…Corresponding markers able to be acquired by the tracking system may be arranged on the robot, in particular, on the instrument arm. In this way, the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “laser alignment”;
Weiss discloses a laser alignment (Weiss, [0082], “The phantom has markers, i.e. first markers, that need to be aligned with the laser bridge”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use the laser alignment of Weiss to align Stawiaski as modified by Magaraggia’s tracking device to the medical device. The motivation for doing so would have been enabling precise positional alignment.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 1, in further view of Stopp et al. (US 2020/0193622).
Regarding claim 9, Stawiaski as modified by Magaraggia with the same motivation from claim 1 discloses the tracking device and the medical device (Magaraggia, [0039], “tracking the pose of the robot or of the instrument arm respectively, the medical system may have a tracking facility arranged in a predetermined, (e.g., constant or fixed), spatial positional relationship to the imaging device…Corresponding markers able to be acquired by the tracking system may be arranged on the robot, in particular, on the instrument arm. In this way, the registration between the coordinate systems of the robot and the imaging device may then be carried out especially simply and reliably with conventionally known methods”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “a measured distance”;
Stopp et al. (hereinafter Stopp) discloses a measured distance between tracking markers and medical device (Stopp, [0075], “the medical instrument may be provided with features of a medical treatment instrument or surgical instrument, such as a tool tip or blade being provided at a known distance with respect to the tracking markers”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to determine Stawiaski as modified by Magaraggia’s pose information between devices based on the distance information of Stopp. The motivation for doing so would have been improving registration accuracy.
Claims 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 1, in further view of Freeman et al. (US 2019/0282324).
Regarding claim 10, Stawiaski discloses obtaining a second series of images of the scene (Stawiaski, [0070], “capturing one or more images at a first exposure, capturing one or more images at a second, different, exposure”); wherein the second series of images includes the medical device, one or more objects in the scene, the tracking device, or some combination thereof (Stawiaski, [0011], “capturing one or more first exposure images of the target space containing the physical object and the tracker at a first exposure with a first of at least one camera”. In addition, in paragraph [0017], “the physical object is a surgical tool”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “perform tracking of the 3D representation in the scene based on visual stimulus in the scene”;
Freeman et al. (hereinafter Freeman) discloses perform tracking of a 3D representation in a scene based on visual stimulus in the scene (Freeman, [0176], “augmented reality device and used to track a position of and manner in which the virtual three-dimensional object interacts with real physical objects within the mixed reality environment”. In addition, in paragraph [0251], “track movement of physical objects (e.g., the wearer's hand, medical devices, at the rescue scene, or the patient) in captured images and continuously or substantially continuously reposition the images of the virtual objects so that positioning of the virtual objects relative to the physical objects is maintained”);
tracking a pose of a 3D representation in a mixed reality visualization over time based on the visual stimulus (Freeman, [0176], “track a position of and manner in which the virtual three-dimensional object interacts with real physical objects within the mixed reality environment. For example, a position and/or orientation of the virtual three-dimensional object(s) can be manipulated in accordance with certain spatially sensitive rules governing the appearance and movement of the virtual object(s)”. In addition, in paragraph [0251], “track movement of physical objects (e.g., the wearer's hand, medical devices, at the rescue scene, or the patient) in captured images and continuously or substantially continuously reposition the images of the virtual objects so that positioning of the virtual objects relative to the physical objects is maintained”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use Freeman’s tracking position of a 3D object corresponding to a real physical object to render Stawiaski as modified by Magaraggia, Nagao and Gibby mixed reality display. The motivation for doing so would have been enabling accurate alignment of virtual and real objects.
Regarding claim 11, Stawiaski discloses obtaining a third series of images of the scene, wherein the third series of images includes the medical device, the tracking device, or both (Stawiaski, [0070], “capturing one or more images at a first exposure, capturing one or more images at a second, different, exposure”); wherein the second series of images includes the medical device, one or more objects in the scene, the tracking device, or some combination thereof (Stawiaski, [0011], “capturing one or more first exposure images of the target space containing the physical object and the tracker at a first exposure with a first of at least one camera”. In addition, in paragraph [0017], “the physical object is a surgical tool”);
Stawiaski as modified by Magaraggia and Nagao with the same motivation from claim 1 discloses re-register the 3D representation (Nagao, [0160], “obtains a transform matrix from a coordinate system of the arm to the patient coordinate system. Next, the endoscopic surgery system 5000 describes a space of the virtual wall in the arm coordinate system using the transform matrix and uses the described space to control the arm”. In addition, in paragraph [0226], “The endoscopic surgery system 5000 repeatedly performs the series of processes described above”);
re-registering, the 3D representation with the medical device (Nagao, [0160], “obtains a transform matrix from a coordinate system of the arm to the patient coordinate system. Next, the endoscopic surgery system 5000 describes a space of the virtual wall in the arm coordinate system using the transform matrix and uses the described space to control the arm”. In addition, in paragraph [0226], “The endoscopic surgery system 5000 repeatedly performs the series of processes described above”);
Stawiaski as modified by Magaraggia, Nagao, Gibby and Freeman with the same motivation from claim 10 discloses generating an updated mixed reality visualization of 3D representation (Freeman, [0176], “a position and/or orientation of the virtual three-dimensional object(s) can be manipulated in accordance with certain spatially sensitive rules governing the appearance and movement of the virtual object(s)”);
Regarding claim 12, Stawiaski as modified by Magaraggia and Nagao with the same motivation from claim 1 discloses re-register the 3D representation using the tracking device (Nagao, [0160], “creates a 3D organ from the CT image…aligns a patient during surgery with the CT image…obtains a transform matrix from a coordinate system of the arm to the patient coordinate system. Next, the endoscopic surgery system 5000 describes a space of the virtual wall in the arm coordinate system using the transform matrix and uses the described space to control the arm”. In addition in paragraph [0163], “a marker 6001 for measurement of an arm position is attached to an arm distal end (for example, the camera head 5005). A position measurement device 6000 measures a position (and a posture) of the arm distal end on the basis of the marker 6001. Furthermore, a marker 6002 for measurement of a position (and a posture) of the patient is attached to the patient as illustrated in FIG. 5. The position measurement device 6000 measures the patient position on the basis of the marker 6002”. The markers are considered the tracking device. In addition, in paragraph [0226], “The endoscopic surgery system 5000 repeatedly performs the series of processes described above”);
Stawiaski as modified by Magaraggia, Nagao, Gibby and Freeman with the same motivation from claim 10 discloses receive a voice command, perform activity based on the voice command (Freeman, [0203], “The device 120 can be configured to record and process the acute care provider's speech to identify and perform the instructed activity”).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 15, in further view of Sela et al. (US 2016/0000515).
Regarding claim 16, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses 3D representation (Nagao, [0160], “creates a 3D organ from the CT image”);
Stawiaski as modified by Magaraggia, Nagao and Gibby with the same motivation from claim 1 discloses a portion of the patient that is aligned to the representation (Gibby, [0057], “as disclosed in FIG. 5E, the AR headset may display in real-time, in the AR headset and based on the registering, the view shown in the photograph of FIG. 5E, which is the inner layer of the patient (e.g., the bones of the patient) from the image data (e.g., the CT images of FIGS. 5C-5D) projected onto actual views of the patient”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “a threshold”;
Sela et al. (hereinafter Sela) discloses within a threshold (Sela, [0078], ““Q” denotes the coordinates of a threshold metric used to determine if the virtual and real objects of interest are misaligned outside of a given tolerance”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to determine the alignment of Stawiaski as modified by Magaraggia, Nagao and Gibby using the concept of Sela’s threshold value. The motivation for doing so would have been providing a threshold value to define an acceptable tolerance for aligment.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski et al. (US 2020/0015909) in view of Magaraggia et al. (US 2020/0229879) in view of Nagao et al. (US 2019/0365499) in view of Gibby et al. (US 2019/0348169), as applied to claim 15, in further view of Soma (US 2017/0143263).
Regarding claim 17, Stawiaski as modified by Magaraggia, Nagao with the same motivation from claim 1 discloses 3D representation of at least a portion of the patient (Nagao, [0160], “creates a 3D organ from the CT image”);
Stawiaski as modified by Magaraggia, Nagao and Gibby with the same motivation from claim 1 discloses acquiring data of the patient in a current position (Gibby, [0071], “at action 406, capturing image data of the patient”);
Stawiaski as modified by Magaraggia, Nagao and Gibby does not expressly disclose “processing the data to generate the real-time 3D representation of at least a portion of the patient in the current position”;
Soma discloses acquiring data of a user in a current position (Soma, [0071], “The image acquisition unit 112 acquires the capture image G1 from the imaging unit 111 and provides the capture image G1 to the image processor 113 and the image generator 117”);
processing the data to generate a real-time representation of at least a portion of the user in the current position (Soma, [0077], “The image processor 113 detects the detected attachment position being the attachment position of the biosignal detection sensor 151 together with the body shape of the user in the capture image G2. FIG. 8 shows detected attachment positions P2 detected by the image processor 113”);
compare the real-time representation to a representation to determine difference data (Soma, [0078], “The comparator 115 compares the defined attachment positions P1 with the detected attachment positions P2…if the position of one of the biosignal detection sensors 151 is different from an appropriate attachment position, the comparator 115 provides that difference to the image generator 117 as a comparison result”);
generate a visual indication based on the difference data (Soma, [0079], “The marker located at the defined attachment position P1 in the display image H2, at which the difference occurs, is shown as a marker M2. The image generator 117 may warn the user by blinking the marker M2 or by changing a color”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use Soma’s visual indication based on difference data to generate navigation information of Stawiaski as modified by Magaraggia, Nagao and Gibby. The motivation for doing so would have been reducing user error and improving accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE ZHAI whose telephone number is (571)270-3740. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at (571) 272 - 7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE ZHAI/Primary Examiner, Art Unit 2611