DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 5/6/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Election/Restrictions
Applicant's election with traverse of Group I, claims 1-14 in the reply filed on 4/30/2026 is acknowledged. The traversal is on the ground(s) that limitations reciting establishing wireless communication in group I are linked to receiving surgical/video data in Group II and that the machine learning limitation recited in claim 8 of Group I is linked to the surgical machine learning process of claim 15 of Group II. This is not found persuasive because the machine learning process of claim 8 of Group I occurs locally at the data adaptors prior to data transmission while the machine learning process of claim 15 of Group II occurs at the central hub after data/video has been received. Further, the machine learning limitation of claim 15 of Group II makes phase predictions of a surgical procedure. No claim in Group I recite matter related to phase predictions.
The requirement is still deemed proper and is therefore made FINAL.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “video sources” and “data sources” in claims 11-13.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Regarding claim 11, “video sources” will be interpreted as cameras or similar generic image detectors as described in ¶0056. “Data sources” will be interpreted as medical devices with sensors as described in ¶0056.
Regarding claims 12 and 13, they are interpreted similarly due to their dependencies.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Positive Statement Regarding - 35 USC § 101
The Examiner’s 35 U.S.C. 101 analysis recognizes that the claimed subject matter is directed to a practical application of a technical solution. The claimed elements, establishing wireless communication to transfer data and processing of surgical and video data, which in light of the specification is understood to mean for example synchronizing data or adding metadata, cannot be reasonably interpreted as an abstract idea such as a mental process. Because the claims recite specific, claimed steps and structural elements that produce a tangible technical result, they are not directed to an abstract idea absent additional inventive concept limitations. Accordingly, the record supports a positive 101 determination for the present claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 10, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS).
Regarding claim 1, Grantcharov discloses a computer-implemented method comprising: establishing wireless communication between one or more data adaptors and a central processing hub in a surgical system (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards. Examiner considers the CCS a “central processing hub”), each of the one or more data adaptors configured to provide surgical data associated with a surgical procedure (¶0253, The data capture devices may provide output data feeds to encoders 174, 176, other data capture devices or a patient observation system 178. The medical or surgical data; ¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on); performing localized processing of the surgical data at the one or more data adaptors prior to sending the surgical data to the central processing hub (¶0258, the smart enabled device or adaptor may incorporate and utilize a SOC device (system-on-chip) or FPGA device (Field Programmable Gate Array) in conjunction with on-board storage, power management and wireless radio(s). It may manage device requirements, device-to-device authentication, storage, communications, content processing, clock synchronization, and time stamping); establishing wireless communication between one or more video adaptors and the central processing hub, where at least one of the one or more video adaptors is configured to provide video associated with the surgical data (¶0257, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards); performing localized processing of the video associated with the surgical data at the one or more of the video adaptors prior to sending the video to the central processing hub (¶0258, smart cameras 1604 … the smart enabled device or adaptor may incorporate and utilize a SOC device (system-on-chip) or FPGA device (Field Programmable Gate Array) in conjunction with on-board storage, power management and wireless radio(s). It may manage device requirements, device-to-device authentication, storage, communications, content processing, clock synchronization, and time stamping); and capturing and processing the surgical data and the video at the central processing hub (¶0261, he CMMS 1620 may aggregate, package, compress and encrypt captured audio, video, medical device data, sensor data, logs, and so on … The CMMS 1620 provides processing and data transformation for clock broadcast for device synchronization; ¶0262, The CCS Server 1614 acts as conduit to black box analytics software and databases).
Regarding claim 2, in which claim 1 is incorporated, Grantcharov discloses receiving video and/or data through one or more wired connections at the central processing hub (¶0259, The black box encoder 1610 … may be resident on the client network connected via Ethernet … the black box encoder 1610 may connect directly to the CCS 1614 to provide data. Examiner considers providing data to the CCS to imply that the CCS “receives data”); and receiving one or more wireless communications at the central processing hub from one or more wireless-enabled sources (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards).
Regarding claim 3, in which claim 1 is incorporated, Grantcharov discloses wherein wireless communication between the one or more data adaptors and the central processing hub passes through a hub adaptor configured to synchronize the surgical data from the one or more data adaptors (¶0260, The Media Management Hub (MMH) 1612 may be a computing machine or server responsible for running the client media management software; ¶0261, The CMMS 1620 provides processing and data transformation for clock broadcast for device synchronization. Examiner considers the MMH as the “hub adaptor”).
Regarding claim 4, in which claim 3 is incorporated, Grantcharov discloses wherein wireless communication between the one or more video adaptors and the central processing hub passes through the hub adaptor (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … the black-box recording device 1600 may be provided by an encoder 1610 that connects via a wireless station 1616 to a media management hub (MMH) 1612 … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards), and hub adaptor is configured to synchronize the surgical data from the one or more data adaptors with the video from the one or more video adaptors (¶0164, all video feeds and audio feeds may be recorded and synchronized for an entire medical procedure; ¶0260, The Media Management Hub (MMH) 1612 may be a computing machine or server responsible for running the client media management software; ¶0261, The CMMS 1620 provides processing and data transformation for clock broadcast for device synchronization).
Regarding claim 10, in which claim 3 is incorporated, Grantcharov discloses a system comprising: a central processing hub (¶0257, This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards. Examiner considers the CCS a “central processing hub”); one or more data adaptors configured to perform localized processing of surgical data associated with a surgical procedure and provide the surgical data to the central processing hub through wireless communication (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards; ¶0258, the smart enabled device or adaptor may incorporate and utilize a SOC device (system-on-chip) or FPGA device (Field Programmable Gate Array) in conjunction with on-board storage, power management and wireless radio(s). It may manage device requirements, device-to-device authentication, storage, communications, content processing, clock synchronization, and time stamping); and one or more video adaptors configured to perform localized processing of video associated with the surgical data and provide the video to the central processing hub through wireless communication (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards; ¶0258, the smart enabled device or adaptor may incorporate and utilize a SOC device (system-on-chip) or FPGA device (Field Programmable Gate Array) in conjunction with on-board storage, power management and wireless radio(s). It may manage device requirements, device-to-device authentication, storage, communications, content processing, clock synchronization, and time stamping).
Regarding claim 11, in which claim 10 is incorporated, Grantcharov discloses one or more video sources and one or more data sources coupled to the central processing hub through wired connections (¶0238, two wall-mounted high-definition wide-angled cameras; two omnidirectional microphones; a laparoscopic camera view; and a vital signs display. These are example data capture devices of a hardware unit. This example application may use an Internet Protocol (“IP”) network in which each data signal may be fed into an Ethernet switch (“ES”). Examiner considers the Ethernet switch a wired connection); and one or more wireless-enabled sources coupled to the central processing hub (¶0257, client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards; ¶0258, the smart enabled devices and adaptors may connect directly to the CCS 1614 to provide data from the operating site via secure client network infrastructure 1618).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5, 6, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS) in view of Essinger et al. (US 2010/0177660) (hereafter, “Essinger”)
Regarding claim 5, in which claim 4 is incorporated, Grantcharov discloses configuring one of the one or more data adaptors or one of the one or more video adaptors (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on) [to operate as] the hub adaptor (¶0260, The Media Management Hub (MMH) 1612 may be a computing machine or server responsible for running the client media management software; ¶0261, The CMMS 1620 provides processing and data transformation for clock broadcast for device synchronization. Examiner considers the MMH as the “hub adaptor”).
However, Grantcharov fails to explicitly disclose interchanging the data or video adaptor with the hub adaptor.
Essinger teaches interchanging the data or video adaptor with the hub adaptor (¶0201, the router device of the present invention may serve multiple functions and dynamically switch and reconfigure into a coordinator device. Essinger teaches interchanging devices on a network, which when combined with Grantcharov’s surgical adaptors and hubs, would yield an adaptor operating as a hub).
Both Grantcharov and Essinger are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Essinger is in the field of networked devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the interchangeable devices of Essinger into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to protect the network against device failure, as suggested by Essinger at ¶0201, these is no need to wait for a network user to find a failed network coordinator and replace it.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Essinger.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Essinger to obtain the invention as specified in claim 5.
Regarding claim 6, in which claim 1 is incorporated, Grantcharov discloses establishing wireless communication between one or more [display adaptors] and the central processing hub (¶0257, Example platform components to provide this capability include autonomous and semi-autonomous smart-enabled devices and adaptors such as medical devices 1602, cameras 1604, microphones 1606, sensors 1608 and so on. … This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards. Examiner considers the CCS a “central processing hub”); and transmitting display data and/or video through one or more [wireless connections to the one or more display adaptors] for output on one or more display devices coupled to the one or more display adaptors (¶0251, These feeds may be displayed on a single interface (e.g., control interface 14) providing a comprehensive overview of the operation).
However, Grantcharov fails to explicitly disclose display adaptors and wireless connections to the one or more display adaptors.
Essinger teaches display adaptors and wireless connections to the one or more display adaptors (Fig. 5A, 5B; ¶0133, As shown in FIG. 5A, the wireless electronic-based display device; ¶0259, display electronic-ink display device in the wireless network. Examiner considers the processing circuitry illustrated in Fig. 5A and 5B to be the “display adaptor”).
Both Grantcharov and Essinger are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Essinger is in the field of networked devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the interchangeable devices of Essinger into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to protect the network against device failure, as suggested by Essinger at ¶0201, these is no need to wait for a network user to find a failed network coordinator and replace it.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Essinger.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Essinger to obtain the invention as specified in claim 6.
Regarding claim 14, in which claim 10 is incorporated, Grantcharov discloses [one or more display adaptors configured to establish wireless communication with the central processing hub] and output display data to one or more display devices (¶0251, These feeds may be displayed on a single interface (e.g., control interface 14) providing a comprehensive overview of the operation).
However, Grantcharov fails to explicitly disclose one or more display adaptors configured to establish wireless communication with the central processing hub.
Essinger teaches one or more display adaptors configured to establish wireless communication with the central processing hub (Fig. 5A, 5B; ¶0133, As shown in FIG. 5A, the wireless electronic-based display device; ¶0259, display electronic-ink display device in the wireless network. Examiner considers the processing circuitry illustrated in Fig. 5A and 5B to be the “display adaptor”).
Both Grantcharov and Essinger are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Essinger is in the field of networked devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the interchangeable devices of Essinger into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to protect the network against device failure, as suggested by Essinger at ¶0201, these is no need to wait for a network user to find a failed network coordinator and replace it.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Essinger.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Essinger to obtain the invention as specified in claim 14.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS) in view of Essinger et al. (US 2010/0177660) (hereafter, “Essinger”) as applied to claim 6 above, and further in view of Torabi (US 2022/0076020).
Regarding claim 7, Grantcharov in view of Essinger discloses the computer-implemented method of claim 6.
However, neither Grantcharov nor Essinger, whether considered individually or in combination, fail to explicitly disclose identifying, by one of the display adaptors, a user of one of the display devices; and customizing an aspect of a user interface or information presented in the user interface to display on one of the display devices based on identifying the user
Torabi teaches identifying, by one of the display adaptors, a user of one of the display devices (¶0081, Upon detecting the second user entering the user seating area, process 900 subsequently generates a dialog window on the monitor of the surgeon console asking the second user to confirm whether he/she is the same user as the first user); and customizing an aspect of a user interface or information presented in the user interface to display on one of the display devices based on identifying the user (¶0082, process 900 can simply generate another dialog window prompting the user to enter his/her identify, which causes robotic surgical system 100 to load new surgeon-console-settings, such as user gaze-tracking settings, user UID-control settings, and user seat settings for the second user. Examiner considers surgeon-console-settings to be “user interface” information).
Grantcharov, Essinger, and Torabi are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks, Essinger is in the field of networked devices, and Torabi is in the field of networked surgical devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the user detection and display of Torabi into the interchangeable devices of Essinger and the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to ensure proper user calibration, as suggested by Torabi at ¶0004, When this user-swap event occurs, the robotic surgical system needs to ensure that the eye tracking settings are recalibrated or reconfigured for the new surgeon.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Essinger and Torabi.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Essinger and Torabi to obtain the invention as specified in claim 7.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS) in view of Sherkat et al. (US 2021/0125722) (hereafter, “Sherkat”).
Regarding claim 8, Grantcharov discloses the computer-implemented method of claim 1.
However, Grantcharov fails to explicitly disclose wherein performing localized processing of the surgical data comprises performing at least a portion of a surgical machine learning process on at least one of the data adaptors
Sherkat teaches wherein performing localized processing of the surgical data comprises performing at least a portion of a surgical machine learning process on at least one of the data adaptors (Fig. 4A; ¶0070, Example internal blocks of the edge ML systems 130 and 150 are shown in FIG. 4A. A quad-core ARM processor 410 (four A53) is an example of a CPU inside the edge system 130, 150. A neural processing unit (NPU) 420 is a hardware accelerator to perform, for example, Tensor operations required for neural network models. The NPU 420 can be obtained from ARM or Samsung, for example. The NPU 420 can perform the forward path (inference) of neural network models).
Both Grantcharov and Sherkat are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Sherkat is in the field of networked medical devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the local machine learning of Sherkat into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to improve performance, as suggested by Sherkat at ¶0070, at a much higher speed and can achieve lower inference time.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Sherkat.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Sherkat to obtain the invention as specified in claim 8.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS) in view of Sherkat et al. (US 2021/0125722) (hereafter, “Sherkat”) further in view of Thienphrapa et al. (US 2022/0358773) (hereafter, “Thienphrapa”).
Regarding claim 9, Grantcharov discloses the computer-implemented method of claim 1.
However, Grantcharov fails to explicitly disclose wherein performing localized processing of the video associated with the surgical data comprises modifying one or more aspects of the video based on performing at least a portion of a surgical machine learning process on at least one of the video adaptors.
Sherkat teaches wherein [performing localized processing of the video associated with the surgical data comprises modifying one or more aspects of the video based on] performing at least a portion of a surgical machine learning process on at least one of the video adaptors (Fig. 4A; ¶0070, Example internal blocks of the edge ML systems 130 and 150 are shown in FIG. 4A. A quad-core ARM processor 410 (four A53) is an example of a CPU inside the edge system 130, 150. A neural processing unit (NPU) 420 is a hardware accelerator to perform, for example, Tensor operations required for neural network models. The NPU 420 can be obtained from ARM or Samsung, for example. The NPU 420 can perform the forward path (inference) of neural network models).
Both Grantcharov and Sherkat are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Sherkat is in the field of networked medical devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the local machine learning of Sherkat into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to improve performance, as suggested by Sherkat at ¶0070, at a much higher speed and can achieve lower inference time.
However, neither Grantcharov nor Sherkat, whether considered individually or in combination, explicitly disclose performing localized processing of the video associated with the surgical data comprises modifying one or more aspects of the video based on a machine learning process.
Thienphrapa teaches performing localized processing of the video associated with the surgical data comprises modifying one or more aspects of the video based on a machine learning process (¶0040, machine learning is applied to the interventional imagery with the annotation overlaid, and an optimized placement of the annotation is identified based on the machine learning).
Grantcharov, Sherkat, and Thienphrapa are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks, Sherkat is in the field of networked medical devices, and Thienphrapa is applied towards using machine learning in medical settings. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the machine learning based overlay of Thienphrapa into the local machine learning of Sherkat and the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to inform medical professionals during a procedure, as suggested by Thienphrapa at ¶0028, informative annotations to assist personnel involved in medical interventions.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Sherkat and Thienphrapa.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Sherkat and Thienphrapa to obtain the invention as specified in claim 9.
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov et al. (US 2018/0122506) (hereafter, “Grantcharov”) (IDS) in view of Shelton et al. (US 2022/0108783) (hereafter, “Shelton”).
Regarding claim 12, in which claim 11 is incorporated, Grantcharov discloses a hub adaptor coupled by a wireless connection to the central processing hub (¶0257, a media management hub (MMH) 1612 storing Client Media Management Software instruction code (CMMS) 1620. This connects to a Central Content Server and management software (CCS) 1614 via client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards. Examiner considers the MMH as a “hub adaptor” and the CCS as a “central processing hub”) and through two or more wireless connections to the one or more data adaptors and the one or more one or more video adaptors (¶0257, client network infrastructure 1618 configured for adoption and utilization of high performance wireless communication standards; ¶0258, The smart enabled devices and adaptors may autonomous or semi-autonomous intelligent devices including but not limited to smart cameras 1604, microphones 1606, data and media converters 1612, encoders 1610, adaptors and sensors 1608 … the smart enabled devices and adaptors may connect directly to the CCS 1614 to provide data from the operating site via secure client network infrastructure 1618 and may receive data, commands, and configuration controls from CCS 1624 directly or via MMH 1612. Examiner considers parts 1604-1612 as the “two or more” data/video adaptors and receiving data/commands/configurations from the MMH to indicate being connected), [wherein the hub adaptor is configured to pair with the central processing hub, the one or more data adaptors, and the one or more one or more video adaptors using a manual mode, a semi-automatic mode, or an automatic mode].
However, Grantcharov fails to explicitly disclose wherein the hub adaptor is configured to pair with the central processing hub, the one or more data adaptors, and the one or more one or more video adaptors using a manual mode, a semi-automatic mode, or an automatic mode.
Shelton teaches wherein the hub adaptor is configured to pair with the central processing hub, the one or more data adaptors, and the one or more one or more video adaptors using a manual mode, a semi-automatic mode, or an automatic mode (¶0165, At 16104, one or more hub connectivity control parameters may be identified. At 16108, a hub connectivity mode may be determined based on the identified hub connectivity control parameter(s). For example, the surgical hub 7006 shown in FIG. 11 may determine the hub connectivity mode based on a hub connectivity control parameter. The hub connectivity mode may be selected from multiple connectivity modes that may be preconfigured, dynamically updated, semi-dynamically updated, periodically updated, or preset. Examiner considers dynamically updated connectivity modes to be “automatic”).
Both Grantcharov and Shelton are analogous to the claimed invention because Grantcharov is in the field of wireless medical device networks and Sherkat is in the field of networked surgical devices. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the local machine learning of Shelton into the medical device network of Grantcharov. The suggestion/motivation for doing so would have been to select compatible devices, as suggested by Shelton at ¶0166, if a surgical instrument lacks the hardware capability to provide indications of instructional information, the surgical hub may switch to a connectivity mode that may disable providing instructional information to the surgical instrument.
This method of improving Grantcharov was within the ordinary ability of one of ordinary skill in the art based on the teachings of Shelton.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Grantcharov with the teachings of Shelton to obtain the invention as specified in claim 12.
Regarding claim 13, in which claim 12 is incorporated, Grantcharov discloses wherein the hub adaptor is configured to perform preprocessing of the surgical data and the video prior to sending the surgical data and the video to the central processing hub (¶0260, The Media Management Hub (MMH) 1612 may be a computing machine or server responsible for running the client media management software; ¶0261, The CMMS 1620 provides processing and data transformation for clock broadcast for device synchronization).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mahadik et al. (US 2021/0196406) discloses a surgical hub with data and video inputs and a touch panel display (Fig. 1; ¶0051, an operating room (OR) hub coupled to one or more medical devices in the operating room and providing an operations user interface).
Wiener et al. (US 2019/0207911) discloses surgical hubs that communicate data to a central cloud server (¶0311, data can be communicated from the surgical hubs to the cloud-based system).
Parihar et al. (US 2019/0125457) discloses wireless communication between modules of a hub (¶0623, facilitate a wireless interactive communication between the modules housed in the hub).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMAO DING whose telephone number is (571)272-7237. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAOMAO DING/Examiner, Art Unit 2676
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights