Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 02/23/2026 and is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 100-119 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent Claims
Step 1 analysis:
Claims 100 and 110 are drawn to a method (i.e., process), and Claim 115 is drawn to a system, which are all within the four statutory categories. (Step 1 – Yes, the claim falls into one of the statutory categories).
Step 2A analysis – Prong One:
Claim 100 recites:
A method for providing remote guidance from a skilled resource to a local user during a medical procedure on a patient for enhancing the quality of the medical procedure performed by the local user, the method comprising:
transmitting information, representative of a procedure being performed by the local user, to the skilled resource, wherein the local user performs the procedure using a medical device;
permitting the skilled resource to transmit feedback to the local user; and
conveying the feedback to the local user via a communication interface.
The series of steps as recited above, excluding the underlined portions, describes managing personal behavior or relationships or interactions between people including following rules or instructions, and therefore fall within the scope of certain methods of organizing human activity. Fundamentally, the method is that of a person interacting with another person that is an expert and sharing feedback from one person to another through a series of instructions, which encompasses a person interacting with another individual including following rules or instructions. Accordingly, the claim recites an abstract idea of managing interactions between people.
Claim 110 recites:
A method for enhancing a device operator's use of a medical device via feedback from a remote source during a medical procedure, the method comprising:
performing a medical procedure using the medical device;
transmitting a video, via a shared application, of the medical procedure from a video-enabled device of the device operator to a remote device of the remote source, the shared application permitting sharing and viewing of the video; and
permitting the remote source control of an operation or attribute of the medical device via the shared application.
The series of steps as recited above, excluding the underlined portions, describes managing personal behavior or relationships or interactions between people including following rules or instructions, and therefore fall within the scope of certain methods of organizing human activity. Fundamentally, the method is that of a person performing a procedure and allowing another person to interact with the procedure while they watch the procedure being performed, which encompasses a person interacting with another individual including following rules or instructions. Accordingly, the claim recites an abstract idea of managing interactions between people.
Claim 115 recites:
A system for enhancing use of a medical device via feedback from a remote source during a medical procedure, the system comprising:
a medical device for performing a medical procedure;
a shared application software for communicating with the medical device;
a camera-enabled first device configured to run the shared application software and permit visual capture of the medical procedure;
a remote second device configured to run the shared application software and receive images or video of the medical procedure from the first device via the shared application, the second device being configured to permit the remote source to provide feedback based on performance of the medical procedure and to control an operation or attribute of the medical device via the shared application software for enhancing use of the device by a device operator.
The series of steps as recited above, excluding the underlined portions, describes managing personal behavior or relationships or interactions between people including following rules or instructions, and therefore fall within the scope of certain methods of organizing human activity. Fundamentally, the method is that of a person receiving feedback from a remote user including allowing visual capture of the medical procedure, allowing the remote user to provide feedback based on the performance of the procedure and allowing control of an operation of the medical device, which encompasses a person interacting with another individual including following rules or instructions. Accordingly, the claim recites an abstract idea of managing interactions between people.
Step 2A analysis – Prong 2:
This judicial exception is not integrated into a practical application. Specifically, independent claims 100, 110, and 115 recite the following additional elements beyond the abstract idea:
Claim 100 recites the additional element of a communication interface, which is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using generic computer components.
Claim 110 recites the following additional elements: using the medical device, transmitting a video, via a shared application, of the medical procedure from a video-enabled device of the device operator to a remote device. The limitations of using the medical device and using a shared application are recited at a high level of generality and amount to no more than mere instructions to apply the exception using generic computer components. The limitation of transmitting a video of the medical procedure from a video-enabled device of the device operator to a remote device is mere data gathering and output recited at a high level of generality, and thus is insignificant extra-solution activity. See MPEP 2106.05(g) (“whether the limitation is significant”). In addition, all uses of the recited judicial exception require such data gathering and output, and, as such, this limitation does not impose any meaningful limits on the claim. This limitations amount to necessary data gathering and outputting. See MPEP 2106.05.
Claim 115 recites the following additional elements: a medical device for performing a medical procedure, a shared application software for communicating with the medical device, a camera-enabled first device configured to run the shared application software, a remote second device configured to run the shared application software and receive images or video of the medical procedure from the first device via the shared application. The limitations of a medical device for performing a medical procedure, a camera-enabled first device configured to run the shared application software, a remote second device configured to run the shared application software are recited at a high level of generality and amount to no more than mere instructions to apply the exception using generic computer components. The limitation of receiving images or video of the medical procedure from the first device via the shared application is mere data gathering and output recited at a high level of generality, and thus is insignificant extra-solution activity. See MPEP 2106.05(g) (“whether the limitation is significant”). In addition, all uses of the recited judicial exception require such data gathering and output, and, as such, this limitation does not impose any meaningful limits on the claim. This limitations amount to necessary data gathering and outputting. See MPEP 2106.05.
The additional elements do not show an improvement to the functioning of a computer or to any other technology, rather the additional elements perform general computing functions and do not indicate how the particular combination improves any technology or provides a technical solution to a technical problem. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, Claims 100, 110, and 115 are directed to an abstract idea without practical application. (Step 2A – Prong 2: No, the additional elements are not integrated into a practical application).
Step 2B analysis:
As discussed above in “Step 2A analysis – Prong 2”, the identified additional elements in Independent Claims 100, 110, and 115 are equivalent to adding the words “apply it” on a generic computer, and/or generally link the use of the judicial exception to a particular technological environment or field of use. Therefore, the claims as a whole do not amount to significantly more than the judicial exception itself.
Additional elements of transmitting a video of the medical procedure from a video-enabled device of the device operator to a remote device and receiving images or video of the medical procedure from the first device via the shared application were found to be insignificant extra-solution activity in Step 2A, Prong Two, because they were determined to be insignificant limitations as necessary data gathering and outputting. However, a conclusion that an additional element is insignificant extra -solution activity in Step 2A, Prong Two should be re-evaluated in Step 2B. See MPEP 2106.05, subsection I.A. At Step 2B, the evaluation of the insignificant extra-solution activity consideration takes into account whether or not the extra-solution activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Here, the claim limitations are similar to receiving and sending information over a network (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); OJP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); See MPEP 2106.05(d)(ll)(i)).
For the role of a computer in a computer implemented invention to be deemed meaningful in the context of this analysis, it must involve more than performance of “well- understood, routine, [and] conventional activities previously known to the industry.” Further, “the mere recitation of a generic computer cannot transform a patent ineligible abstract idea into a patent-eligible invention.” The applicant’s specification discloses: the interface may be an interface of an application or program or a website that is accessed by both devices (See specification [0046]). The electronic devices 104, 106, as well as any other electronic devices that may be in communication with at least the local electronic device 104 and/or the remote electronic device 106, can include mobile phones (i.e., smart phones), tablets, display screens (such as smart TVs and other smart screens), as well as other display or non-display communication devices that can communicate (See specification [0039]).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as discussed above with respect to integration of the abstract idea into a practical application, using the additional elements to perform the steps for providing feedback amount to no more than using computer related devices to implement the abstract idea.
The use of a computer or processor to merely automate or implement the abstract idea cannot provide significantly more than the abstract idea itself. (See MPEP 2106.05(f) where mere instructions to apply an exception does not render an abstract idea patent eligible). There is no indication that the additional limitations alone or in combination improves the functioning of a computer or any other technology, improves another technology or technical field, or effects a transformation or reduction of a particular article to a different state or thing. Therefore, the claims are not patent eligible. The Examiner has therefore determined that no additional element, or combination of additional claims elements is/are sufficient to ensure the claims amount to significantly more than the abstract idea identified above (Step 2B: Independent claims - NO).
Dependent Claims
Dependent Claims 101-109, 111-114, and 116-119 are directed towards elements used to describe the transmission of information and permittance of feedback. These elements include: (claim 101) wherein the permitting the skilled resource to transmit feedback to the local user comprises permitting the skilled resource to indicate a suggested position or movement of the medical device; (claim 102) wherein the transmitting information comprises sending a video feed to the skilled resource of the local user performing the medical procedure or providing one or more images to the skilled resource; (claim 103) wherein the permitting the skilled resource to transmit feedback comprises overlaying notation over the information transmitted; (claim 104) wherein the conveying the feedback comprises providing visual feedback to the local user using a first communication device or transmitting an audio instruction to the local user; (claim 105) wherein the conveying the feedback comprises providing a graphic overlay of an example routine, target area on the patient, position of the medical device, or movement of the medical device; (claim 106) wherein the skilled resource comprises a skilled user of the medical device or an artificial intelligence software; (claim 107 and 112) permitting the skilled resource to transmit an example routine to the local user for viewing; (claim 108) permitting the skilled resource to transmit feedback comprises permitting control of at least one attribute of the medical device by the skilled resource; (claim 109) permitting the skilled resource to transmit feedback to the local user comprises permitting the skilled resource to mark a virtual target on the patient or annotate the video feed with a symbol, line, or highlight tool; (claim 111) permitting the remote source to transmit feedback to the device operator via a notation overlaid onto the video or a marking of a virtual target on a patient or annotate the video with a symbol, line, or highlight tool; (claim 113) permitting the remote source control comprises permitting the remote source to change a mode of the medical device; (claim 114) wherein the permitting the remote source control comprises, in response to a signal from the remote source, generating haptic feedback via the medical device to allow the remote source to communicate with the medical device operator; (claim 116) the second device being configured to permit the remote source to provide feedback allows the remote source to provide visual feedback using the remote second device; (claim 117) wherein the visual feedback comprises a graphic overlay of an example routine, target area on a patient, position of the medical device, movement of the medical device, or an animation of at least one suggested position or movement of the medical device; (claim 118) wherein the first device or the second device is a smartphone or a tablet; (claim 119) wherein the remote source comprises a skilled user of the medical device or an artificial intelligence software.
The elements of claims 101, 103-109, 11-114, 116-117, and 119 describe managing personal behavior or relationships or interactions between people including following rules or instructions, and therefore fall within the same scope of certain methods of organizing human activity as previously described for the independent claims.
This judicial exception is not integrated into a practical application. Specifically, the dependent
claims recite the following additional elements beyond the abstract idea: (claim 102) wherein the transmitting information comprises sending a video feed to the skilled resource of the local user performing the medical procedure or providing one or more images to the skilled resource; (claim 104) using a first communication device; (claim 106 and 119) the skilled resource/the remote source comprises an artificial intelligence software; (claim 118) wherein the first device or second device is a smartphone or a tablet.
Claim 102 recites limitations that amount to mere data gathering and output recited at a high level of generality, and thus is insignificant extra-solution activity. The limitation of claim 118 amounts to generally linking the use of the abstract idea to a particular technological environment, and therefore, does not meaningfully limit the abstract idea (See MPEP 2106.05(e)). The limitation of claim 104 is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using generic computer components. The limitation of claims 106 and 119 comprising an artificial intelligence software is recited at a high level of generality such that the use of the models to carry out the abstract idea amounts to using a mathematical algorithm to apply the abstract idea, which amounts to mere instructions to apply the exception, as per MPEP 2106.05(f)(2). The limitations do not impose any meaningful limits on practicing the abstract idea, and therefore do not integrate the abstract idea into a practical application (see MPEP 2106.05(f)).
The additional elements do not show an improvement to the functioning of a computer or to
any other technology, rather the additional elements perform general computing functions and do not
indicate how the particular combination improves any technology or provides a technical solution to a
technical problem. Accordingly, these additional elements, when considered separately and as an
ordered combination, do not integrate the abstract idea into a practical application because they do not
impose any meaningful limits on practicing the abstract idea. Therefore, the dependent claims are
directed to an abstract idea without practical application. (Step 2A – Prong 2: No, the additional
elements are not integrated into a practical application).
As discussed above, the identified additional elements in the dependent claims are equivalent to adding the words “apply it” on a generic computer, and/or generally link the use of the judicial exception to a particular technological environment or field of use. Therefore, the claims as a whole do not amount to significantly more than the judicial exception itself.
Additional element of claim 102, wherein the transmitting information comprises sending a video feed to the skilled resource of the local user performing the medical procedure or providing one or more images to the skilled resource was found to be insignificant extra-solution activity in Step 2A, Prong Two, because it were determined to be insignificant limitations as necessary data gathering and outputting. However, a conclusion that an additional element is insignificant extra -solution activity in Step 2A, Prong Two should be re-evaluated in Step 2B. See MPEP 2106.05, subsection I.A. At Step 2B, the evaluation of the insignificant extra-solution activity consideration takes into account whether or not the extra-solution activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Here, the claim limitations are similar to receiving and sending information over a network (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); OJP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); See MPEP 2106.05(d)(ll)(i)).
For the role of a computer in a computer implemented invention to be deemed meaningful in
the context of this analysis, it must involve more than performance of “well-understood, routine, [and]
conventional activities previously known to the industry.” Further, “the mere recitation of a generic
computer cannot transform a patent ineligible abstract idea into a patent-eligible invention.”
The use of a computer or processor to merely automate or implement the abstract idea cannot
provide significantly more than the abstract idea itself. (See MPEP 2106.05(f) where mere instructions to apply an exception does not render an abstract idea patent eligible). There is no indication that the
additional limitations alone or in combination improves the functioning of a computer or any other
technology, improves another technology or technical field, or effects a transformation or reduction of a
particular article to a different state or thing. Therefore, the claims are not patent eligible.
The Examiner has therefore determined that no additional element, or combination of
additional claims elements is/are sufficient to ensure the claims amount to significantly more than the
abstract idea identified above (Step 2B: Dependent claims - NO).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 100-109 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Pelissier et al. (US 2020/0268344) (hereinafter Pelissier).
Regarding Claim 100, Pelissier teaches the following:
A method for providing remote guidance from a skilled resource to a local user during a medical procedure on a patient for enhancing the quality of the medical procedure performed by the local user (Abstract, [0002], [0015], [0023] A method for providing visual feedback from a remote expert to an ultrasound system user to help improve their scanning technique. Methods that include adaptations for providing and receiving remote feedback to an operator performing an ultrasound imaging procedure in real-time.), the method comprising:
transmitting information, representative of a procedure being performed by the local user, to the skilled resource, wherein the local user performs the procedure using a medical device ([0017], [0018], [0023], [0062], and Fig. 1: acquiring ultrasound image data acquisition information at the first location and transmitting the ultrasound image data and ultrasound image data acquisition information to a second location. A local user interface is in communication with the medical imaging device and patient imaging device and is configured to transmit the ultrasound image data and patient image data to a remote interface located remotely. System 100 comprises apparatus 100A at a location where a user performs an ultrasound scan on a patient and apparatus 100B at a remote location. An expert may use apparatus 100B to review information about an ultrasound procedure in real time as the ultrasound procedure is being performed at apparatus 100A and to provide real time feedback to a user of apparatus 100A.);
permitting the skilled resource to transmit feedback to the local user ([0072] remote interface is a general purpose device configured by software to allow a remote expert to review an ultrasound procedure and provide feedback); and
conveying the feedback to the local user via a communication interface ([0063], [0072] The remote expert may provide feedback by interacting with a remote interface. The expert may, for example, use system to provide feedback on how to capture a high quality image. This feedback may include adjustments to the type or amount of acoustic coupling used, gross positioning, fine positioning, or imaging parameters.).
Regarding Claim 101, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the permitting the skilled resource to transmit feedback to the local user comprises permitting the skilled resource to indicate at least one suggested position or movement of the medical device ([0063], [0072] remote interface is a general purpose device configured by software to allow a remote expert to review an ultrasound procedure and provide feedback. The remote expert may provide feedback by interacting with a remote interface. The expert may, for example, use system to provide feedback on how to capture a high quality image. This feedback may include adjustments to the type or amount of acoustic coupling used, gross positioning, fine positioning, or imaging parameters.).
Regarding Claim 102, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the transmitting information comprises sending a video feed to the skilled resource of the local user performing the medical procedure or providing one or more images to the skilled resource (Fig. 2, [0028], [0068], [0071] When remote feedback is desired apparatus 100A (local user) is caused to transmit one or both of the ultrasound imaging data and the video stream through a data communication network to remote apparatus 100B (expert). A camera operable to generate a video stream wherein the data interface is configured to transmit the video stream from the camera to the remote interface. In some embodiments the patient images may comprise a sequence of still images.).
Regarding Claim 103, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the permitting the skilled resource to transmit feedback comprises the skilled resource overlaying notation over the information transmitted to the skilled resource ([0141], [0148], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In some embodiments remote interface is configured to allow the expert to select predetermined annotations from a set of predetermined annotations and to drag and drop the selected annotations at desired locations on a frozen image. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match.).
Regarding Claim 104, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the conveying the feedback comprises providing visual feedback to the local user using a first communication device ([0015], [0072] A method for providing visual feedback from a remote expert to an ultrasound system user using a remote interface) or transmitting an audio instruction to the local user ([0117], [0133] audio messages may be provided. In some embodiments the audio messages may include synthesized or pre-recorded speech messages such as 'scan left'; 'scan right'; 'tilt forward' etc. Such audio messages may optionally be delivered privately to a user, for example by way of an earpiece. In some embodiments the earpiece communicates wirelessly (e.g. by Bluetooth™) with local user interface. In addition to one or more other kinds of feedback, local interface device and remote user interface may optionally be configured to support two-way voice communication between the user of apparatus 100A and the expert at apparatus 100B.).
Regarding Claim 105, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the conveying the feedback comprises providing a graphic overlay ([0073], [0141], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A (local user) sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match.) of an example routine ([0078] In some embodiments remote interface includes a demonstration device which can be positioned like probe and is equipped with a position sensor. The expert may manipulate the demonstration device to demonstrate a particular technique while the position and orientation of the demonstration device is tracked. The tracked position and orientation of the demonstration device may be transmitted to apparatus 100A which may generate an animation showing the user of apparatus 100A how the expert is manipulating the demonstration device.), target area on the patient, position of the medical device, or movement of the medical device ([0025], [0073], [0141], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A (local user) sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match. In some embodiments the icons include one or more of: icons depicting tilting the probe forward and backward; icons depicting rocking the probe right and left; icons depicting rotating the probe clockwise and counterclockwise; and icons depicting moving the probe to the left, right, forward and backward).
Regarding Claim 106, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the skilled resource comprises a skilled user of the medical device or an artificial intelligence software ([0061] An ultrasound system may provide features that allow a person at a remote site (e.g. an expert ultrasound user) to review the actions of a person).
Regarding Claim 107, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, further comprising permitting the skilled resource to transmit an example routine to the local user for viewing ([0078] The remote interface includes a demonstration device which can be positioned like the probe and is equipped with a position sensor. The demonstration device may be a probe or a replica of a probe for example. The expert may manipulate the demonstration device to demonstrate a particular technique while the position and orientation of the demonstration device is tracked. The tracked position and orientation of the demonstration device may be transmitted to apparatus 100A which may generate an animation showing the user of apparatus 100A how the expert is manipulating the demonstration device.).
Regarding Claim 108, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the permitting the skilled resource to transmit feedback comprises permitting control of at least one attribute of the medical device by the skilled resource ([0075] patient imaging device is adjustable (for example has adjustable pan and/or zoom) and remote interface device 118 includes controls that allow the expert to remotely adjust patient imaging device 104 to obtain a clear view of the procedure or a view that can best help the user to see and understand how to best conduct a scan.).
Regarding Claim 109, Pelissier teaches the method of claim 100, and further teaches:
The method of Claim 100, wherein the permitting the skilled resource to transmit feedback to the local user comprises permitting the skilled resource to mark a virtual target on the patient or annotate the video feed with a symbol, line, or highlight tool ([0144], [0148] Some embodiments provide an annotation feature that may be used by an expert to annotate images (e.g. frames from the ultrasound imaging data generated by ultrasound imaging apparatus and/or the video stream from patient imaging device). the expert may use remote interface to highlight a particular area of an ultrasound image or a particular area within the video stream from patient imaging device. Highlighting may be performed on 'live' images or on a selected frozen frame of the ultrasound image data or the video stream.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 110-119 are rejected under 35 U.S.C. 103 as being unpatentable over Pelissier et al. (US 2020/0268344) (hereinafter Pelissier) in view of Starobinets et al. (US 2022/0319721) (Hereinafter Starobinets).
Regarding Claim 110, Pelissier teaches the following:
A method for enhancing a device operator's use of a medical device via feedback from a remote source during a medical procedure (Abstract, [0002], [0015], [0023] A method for providing visual feedback from a remote expert to an ultrasound system user to help improve their scanning technique. Methods that include adaptations for providing and receiving remote feedback to an operator performing an ultrasound imaging procedure in real-time.), the method comprising:
performing a medical procedure using the medical device ([0062] System comprises apparatus 100A at a location where a user performs an ultrasound scan on a patient);
transmitting a video, via a [shared] application ([0176] and Fig. 11: local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality. Fig. 11 illustrates an example embodiment wherein local user interface 102 is provided by a cellular telephone 1202 configured with an app 1203 that includes predetermined messages 1204 in an app database 1205. Remote interface 118 is provided by a cellular telephone 1218 executing an app 1219. App 1219 provides controls that enable the expert to provide predetermined feedback messages to a user as described herein.), of the medical procedure from a video-enabled device of the device operator to a remote device of the remote source, the [shared] application permitting sharing and viewing of the video (Fig. 2, [0028], [0068], [0071], [0176] When remote feedback is desired, apparatus 100A (local user) is caused to transmit one or both of the ultrasound imaging data and the video stream through a data communication network to remote apparatus 100B (expert). A camera operable to generate a video stream wherein the data interface is configured to transmit the video stream from the camera to the remote interface. In some embodiments the patient images may comprise a sequence of still images. Local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality.); and
permitting the remote source control of an operation or attribute of the medical device via the [shared] application ([0075], [0176] patient imaging device is adjustable (for example has adjustable pan and/or zoom) and remote interface device 118 includes controls that allow the expert to remotely adjust patient imaging device 104 to obtain a clear view of the procedure or a view that can best help the user to see and understand how to best conduct a scan. Local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality.).
However, Pelissier does not explicitly disclose the following that is met by Starobinets:
via a shared application ([0034] the natural language communication pathway may be provided via an ROCC (Radiology Operations Command Center) device, such as a mobile device (e.g., a tablet computer or a smartphone). For example, an “app” can run on the ROCC device (operable by the local operator LO) and the remote workstation (operable by the remote expert RE) to allow communication (e.g., audio chats, video chats, and so forth) between the local operator and the remote expert.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to have combined the method including transmitting information between a local device and remote device, as taught by Pelissier, with the transmission occurring via a shared application, as taught by Starobinets, because the claimed invention is only a combination of these well-known elements which would have performed the same function in combination as each did separately. Pelissier already discloses communication between the local user and remote user using apps on each user device. Including the shared app that is run on the local device and on the remote device for communication between the two, as taught by Starobinets, performs the same function of allowing communication between a local user and a remote expert through the use of apps, as taught by Pelissier. Therefore, the results would have been predictable to one of ordinary skill in the art (MPEP 2143).
Regarding Claim 111, the combination of Pelissier and Starobinets teaches the method of claim 110, and Pelissier further teaches:
The method of Claim 110, further comprising permitting the remote source to transmit feedback to the device operator via (i) a notation overlaid onto the video transmitted to the remote source ([0073], [0135], [0141], [0148], [0152], The remote expert may drag and drop the icon onto the display of an ultrasound image stream or video stream at the remote interface. FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. The expert may operate a control that selects a frame of the video stream to freeze. Remote interface is configured to allow the expert to select predetermined annotations from a set of predetermined annotations and to drag and drop the selected annotations at desired locations on a frozen image. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match) or (ii) a marking of a virtual target on a patient or annotate the video with a symbol, line, or highlight tool ([0025], [0073], [0141], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A (local user) sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match. In some embodiments the icons include one or more of: icons depicting tilting the probe forward and backward; icons depicting rocking the probe right and left; icons depicting rotating the probe clockwise and counterclockwise; and icons depicting moving the probe to the left, right, forward and backward).
Regarding Claim 112, the combination of Pelissier and Starobinets teaches the method of claim 110, and Pelissier further teaches:
The method of Claim 110, further comprising permitting the remote source to transmit an example routine to the device operator for viewing ([0078] The remote interface includes a demonstration device which can be positioned like the probe and is equipped with a position sensor. The demonstration device may be a probe or a replica of a probe for example. The expert may manipulate the demonstration device to demonstrate a particular technique while the position and orientation of the demonstration device is tracked. The tracked position and orientation of the demonstration device may be transmitted to apparatus 100A which may generate an animation showing the user of apparatus 100A how the expert is manipulating the demonstration device.).
Regarding Claim 113, the combination of Pelissier and Starobinets teaches the method of claim 110, and Pelissier further teaches:
The method of Claim 110, wherein the permitting the remote source control comprises permitting the remote source to change a mode of the medical device ([0075] patient imaging device is adjustable (for example has adjustable pan and/or zoom) and remote interface device includes controls that allow the expert to remotely adjust patient imaging device to obtain a clear view of the procedure or a view that can best help the user to see and understand how to best conduct a scan. For example, the remote user may adjust pan and/or zoom of patient imaging device).
Regarding Claim 114, the combination of Pelissier and Starobinets teaches the method of claim 110, and Pelissier further teaches:
The method of Claim 110, wherein the permitting the remote source control comprises, in response to a signal from the remote source, generating haptic feedback via the medical device to allow the remote source to communicate with the medical device operator ([0115], [0119] If the monitored motion does not correspond to the message received from remote interface then a warning signal may be provided to the user. The warning signal may, for example, comprise a sound, tactile feedback or the like. For example, probe may be controlled to deliver a vibration or other haptic feedback to a user. Tactile feedback may be provided. In some embodiments the tactile feedback is provided by transducers such as piezoelectric transducers and/or heaters on a housing of the probe that may be selectively actuated to change the texture, temperature and/or vibration patterns sensed by a user holding the probe in a manner that maps intuitively to suggested movements).
Regarding Claim 115, Pelissier teaches the following:
A system for enhancing use of a medical device via feedback from a remote source during a medical procedure (Abstract, [0002], [0015], [0023] A system for providing visual feedback from a remote expert to an ultrasound system user to help improve their scanning technique. Methods that include adaptations for providing and receiving remote feedback to an operator performing an ultrasound imaging procedure in real-time.), the system comprising:
a medical device for performing a medical procedure ([0062] System comprises apparatus 100A at a location where a user performs an ultrasound scan on a patient);
a [shared] application software for communicating with the medical device ([0176] and Fig. 11: local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality. Fig. 11 illustrates an example embodiment wherein local user interface 102 is provided by a cellular telephone 1202 configured with an app 1203 that includes predetermined messages 1204 in an app database 1205. Remote interface 118 is provided by a cellular telephone 1218 executing an app 1219. App 1219 provides controls that enable the expert to provide predetermined feedback messages to a user as described herein. );
a camera-enabled first device configured to run the [shared] application software and permit visual capture of the medical procedure ([0068], [0069], [0176] A patient imaging device captures patient images showing probe and patient P. The patient images may comprise a video stream. In some embodiments the patient images may comprise a sequence of still images. Imaging device may, for example, comprise a suitable video camera, web camera, security camera or the like. where local user interface device comprises a tablet, laptop or cellular telephone equipped with a camera, that camera may be configured to operate as patient imaging device. Patient imaging device transmits the video stream to local user interface device. Local user interface device may display the ultrasound imaging data and/or video stream from patient imaging device to the operator on a display 102A. Local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality.);
a remote second device configured to run the [shared] application software and receive images or video of the medical procedure from the first device via the [shared] application ([0168]-[0170], [0176] The ultrasound imaging data and video stream are transmitted to a second location from the local user, and the ultrasound imaging data and video stream are received and displayed at the second location. Local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality.), the second device being configured to permit the remote source to provide feedback based on performance of the medical procedure ([0062], [0072] System 100 comprises apparatus 100A at a location where a user performs an ultrasound scan on a patient and apparatus 100B at a remote location. An expert may use apparatus 100B to review information about an ultrasound procedure in real time as the ultrasound procedure is being performed at apparatus 100A and to provide real time feedback to a user of apparatus 100A. The remote interface is a general purpose device configured by software to allow a remote expert to review an ultrasound procedure and provide feedback) and to control an operation or attribute of the medical device via the [shared] application software for enhancing use of the device by a device operator ([0075], [0176] patient imaging device is adjustable (for example has adjustable pan and/or zoom) and remote interface device 118 includes controls that allow the expert to remotely adjust patient imaging device 104 to obtain a clear view of the procedure or a view that can best help the user to see and understand how to best conduct a scan. Local user interface 102 and/or remote interface 118 are provided by general purpose computing devices. Such computing devices may execute software (e.g. apps) that cause them to provide functionality.).
However, Pelissier does not explicitly disclose the following that is met by Starobinets:
a shared application ([0034] the natural language communication pathway may be provided via an ROCC (Radiology Operations Command Center) device, such as a mobile device (e.g., a tablet computer or a smartphone). For example, an “app” can run on the ROCC device (operable by the local operator LO) and the remote workstation (operable by the remote expert RE) to allow communication (e.g., audio chats, video chats, and so forth) between the local operator and the remote expert.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to have combined the method including transmitting information between a local device and remote device, as taught by Pelissier, with the transmission occurring via a shared application, as taught by Starobinets, because the claimed invention is only a combination of these well-known elements which would have performed the same function in combination as each did separately. Pelissier already discloses communication between the local user and remote user using apps on each user device. Including the shared app that is run on the local device and on the remote device for communication between the two, as taught by Starobinets, performs the same function of allowing communication between a local user and a remote expert through the use of apps, as taught by Pelissier. Therefore, the results would have been predictable to one of ordinary skill in the art (MPEP 2143).
Regarding Claim 116, the combination of Pelissier and Starobinets teaches the system of claim 115, and Pelissier further teaches:
The system of Claim 115, wherein the second device being configured to permit the remote source to provide feedback allows the remote source to provide visual feedback using the remote second device ([0015], [0072] Providing visual feedback from a remote expert to an ultrasound system user using a remote interface that is configured to allow a remote expert to review an ultrasound procedure and provide feedback. The remote interface device may comprise a smartphone, tablet, laptop, or personal computer configured by software to perform the functions of the remote interface).
Regarding Claim 117, the combination of Pelissier and Starobinets teaches the system of claim 116, and Pelissier further teaches:
The system of Claim 116, wherein the visual feedback comprises a graphic overlay ([0073], [0141], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A (local user) sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match.) of an example routine ([0078] In some embodiments remote interface includes a demonstration device which can be positioned like probe and is equipped with a position sensor. The expert may manipulate the demonstration device to demonstrate a particular technique while the position and orientation of the demonstration device is tracked. The tracked position and orientation of the demonstration device may be transmitted to apparatus 100A which may generate an animation showing the user of apparatus 100A how the expert is manipulating the demonstration device.), target area on a patient, position of the medical device, movement of the medical device ([0025], [0073], [0141], [0152], FIG. 6 shows an example remote user interface with drag and drop graphical elements that a remote user can select to provide feedback. In the example, the expert drags and drops a targeting icon onto patient video stream to indicate the desired location of the probe. In some embodiments remote interface sends a plurality of messages for a drag and drop event so that a user at apparatus 100A (local user) sees where the expert has dragged the icon in real time while the icon is being dragged by the expert at remote interface. Additionally, the frozen reference image could be displayed adjacent to the live ultrasound image, or overlaid with a transparency on display so that the local user can try to adjust the position of probe to match. In some embodiments the icons include one or more of: icons depicting tilting the probe forward and backward; icons depicting rocking the probe right and left; icons depicting rotating the probe clockwise and counterclockwise; and icons depicting moving the probe to the left, right, forward and backward), or an animation of at least one suggested position or movement of the medical device ([0139] In some embodiments, processing the message comprises generating an animation illustrating movement of the probe from the starting to the ending position. Such an animation may assist the user to see how to reposition the probe for better results).
Regarding Claim 118, the combination of Pelissier and Starobinets teaches the system of claim 115, and Pelissier further teaches:
The system of Claim 115, wherein the first device ([0066], [0070] A user of apparatus 100A may request remote assistance from an expert using a local interface device. The local user interface may comprise of a tablet computer, a smart phone, a laptop computer, a personal computer, a computer workstation or the like.) or the second device is a smartphone or a tablet ([0072] remote interface device may comprise a smartphone, tablet, laptop, or personal computer).
Regarding Claim 119, the combination of Pelissier and Starobinets teaches the system of claim 115, and Pelissier further teaches:
The system of Claim 115, wherein the remote source comprises a skilled user of the medical device or an artificial intelligence software ([0061] An ultrasound system may provide features that allow a person at a remote site (e.g. an expert ultrasound user) to review the actions of a person).
Conclusion
The relevant art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Leyvi et al. (WO 2018/153979 A1) discloses a remotely controlled ultrasonic imaging system that allows a remotely located diagnostic imaging expert to view transmitted ultrasound images and control the ultrasound functionality.
Portable Ultrasound Remote Guidance Systems describes remote expert guidance advantages and examples of these systems for real-time shared visualization of ultrasounds being performed by a user.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXIS K VAN DUZER whose telephone number is (571)270-5832. The examiner can normally be reached Monday thru Thursday 8-5 CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long can be reached at (571) 270-5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.K.V./Examiner, Art Unit 3682
/EVANGELINE BARR/Primary Examiner, Art Unit 3682
Prosecution Insights