DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/28/2026 has been entered.
Response to Arguments
Regarding 35 U.S.C. 112(a)
Applicant's arguments filed 01/28/2026 have been fully considered but they are not persuasive. For example, applicant argues “the subject specification describes that "[t]he processor of the controller 200 realizes a reception step, a display step, and an output step by executing a diagnostic program 231 or the like stored in the storage section 230 or the like." See paragraph [0039] of the subject specification, emphasis added. Therefore, paragraph [0039] of the subject specification describes that the abovementioned operations are performed by executing a diagnostic program 231. The subject specification then describes that "[t]he diagnostic program 231 for realizing the reception step, the display step, and the output step may be stored, for example, in the storage section 140 of the diagnostic apparatus 10. In this case, the controller 130 of the diagnostic apparatus 10 functions as an output section, and outputs the diagnostic program 231 to the display terminal 20 via the wireless network N." See paragraph [0040] of the subject specification, emphasis added. Therefore, in one embodiment where the diagnostic program 231 is stored in the storage section 140 of the diagnostic apparatus 10, the ultrasound diagnostic apparatus 10 is caused to acquire a reception time at which the image data is received, calculate a delay time, set the delay time, calculate a difference, determine whether the difference exceeds a threshold value and change the display style of displaying as recited by the claims” (REMARKS pg. 11-17). Examiner respectfully disagrees in that applicant has provided only support that the controller 200 (i.e. of the display terminal) executes the diagnostic program 231 where the diagnostic program causes the reception step, display step, and output step accordingly as noted by applicant. It is noted that the disclosure in [0040] merely provides support that the diagnostic program may be stored in the ultrasound apparatus 10, however, further discloses that the diagnostic program 231 is output to the display terminal. It is noted that such disclosure in [0040] only provides support for the ultrasound apparatus storing the diagnostic program and outputting to the display terminal for the display terminal to execute the diagnostic program and does not implicitly nor explicitly provide any support that the ultrasound apparatus executes the diagnostic program as argued by applicant. Furthermore, it is noted that for the ultrasound apparatus to execute the diagnostic program including receiving the reception time by the display terminal, such reception would require extra steps of the display terminal communicating the reception time to the ultrasound apparatus which is not implicitly nor explicitly disclosed by the originally filed application. Applicant’s arguments are therefore not found persuasive for the reasons listed above.
Regarding 35 U.S.C. 112(b)
Applicant's arguments filed 01/28/2026 have been fully considered but they are not persuasive. For example, applicant argues “the independent claims 1 and 7-10 have ben amended to define the relationship among the ‘delay time’, ‘transmission delay’ and ‘delay information’”. Examiner notes that while the claims have been amended to define a relationship between the transmission delay and the delay time, it is noted that the claim remains unclear as the delay time which is a magnitude of the transmission delay may be the same as the delay information indicating that a transmission delay of the image data is occurring under its broadest reasonable interpretation, however, are recited differently thus remaining to be unclear as to whether the claim intends to display the delay time or if the delay information is something different/distinct from the delay time. For at least these reasons, applicant’s arguments are not found persuasive and the 112(b) rejection is maintained/updated in view of the amendments.
Furthermore, no amendments/arguments are made with respect to the setting of the delay time claims 1 and 7-10 and the rejection of claims 2 and 5. Examiner notes that the previously set forth rejections are maintained.
Regarding prior art
Applicant's arguments filed 01/28/2026 have been fully considered but they are not persuasive. For example, applicant argues “that Huang teaches that ‘the operation end extracts end specific time displayed in a clock area through an image recognition algorithm and compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge”. See page 3 16th paragraph of Huang… that is, the subject to be compared with the threshold value is the difference between the specific time and the time on a computer, not the difference between the relative time differences at the different times” (REMARKS pg. 14-15). Examiner respectfully disagrees, in that applicant appears to misunderstand examiner’s interpretation of the art. First it is noted that the difference of the time is considered to be the delay time calculated at a second time and the threshold value is considered to be a delay time calculated at a first time. In other words, the threshold value is necessarily a delay time otherwise comparison between the difference of the times calculated during the operation and the threshold value would not be an accurate comparison and that the threshold value is necessarily calculated at a first time otherwise the threshold value could not exist. Second, by comparing the difference of the time to the threshold and determining if the difference of the time is larger or smaller than the threshold, it is necessarily determined whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (e.g. 0). In other words, if the threshold (i.e. reference delay time) is 1 ms and the delay time is 2 ms it is determining that the delay time is larger (by comparing and determining the difference between the two is 1 ms which is greater than 0) and thus determines that the difference exceeds a threshold value. In other words, it is noted that the threshold value disclosed in Huang is not interpreted as the threshold value of the recited claim language, but rather the reference time and through comparison of the reference time (i.e. the threshold value) and the delay time calculated at the second time it is noted that a determination of whether the difference between the two exceeds a threshold value is performed in order to determine if the delay time is larger than the reference time. For at least these reasons, applicant’s arguments against the teachings of Huang are not found persuasive.
Applicant’s arguments with respect to claims 20 and 21 have been considered but are moot in view of the new ground of rejection necessitated by amendment.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 8, 10-11, 17 and 19 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 8 recites the limitation “wherein the program further causes the ultrasound diagnostic apparatus to perform: ….acquiring a reception time at which the image data is received, calculating a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time, wherein the program further causes the ultrasound diagnostic apparatus to perform: setting the delay time calculated at a first time as a reference delay time; calculating the delay time at a second time after the first time; calculating a difference between the reference delay time and the delay time calculated at the second time; determining whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value; and changing a display style of displaying the delay information when the difference exceeds the threshold value”. Examiner notes that the limitations are directed to a program that causes the ultrasound diagnostic apparatus to perform the above functions in which the ultrasound diagnostic apparatus is specifically recited as being a separate unit from an ultrasound image display terminal. Applicant’s originally filed specification provides support that the ultrasound image display terminal (to which the time information is output and data for displaying the delay information is output by the ultrasound diagnostic apparatus) is configured for performing the above recited functions, however, there is not sufficient support that the ultrasound diagnostic apparatus is caused to acquire a reception time at which the image data is received, calculate a delay time, set the delay time calculate a difference, determine whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value and change the display style of displaying as recited by the claims. For at least these reasons, the limitation constitutes new matter as a person having ordinary skill in the art would not have recognized the inventor had possession of the claimed invention at the original time of filing.
Claim 10 is rejected for the same reasons listed above with respect to claim 8. Specifically applicant’s originally filed specification does not provide sufficient support that a hardware processor included in the ultrasound diagnostic apparatus that is a separate unit from an ultrasound image display terminal acquires the time stamp and a reception time at which the image data is receive by the ultrasound image display terminal, and calculates a delay time using the time stamp and the reception time, sets the delay time, calculates the delay time, calculates a difference, determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value, and changes a display style. Such recited functions are specifically disclosed as being performed by the hardware processor (controller 200) of the ultrasound image display terminal. For at least these reasons, the limitation constitutes new matter as a person having ordinary skill in the art would not have recognized the inventor had possession of the claimed invention at the original time of filing.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 7-10 recite the limitation “calculates/calculating a delay time”. It is unclear if the delay time is the same as the delay information recited previously or if these are different/distinct features. In other words, while the claim recites the delay information indicating that a transmission delay of the image data is occurring and that the delay time is a magnitude of the transmission delay, it would therefore appear that the delay information may be the same as or may be different/distinct from the delay time, but the claim does not clearly set forth what the relationship between the two is. For examination purposes, it has been interpreted to mean that the delay time may be the same as or different from the delay information, however, clarification is required.
Claims 1 and 7-10 recite the limitation “sets/setting the delay time calculated at a first time as a reference delay time”. Examiner notes that “the delay time” holds antecedent basis to the delay time using the time stamp and the reception time, thus making it unclear if the delay time calculated as a first time is the same or different from the delay time recited previously. Although the delay times are calculated at first and second times respectively such calculation timing does not make clear that the delay times are different/distinct or if they are the same values just calculated at different timings. For examination purposes, it has been interpreted that the reference delay time is different from the previously recited delay time, however, clarification is required.
Claim 2 recites the limitation “determines whether the transmission delay is occurring”. It is unclear if the determination of the transmission delay is occurring is the same as or corresponds with the calculation of the delay time. In other words, it is unclear if the transmission delay is determined to be occurring as a result of the calculation of the delay time or if this is a separate/distinct features/determination. For examination purposes, it has been interpreted to mean the same or different as the calculation of the delay time, however, clarification is required.
Claim 5 recites the limitation “calculates, based on the reference delay time and the delay time calculated at the second time, a difference time indicating whether the transmission delay is occurring”. It is unclear if the difference time is the same as the difference between the reference delay time and the delay time recited in claim 1 or if this is a different/distinct difference time (e.g. one that is merely based on the reference delay time and the delay time but is not a difference between the two). For examination purposes, it has been interpreted to mean the same as the difference, however, clarification is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 12-13, and 15-19 are rejected under 35 U.S.C. 103 as unpatentable over Kottenstette et al. (US 20230128665 A1), hereinafter Kottenstette in view of Foreign Huang (CN 113422708 A) hereinafter Huang. Examiner notes that citations to Huang are with reference to the translated copy provided herein.
Regarding claim 1,
Kottenstette discloses an ultrasound diagnostic system (at least fig. 4 (200) and corresponding disclosure in at least [0099] considered an ultrasound diagnostic system as it comprises an IVUS system 254 and imaging system 248 which may be ultrasound as disclosed in [0089]) comprising:
An ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that generates image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)); and
An ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]) that is a separate unit from the ultrasound diagnostic apparatus (204) (see at least fig. 4 depicting separate units) and displays an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]) and/or the IVUS system);
Wherein the ultrasound diagnostic apparatus (204) and the ultrasound image display terminal (202) are connected with one another via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), and
Wherein the ultrasound diagnostic system (200) further comprises:
A transmitter (at least fig. 4 (214 and/or 210) and corresponding disclosure in at least [0102]-[0103] where the control module which receives data for transmitting over the network 206 through local firewall 210 is considered a transmitter) that transmits the image data from the ultrasound diagnostic apparatus (204) to the ultrasound image display terminal (202) ([0103] which discloses local command and control module 214 uses the timestamp information from the local reference clock 226 to timestamp the image data received from the first video capture and scaling device 242 and further discloses the timestamped images (i.e. image data) and hemodynamic data may be transmitted via network 206 to the remote command and control module 212 in the control center 202. See also [0168] The local command and control module 214 then sends the image frame capture time Tf to the control center 202 along with the captured image frame); and
A hardware processor (at least fig. 4 (212 and 216) and corresponding disclosure in at least [0100] which discloses in an example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented on separate hardware (e.g., computer systems). In another example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented as separate software components or logical subsystem components on the same computer system) configured to output delay information indicating that a transmission delay of the image data is occurring ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console and further discloses the graphical user interface 460 for the control center [0165] which discloses the image feedback delay Timage refers to the delay or latency associated with the transmission of image and/or hemodynamic data from the robotic medical device system. See also [0134] which discloses display of images at the remote site display 240 (controlled by remote command and control module as disclosed in [0105]) shown in fig. 4 may be configured to display the image display time and the command and control signal delay time) and display the delay information on the ultrasound image display terminal ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console. See also fig. 14 (1106) and corresponding disclosure in at least [0135]),
wherein the ultrasound diagnostic apparatus adds a time stamp to the image data, wherein the hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of the transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202), and
Wherein the hardware processor changes a display style of displaying the delay information when a difference between a reference delay time and the delay time exceeds a threshold value (see figs. 6-7 and 14 where the delay of the transmitted images is displayed examiner notes that the value changes with the delay thus the display style (e.g. characters of the delay value) are changed at all times including when a difference exceeds a threshold).
Kotenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates the difference between the reference delay time and the delay time calculated at the second time, determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein an apparatus adds a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
Wherein a hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
Wherein the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, and determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Examiner further notes that the system determines a difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
Regarding claim 2,
Kottenstette further teaches wherein the hardware processor determines whether the transmission delay is occurring ([0166] which discloses the image feedback delay Timage may be computed at the remote command and control module 212 and the image feedback delay Timage may be computed as the difference between the time (Tpatientside) (i.e. reference delay time information) at which an image frame is captured (e.g., indicated via a time stamp) and the time (Tcontrolside) at which the captured image frame is received at the control center 202. Where such a computation requires acquiring a second time (i.e. Tcontrolside) at a second time different from the first time(i.e. the time at which the captured image frame is received, such a different in time necessarily determines whether the transmission delay is occurring), and outputs the delay information based on a result of the determination (([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console and further discloses the graphical user interface 460 for the control center [0165] which discloses the image feedback delay Timage refers to the delay or latency associated with the transmission of image and/or hemodynamic data from the robotic medical device system. See also [0134] which discloses display of images at the remote site display 240 (controlled by remote command and control module as disclosed in [0105]) shown in fig. 4 may be configured to display the image display time and the command and control signal delay time where such displaying of information is based on a result of the determination of Timage).
Regarding claim 3,
Kottenstette further discloses wherein the hardware processor outputs the delay information in response to a predetermined condition being satisfied ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that the start of the catheterization during the procedure is considered a predetermined condition that has been satisfied).
Regarding claim 4,
Kottenstette further discloses wherein the predetermined condition is a puncture mode ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that such catheterization is considered a puncture mode in it’s broadest reasonable interpretation and has been satisfied at the start of the procedure).
Regarding claim 5,
Kottenstette further discloses wherein the delay information is delay information indicating that the transmission delay is occurring at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal ([0165] which discloses Timage refers to the delay or latency associated with the transmission of image from the robotic medical device to the control center thus is occurring at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal), and
Wherein the hardware processor
Calculates based on the reference delay time and the delay time calculated at the second time a difference time indicating whether the transmission delay is occurring ([0166] which discloses the image feedback delay Timage may be computed at the remote command and control module 212 and the image feedback delay Timage may be computed as the difference between the time (Tpatientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (Tcontrolside) at which the captured image frame is received at the control center 202)
Additionally/alternatively, Huang teaches calculating, based on the reference delay time and the delay time calculated at the second time, a difference time indicating whether a transmission ndelay is occurring (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge).
Examiner notes that the delay information thus indicates that the transmission delay is occurring at at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal in the modified system.
Regarding claim 6,
Kottenstette further discloses wherein the processor outputs the delay information in response to the difference time being equal to or greater than a predetermined threshold value ([0134] In another example embodiment, a display of images at the remote site (e.g., on display 240 shown in FIG. 4) may be configured to display the image delay time and the command and control signal delay time as shown in FIG. 14. See at least fig. 14 depicting a difference time (i.e. the image delay time) being 17ms which is equal to or greater than a predetermined threshold value (e.g. 0), thus the processor outputs the delay information in response to such a difference time being equal to or greater than 0).
Regarding claim 7,
Kottenstette discloses a non-transitory computer-readable medium storing a program ([0270] which discloses at least one memory may include or store computer program code, and the at least one memory and the computer program code may be configured to, with at least one processor, cause a robotic medical device system to perform the necessary tasks) that causes an ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) and is connected with the ultrasound diagnostic apparatus via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), the ultrasound diagnostic apparatus generating image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)), to perform:
Receiving the image data transmitted from the ultrasound diagnostic apparatus ([0100] which discloses first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204);
Displaying an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 ([0089] which discloses imaging system may be ultrasound) and/or the IVUS system); and
Outputting delay information indicating that a transmission delay of the image data is occurring ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console and further discloses the graphical user interface 460 for the control center [0165] which discloses the image feedback delay Timage refers to the delay or latency associated with the transmission of image and/or hemodynamic data from the robotic medical device system See also [0134] which discloses display of images at the remote site display 240 (controlled by remote command and control module as disclosed in [0105]) shown in fig. 4 may be configured to display the image display time and the command and control signal delay time), and displaying the delay information on the ultrasound image display terminal ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console. See also fig. 14 (1106) and corresponding disclosure in at least [0135]),
wherein the program further causes the ultrasound image display terminal to perform:
receiving a time stamp added to the image data ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202);
acquiring a reception time at which the image data is received by the ultrasound image display terminal ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202); and
calculating a delay time which is a magnitude of the transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202),
wherein the program further causes the ultrasound image display terminal to perform:
changing a display style of displaying the delay information when a difference between a reference delay time and the delay time calculated at the second time exceeds the threshold value (see figs. 6-7 and 14 where the delay of the transmitted images is displayed examiner notes that the value changes with the delay thus the display style (e.g. characters of the delay value) are changed at all times including when a difference exceeds a threshold).
Kottenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal and wherein the program further causes the ultrasound diagnostic apparatus to perform: setting the delay time calculated at a first time as a reference delay time; calculating the delay time at a second time after the first time; calculating the difference between the reference delay time and the delay time calculated at the second time; determining whether the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein a program causes an terminal to receive a time stamp added to image data, the time stamp including a transmission time at which the image data is transmitted from an execution end to an operation end (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
acquire the time stamp and a reception time at which the image data is received by the operation end, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
set the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time,
calculate a difference between the reference delay time and the delay time calculated at the second time, and
determine whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Examiner notes that the system determines a difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
Regarding claim 8.
Kottenstette discloses a non-transitory computer-readable medium storing a program ([0270] which discloses at least one memory may include or store computer program code, and the at least one memory and the computer program code may be configured to, with at least one processor, cause a robotic medical device system to perform the necessary tasks) that causes an ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]), is connected with the ultrasound image display terminal via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), and generates image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound), the ultrasound image display terminal displaying an ultrasound image ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]), to perform:
Transmitting the image data to the ultrasound image display terminal ([0103] which discloses local command and control module 214 uses the timestamp information from the local reference clock 226 to timestamp the image data received from the first video capture and scaling device 242 and further discloses the timestamped images (i.e. image data) and hemodynamic data may be transmitted via network 206 to the remote command and control module 212 in the control center 202); and
Outputting, to the ultrasound image display terminal, time information for calculating delay information indicating that a transmission delay of the image data is occurring (([0168] The local command and control module 214 records the image frame capture time Tf, which indicates the time at which an image frame is captured at the patient side (e.g., via a time stamp). The local command and control module 214 then sends the image frame capture time Tf to the control center 202 along with the captured image frame), and outputting data for displaying the delay information on the ultrasound image display terminal ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console. See also fig. 14 (1106) and corresponding disclosure in at least [0135]),
wherein the program further causes the ultrasound diagnostic apparatus to perform:
adding a time stamp to the image data ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202);
acquiring a reception time at which the image data is received by the ultrasound image display terminal ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202); and
calculating a delay time which is a magnitude of the transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202),
wherein the program further causes the ultrasound diagnostic apparatus to perform:
changing a display style of displaying the delay information when a difference between a reference delay time and the delay time exceeds the threshold value (see figs. 6-7 and 14 where the delay of the transmitted images is displayed examiner notes that the value changes with the delay thus the display style (e.g. characters of the delay value) are changed at all times including when a difference exceeds a threshold).
Kottenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal and wherein the program further causes the ultrasound diagnostic apparatus to perform: setting the delay time calculated at a first time as a reference delay time; calculating the delay time at a second time after the first time; calculating the difference between the reference delay time and the delay time calculated at the second time; determining whether the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein a program causes an apparatus to add a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
acquire the time stamp and a reception time at which the image data is received by the operation end, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
set the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time,
calculate a difference between the reference delay time and the delay time calculated at the second time, and
determine whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
Regarding claim 9,
Kottenstette discloses an ultrasound image display terminal (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound diagnostic apparatus (at least fig. 4 (202) and corresponding disclosure in at least [0099]) and is connected with the ultrasound diagnostic apparatus via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), the ultrasound diagnostic apparatus generating image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)), comprising:
a receiver (at least fig. 4 (212) and corresponding disclosure in at least [0100]) that receives the image data transmitted from the ultrasound diagnostic apparatus ([0100] which discloses first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204);
a display (at least fig. 4 (240) and corresponding disclosure in at least [0100]) that displays an ultrasound image based on the image data (see at least fig. 4 depicting separate units) and displays an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]) and/or the IVUS system);
and a hardware processor (at least fig. 4 (212 and 216) and corresponding disclosure in at least [0100] which discloses in an example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented on separate hardware (e.g., computer systems). In another example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented as separate software components or logical subsystem components on the same computer system) that outputs delay information indicating that a transmission delay of the image data is occurring ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console and further discloses the graphical user interface 460 for the control center [0165] which discloses the image feedback delay Timage refers to the delay or latency associated with the transmission of image and/or hemodynamic data from the robotic medical device system. See also [0134] which discloses display of images at the remote site display 240 (controlled by remote command and control module as disclosed in [0105]) shown in fig. 4 may be configured to display the image display time and the command and control signal delay time) and displays the delay information on the display ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console. See also fig. 14 (1106) and corresponding disclosure in at least [0135]),
wherein the hardware processor receives a time stamp added to the image data, and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of the transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202), and
Wherein the hardware processor changes a display style of displaying the delay information when the difference between the reference delay time and a delay time calculated at a second time exceeds a threshold value (see figs. 6-7 and 14 where the delay of the transmitted images is displayed examiner notes that the value changes with the delay thus the display style (e.g. characters of the delay value) are changed at all times including when the difference exceeds a threshold).
Kotenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein an apparatus adds a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from an execution end to an operation end (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
Wherein a hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
Wherein the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, and determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
Regarding claim 10,
An ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]), is connected with the ultrasound image display terminal (202) via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), and generates image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound), the ultrasound image display terminal displaying an ultrasound image ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]) and/or the IVUS system), comprising:
a transmitter (at least fig. 4 (214 and/or 210) and corresponding disclosure in at least [0102]-[0103] where the control module which receives data for transmitting over the network 206 through local firewall 210 is considered a transmitter) that transmits the image data to the ultrasound image display terminal ([0103] which discloses local command and control module 214 uses the timestamp information from the local reference clock 226 to timestamp the image data received from the first video capture and scaling device 242 and further discloses the timestamped images (i.e. image data) and hemodynamic data may be transmitted via network 206 to the remote command and control module 212 in the control center 202. See also [0168] The local command and control module 214 records the image frame capture time Tf, which indicates the time at which an image frame is captured at the patient side (e.g., via a time stamp). The local command and control module 214 then sends the image frame capture time Tf to the control center 202 along with the captured image frame);
and a hardware processor (at least fig. 4 (214) and corresponding disclosure in at least [0102] which discloses the local command and control module 214 and the local controller 218 are implemented on separate hardware (e.g., computer systems). In another example embodiment, the local firewall 210, the local command and control module 214 and the local controller 218 are implemented as separate software components or logical subsystem components on the same computer system) that outputs, to the ultrasound image display terminal, time information for calculating delay information indicating that a transmission delay of the image data is occurring (([0168] The local command and control module 214 records the image frame capture time Tf, which indicates the time at which an image frame is captured at the patient side (e.g., via a time stamp). The local command and control module 214 then sends the image frame capture time Tf to the control center 202 along with the captured image frame).
and outputs data for displaying the delay information on the ultrasound image display terminal ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console. See also fig. 14 (1106) and corresponding disclosure in at least [0135]),
Wherein the hardware processor adds a time stamp to the image data, wherein the hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of the transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202), and
Wherein the hardware processor changes a display style of displaying the delay information when the difference exceeds a threshold value (see figs. 6-7 and 14 where the delay of the transmitted images is displayed examiner notes that the value changes with the delay thus the display style (e.g. characters of the delay value) are changed at all times including when the difference exceeds a threshold).
Kotenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value, and changes a display style of displaying the delay information when the difference exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein an apparatus adds a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
Wherein a hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
Wherein the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, and determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
Regarding claim 12,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Huang as applied to claim 1 above further teaches wherein the first time is a time during pre-scanning (examiner notes that the threshold value is a value that is calculated before the procedure thus is calculated at a first time considered to be during a pre-scanning), and Kottenstet further teaches wherein the image transfer occurs during a puncture process ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that such catheterization is considered a puncture process in its broadest reasonable interpretation).
Examiner thus notes that in the modified system the second time is a time during a puncture process.
Regarding claim 13,
Kotenstette, as modified, further teaches wherein the hardware processor does not display the delay information in the pre-scanning even if the transmission delay is occurring (Examiner notes that before the procedure no delay information is being displayed. Furthermore, Kotenstette further teaches the hardware processor does not display the delay information for example with respect to fig. 8-10, thus a person having ordinary skill in the art would recognize that the hardware processor functions to not display the delay information at all times thus would function to not display the delay information in pre-scanning even if the transmission delay is occurring).
Regarding claim 15,
Kotenstette further teaches wherein the ultrasound image display terminal includes a display part configured to display a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Regarding claim 16,
Kotenstette further teaches wherein the program further causes the ultrasound image display terminal to perform: displaying, by a display part included in the ultrasound image display terminal, a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Regarding claim 17,
Kotenstette further teaches wherein the program further causes the ultrasound diagnostic apparatus to perform: outputting data for displaying, by a display part included in the ultrasound image display terminal, a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay)) (Additionally, alternatively “for displaying….” Is considered intended use of the data and the output data (i.e. image data or time/stamp data) is capable of being used for displaying a display area including a level meter indicating a magnitude of the transmission delay the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay thus reads on claim as currently recited), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Regarding claim 18,
Kotenstette further teaches wherein the display displays a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle which is configured to move (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Regarding claim 19,
Kotenstette further teaches wherien the hardware processor outputs data for displaying, by a display part included in the ultrasound image display terminal, a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay) (Additionally, alternatively the image data/time stamp is considered to be data which is output by the hardware processor where “for displaying….” Is considered intended use of the data and the output data (i.e. image data or time/stamp data) would be capable of being used for displaying a display area including a level meter indicating a magnitude of the transmission delay the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay thus reads on claim as currently recited), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette and Huang as applied to claim 10 above and further in view of Park et al. (US 20210282750 A1), hereinafter Park.
Regarding claim 11,
Kottenstette teaches the elements of claim 10 as previously stated. Kottenstette further teaches wherein the ultrasound image display terminal stores a program ([0270] which discloses at least one memory may include or store computer program code, and the at least one memory and the computer program code may be configured to, with at least one processor, cause a robotic medical device system to perform the necessary tasks) that causes the ultrasound image display terminal (204) to perform:
receiving the image data transmitted from the ultrasound diagnostic apparatus ([0100] which discloses first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204);
displaying an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 ([0089] which discloses imaging system may be ultrasound) and/or the IVUS system);
and outputting the delay information indicating that the transmission delay of the image data is occurring ([0125] which discloses each graphical user interface 460, 462 displays the delay 464, 466 of the transmitted images from the robotic medical device system to the control console and further discloses the graphical user interface 460 for the control center [0165] which discloses the image feedback delay Timage refers to the delay or latency associated with the transmission of image and/or hemodynamic data from the robotic medical device system See also [0134] which discloses display of images at the remote site display 240 (controlled by remote command and control module as disclosed in [0105]) shown in fig. 4 may be configured to display the image display time and the command and control signal delay time).
Kottenstette fails to explicitly teach wherein the storage of the ultrasound diagnostic apparatus stores the program and wherein the hardware processor outputs the program to the ultrasound image display terminal.
Park, in a similar field of endeavor involving medical imaging systems, teaches a medical apparatus ([0054] which discloses external apparatus may be a medical apparatus) which is wirelessly connected to a separate ultrasound image display terminal (at least fig. 1 (100) and corresponding disclosure in at least [0046]) wherein the medical apparatus comprises a storage that stores a program that controls the entire operations of the ultrasound image display terminal ([0059] which discloses a program for controlling the ultrasound diagnosis apparatus 100 may be installed in the external apparatus. The program may include command languages to perform the entire operations of the controller 120), wherein a hardware processor of the medical apparatus outputs the control program to the ultrasound image display terminal ([0059] which discloses the program for controlling the ultrasound diagnosis apparatus 100 may be installed in the external apparatus. Examiner notes that in order for the ultrasound diagnosis apparatus 100 to perform the operations therein, the program would necessarily be output from the external apparatus (and thus a hardware processor thereof) to the ultrasound image display terminal).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include storing the program for controlling the ultrasound image display terminal as taught by Park in order to reduce the amount of storage required of the ultrasound image display terminal. Furthermore, such a modification amounts to merely a rearrangement of parts (i.e. location where the program is stored), where modifying the position of the program for controlling the image display terminal would not have modified the operation of the device thus rendering the claim obvious (MPEP 2144).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette and Huang as applied to claim 1 above and further in view of Lan (US 20170099237 A1), hereinafter Lan.
Regarding claim 14,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kotenstette, as currently modified, fails to explicitly teach wherein the hardware updates the reference delay time when the calculated delay time is shorter than the reference delay time.
Lan, in a similar field of endeavor involving data transmission, teaches updating a reference delay time when a calculated delay time is shorter than the reference delay time ([0057] determining whether the transmission delay in step 101 is shorter than the minimum transmission delay within a preset time period, and updating the minimum transmission delay within the preset time period to the transmission delay in step 101 if yes)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified, to include updating the reference delay time as taught by Lan in order to ensure that the data transfer is occurring with the smallest possible timing. In other words, when the system of Kotenstette, as currently modified, determines that the delay time is shorter than the reference delay time, a person having ordinary skill in the art would have recognized that the system is thereby capable of transmitted data with a smaller delay amount. Thus, by updating the reference delay according to the shorter calculated transmission delay, the system may function to determine whether the transmission delay is working within its capabilities or if the delay is larger than what the system is capable of. A person having ordinary skill in the art would therefore be motivated to modify the reference delay time to match the calculated delay time to provide a more enhanced system/operating status thereof.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette and Huang as applied to claim 1 above and further in view of Dickie (US 20220386313 A1), hereinafter Dickie.
Regarding claim 20,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kottenstette further teaches wherein the hardware processor displays the delay information on the ultrasound image display terminal when the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value
Kotenstette fails to explicitly teach wherein the hardware processor does not display the delay information when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less and hides the delay information that has been displayed on the ultrasound image display terminal when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less.
Nonetheless, Dickie in a similar field of endeavor involving ultrasound data transmission, teaches wherein a hardware processor does not display delay information indicating a transmission delay is occurring ([0050] which discloses QoS (Quality of Service) can be impacted by several related aspects of the network service, such as packet loss, bit rate, throughput, transmission delay, and availability) when a quality of service is greater than a threshold (see at least fig. 7 (706) and corresponding disclosure in at least [0113]), displays the delay information on a display terminal when the quality of service is less than the threshold value (at least fig. 7 (708) and corresponding disclosure in at least [0114]. Examiner notes that the prompt offering to optimize the wireless network is considered delay information indicating a transmission delay is occurring (i.e. the quality of service is low which is affected by transmission delay as disclosed in [0050]), and hides the delay information that has been displayed on the ultrasound image display terminal when the quality of service is greater than the threshold value (at least fig. 7 depicting a loop where it is noted that the prompt would be hidden once the measure quality of service parameter is not found to be less than the expected quality of service and the prompt is not displayed again).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified to include displaying delay information as taught by Dickie in order to allow a user to optimize the wireless network when it is determined that the quality of service between the apparatus and the display terminal is low (Dickie [0117]-[0118]). Such a modification would allow for enhancing the connection between the apparatus and the display terminal and therefore enhancing the overall image transferring procedure accordingly. Examiner notes that the quality of service being above an expected threshold in the modified system would correspond with difference between the reference time delay and the delay time calculated at a second time as determined by Huang being a threshold value or less (i.e. not overlarge). Additionally/alternatively it would have been obvious to a person having ordinary skill in the art before the effective filing date to not display and/or hide the delay information of Kotenstette when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less (as determined by Huang) in order to ensure that the delay information is only presented to the user when the transmission delay is overlarge thus allowing a user to readily recognize when an overlarge transmission delay is occurring and the ultrasound image data transfer is being impacted. Such a modification would enhance the overall procedure of Kotenstette.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette and Huang as applied to claim 1 above and further in view of Alam et al. (US 20210351991 A1), hereinafter Alam.
Regarding claim 21,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kotenstette further teaches wherein the ultrasound image display terminal includes a display part configured to display a display area including a level meter (see at least fig. 14 (1106)) indicating a magnitude of the transmission delay (1106 displays the text values of the delay), the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay (see at least fig. 14 (1106) depicting a bar (i.e. rectangle having text information)) configured to move (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig. 14 (1106) the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Kottenstette fails to explicitly teach wherein the hardware processor displays the delay information by at least one of: changing a color of the bar or the needle of the level meter; or changing a color of a frame surrounding a periphery of the display area according to the magnitude of the transmission delay.
Nonetheless, Alam, in a similar field of endeavor involving wireless communications, teaches displaying delay information by changing a color of a frame (at least fig. 6A (618) and corresponding disclosure in at least [0038]) surrounding a periphery of a display area (at least fig. 6A (620) and corresponding disclosure in at least [0038]) according to a magnitude of a transmission delay ([0026] which discloses latency data above some high latency threshold corresponds to a red frame-based indicator, latency below the high latency threshold and above some low latency threshold corresponds to a yellow-frame-based indicator, and latency information below low latency threshold corresponds to a green frame-based indicator).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include changing a color of a frame surrounding a periphery of a display area as taught by Alam in order to convey valuable information to a user, by relaying the network condition and the type and quality of service available thereby helping the user set more accurate expectations and leading to an improved procedure (Alam [0058]). Furthermore, it would have been obvious to change a color of the bar of Kottenstette in the same manner taught by Alam for the same reasons. Such a modification would allow a user to readily understand the extent of the transmission delay and whether it is relatively high or low.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BROOKE L KLEIN whose telephone number is (571)270-5204. The examiner can normally be reached Mon-Fri 7:30-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at 5712700552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BROOKE LYN KLEIN/Primary Examiner, Art Unit 3797