DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding 35 U.S.C. 112
Examiner notes that the 112(a) and 112(b) rejections are withdrawn in view of the amendments to the claims.
Regarding prior art
Applicant’s arguments with respect to the claims have been considered but are moot in view of the new grounds of rejection necessitated by amendment. Specifically new teachings are applied to teach the setting of the first delay time calculated according to the claim as a reference delay time.
Regarding applicant’s arguments that Huang fails to teach comparing the difference between the magnitudes of the delays at different time with the threshold value, examiner respectfully disagrees in that as noted in previous actions, determining whether one delay time is larger than another is necessarily a comparison of the difference (i.e. the difference between the one delay time and the other) to a threshold value (i.e. 0). In other words, determining DT2>DT1 is the same as determining DT2-DT1>0 where DT2 is considered a second delay time and DT1 is a first delay time and 0 is considered a predetermined allowable range or threshold value.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 7, 9, 12-13, 15-16 and 18 are rejected under 35 U.S.C. 103 as unpatentable over Kottenstette et al. (US 20230128665 A1), hereinafter Kottenstette in view of Foreign Huang (CN 113422708 A) hereinafter Huang and Ubukata (US 20170292921 A1), hereinafter Ubukata.
Regarding claim 1,
Kottenstette discloses an ultrasound diagnostic system (at least fig. 4 (200) and corresponding disclosure in at least [0099] considered an ultrasound diagnostic system as it comprises an IVUS system 254 and imaging system 248 which may be ultrasound as disclosed in [0089]) comprising:
An ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that generates image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)); and
An ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]) that is a separate unit from the ultrasound diagnostic apparatus (204) (see at least fig. 4 depicting separate units) and displays an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]) and/or the IVUS system);
Wherein the ultrasound diagnostic apparatus (204) and the ultrasound image display terminal (202) are connected with one another via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), and
Wherein the ultrasound diagnostic system (200) further comprises:
A transmitter (at least fig. 4 (214 and/or 210) and corresponding disclosure in at least [0102]-[0103] where the control module which receives data for transmitting over the network 206 through local firewall 210 is considered a transmitter) that transmits the image data from the ultrasound diagnostic apparatus (204) to the ultrasound image display terminal (202) ([0103] which discloses local command and control module 214 uses the timestamp information from the local reference clock 226 to timestamp the image data received from the first video capture and scaling device 242 and further discloses the timestamped images (i.e. image data) and hemodynamic data may be transmitted via network 206 to the remote command and control module 212 in the control center 202. See also [0168] The local command and control module 214 then sends the image frame capture time Tf to the control center 202 along with the captured image frame); and
A hardware processor (at least fig. 4 (212 and 216) and corresponding disclosure in at least [0100] which discloses in an example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented on separate hardware (e.g., computer systems). In another example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented as separate software components or logical subsystem components on the same computer system) configured to output delay information indicating that a transmission delay of the image data is occurring (See at least fig. 20 (2002) and corresponding disclosure in at least [0240] and fig. 21 (2104) and disclosure in at least [0248] Examiner notes that the pop-up of high/medium latency indicates that a transmission delay of the image data is occurring as [0189] discloses a total delay T.sub.delay greater than or equal to the threshold acceptable delay t.sub.acceptable, but less than a maximum threshold delay t.sub.disable ( t.sub.acceptable ≤T.sub.delay < t.sub.disable). As discussed herein, this condition may be referred to as a “medium latency condition.” and [0190] discloses a situation in which the total delay T.sub.delay is greater than or equal to the maximum delay threshold t.sub.dísable may be referred to as a “high latency condition.” and [0135] which discloses the total delay is the sum of the image delay time and the command and control signal delay time. In another example, the total delay may be displayed as a drop down overlay on the display and the total delay (which may be displayed as a drop down overlay on the display). Examiner notes that the total delay indicates that a transmission delay of the image data is occurring since it is the sum of the image delay time and the command and control delay time) and display the delay information on the ultrasound image display terminal (See at least figs. 20 and 21 or fig. 15 (1206)),
wherein the ultrasound diagnostic apparatus adds a first time stamp to the image data of a first frame, the first time stamp including a time at which the image data of the first frame is acquired,
wherein the hardware processor acquires the first time stamp and a first reception time at which the image data of the first frame is received by the ultrasound image display terminal, and calculates a first delay time which is a magnitude of the transmission delay, the first delay time being a difference between a first stamped time indicated by the time stamp and the first reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202),
wherein the ultrasound diagnostic apparatus adds a second time stamp to the image data of a second frame after the first frame, the second time stamp including a time at which the image data of the second frame is acquired ([0136] which discloses the total delay and frame rates may be updated in real-time. Examiner notes that such a total delay being updated in real time would require first and second imaging delays to be calculated, thus second time stamps to be added to a second frame (i.e. in real time)),
wherein the hardware processor acquires the second time stamp and a second reception time at which the image data of the second frame is received by the ultrasound image display terminal, and calculates a second delay time which is a magnitude of the transmission delay, the second delay time being a difference between a second stamped time indicated by the second time stamp and the second reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202)
Wherein the hardware processor changes a display style of displaying the delay information when a difference between a reference delay time and the delay time exceeds a predetermined allowable range (see at least figs. 20 and 21 and/or fig. 15, where the hardware processor changes the display style (i.e. value/text) of displaying the delay information (i.e. medium/high latency) or the value of the total delay in fig. 15 would change when a difference between a reference delay time and a delay time exceeds a pre-determined allowable range), and
Wherein the first delay time and the second delay time are different from the delay information (See at least figs. 20, 21 and 15 in which the delay information is different from the first/second delay time (i.e. the imaging delay))
Kotenstette fails to explicitly teach the time stamps including a transmission time at which the image data of the first frame is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, setting the first delay time as a reference delay time, calculating a difference between the reference delay time and the second delay time, determining whether the difference between the reference delay time and the second delay time exceeds the predetermined allowable range.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein an apparatus adds a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
Wherein a hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
Wherein the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, and determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Examiner further notes that the system determines a difference exceeds a predetermined allowable range (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
The modified system fails to explicitly teach setting the first delay time as the reference delay time (i.e. the threshold value of Huang).
Nonetheless, Ubukata, in a similar field of endeavor involving medical imaging delays, teaches a processor configured to set a calculated first delay time as a reference delay time ([0097]), calculate a difference between the reference delay time and a second calculated delay time, determine whether the difference between the reference delay time and the second delay time exceeds a predetermined allowable range and notifies a user when the difference exceeds the predetermined allowable range ([0102]-[0103]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified, to include setting the first delay time as a reference delay time in the same manner as taught by Ubukata, in order to detect an abnormality in the system. Such a modification would allow for comparing the currently/second delay time with at least one preliminarily acquired delay time to alert a user when the second delay time is abnormal with respect to the system thereby ensuring the system is functioning in a same manner which it has been previously determined to be functioning. Furthermore, such a modification amounts to merely a simple modification of the threshold value of the modified system such that it is simply substituted for a previously calculated delay time yielding predictable results with respect to determination of large delays in a medical imaging system thereby rendering the claim obvious (MPEP 2143).
Regarding claim 3,
Kottenstette further discloses wherein the hardware processor outputs the delay information in response to a predetermined condition being satisfied ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that the start of the catheterization during the procedure is considered a predetermined condition that has been satisfied).
Regarding claim 4,
Kottenstette further discloses wherein the predetermined condition is a puncture mode ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that such catheterization is considered a puncture mode in it’s broadest reasonable interpretation and has been satisfied at the start of the procedure).
Regarding claim 5,
Kottenstette further discloses wherein the delay information is delay information indicating that the transmission delay is occurring at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal ([0165] which discloses Timage refers to the delay or latency associated with the transmission of image from the robotic medical device to the control center thus is occurring at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal), and
Wherein the hardware processor
Calculates based on the reference delay time and the delay time calculated at the second time a difference time indicating whether the transmission delay is occurring ([0166] which discloses the image feedback delay Timage may be computed at the remote command and control module 212 and the image feedback delay Timage may be computed as the difference between the time (Tpatientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (Tcontrolside) at which the captured image frame is received at the control center 202)
Additionally/alternatively, Huang teaches calculating, based on the reference delay time and the delay time calculated at the second time, a difference time indicating whether a transmission ndelay is occurring (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge).
Examiner notes that the delay information thus indicates that the transmission delay is occurring at at least one of the ultrasound diagnostic apparatus and the ultrasound image display terminal in the modified system.
Regarding claim 7,
Kottenstette discloses a non-transitory computer-readable medium storing a program ([0270] which discloses at least one memory may include or store computer program code, and the at least one memory and the computer program code may be configured to, with at least one processor, cause a robotic medical device system to perform the necessary tasks) that causes an ultrasound image display terminal (at least fig. 4 (202) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound diagnostic apparatus (at least fig. 4 (204) and corresponding disclosure in at least [0099]) and is connected with the ultrasound diagnostic apparatus via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), the ultrasound diagnostic apparatus generating image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)), to perform:
Receiving the image data transmitted from the ultrasound diagnostic apparatus ([0100] which discloses first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204);
Displaying an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 ([0089] which discloses imaging system may be ultrasound) and/or the IVUS system); and
Outputting delay information indicating that a transmission delay of the image data is occurring (See at least fig. 20 (2002) and corresponding disclosure in at least [0240] and fig. 21 (2104) and disclosure in at least [0248] Examiner notes that the pop-up of high/medium latency indicates that a transmission delay of the image data is occurring as [0189] discloses a total delay T.sub.delay greater than or equal to the threshold acceptable delay t.sub.acceptable, but less than a maximum threshold delay t.sub.disable ( t.sub.acceptable ≤T.sub.delay < t.sub.disable). As discussed herein, this condition may be referred to as a “medium latency condition.” and [0190] discloses a situation in which the total delay T.sub.delay is greater than or equal to the maximum delay threshold t.sub.dísable may be referred to as a “high latency condition.” and [0135] which discloses the total delay is the sum of the image delay time and the command and control signal delay time. In another example, the total delay may be displayed as a drop down overlay on the display and the total delay (which may be displayed as a drop down overlay on the display). Examiner notes that the total delay indicates that a transmission delay of the image data is occurring since it is the sum of the image delay time and the command and control delay time) and display the delay information on the ultrasound image display terminal (See at least figs. 20 and 21 or fig. 15 (1206)),
wherein the program further causes the ultrasound image display terminal to perform:
receiving a first time stamp added to the image data of a first frame ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202);
acquiring a first reception time at which the image data of the first frame is received by the ultrasound image display terminal ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202); and
calculating a first delay time which is a magnitude of the transmission delay, the first delay time being a difference between a first stamped time indicated by the first time stamp and the first reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202),
wherein the program further causes the ultrasound image display terminal to perform:
receiving a second time stamp added to the image data of a second frame after the first frame ([0136] which discloses the total delay and frame rates may be updated in real-time. Examiner notes that such a total delay being updated in real time would require first and second imaging delays to be calculated, thus second time stamps to be added to a second frame (i.e. in real time)),
acquiring a second reception time at which the image data of the second frame is received by the ultrasound image display terminal ([0136] which discloses the total delay and frame rates may be updated in real-time. Examiner notes that such a total delay being updated in real time would require first and second imaging delays to be calculated, thus a second reception time is received (i.e. in real time))
calculating a second delay time which is a magnitude of the transmission delay, the second delay time being a difference between a second stamped time indicated by the second time stamp and the second reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202)
changing a display style of displaying the delay information when a difference between a reference delay time and the delay time calculated at the second time exceeds the threshold value (see at least figs. 20 and 21 and/or fig. 15, where the hardware processor changes the display style (i.e. value/text) of displaying the delay information (i.e. medium/high latency) or the value of the total delay in fig. 15 would change when a difference between a reference delay time and a delay time exceeds a pre-determined allowable range)
Wherein the first delay time and the second delay time are different from the delay information (See at least figs. 20, 21 and 15 in which the delay information is different from the first/second delay time (i.e. the imaging delay))
Kotenstette fails to explicitly teach the time stamps including a transmission time at which the image data of the first frame is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, setting the first delay time as a reference delay time, calculating a difference between the reference delay time and the second delay time, determining whether the difference between the reference delay time and the second delay time exceeds the predetermined allowable range.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein a program causes an terminal to receive a time stamp added to image data, the time stamp including a transmission time at which the image data is transmitted from an execution end to an operation end (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
acquiring the time stamp and a reception time at which the image data is received by the operation end, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
calculates the delay time at a second time after the first time, calculate a difference between a reference delay time and the delay time calculated at the second time, and
determine whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Examiner notes that the system determines a difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
The modified system fails to explicitly teach setting the first delay time as the reference delay time (i.e. the threshold value of Huang).
Nonetheless, Ubukata, in a similar field of endeavor involving medical imaging delays, teaches setting a calculated first delay time as a reference delay time ([0097]), calculating a difference between the reference delay time and a second calculated delay time, determining whether the difference between the reference delay time and the second delay time exceeds a predetermined allowable range and notifies a user when the difference exceeds the predetermined allowable range ([0102]-[0103]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified, to include setting the first delay time as a reference delay time in the same manner as taught by Ubukata, in order to detect an abnormality in the system. Such a modification would allow for comparing the currently/second delay time with at least one preliminarily acquired delay time to alert a user when the second delay time is abnormal with respect to the system thereby ensuring the system is functioning in a same manner which it has been previously determined to be functioning. Furthermore, such a modification amounts to merely a simple modification of the threshold value of the modified system such that it is simply substituted for a previously calculated delay time yielding predictable results with respect to determination of large delays in a medical imaging system thereby rendering the claim obvious (MPEP 2143).
Regarding claim 9,
Kottenstette discloses an ultrasound image display terminal (at least fig. 4 (204) and corresponding disclosure in at least [0099]) that is a separate unit from an ultrasound diagnostic apparatus (at least fig. 4 (202) and corresponding disclosure in at least [0099]) and is connected with the ultrasound diagnostic apparatus via a wireless network (at least fig. 4 (206) and corresponding disclosure in at least [0099] which discloses the control center 202 and the robotic medical device system are in communication over a network 202, where the network may be a VPN. Examiner notes that such a network is understood to be a wireless network), the ultrasound diagnostic apparatus generating image data ([0102] control module 214 also receive images from an imaging system 248 where it is noted that images or data therefrom are necessarily generated by the imaging system 248 and therefore the ultrasound diagnostic apparatus 204 in order to be received by the control module. See also [0089] describes imaging system may be any medical imaging system that may be used in conjunction with a catheter based medical procedure (e.g. ultrasound)), comprising:
a receiver (at least fig. 4 (212) and corresponding disclosure in at least [0100]) that receives the image data transmitted from the ultrasound diagnostic apparatus ([0100] which discloses first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204);
a display (at least fig. 4 (240) and corresponding disclosure in at least [0100]) that displays an ultrasound image based on the image data (see at least fig. 4 depicting separate units) and displays an ultrasound image based on the image data ([0100] which discloses a first display 240 in the control center 202 is coupled to the remote command and control module 212 and may be used to display data and images received from the robotic medical device system 204, thus the ultrasound image display terminal displays ultrasound images from the imaging system 248 which may be an ultrasound imaging system ([0089]) and/or the IVUS system);
and a hardware processor (at least fig. 4 (212 and 216) and corresponding disclosure in at least [0100] which discloses in an example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented on separate hardware (e.g., computer systems). In another example embodiment, the remote firewall 208, the remote command and control module 212 and the remote controller 216 are implemented as separate software components or logical subsystem components on the same computer system) that outputs delay information indicating that a transmission delay of the image data is occurring (See at least fig. 20 (2002) and corresponding disclosure in at least [0240] and fig. 21 (2104) and disclosure in at least [0248] Examiner notes that the pop-up of high/medium latency indicates that a transmission delay of the image data is occurring as [0189] discloses a total delay T.sub.delay greater than or equal to the threshold acceptable delay t.sub.acceptable, but less than a maximum threshold delay t.sub.disable ( t.sub.acceptable ≤T.sub.delay < t.sub.disable). As discussed herein, this condition may be referred to as a “medium latency condition.” and [0190] discloses a situation in which the total delay T.sub.delay is greater than or equal to the maximum delay threshold t.sub.dísable may be referred to as a “high latency condition.” and [0135] which discloses the total delay is the sum of the image delay time and the command and control signal delay time. In another example, the total delay may be displayed as a drop down overlay on the display and the total delay (which may be displayed as a drop down overlay on the display). Examiner notes that the total delay indicates that a transmission delay of the image data is occurring since it is the sum of the image delay time and the command and control delay time) and display the delay information on the ultrasound image display terminal (See at least figs. 20 and 21 or fig. 15 (1206),
wherein the hardware processor receives a first time stamp added to the image data of the first frame,
wherein the hardware processor acquires the first time stamp and a first reception time at which the image data of the first frame is received by the ultrasound image display terminal, and calculates a first delay time which is a magnitude of the transmission delay, the first delay time being a difference between a first stamped time indicated by the first time stamp and the first reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202), and
wherein the hardware processor receives a second time stamp added to the image data of a second frame after the first frame ([0136] which discloses the total delay and frame rates may be updated in real-time. Examiner notes that such a total delay being updated in real time would require first and second imaging delays to be calculated, thus second time stamps to be added to a second frame (i.e. in real time) and received accordingly).
Wherein the hardware processor acquires the second time stamp and a second reception time at which the image data of the second frame is received by the ultrasound image display terminal, and calculates a second delay time which is a magnitude of the transmission delay, the second delay time being a difference between a second stamped time indicated by the second time stamp and the second reception time ([0166] which discloses In yet another, more specific, example, the image feedback delay T.sub.image may be computed as the difference between the time (T.sub.patientside) at which an image frame is captured (e.g., indicated via a time stamp) and the time (T.sub.controlside) at which the captured image frame is received at the control center 202)
Wherein the hardware processor changes a display style of displaying the delay information when a difference between a reference delay time and the delay time exceeds a predetermined allowable range (see at least figs. 20 and 21 and/or fig. 15, where the hardware processor changes the display style (i.e. value/text) of displaying the delay information (i.e. medium/high latency) or the value of the total delay in fig. 15 would change when a difference between a reference delay time and a delay time exceeds a pre-determined allowable range), and
Wherein the first delay time and the second delay time are different from the delay information (See at least figs. 20, 21 and 15 in which the delay information is different from the first/second delay time (i.e. the imaging delay))
Kotenstette fails to explicitly teach the time stamp including a transmission time at which the image data is transmitted from the ultrasound diagnostic apparatus to the ultrasound image display terminal, the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value.
Huang, in a similar field of endeavor involving ultrasound image transmission, teaches wherein an apparatus adds a time stamp to image date, the time stamp including a transmission time at which the image data is transmitted from an execution end to an operation end (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area. Examiner notes that the clock picture is a time stamp and necessarily includes a transmission time (i.e. time of sending the data packet) in order to evaluate the delay time thereof),
Wherein a hardware processor acquires the time stamp and a reception time at which the image data is received by the ultrasound image display terminal, and calculates a delay time which is a magnitude of a transmission delay, the delay time being a difference between a stamped time indicated by the time stamp and the reception time (pg. 3 16th paragraph which discloses calculating the time difference between the time of sending the data packet and the time of receiving the data packet. The video image transmitted from the execution end to the operation end contains a clock picture and the operation end extracts a specific time displayed in a clock area), and
Wherein the hardware processor sets the delay time calculated at a first time as a reference delay time, calculates the delay time at a second time after the first time, calculates a difference between the reference delay time and the delay time calculated at the second time, and determines whether the difference between the reference delay time and the delay time calculated at the second time exceeds a threshold value (pg. 3 16th paragraph which discloses compares the specific time with the time on a computer of the operation end, and if the difference of the time is larger than a set threshold value, the delay is judged to be overlarge; (examiner notes that the set threshold value considered to be a delay time calculated at a first time as a reference time as a person having ordinary skill in the art would have recognized that a threshold value corresponds with a delay time in order to make a comparison thereto and must be calculated at a first time in order to have such a threshold value). Difference exceeds a threshold (e.g. 0) by determining if the delay is larger than the threshold which would indicate that the difference between the delay and the threshold is greater than 0)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include a transmission time as taught by Huang and further to include calculating a difference between a reference delay time and the delay time as taught by Huang in order to identify when there is a condition or interruption occurring between the transmission side and the execution side and/or the network in between (Huang pg. 3 8th paragraph) . Such a modification would allow for sensing such conditions of a network in real time, effectively avoiding misoperation and improving the safety of the procedure (Huang pg. 3 6th paragraph). A person having ordinary skill in the art would have thus recognized the benefit of including a transmission time and calculating a difference between the reference time and the delay time as taught by Huang in order to provide additional network information to a user (e.g. an image delay occurring specifically because of a network latency between two terminals), thereby enhancing the overall safety of the procedure and understanding of delay issues by the user.
The modified system fails to explicitly teach setting the first delay time as the reference delay time (i.e. the threshold value of Huang).
Nonetheless, Ubukata, in a similar field of endeavor involving medical imaging delays, teaches a processor configured to set a calculated first delay time as a reference delay time ([0097]), calculate a difference between the reference delay time and a second calculated delay time, determine whether the difference between the reference delay time and the second delay time exceeds a predetermined allowable range and notifies a user when the difference exceeds the predetermined allowable range ([0102]-[0103]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified, to include setting the first delay time as a reference delay time in the same manner as taught by Ubukata, in order to detect an abnormality in the system. Such a modification would allow for comparing the currently/second delay time with at least one preliminarily acquired delay time to alert a user when the second delay time is abnormal with respect to the system thereby ensuring the system is functioning in a same manner which it has been previously determined to be functioning. Furthermore, such a modification amounts to merely a simple modification of the threshold value of the modified system such that it is simply substituted for a previously calculated delay time yielding predictable results with respect to determination of large delays in a medical imaging system thereby rendering the claim obvious (MPEP 2143).
Regarding claim 12,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kottenstette, as modified, further teaches wherein the first frame is generated during pre-scanning and the second frame is generated during a puncture process ([0066] which discloses catheterization procedures starts by gaining access into the appropriate vessel such as an artery or vein, with a sheath or guide catheter using standard percutaneous techniques. Examiner notes that any images are considered to be during pre-scanning in its broadest reasonable interpretation and during a puncture process).
Regarding claim 13,
Kotenstette, as modified, further teaches wherein the hardware processor does not display the delay information in the pre-scanning even if the transmission delay is occurring (Kottenstette teaches the hardware processor does not display the delay information for example with respect to fig. 8-10, thus a person having ordinary skill in the art would recognize that the hardware processor functions to not display the delay information at all times thus would function to not display the delay information in pre-scanning even if the transmission delay is occurring).
Regarding claim 15,
Kotenstette further teaches wherein the ultrasound image display terminal includes a display part configured to display a display area including a level meter (see at least fig. 15, 20, and 21)) indicating a magnitude of the transmission delay (see fig. 15 which depicts the text values of the total delay and figs. 20 and 21 depicting text indicating a magnitude of the transmission delay is high or medium), the level meter including a bar or a needle which is configured to move (see at least fig. 15 and 20 and 21 depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig.15, 20 and 21 the delay information is displayed in different characters (i.e. number values or high/medium latency) according to the magnitude of the transmission delay).
Regarding claim 16,
Kotenstette further teaches wherein the program further causes the ultrasound image display terminal to perform: displaying, by a display part included in the ultrasound image display terminal, a display area including a level meter (see at least fig. 15, 20, and 21)) indicating a magnitude of the transmission delay (see fig. 15 which depicts the text values of the total delay and figs. 20 and 21 depicting text indicating a magnitude of the transmission delay is high or medium), the level meter including a bar or a needle which is configured to move (see at least fig. 15 and 20 and 21 depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig.15, 20 and 21 the delay information is displayed in different characters (i.e. number values or high/medium latency) according to the magnitude of the transmission delay).
Regarding claim 18,
Kotenstette further teaches wherein the display displays a display area including a level meter (see at least fig. 15, 20, and 21)) indicating a magnitude of the transmission delay (see fig. 15 which depicts the text values of the total delay and figs. 20 and 21 depicting text indicating a magnitude of the transmission delay is high or medium), the level meter including a bar or a needle which is configured to move (see at least fig. 15 and 20 and 21 depicting a bar (i.e. rectangle having text information)) configured to move according to the magnitude of the transmission delay (i.e. by changing the text value of the magnitude of delay)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least fig.15, 20 and 21 the delay information is displayed in different characters (i.e. number values or high/medium latency) according to the magnitude of the transmission delay).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette, Huang, and Ubukata as applied to claim 1 above and further in view of Lan (US 20170099237 A1), hereinafter Lan.
Regarding claim 14,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kotenstette, as currently modified, fails to explicitly teach wherein the hardware updates the reference delay time when the calculated delay time is shorter than the reference delay time.
Lan, in a similar field of endeavor involving data transmission, teaches updating a reference delay time when a calculated delay time is shorter than the reference delay time ([0057] determining whether the transmission delay in step 101 is shorter than the minimum transmission delay within a preset time period, and updating the minimum transmission delay within the preset time period to the transmission delay in step 101 if yes)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified, to include updating the reference delay time as taught by Lan in order to ensure that the data transfer is occurring with the smallest possible timing. In other words, when the system of Kotenstette, as currently modified, determines that the delay time is shorter than the reference delay time, a person having ordinary skill in the art would have recognized that the system is thereby capable of transmitted data with a smaller delay amount. Thus, by updating the reference delay according to the shorter calculated transmission delay, the system may function to determine whether the transmission delay is working within its capabilities or if the delay is larger than what the system is capable of. A person having ordinary skill in the art would therefore be motivated to modify the reference delay time to match the calculated delay time to provide a more enhanced system/operating status thereof.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette, Huang, and Ubukata as applied to claim 1 above and further in view of Dickie (US 20220386313 A1), hereinafter Dickie.
Regarding claim 20,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kottenstette further teaches wherein the hardware processor displays the delay information on the ultrasound image display terminal when the difference between the reference delay time and the delay time calculated at the second time exceeds the threshold value
Kotenstette fails to explicitly teach wherein the hardware processor does not display the delay information when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less and hides the delay information that has been displayed on the ultrasound image display terminal when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less.
Nonetheless, Dickie in a similar field of endeavor involving ultrasound data transmission, teaches wherein a hardware processor does not display delay information indicating a transmission delay is occurring ([0050] which discloses QoS (Quality of Service) can be impacted by several related aspects of the network service, such as packet loss, bit rate, throughput, transmission delay, and availability) when a quality of service is greater than a threshold (see at least fig. 7 (706) and corresponding disclosure in at least [0113]), displays the delay information on a display terminal when the quality of service is less than the threshold value (at least fig. 7 (708) and corresponding disclosure in at least [0114]. Examiner notes that the prompt offering to optimize the wireless network is considered delay information indicating a transmission delay is occurring (i.e. the quality of service is low which is affected by transmission delay as disclosed in [0050]), and hides the delay information that has been displayed on the ultrasound image display terminal when the quality of service is greater than the threshold value (at least fig. 7 depicting a loop where it is noted that the prompt would be hidden once the measure quality of service parameter is not found to be less than the expected quality of service and the prompt is not displayed again).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kotenstette, as currently modified to include displaying delay information as taught by Dickie in order to allow a user to optimize the wireless network when it is determined that the quality of service between the apparatus and the display terminal is low (Dickie [0117]-[0118]). Such a modification would allow for enhancing the connection between the apparatus and the display terminal and therefore enhancing the overall image transferring procedure accordingly. Examiner notes that the quality of service being above an expected threshold in the modified system would correspond with difference between the reference time delay and the delay time calculated at a second time as determined by Huang being a threshold value or less (i.e. not overlarge). Additionally/alternatively it would have been obvious to a person having ordinary skill in the art before the effective filing date to not display and/or hide the delay information of Kotenstette when the difference between the reference delay time and the delay time calculated at the second time is the threshold value or less (as determined by Huang) in order to ensure that the delay information is only presented to the user when the transmission delay is overlarge thus allowing a user to readily recognize when an overlarge transmission delay is occurring and the ultrasound image data transfer is being impacted. Such a modification would enhance the overall procedure of Kotenstette.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette, Huang, and Ubukata as applied to claim 1 above and further in view of Alam et al. (US 20210351991 A1), hereinafter Alam.
Regarding claim 21,
Kotenstette, as modified, teaches the elements of claim 1 as previously stated. Kotenstette further teaches wherein the ultrasound image display terminal includes a display part configured to display a display area including a level meter (see at least fig. 15, 20, and 21) indicating a magnitude of the transmission delay (see at least figs. 15, 20, and 21), the level meter including a bar or a needle which is configured to move according to the magnitude of the transmission delay (see at least fig. 15, 20 and 21 depicting a bar (i.e. rectangle having text information)) configured to move (i.e. by changing the text value of the magnitude of delay or high/low latency)), and the hardware processor displays the delay information by at least displaying text information in the display area in different characters according to the magnitude of the transmission delay (See at least figs. 15, 20, or 21, the delay information is displayed in different characters (i.e. number values) according to the magnitude of the transmission delay).
Kottenstette fails to explicitly teach wherein the hardware processor displays the delay information by at least one of: changing a color of the bar or the needle of the level meter; or changing a color of a frame surrounding a periphery of the display area according to the magnitude of the transmission delay.
Nonetheless, Alam, in a similar field of endeavor involving wireless communications, teaches displaying delay information by changing a color of a frame (at least fig. 6A (618) and corresponding disclosure in at least [0038]) surrounding a periphery of a display area (at least fig. 6A (620) and corresponding disclosure in at least [0038]) according to a magnitude of a transmission delay ([0026] which discloses latency data above some high latency threshold corresponds to a red frame-based indicator, latency below the high latency threshold and above some low latency threshold corresponds to a yellow-frame-based indicator, and latency information below low latency threshold corresponds to a green frame-based indicator).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kottenstette to include changing a color of a frame surrounding a periphery of a display area as taught by Alam in order to convey valuable information to a user, by relaying the network condition and the type and quality of service available thereby helping the user set more accurate expectations and leading to an improved procedure (Alam [0058]). Furthermore, it would have been obvious to change a color of the bar of Kottenstette in the same manner taught by Alam for the same reasons. Such a modification would allow a user to readily understand the extent of the transmission delay and whether it is relatively high or low.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BROOKE L KLEIN whose telephone number is (571)270-5204. The examiner can normally be reached Mon-Fri 7:30-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BROOKE LYN KLEIN/Primary Examiner, Art Unit 3797
