Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to amendments and remarks filed on 09/05/2025. Claims 1-9 are considered in this office action. Claims 1, 3-4, 6-7, and 9 have been amended. Claims 1-9 are pending examination. The previous objection to the specification has been withdrawn in light of the instant amendments. Applicant's amendment necessitated new grounds of rejection therefore, claims 1-9 are rejected.
Response to Arguments
Applicant presents the following arguments regarding the previous office action:
Kaknjo does not teach or suggest: “receiving from an ROV a sensor data stream; " controlling a display, based on the sensor data stream, to render an image of a first configurable visual indicator in the ROV; " receiving from the ROV an indication of a selected state of the first configurable visual indicator in the ROV; and " controlling a second configurable visual indicator to provide a visual signal based on the indication of the selected state of the first configurable visual indicator. So, therefore, Kaknjo does not teach or suggest:
receive from the remotely operated vehicle a sensor data stream; control a display, based on the sensor data stream, to render an image of a first configurable visual indicator in the remotely operated vehicle; and at least one of: receive from the remotely operated vehicle an indication of a selected state of the first configurable visual indicator in the remotely operated vehicle, and control a second configurable visual indicator to provide a visual signal based on the indication of the selected state of the first configurable visual indicator, to enable an operator to determine whether the image of the first configurable visual indicator displayed on the display and the visual signal provided by the second configurable visual indicator are in corresponding states to thereby determine whether a malfunction in the sensor data stream has occurred, the sensor data stream being dependent on an actual state of the first configurable visual indicator; and
as recited in the amended claim 1.
Applicant’s arguments with respect to the independent claims specifically claim 1 has been fully considered and is moot in light of new grounds for rejection below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-5, and 7-8 are all rejected under 35 U.S.C. 103 as being unpatentable
over Altman (WO2019180700A1) in view of Kaknjo et al (Real-Time Video Latency Measurement between a Robot and Its Remote Control Station: Causes and Mitigation), further in view of Murzyn (US20190244446A1).
Regarding claim 1, Altman discloses, an apparatus comprising at least one processing core, at least one memory including computer program code (Altman, 0022, Lines 6-14, primary vehicle 110 may comprise an AI module 119; and the remote server 170 may comprise or may control, or may be associated with, a remote AI module 179. [0024] In accordance with the present invention, data is sensed by the multiple sensors, such as vehicular sensors 111 of primary vehicle 110, and/or vehicular sensors 151 of secondary vehicle 150, and/or sensors 161 of infrastructure elements 161. The sensed data is exchanged, transmitted and/or received among entities in the system via transceivers (132, 152, 162, 174); including sensed data, raw data, partially-processed data, fully processed data, data from external sources, and/or commands or signals that are generated based on analysis of data) … (Altman, 0010, Lines 1-5, primary vehicle 110 may comprise one or more sensors 111 which may be of one or more types and models, for example, imagers, cameras, microphones, image acquisition units, video acquisition units, distance detectors, LIDARs, proximity sensors, RADARs or the like; which are able to continually and/or intermittently sense the surrounding of the vehicle. A vehicular storage unit), and the computer program code being configured to, with the at least one processing core, cause the apparatus at least to: perform as a remote driving station to a remotely operated vehicle by providing operating instructions to the remotely operated vehicle (Altman, 0017, Lines 1-7, the remotely-generated driving commands are transmitted from the remote server 170 (or from a transmission unit associated with it or controlled by it), directly or indirectly, to the vehicular transceiver(s) 132. A vehicular tele-driving processor 133 analyzes the incoming or received signals or messages of tele-operation commands, and optionally converts or translates them into locally-actionable commands that the vehicular system and the various Vehicular Operation Units 124 can then execute), control a display based on the sensor data stream, (0059, the remote component shall be able to display this sparse representation in a way sufficient for the operator to be able to take over the dynamic driving of the vehicle), and at least one of: receive from the remotely operated vehicle (00137, a vehicular Artificial Intelligence (Al) unit, that is configured: (a) to receive inputs from a plurality of vehicular sensors of a vehicle, (b) to locally process within said vehicle at least a first portion of said inputs, (c) to wirelessly transmit via a vehicular wireless transmitter at least a second portion of said inputs to a remote tele-driving processor located externally to said vehicle). However Altman does not explicitly dis close the remaining limitations of claim 1.
Nevertheless, Kaknjo who is in the same field of endeavor of real-time video latency of remotely controlled vehicles discloses, receive from the remotely operated vehicle a sensor data stream (Kaknjo, Introduction, Paragraph 3, Lines 31-37, when controlling the mini ROV (locally or remotely), the operator has the ability to input control commands while relying on feedback information from on-board sensors. One of the most valuable feedback sensors for the operator is the video stream, which is normally displayed on one of the operator’s screens when operating the ROV), the sensor data stream being dependent on an actual state of the first configurable visual indicator (Kaknjo, Proposed Video Latency Measurement Methodology, Section 3.2, Lines 1-7, to eliminate the effect of screen refresh rate on the measurements, an alternative approach was used for measuring capture-to-display latency. In this case, an NTP server was used for generating a visual event which was detectable by a web camera. The event is generated by an LED which is connected to the source of a PPS (pulse per second) signal from a GPS receiver); and image at least a part of the display to generate image data (See Kaknjo, Proposed Video Latency Measurement Methodology, Figure 6, Test configuration with web camera and PPS LED from NTP server),
PNG
media_image1.png
317
424
media_image1.png
Greyscale
and digitize the image data and provide to the remotely operated vehicle an indication of the digitization (Kaknjo, Proposed Video Latency Measurement Methodology, Paragraph 9, Lines 7-14, to measure typical capture-to-display latency of a camera, two configurations (with node and integrated web camera) were tested (Tests 1 and 2 in a configuration with a screen and Test3 in a configuration with a PPS LED diode; see Table 2). The web camera used in the tests is an integrated webcam of a Dell Inspiron 7720, with 0.92M (megapixel) and maximum resolution of 1280x720 pixels).
One of ordinary skill in the art prior to the effective filing date of the given invention
would have been motivated to combine Altman’s disclosure with Kaknjo’s teaching. This
combination would enhance Altman’s system by using a visual LED as a source of visual event and a photo transistor to detect the event. In achieving this an operator would know when latency or a malfunction has occurred and take steps to correct the issue.
Justification for combining Altman’s disclosure with Kaknjo’s teaching not only comes from the state of the art but from Kaknjo (Kaknjo, Proposed Video Latency Measurement Methodology, Paragraph 15-16, Lines 1-18 by using the LED as a visual event source, the effect of screen refresh rate is removed from the measurements and the maximum error now depends solely on the camera frames-per-second value. For a frame per second value of𝑓𝐶25 = 25𝐻𝑧, the maximum possible error will be Δ𝑇25 =𝑇𝐶30 = 40𝑚𝑠.This approach with the PPS LED is not used in the rest of experiments, despite the fact that it represents a way to eliminate inaccuracy caused by the screen refresh rate. One reason for this is that using the PPS LED requires a more complicated setup than the one in which the display is used. Not all NTP servers have a PPS output or LED connected. The second reason was that the sole purpose of the tests is to measure latency which can be experienced by an operator who is looking at the display and controlling the remote robot mostly based on the video feedback, i.e., user-perceived latency).However, even the combination of Altman and Kaknjo does not disclose some aspects of amended claim 1.
Furthermore, Murzyn who is in the same field of endeavor of cluster monitoring systems discloses, rendering an image of a first configurable visual indicator in the remotely operated vehicle (0007, monitoring operation of an instrument cluster configured to display vehicle operating information to a driver. the system includes a camera pointed at the instrument cluster to capture images of operation of the instrument cluster), an indication of a selected state of the first configurable visual indicator in the remotely operated vehicle (0018, the instrument cluster 12 also includes a plurality of tell-tales 30A-30U. The telltales 30A-30U include text and/or graphics that are illuminated in order to convey vehicle operating information to the driver), and control a second configurable visual indicator to provide a visual signal based on the indication of the selected state of the first configurable visual indicator (0007, a fault condition notification module is configured to generate a fault notice when there is a discrepancy between operation of the instrument cluster as captured in the images and the predetermined, expected operation of the instrument cluster) … (0021, the control module 50 may also receive vehicle operating information from any suitable sensors or other onboard control modules of the vehicle. The vehicle operating information includes the vehicle operating information displayed by the instrument cluster 12 at the center dial 20, the sub-dials 22A-22D, and the tell-tales 30A-30U), to enable an operator to determine whether the image of the first configurable visual indicator displayed on the display (0016, the system 10 monitors operation of the instrument cluster 12 with any suitable camera 14. Although FIG. 1 illustrates a single camera 14, any suitable number of cameras may be included as appropriate in order to monitor a desired area of the instrument cluster 12. The camera 14 may be included in a common housing 16 with the instrument cluster 12. Alternatively, the camera 14 may be arranged at any other suitable location as long as the camera 14 is pointed at (directly or indirectly) the instrument cluster 12 in order to visually capture operation thereof), and the visual signal provided by the second configurable visual indicator (0023, the fault notice generated by the fault condition notification module 52 may be any suitable fault notice, such as a fault notice conveyed to the driver on a redundant display screen in the passenger cabin, a fault warning chime generated by a speaker in the passenger cabin, and/or a fault message transmitted to the driver), are in corresponding states to thereby determine whether a malfunction in the sensor data stream has occurred (0022, the control module 50 compares operation of the instrument cluster 12 as captured in the images taken by the camera 14, with expected operation thereof, and previous operation thereof, to identify fault conditions).
One of ordinary skill in the art prior to the effective filing date of the given invention
would have been motivated to combine the combination of Altman and Kaknjo with Murzyn. By placing a configurable visual indicator in the vehicles camera field of view, streaming its image in the sensor data to a remote display and transmitting the corresponding vehicle state information so the remote station can control a second visual indicator (fault signal) based on the selected state and the comparison enables the operator to determine whether the image of the first indicator and the second indicators state correspond and to infer a malfunctions in the camera/sensor data stream when they do not.
Justification for combining the combination of Altman and Kaknjo’s teaching with Murzyn not only comes from the state of the art but from Murzyn (0042, it will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms and that neither should be construed to limit the scope of the disclosure).
Regarding claim 2, Altman, Kaknjo and Murzyn disclose the apparatus according to claim 1 as discussed supra. Furthermore, Altman discloses, the apparatus is configured to provide to the remotely operated vehicle at least one of a timestamp received from the remotely operated vehicle (Altman, 0073, Lines 1-7, synchronization or another type of coordination between the teleoperators and/or between the teleoperators and the vehicular AI component or vehicular processors and/or via another entity (e.g., a central AI component or layer) may be done, for example, via timestamps, IDs, packet IDs, a control channel, a feedback channel, a feedback loop, codes, control messages, transmission and reception of acknowledgement (ACK) packets or codes, transmission and reception of Negative Acknowledgement (NACK) packets or codes, collision-avoidance schemes, ordered list of priorities or prevailing rules, and/or other means), and an identity of the remotely operated vehicle, received in the apparatus from the remotely operated vehicle (Altman, 00123, Lines 21-27, control data or messaging data or feedback data or other meta-data (e.g., time-stamp / date-stamp, a unique identifier of the vehicle that this transmission is intended to, a unique identifier of the remote AI unit or the remote tele-operator that provided or transmitted the data, an indication whether the sent data is a strict command that must be followed or whether it is only an option or a suggestion or a recommendation that the recipient vehicle may or may not accept or reject or modify, or the like), and/or other types of data or signals). Justification for combining these disclosures is the same reasoning given in claim 1.
Regarding claim 4, Altman discloses a method comprising: performing, by an apparatus, as a remote driving station to a remotely operated vehicle by providing operating instructions to the remotely operated vehicle (Altman, 0017, Lines 1-7, the remotely-generated driving commands are transmitted from the remote server 170 (or from a transmission unit associated with it or controlled by it), directly or indirectly, to the vehicular transceiver(s) 132. A vehicular tele-driving processor 133 analyzes the incoming or received signals or messages of tele-operation commands, and optionally converts or translates them into locally-actionable commands that the vehicular system and the various Vehicular Operation Units 124 can then execute), control a display based on the sensor data stream, (0059, the remote component shall be able to display this sparse representation in a way sufficient for the operator to be able to take over the dynamic driving of the vehicle), and at least one of: receive from the remotely operated vehicle (00137, a vehicular Artificial Intelligence (Al) unit, that is configured: (a) to receive inputs from a plurality of vehicular sensors of a vehicle, (b) to locally process within said vehicle at least a first portion of said inputs, (c) to wirelessly transmit via a vehicular wireless transmitter at least a second portion of said inputs to a remote tele-driving processor located externally to said vehicle). However Altman does not explicitly dis close the remaining limitations of claim 4.
Nevertheless, Kaknjo who is in the same field of endeavor of real-time video latency of remotely controlled vehicles discloses, receive from the remotely operated vehicle a sensor data stream (Kaknjo, Introduction, Paragraph 3, Lines 31-37, when controlling the mini ROV (locally or remotely), the operator has the ability to input control commands while relying on feedback information from on-board sensors. One of the most valuable feedback sensors for the operator is the video stream, which is normally displayed on one of the operator’s screens when operating the ROV), the sensor data stream being dependent on an actual state of the first configurable visual indicator (Kaknjo, Proposed Video Latency Measurement Methodology, Section 3.2, Lines 1-7, to eliminate the effect of screen refresh rate on the measurements, an alternative approach was used for measuring capture-to-display latency. In this case, an NTP server was used for generating a visual event which was detectable by a web camera. The event is generated by an LED which is connected to the source of a PPS (pulse per second) signal from a GPS receiver); and image at least a part of the display to generate image data (See Kaknjo, Proposed Video Latency Measurement Methodology, Figure 6, Test configuration with web camera and PPS LED from NTP server), and digitize the image data and provide to the remotely operated vehicle an indication of the digitization (Kaknjo, Proposed Video Latency Measurement Methodology, Paragraph 9, Lines 7-14, to measure typical capture-to-display latency of a camera, two configurations (with node and integrated web camera) were tested (Tests 1 and 2 in a configuration with a screen and Test3 in a configuration with a PPS LED diode; see Table 2). The web camera used in the tests is an integrated webcam of a Dell Inspiron 7720, with 0.92M (megapixel) and maximum resolution of 1280x720 pixels).
However, even the combination of Altman and Kaknjo does not disclose some aspects of amended claim 4.
Furthermore, Murzyn who is in the same field of endeavor of cluster monitoring systems discloses, rendering an image of a first configurable visual indicator in the remotely operated vehicle (0007, monitoring operation of an instrument cluster configured to display vehicle operating information to a driver. the system includes a camera pointed at the instrument cluster to capture images of operation of the instrument cluster), an indication of a selected state of the first configurable visual indicator in the remotely operated vehicle (0018, the instrument cluster 12 also includes a plurality of tell-tales 30A-30U. The telltales 30A-30U include text and/or graphics that are illuminated in order to convey vehicle operating information to the driver), and control a second configurable visual indicator to provide a visual signal based on the indication of the selected state of the first configurable visual indicator (0007, a fault condition notification module is configured to generate a fault notice when there is a discrepancy between operation of the instrument cluster as captured in the images and the predetermined, expected operation of the instrument cluster) … (0021, the control module 50 may also receive vehicle operating information from any suitable sensors or other onboard control modules of the vehicle. The vehicle operating information includes the vehicle operating information displayed by the instrument cluster 12 at the center dial 20, the sub-dials 22A-22D, and the tell-tales 30A-30U), to enable an operator to determine whether the image of the first configurable visual indicator displayed on the display (0016, the system 10 monitors operation of the instrument cluster 12 with any suitable camera 14. Although FIG. 1 illustrates a single camera 14, any suitable number of cameras may be included as appropriate in order to monitor a desired area of the instrument cluster 12. The camera 14 may be included in a common housing 16 with the instrument cluster 12. Alternatively, the camera 14 may be arranged at any other suitable location as long as the camera 14 is pointed at (directly or indirectly) the instrument cluster 12 in order to visually capture operation thereof), and the visual signal provided by the second configurable visual indicator (0023, the fault notice generated by the fault condition notification module 52 may be any suitable fault notice, such as a fault notice conveyed to the driver on a redundant display screen in the passenger cabin, a fault warning chime generated by a speaker in the passenger cabin, and/or a fault message transmitted to the driver), are in corresponding states to thereby determine whether a malfunction in the sensor data stream has occurred (0022, the control module 50 compares operation of the instrument cluster 12 as captured in the images taken by the camera 14, with expected operation thereof, and previous operation thereof, to identify fault conditions). Justification for combining these disclosures is the same reasoning given in claim 1.
Regarding claim 5, Altman, Kaknjo and Murzyn disclose the method according to claim 4, as discussed supra. Furthermore, Altman discloses, the apparatus is configured to provide to the remotely operated vehicle at least one of a timestamp received from the remotely operated vehicle (Altman, 0073, Lines 1-7, synchronization or another type of coordination between the teleoperators and/or between the teleoperators and the vehicular AI component or vehicular processors and/or via another entity (e.g., a central AI component or layer) may be done, for example, via timestamps, IDs, packet IDs, a control channel, a feedback channel, a feedback loop, codes, control messages, transmission and reception of acknowledgement (ACK) packets or codes, transmission and reception of Negative Acknowledgement (NACK) packets or codes, collision-avoidance schemes, ordered list of priorities or prevailing rules, and/or other means), and an identity of the remotely operated vehicle, received in the apparatus from the remotely operated vehicle (Altman, 00123, Lines 21-27, control data or messaging data or feedback data or other meta-data (e.g., time-stamp / date-stamp, a unique identifier of the vehicle that this transmission is intended to, a unique identifier of the remote AI unit or the remote tele-operator that provided or transmitted the data, an indication whether the sent data is a strict command that must be followed or whether it is only an option or a suggestion or a recommendation that the recipient vehicle may or may not accept or reject or modify, or the like), and/or other types of data or signals). Justification for combining these disclosures is the same reasoning given in claim 1.
Regarding claim 7, Altman discloses a non-transitory computer readable medium having stored thereon a set of computer readable instructions that, when executed by at least one processor, cause an apparatus to at least: perform as a remote driving station to a remotely operated vehicle by providing operating instructions to the remotely operated vehicle (Altman, 0017, Lines 1-7, the remotely-generated driving commands are transmitted from the remote server 170 (or from a transmission unit associated with it or controlled by it), directly or indirectly, to the vehicular transceiver(s) 132. A vehicular tele-driving processor 133 analyzes the incoming or received signals or messages of tele-operation commands, and optionally converts or translates them into locally-actionable commands that the vehicular system and the various Vehicular Operation Units 124 can then execute), control a display based on the sensor data stream, (0059, the remote component shall be able to display this sparse representation in a way sufficient for the operator to be able to take over the dynamic driving of the vehicle), and at least one of: receive from the remotely operated vehicle (00137, a vehicular Artificial Intelligence (Al) unit, that is configured: (a) to receive inputs from a plurality of vehicular sensors of a vehicle, (b) to locally process within said vehicle at least a first portion of said inputs, (c) to wirelessly transmit via a vehicular wireless transmitter at least a second portion of said inputs to a remote tele-driving processor located externally to said vehicle). However Altman does not explicitly dis close the remaining limitations of claim 7.
Nevertheless, Kaknjo who is in the same field of endeavor of real-time video latency of remotely controlled vehicles discloses, receive from the remotely operated vehicle a sensor data stream (Kaknjo, Introduction, Paragraph 3, Lines 31-37, when controlling the mini ROV (locally or remotely), the operator has the ability to input control commands while relying on feedback information from on-board sensors. One of the most valuable feedback sensors for the operator is the video stream, which is normally displayed on one of the operator’s screens when operating the ROV), the sensor data stream being dependent on an actual state of the first configurable visual indicator (Kaknjo, Proposed Video Latency Measurement Methodology, Section 3.2, Lines 1-7, to eliminate the effect of screen refresh rate on the measurements, an alternative approach was used for measuring capture-to-display latency. In this case, an NTP server was used for generating a visual event which was detectable by a web camera. The event is generated by an LED which is connected to the source of a PPS (pulse per second) signal from a GPS receiver); and image at least a part of the display to generate image data (See Kaknjo, Proposed Video Latency Measurement Methodology, Figure 6, Test configuration with web camera and PPS LED from NTP server), and digitize the image data and provide to the remotely operated vehicle an indication of the digitization (Kaknjo, Proposed Video Latency Measurement Methodology, Paragraph 9, Lines 7-14, to measure typical capture-to-display latency of a camera, two configurations (with node and integrated web camera) were tested (Tests 1 and 2 in a configuration with a screen and Test3 in a configuration with a PPS LED diode; see Table 2). The web camera used in the tests is an integrated webcam of a Dell Inspiron 7720, with 0.92M (megapixel) and maximum resolution of 1280x720 pixels).
However, even the combination of Altman and Kaknjo does not disclose some aspects of amended claim 7.
Furthermore, Murzyn who is in the same field of endeavor of cluster monitoring systems discloses, rendering an image of a first configurable visual indicator in the remotely operated vehicle (0007, monitoring operation of an instrument cluster configured to display vehicle operating information to a driver. the system includes a camera pointed at the instrument cluster to capture images of operation of the instrument cluster), an indication of a selected state of the first configurable visual indicator in the remotely operated vehicle (0018, the instrument cluster 12 also includes a plurality of tell-tales 30A-30U. The telltales 30A-30U include text and/or graphics that are illuminated in order to convey vehicle operating information to the driver), and control a second configurable visual indicator to provide a visual signal based on the indication of the selected state of the first configurable visual indicator (0007, a fault condition notification module is configured to generate a fault notice when there is a discrepancy between operation of the instrument cluster as captured in the images and the predetermined, expected operation of the instrument cluster) … (0021, the control module 50 may also receive vehicle operating information from any suitable sensors or other onboard control modules of the vehicle. The vehicle operating information includes the vehicle operating information displayed by the instrument cluster 12 at the center dial 20, the sub-dials 22A-22D, and the tell-tales 30A-30U), to enable an operator to determine whether the image of the first configurable visual indicator displayed on the display (0016, the system 10 monitors operation of the instrument cluster 12 with any suitable camera 14. Although FIG. 1 illustrates a single camera 14, any suitable number of cameras may be included as appropriate in order to monitor a desired area of the instrument cluster 12. The camera 14 may be included in a common housing 16 with the instrument cluster 12. Alternatively, the camera 14 may be arranged at any other suitable location as long as the camera 14 is pointed at (directly or indirectly) the instrument cluster 12 in order to visually capture operation thereof), and the visual signal provided by the second configurable visual indicator (0023, the fault notice generated by the fault condition notification module 52 may be any suitable fault notice, such as a fault notice conveyed to the driver on a redundant display screen in the passenger cabin, a fault warning chime generated by a speaker in the passenger cabin, and/or a fault message transmitted to the driver), are in corresponding states to thereby determine whether a malfunction in the sensor data stream has occurred (0022, the control module 50 compares operation of the instrument cluster 12 as captured in the images taken by the camera 14, with expected operation thereof, and previous operation thereof, to identify fault conditions). Justification for combining these disclosures is the same reasoning given in claim 1.
Regarding claim 8, Altman, Kaknjo and Murzyn disclose the non-transitory computer readable medium according to claim 7 as discussed supra. Furthermore, Altman discloses, the apparatus is configured to provide to the remotely operated vehicle at least one of a timestamp received from the remotely operated vehicle (Altman, 0073, Lines 1-7, synchronization or another type of coordination between the teleoperators and/or between the teleoperators and the vehicular AI component or vehicular processors and/or via another entity (e.g., a central AI component or layer) may be done, for example, via timestamps, IDs, packet IDs, a control channel, a feedback channel, a feedback loop, codes, control messages, transmission and reception of acknowledgement (ACK) packets or codes, transmission and reception of Negative Acknowledgement (NACK) packets or codes, collision-avoidance schemes, ordered list of priorities or prevailing rules, and/or other means), and an identity of the remotely operated vehicle, received in the apparatus from the remotely operated vehicle (Altman, 00123, Lines 21-27, control data or messaging data or feedback data or other meta-data (e.g., time-stamp / date-stamp, a unique identifier of the vehicle that this transmission is intended to, a unique identifier of the remote AI unit or the remote tele-operator that provided or transmitted the data, an indication whether the sent data is a strict command that must be followed or whether it is only an option or a suggestion or a recommendation that the recipient vehicle may or may not accept or reject or modify, or the like), and/or other types of data or signals). Justification for combining these disclosures is the same reasoning given in claim 1.
Claims 3, 6, and 9 are all rejected under 35 U.S.C. 103 as being unpatentable
over Altman (WO2019180700A1) in view of Kaknjo et al (Real-Time Video Latency Measurement between a Robot and Its Remote Control Station: Causes and Mitigation) further in view of Murzyn (US20190244446A1), further in view of Trank et al (US20220147042A1).
Regarding claim 3, Altman, Kaknjo, and Murzyn disclose the apparatus according to claim 1 as discussed supra. Furthermore, Kaknjo discloses, the indication of the digitization indicates either a state of the configurable visual indicator as rendered on the display based on the sensor data stream (See fig 1.) and (Kaknjo, Proposed Video Latency Measurement Methodology, Paragraph 9, Lines 7-14, to measure typical capture-to-display latency of a camera, two configurations (with node and integrated web camera) were tested (Tests 1 and 2 in a configuration with a screen and Test3 in a configuration with a PPS LED diode; see Table 2). The web camera used in the tests is an integrated webcam of a Dell Inspiron 7720, with 0.92M (megapixel) and maximum resolution of 1280x720 pixels). However neither Kaknjo nor Altman nor Murzyn explicitly disclose the use of a nonce rendered on the display.
Nevertheless, Trank who is in the same field of endeavor of real-time data and video streaming for remotely controlled vehicles discloses, a nonce as rendered on the display based on the sensor data stream (Trank, 0027, Lines 1-7, in another embodiment, the G-SRMCD may be programmed and configured further to use the GPGPU parallel processing resources provided by the embedded system-on-module to computationally embed a 64-bit binary number representing an NTP timestamp into the corner pixels of an image frame in the video stream by replacing the images original pixels with solid black or white pixels representing zeros or ones of the 64-bit binary number).
One of ordinary skill in the art prior to the effective filing date of the given invention
would have been motivated to combine the combination of Altman, Kaknjo, and Murzyn with Trank’s disclosure. This combination would enhance the system by using yet another means for detecting sensor malfunctions and latency. In achieving this an operator would know when latency or a malfunction has occurred and take steps to correct the issue.
Justification for combining the combination of Altman, Murzyn, and Kaknjo with Trank’s disclosure not only comes from the state of the art but from Trank (Trank, 0090, Lines 6-7, it will be obvious to persons skilled in the art to make various changes and modifications to the invention described herein).
Regarding claim 6, the applicant’s claim has similar limitations to claim 3 and
therefore is rejected for the same reasons set forth by the Examiner.
Regarding claim 9, the applicant’s claim has similar limitations to claim 3 and
therefore is rejected for the same reasons set forth by the Examiner.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Altman, Murzyn, and Kaknjo to incorporate Trank’s teachings. This would serve to enhance the invention by integrating another means for detecting sensor malfunctions and latency, utilizing already present components such as the display.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANE E DOUGLAS whose telephone number is (703)756-1417. The examiner can normally be reached Monday - Friday 7:30AM - 5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached on (571) 272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.E.D./Examiner, Art Unit 3665
/CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights