Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims of U.S. Patent No. 12/271,192. Although the claims at issue are not identical, they are not patentably distinct from each other because they have claimed similar subject matter, i.e. GPU parallel processing and reduced bit rate messaging.
Instant Application
US App. No.: 12/271,192
Regarding claim 1, instant application claims a method, comprising:
providing a remote monitoring control device including a processor;
configuring the remote monitoring control device to communicate with a remote monitoring control center; configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit;
transferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; and
wherein the remote monitoring control device uses an accelerated video encoder that encodes a video stream into a reduced bitrate bitstream with one of several video codecs using GPU parallel processing on an embedded system-on-module.
using the remote monitoring control device to format an encoded bitstream of the reduced bitrate bitstream into RTP or SRTP messages;
2. A remote monitoring and control system capable of carrying out the method recited in Claim 1.
3. The remote monitoring control system of claim 2, wherein when the camera sensor is greater than 30 cm from the remote control module of the remote monitoring control device, the remote monitoring control device uses FPD-Link III serializer and deserializer modules to transport high bandwidth data of the near real-time video stream from the camera sensor to the remote monitoring control device.
4. The remote monitoring control system of claim 3, wherein the FPD-Link III serializer and deserializer modules are GMSL or GMSL2 serializer and deserializer modules.
5. The remote monitoring control system of claim 2, a communication protocol is used to establish encrypted audio and video streams and encrypted command and response data with the remote monitoring control center, and wherein the communication protocol is used as a transport for WebRTC "RTCPeerConnection" methods in order to negotiate and establish encrypted SRTP streams transporting audio and video, and command and response data via a WebRTC data channel.
6. The remote monitoring control system of claim 2, wherein:the autonomous unit is one of a plurality of autonomous units; andthe remote monitoring control center is configured to communicate with a remote- control operator via an WebRTC-enabled browser of the remote-control operator, the WebRTC-enabled browser configured to display multiple video streams from the plurality of autonomous units; andwherein the WebRTC data channel is configured to exchange data between the remote monitoring control device and the remote monitoring control center, the data including control instructions to the autonomous unit that is a vehicle, a robot, or a drone.
7. A method, comprising:providing a remote monitoring control device including a processor;configuring the remote monitoring control device to communicate with a remote monitoring control center;configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit; andtransferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; andusing the remote monitoring control device to establish a secure websocket connection to an internet cloud-based server to perform a WebRTC session negotiation with the remote monitoring control center, and to establish RTP/UDP, SRTP/UDP, andSCTP data channel paths for the exchange of camera image streams and command and control data with the remote monitoring control center.
8. The method of claim 7, wherein transferring the processed data stream from the remote monitoring control device to the remote monitoring control center includes transferring a high bandwidth audio, video, and data streams via a commercial public LTE/4G/5G network, a private LTE network, or a Wi-Fi network, and wherein the remote monitoring control device is programmed to read camera image data of the processed data stream to place a camera image frame bitstream directly into a zero-copy EGL context, and wherein the remote monitoring control device provides Bayer-to-YUV conversion using a hardware-based image signal processor of an embedded system-on-module.
9. The method of claim 7, wherein the remote monitoring control device uses a General- Purpose Computing on GPUs parallel computing resources environment to computationally combine image frame bitstreams from the at least one camera, which includes two or more cameras, into a single seamless image stream with a panorama, the single seamless image stream being larger than a single camera image stream and the panorama being larger than a single camera panorama.
10. The method of claim 7, wherein the remote monitoring control device uses an accelerated video encoder that encodes a video stream into a reduced bitrate bitstream with one of several video codecs using GPU parallel processing on an embedded system-on-module.
11. The method of claim 7, further comprising using the remote monitoring control device to format an encoded bitstream of the reduced bitrate bitstream into RTP or SRTP messages, and using the remote monitoring control device to negotiate a transmission of RTP/SRTP audio and video streams of the encoded bitstream to the remote monitoring control center using ICE protocols.
12. The method of claim 7, further comprising using the remote monitoring control device to establish a two-way data exchange on a data channel of the processed data stream to exchange timestamped command, control, and telemetry feedback messages between the remote monitoring control device and the remote monitoring control center.
13. The method of claim 7, further comprising using the remote monitoring control device to support a WebRTC data channel of the processed data stream to exchange status and alarm messages between the remote monitoring control device and the remote monitoring control center that indicate a loss of incoming signal, a late arrival of command messages, or a safety alert.
14. The method of claim 7, wherein the remote monitoring control device uses a plurality of timestamps contained in a plurality of messages carried by a WebRTC data channel of the processed data stream to measure a delay between the remote monitoring control device and the remote monitoring control center for safe teleoperation of the autonomous unit that is a vehicle, a robot, or a drone.
15. The method of claim 7, further comprising transferring a data of a passenger of the autonomous unit via a human interface from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
16. The method of claim 7, further comprising transferring a data of a third party external to the autonomous unit via an external interface, which is disposed at a surface or in a surface portion of the autonomous unit, from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
17. A method, comprising:providing a remote monitoring control device including a processor;configuring the remote monitoring control device to communicate with a remote monitoring control center;configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit; andtransferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; andtransferring a data of a third party external to the autonomous unit via an external interface, which is disposed at a surface or in a surface portion of the autonomous unit, from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
18. The method of claim 17, wherein the remote monitoring control device uses an accelerated video encoder that encodes a video stream into a reduced bitrate bitstream with one of several video codecs using GPU parallel processing on an embedded system-on-module.
19. The method of claim 17, further comprising:using the remote monitoring control device to format an encoded bitstream of the reduced bitrate bitstream into RTP or SRTP messages;using the remote monitoring control device to negotiate a transmission of RTP/SRTP audio and video streams of the encoded bitstream to the remote monitoring control center using ICE protocols; andusing the remote monitoring control device to establish a two-way data exchange on a data channel of the processed data stream to exchange timestamped command, control, and telemetry feedback messages between the remote monitoring control device and the remote monitoring control center.
20. The method of claim 17, further comprising using the remote monitoring control device to support a WebRTC data channel of the processed data stream to exchange status and alarm messages between the remote monitoring control device and the remote monitoring control center that indicate a loss of incoming signal, a late arrival of command messages, or a safety alert.
21. The method of claim 17, further comprising transferring a data of a passenger of the autonomous unit via a human interface from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
1. A method, comprising:
providing a remote monitoring control device including a processor;
configuring the remote monitoring control device to communicate with a remote monitoring control center;
configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit;
transferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; and
wherein the remote monitoring control device is programmed to read camera image data of the processed data stream to place a camera image frame bitstream directly into a zero-copy EGL context.
4. The method of claim 1, wherein the remote monitoring control device uses a General-Purpose Computing on GPUs parallel computing resources environment to computationally combine image frame bitstreams from the at least one camera, which includes two or more cameras, into a single seamless image stream with a panorama, the single seamless image stream being larger than a single camera image stream and the panorama being larger than a single camera panorama.
6. The method of claim 5, further comprising using the remote monitoring control device to format an encoded bitstream of the reduced bitrate bitstream into RTP or SRTP messages.
2. The method of claim 1, wherein transferring the processed data stream from the remote monitoring control device to the remote monitoring control center includes transferring a high bandwidth audio, video, and data streams via a commercial public LTE/4G/5G network, a private LTE network, or a Wi-Fi network.
3. The method of claim 1, wherein the remote monitoring control device provides Bayer-to-YUV conversion using a hardware-based image signal processor of an embedded system-on-module.
5. The method of claim 1, wherein the remote monitoring control device uses an accelerated video encoder that encodes a video stream into a reduced bitrate bitstream with one of several video codecs using GPU parallel processing on an embedded system-on-module.
7. The method of claim 6, further comprising using the remote monitoring control device to negotiate a transmission of RTP/SRTP audio and video streams of the encoded bitstream to the remote monitoring control center using ICE protocols.
8. The method of claim 1, further comprising using the remote monitoring control device to establish a secure websocket connection to an internet cloud-based server to perform a WebRTC session negotiation with the remote monitoring control center, and to establish RTP/UDP, SRTP/UDP, and SCTP data channel paths for the exchange of camera image streams and command and control data with the remote monitoring control center.
9. The method of claim 1, further comprising using the remote monitoring control device to establish a two-way data exchange on a data channel of the processed data stream to exchange timestamped command, control, and telemetry feedback messages between the remote monitoring control device and the remote monitoring control center.
10. The method of claim 1, further comprising using the remote monitoring control device to remotely set up controllable camera and video controls of the camera sensor.
11. The method of claim 1, further comprising transferring a data of a passenger of the autonomous unit via a human interface from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
12. The method of claim 1, further comprising transferring a data of a third party external to the autonomous unit via an external interface, which is disposed at a surface or in a surface portion of the autonomous unit, from the remote monitoring control device to the remote monitoring control center, the data instructing the remote monitoring control center to increase monitoring of the autonomous unit or assume control of the autonomous unit.
13. A remote monitoring and control system capable of carrying out the method recited in claim 1.
14. A method, comprising:
providing a remote monitoring control device including a processor;
configuring the remote monitoring control device to communicate with a remote monitoring control center;
configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit;
transferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; and
using the remote monitoring control device to support a WebRTC data channel of the processed data stream to exchange status and alarm messages between the remote monitoring control device and the remote monitoring control center that indicate a loss of incoming signal, a late arrival of command messages, or a safety alert.
15. A remote monitoring and control system capable of performing the method of claim 14.
16. The method of claim 14, wherein the remote monitoring control device is one of a plurality of remote monitoring control devices, and the collection device is one of a plurality of collection devices, each remote monitoring control device configured to communicate with at least one collection device from the plurality of collection devices.
17. A method, comprising:
providing a remote monitoring control device including a processor;
configuring the remote monitoring control device to communicate with a remote monitoring control center;
configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit;
transferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; and
wherein the remote monitoring control device uses a plurality of timestamps contained in a plurality of messages carried by a WebRTC data channel of the processed data stream to measure a delay between the remote monitoring control device and the remote monitoring control center for safe teleoperation of the autonomous unit that is a vehicle, a robot, or a drone.
18. A remote monitoring and control system capable of performing the method of claim 17.
19. The method of claim 17, wherein the remote monitoring control device is one of a plurality of remote monitoring control devices, and the collection device is one of a plurality of collection devices, each remote monitoring control device configured to communicate with at least one collection device from the plurality of collection devices.
Allowable Subject Matter
The following is an examiner’s statement of reasons for allowance:
US 2017/0192423 A1 to Cruise Automation, Inc. discloses systems and methods are provided for remotely assisting an autonomous vehicle. The method includes: aggregating sensor data from the autonomous vehicle; identifying an assistance-desired scenario; generating an assistance request based on the sensor data; transmitting the assistance request to a remote assistance interface; and receiving and processing a response to the assistance request. The remote assistance interface includes a remote assistance interface that is used in generating the response to the assistance request.
US 2017/0090476 A1 to Uber Technologies, Inc. discloses autonomous vehicle comprising:
multiple sensors;
a primary control sub-system to control the vehicle, including to (i) receive sensor input from at least some of the multiple sensors of the vehicle, (ii) determine, from the sensor input, a vehicle action or vehicle, and (iii) signal one or more control parameters corresponding to the determined action or state to a corresponding one or more vehicle interfaces of the vehicle; and
an auxiliary control unit, operating independently of the primary control system, (i) to monitor for and detect a specific set of conditions or events from one or more of the multiple sensors, and (ii) to generate a vehicle response output in response to monitoring for the specific set of conditions or events.
However, none of the cited prior art disclose “providing a remote monitoring control device including a processor; configuring the remote monitoring control device to communicate with a remote monitoring control center; configuring the remote monitoring control device to communicate with a collection device, which includes a camera sensor including at least one of a sensor or at least one USB or Ethernet camera, disposed at an autonomous unit; and transferring a processed data stream including a near real-time video stream of the collection device from the remote monitoring control device to the remote monitoring control center; and using the remote monitoring control device to format an encoded bitstream of the reduced bitrate bitstream into RTP or SRTP messages; and wherein the remote monitoring control device uses an accelerated video encoder that encodes a video stream into a reduced bitrate bitstream with one of several video codecs using GPU parallel processing on an embedded system-on-module.”
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK F HUANG whose telephone number is (571)272-0701. The examiner can normally be reached Monday-Friday, 8:30 am - 6:00 pm (Eastern Time), Federal Alternative First Friday Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at (571)272-2988.. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK F HUANG/ Primary Examiner, Art Unit 2485
Prosecution Insights