DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments/Amendments
The amendment filed October 7th, 2025 has been entered. Claims 1,2,4-14, and 16-20 are currently pending in the Application.
Applicant’s arguments with respect to the rejection of claims under 35 U.S.C 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-9, 11-16, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Publication No. 20170227965, to Decenzo et al. (hereinafter Decenzo).
Regarding claim 1, Decenzo teaches, A remote drone pilot system comprising: (See at least paragraph [0046] ”A system including a drone or unmanned vehicle configured to perform surveillance of a premises. The drone surveillance includes autonomous navigation and/or remote or optional piloting around the premises.” ).
a pilot endpoint system comprising a pilot endpoint and a controller connected to the pilot endpoint; (See at least paragraph [0113] ”The PAD system includes a remote client device that provides a user or remote operator access to the network. In various embodiments, an operator may interact with the PAD as well as any components of the network environment. For example, remote client device may be used to control the PAD.” ). Further, (See at least paragraph [0114] ”The PAD of an embodiment is configured to receive control signals from the remote client devices.” ). a control endpoint system (See at least paragraph [0125] ”FIG. 2 is a block diagram of a connected device system showing components of the connected device gateway at the premises and the session server in the cloud-based server environment, under an embodiment. ” ). comprising a control endpoint, (See at least paragraph [0125] ”the device manager is configured to manage communications with devices at the premises.” ). a signal adaptor connected to the control endpoint, (See at least paragraph [0125] ”The connected device gateway, which is also referred to herein as “Cloud Hub” in some embodiments, comprises a processor that includes or is coupled to one or more logical components that include a server connection manager, a device manager, a rules engine and a communication protocol manager (e.g., wired, wireless, etc.). The communication protocol manager is coupled to the transceivers or radios of the connected device gateway that are configured to communicate with the various connected devices at the premises. The server connection manager is configured to communicate with servers coupled to the WAN, while the device manager is configured to manage communications with devices at the premises..” ). and a transmitter connected to the signal adaptor; (See at least paragraph [0125] ”The communication protocol manager is coupled to the transceivers or radios of the connected device gateway that are configured to communicate with the various connected devices” ).
a remote bridge, including a server; (See at least paragraph [0129-0130] ”The server environment of an embodiment also includes a bridge server configured to provide an open communications interface between the smart devices and/or the connected devices and the security system. Any device can be a plugin or a subscriber to the bridge server, but the embodiment is not so limited.FIG. 3 is a block diagram of an example connected device system including a bridge server, under an embodiment. FIG. 4 is a block diagram of a system comprising a bridge server in communication with devices and an application server and gateway server, under an embodiment. With reference to these figures, the bridge server includes an event bus (e.g., bidirectional event bus) coupled to a set of device-specific plugins (e.g., location adapter, Nest adapter, etc.) that each corresponds to a particular device or type of device. Each plugin comprises code written to an API that corresponds to that device. Each plugin puts events for its corresponding device onto the event bus (e.g., Nest thermostat, change temperature, etc.) and receives data via the event bus. The plugins of an embodiment include but are not limited to an API plugin, a UI plugin, and a card UI.” ).
and a drone, wherein: the drone is configured to communicate with the transmitter of the control endpoint system to receive and send drone operating data to the control endpoint system and to communicate drone video data to the control endpoint system, (See at least paragraph [0065] ”The PAD of this embodiment is configured to communicate (e.g., exchange data, etc.) with components of the server environment via one or more of the broadband communicator, the cellular communicator, and the connected device gateway, as described in detail herein. The PAD of an alternative embodiment is configured to communicate directly with components of the server environment via one or more on-board communication components (e.g., cellular transceiver, etc.), but is not so limited.” ). Further, (See at least paragraph [0049] ”The system of an embodiment also includes drone stations configured to be bases and/or charging stations for one or more of drones. The system also includes a gateway and/or a server that is in communication with the drones via one or more data or communication networks (e.g., broadband, cellular, WiFi, etc.) as described in detail herein. The gateway/server receives signals from the drones, and the signals include sensor or surveillance data as well as drone data. The gateway of an embodiment includes the drone base station but is not so limited.” ).
the pilot endpoint is configured to communicate with the control endpoint only through the remote bridge, (See at least paragraph [0144] ”The system of this example includes an integrated or combined device application hosted on a remote client device to provide integrated access to the three connected devices. In an embodiment, the combined application communicates 601/602/603 with the corresponding device(s) through the bridge server, which communicates 601/602/603 directly with the connected device gateway at the premises. Additionally, the connected device gateway is configured to synchronize between connected devices at the local premises and connected devices at a remote premises.” ). Further, (See at least paragraph [0116] ”a thermostat in a home can connect to a corresponding cloud server and relay state information to the cloud service of a connected light switch at the same premises. In this way a state change in one device can trigger actions in other devices using the ‘cloud relay’ mechanism”).
the remote bridge is configured to pass all communications between the pilot endpoint and the control endpoint using an internet protocol, thereby allowing the drone operating data and drone video data to be communicated amongst the pilot endpoint, the control endpoint, and the drone in real-time, (See at least paragraph [0221] ”The integrated security system does not need any special rules at the customer premise and/or at the security system provider central station because the integrated security system makes outgoing connections using TCP over the standard HTTP and HTTPS ports. Provided outbound TCP connections are allowed then no special requirements on the firewalls are necessary.” ). Further, (See at least paragraph [0164] ”The Cloud Hub Firmware of an example embodiment includes but is not limited to the following components: SMA Client: an always-on (i.e., always-TCP-connected) SMA client, ” ).
and the remote bridge further includes an artificial intelligence system engine that is arranged to conduct real-time analysis on the drone video data and the drone operating data. (See at least paragraph [0054-0055] ”The drone is launched, and as the drone flies the programmed pattern the onboard sensors collect sensor data. The gateway/server and/or a central monitoring station receives signals from various sensors (e.g., camera, microphone, etc.) on the drones. The server applies analytics to the sensor data by comparing for instance, current images captured by the drone at a particular location to stored images previously taken at the same location to detect features in the current images that are different than expected based on the stored images.” ). Further, (See at least paragraph [0055] ”In the case where an unacceptable level of feature differences is detected that could be an indication of detected movement of a person or thing, within the field of view of the images, the analytics outputs an indication. In the case where the gateway/server performs the processing, the gateway/server notifies the monitoring station or where the monitor station is performing the processing, monitoring station alerts the operator. The processing upon detection of an unacceptable level of feature differences, modifies the drone navigation pattern. If intrusion is detected by the drone and/or other sensors within a facility, such as a window being opened or a glass break detector or contact switch being asserted on an intrusion detection system, the drone can be immediately guided (e.g., autonomously, by the operator, etc.) to that location providing similar surveillance until the incident is evaluated and handled.” ).
Regarding claim 2, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the remote bridge includes a broker module arranged to authenticate a connection between the remote bridge and each of the control endpoint and a remote endpoint. (See at least paragraph [0157] ”The Session Server of an embodiment is configured to use a gateway registry service to route incoming UDP packets from the CPE to the proper LWGW instance via a one to one mapping of CPE-unique IDs to site IDs. With the addition of the Cloud Hub, a second CPE-unique ID is used which is mapped to the same LWGW instance as the primary SMA client's CPE-unique ID. To accomplish this the Device Registry service is leveraged, and this registry maintains a mapping of CPE ID and device type to site ID. The session server is configured to use this Device Registry to properly route income packets but is not so limited.”). Further, (See at least paragraph [0167] ”The process for device installation and bootstrapping includes a first step that couples or connects the Cloud Hub to the Registry Gateway (e.g., via the pre-configured Registry Gateway URL) and retrieves its assigned siteID and the Credential Gateway URL. A second step includes the Cloud Hub retrieving its master key from the Credential Gateway using its siteID and Activation Key. The process comprises a third step in which the Cloud Hub retrieves Session Gateway Information from the Credential Gateway. At the end of the Bootstrap phase, the Cloud Hub has obtained its master key and its Session Gateway address from the iControl Gateway.”).
Regarding claim 4, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the controller includes a joystick, wherein each of the pilot endpoint and the control endpoint comprise a graphical user interface configured to display the drone video data and at least some of the drone operating data. (See at least paragraph [0048] ”The drone may be programmed in advance using weigh points and simultaneously be controlled with a radio control transmitter so that real time manipulation and control of the drone may occur based on particular operation of the drone that is desired.”).Further, (See at least paragraph [0046] ”A system including a drone or unmanned vehicle configured to perform surveillance of a premises. The drone surveillance includes autonomous navigation and/or remote or optional piloting around the premises. The drone includes a controller coupled to a plurality of sensors configured to collect drone data and security data at the premises, wherein the controller is configured to generate control data for the drone and the premises using the drone data and the security data. A remote device coupled to the drone includes a user interface configured to present the drone data, the security data, and/or the control data.”).
Regarding claim 5, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the drone includes a camera,wherein the drone video data from the drone is arranged to be communicated to a video device on the signal adaptor. (See at least paragraph [0286] ”The video routing engine 322 is responsible for delivering seamless video streams to the user with zero-configuration. Through a multi-step, staged approach the video routing engine uses a combination of UPnP port-forwarding, relay server routing and STUN/TURN peer-to-peer routing.”).
Regarding claim 6, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the pilot endpoint system further includes an audio device, wherein the audio device is connected to the pilot endpoint such that commands can be input into the pilot endpoint verbally via the audio device. (See at least paragraph [0214] ”2-way voice over IP and GSM.”). Further, (See at least paragraph [0116] ”In the conventional art these devices typically include an IP protocol connection to a server remote to the premise (‘in the cloud’). This server often provides remote access and control of the device through mobile apps running on phones or tablets. In some cases the connected devices communicate through this ‘cloud’ server to other devices through their own servers ‘in the cloud’. By way of example, a thermostat in a home can connect to a corresponding cloud server and relay state information to the cloud service of a connected light switch at the same premises. In this way a state change in one device can trigger actions in other devices using the ‘cloud relay’ mechanism. Further, high bandwidth media applications (e.g., video, voice, etc.) ”).
Regarding claim 7, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the pilot endpoint comprises a plurality of pilot endpoints, (See at least paragraph [0226] and [FIG.2] ”The iConnect servers 104 support a diverse collection of clients 120 ranging from mobile devices, to PCs, to in-home security devices, to a service provider's internal systems. Most clients 120 are used by end-users, but there are also a number of clients 120 that are used to operate the service.”). wherein a first of the plurality of pilot endpoints is arranged to provide commands to control the drone, wherein a second of the plurality of pilot endpoints is configured to only view the drone video data and the drone operating data, (See at least paragraph [0237] ”Users can only access the networks to which they have been granted permission.”). Further, (See at least paragraph [0230-0231] ”mobile device 206 (e.g., PDA, mobile phone, etc.) accessing the integrated security system Mobile Portal. This type of client 206 is used by end-users to view system status and perform operations on devices (e.g., turning on a lamp, arming a security panel, etc.) rather than for system configuration tasks such as adding a new device or user. PC or browser-based “widget” containers 208 that present integrated security system service content, as well as other third-party content, in simple, targeted ways (e.g. a widget that resides on a PC desktop and shows live video from a single in-home camera). “Widget” as used herein means applications or programs in the system. ”).
wherein the drone video data and the drone operating data is displayed identically on a graphical user interface display on each of the first and second of the plurality of pilot endpoints. (See at least paragraph [0260] ”An iControl Web Portal Application 272 runs on PC browsers and delivers the web-based interface to the integrated security system service. ”).
Regarding claim 8, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the signal adaptor is connected to the control endpoint by a first USB connection and the controller is connected to the pilot endpoint by a second USB connection. (See at least paragraph [0218] and [FIG.15 Item 408] ”USB 2.0 via ports on the gateway;”). Further, (See at least paragraph [0113] ”The remote client device includes at least one of personal computers, smart telephones, tablet computers, mobile devices, and other processing devices.”).
Regarding claim 9, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the drone, the remote bridge, the control endpoint system, and the pilot endpoint system are arranged to operate with a connection latency of about or less than 150 ms. (See at least paragraph [0282] ”provide short latency between device triggers and actions.”). Further, (See at least paragraph [0048] ”The drone may be programmed in advance using weigh points and simultaneously be controlled with a radio control transmitter so that real time manipulation and control of the drone may occur based on particular operation of the drone that is desired.”). Further, (See at least paragraph [0080] ”live streaming”).
Regarding claim 10, Decenzo discloses the claimed features of claim 1, and Decenzo further discloses, wherein the drone, the remote bridge, the control endpoint system, and the pilot endpoint system are arranged to operate with a connection latency of between about 100 ms and about 150 ms. (See at least paragraph [0282] ”provide short latency between device triggers and actions.”). Further, (See at least paragraph [0048] ”The drone may be programmed in advance using weigh points and simultaneously be controlled with a radio control transmitter so that real time manipulation and control of the drone may occur based on particular operation of the drone that is desired.”). Further, (See at least paragraph [0080] ”live streaming”).
Regarding claim 11, which is commensurate to claim 1, which Decenzo discloses, and claim 11 further includes the limitation wherein Decenzo discloses, wherein steps (b)-(i) occur with a connection latency of about or less than 150 ms. (See at least paragraph [0282] ”provide short latency between device triggers and actions.”). Further, (See at least paragraph [0048] ”The drone may be programmed in advance using weigh points and simultaneously be controlled with a radio control transmitter so that real time manipulation and control of the drone may occur based on particular operation of the drone that is desired.”). Further, (See at least paragraph [0080] ”live streaming”).
Regarding claim 12, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, wherein connecting the pilot endpoint to the control endpoint through the remote bridge includes authenticating the pilot endpoint and the control endpoint using a broker module of the server of the remote bridge. (See at least paragraph [0240] ”A Registry Manager 220 defines and manages users and networks. ”). Further, (See at least paragraph [0157] ”The Session Server of an embodiment is configured to use a gateway registry service to route incoming UDP packets from the CPE to the proper LWGW instance via a one to one mapping of CPE-unique IDs to site IDs. With the addition of the Cloud Hub, a second CPE-unique ID is used which is mapped to the same LWGW instance as the primary SMA client's CPE-unique ID. To accomplish this the Device Registry service is leveraged, and this registry maintains a mapping of CPE ID and device type to site ID. The session server is configured to use this Device Registry to properly route income packets but is not so limited.”). Further, (See at least paragraph [0167] ”The process for device installation and bootstrapping includes a first step that couples or connects the Cloud Hub to the Registry Gateway (e.g., via the pre-configured Registry Gateway URL) and retrieves its assigned siteID and the Credential Gateway URL. A second step includes the Cloud Hub retrieving its master key from the Credential Gateway using its siteID and Activation Key. The process comprises a third step in which the Cloud Hub retrieves Session Gateway Information from the Credential Gateway. At the end of the Bootstrap phase, the Cloud Hub has obtained its master key and its Session Gateway address from the iControl Gateway.”).
Regarding claim 13, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, wherein the remote bridge is a first remote bridge, and wherein the server includes a search module, and further comprising:determining using the search module a desirable second remote bridge from a plurality of second remote bridges; andtransferring the control endpoint and the pilot endpoint to be connected through the desirable second remote bridge. (See at least paragraph [0201-0202] ”When a UDP SMA message arrives at a session server, if the LWGW corresponding to the CPE-unique ID message is not already running on the given session server, then the session server initiates a new LWGW instance there, and if the corresponding LWGW is currently running on another session server, it will be gracefully shut down. In this way, the LWGW can move from one session server to another.”).
Regarding claim 14, Decenzo discloses the claimed features of claim 13, and Decenzo further discloses, wherein the search module determines the desirable second remote bridge based on one or more of shortest latency between the control endpoint and the pilot endpoint, shortest physical distance between the control endpoint and the pilot endpoint, or geographical proximity to the control endpoint or the pilot endpoint. (See at least paragraph [0157] ”The Session Server of an embodiment is configured to use a gateway registry service to route incoming UDP packets from the CPE to the proper LWGW instance via a one to one mapping of CPE-unique IDs to site IDs.”). Further, (See at least paragraph [0282] ”Though the automation rules are programmed and reside at the portal/server level, they are cached at the gateway level in order to provide short latency between device triggers and actions.”).
Regarding claim 16, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, wherein conducting the real-time analysis further includes classifying the at least one of the number of objects viewed, the frequency of objects viewed, or the particular targeted object to detect a component,sub-assembly, or system on the drone video data and create a report of a classification. (See at least paragraph [0057] “feature recognition”). Further, (See at least paragraph [0060] ”The drone captures images of the deviation and reports them as troubles during the premises disarmed period or alarm during the premises armed period.”).
Regarding claim 17, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, further comprising transmitting the command verbally to the pilot endpoint using an audio device including a microphone. (See at least paragraph [0214] ”2-way voice over IP and GSM.”). Further, (See at least paragraph [0116] ”In the conventional art these devices typically include an IP protocol connection to a server remote to the premise (‘in the cloud’). This server often provides remote access and control of the device through mobile apps running on phones or tablets. In some cases the connected devices communicate through this ‘cloud’ server to other devices through their own servers ‘in the cloud’. By way of example, a thermostat in a home can connect to a corresponding cloud server and relay state information to the cloud service of a connected light switch at the same premises. In this way a state change in one device can trigger actions in other devices using the ‘cloud relay’ mechanism. Further, high bandwidth media applications (e.g., video, voice, etc.) ”).
Regarding claim 18, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, wherein the pilot endpoint is a first pilot endpoint, and further comprising connecting a second pilot endpoint to the control endpoint, wherein the first pilot endpoint communicates commands to pilot the drone and the second pilot endpoint communicates commands to control the operation of a camera on the drone providing the drone video data. (See at least paragraph [0057] ”Alternative techniques include an operator at the monitoring facility manually controlling the drone to a specific location or having the drone hover at the location.”). Further, (See at least paragraph [0283] ”DeviceConnect 310 includes definitions of all supported devices (e.g., cameras, security panels, sensors, etc.) using a standardized plug-in architecture. The DeviceConnect module 310 offers an interface that can be used to quickly add support for any new device as well as enabling interoperability between devices that use different technologies/protocols. For common device types, pre-defined sub-modules have been defined, making supporting new devices of these types even easier. SensorConnect 312 is provided for adding new sensors, CameraConnect 316 for adding IP cameras, and PanelConnect 314 for adding home security panels.”).
Regarding claim 19, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, further comprising:multiplexing the drone video data communicated from the control endpoint using the remote bridge; and displaying the multiplexed drone video data on a graphical user interface of each of a plurality of additional pilot endpoints. (See at least paragraph [0265] ”A Content Manager Application Component 276 delivers content to a variety of clients.”).
Regarding claim 20, Decenzo discloses the claimed features of claim 11, and Decenzo further discloses, wherein the drone is an unmanned aerial vehicle (See at least paragraph [0070] “The PAD of an embodiment includes, but is not limited to, an unmanned aerial vehicles (UAV), a fixed-wing UAVs, rotorcraft, rovers, rolling robots, walking robots, crawling robots, surface vehicles, submersibles, boats, and hovercraft, for example. ”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Wesam Almadhrhi whose telephone number is (571) 270-3844. The examiner can normally be reached on 7:30 AM - 5PM Mon-Fri Eastern Alt Fri.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached on (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESAM NMN ALMADHRHI/Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666