DETAILED ACTION
Status of Claims:
Claims 1 – 7, 9 – 15, 17, and 19 – 23 are pending.
Claims 1 – 3, 7, 9 – 11, 15, and 19 – 20 are amended.
Claims 8, 16, and 18 are cancelled.
Claims 21 – 23 are new.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments in the amendments, filed 05/11/2026, have been fully considered and they are persuasive. A new ground(s) of rejection is presented based on the Applicant’s amendments.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 – 2, 9 – 10, 17, and 21 – 23 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Cardillo (US 12266249).
As per claim 1, a computer system comprising:
a processing system including one or more processing devices and one or more computer storage devices, the processing system processing computer program instructions that configure the processing system to implement:
an application processing data and streaming media data (The video may be utilized to send camera data channel (streaming media data) generated by the one or more cameras 112 to the premises management application ... The data channel may be configured to exchange any type of data (application processing data) … between the premises management application 113 and the one or more cameras 112, See Col. 6, Lines 15 - 42);
a setup protocol client in communication with the application and responsive to a request from the application to establish a real-time streaming connection using a real-time streaming protocol with another device over a computer network (The premises management application 113 and/or the user device 106 may be configured to cause a connection to be established between the premises management application 113 and the one or more cameras 112 based on determining the first event. The connection may immediately be established based on determining the first event … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 5 - 42), the real-time streaming connection including one or more streaming media channels and one or more bidirectional data channels within the real-time streaming connection (The connection may comprise a video channel, a data channel, a combination thereof (e.g., combined as one channel, or via two separate channels) … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42); and
a real-time streaming client in communication with the application and configured to:
receive the streaming media data from the application and transmit the streaming media data to the other device over the one or more streaming media channels within the real-time streaming connection (The video channel (e.g., or a streaming channel, a media channel, an audio channel, a combined video and audio channel) may be configured to exchange audio, video, and/or image data between the premises management application 113 and the one or more cameras 112, See Col. 6, Lines 15 - 42),
receive the data from the application and to communicate the data to the other device over the one or more bidirectional data channels within the real-time streaming connection (The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112. The data channel may be unidirectional or bidirectional (e.g., both the premises management application 113 and the one or more cameras 112 can send and receive data in the same data channel), See Col. 6, Lines 15 - 42), and
receive other data from the other device over the one or more bidirectional data channels within the real-time streaming connection and communicate the other data to the application (The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112. The data channel may be unidirectional or bidirectional (e.g., both the premises management application 113 and the one or more cameras 112 can send and receive data in the same data channel), See Col. 6, Lines 15 - 42).
As per claim 2, the computer system of claim 1, wherein the real-time streaming client transmits the streaming media data over the one or more streaming media channels while communicating data to and from the application and the other device over the one or more bidirectional data channels (The video channel (e.g., or a streaming channel, a media channel, an audio channel, a combined video and audio channel) may be configured to exchange audio, video, and/or image data between the premises management application 113 and the one or more cameras 112 … The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112 … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42).
As per claim 9, a computer system comprising:
a processing system including one or more processing devices and one or more computer storage devices, the processing system processing computer program instructions that configure the processing system to implement:
an application configured to process data and streaming media data (The video may be utilized to send camera data channel (streaming media data) generated by the one or more cameras 112 to the premises management application ... The data channel may be configured to exchange any type of data (application processing data) … between the premises management application 113 and the one or more cameras 112, See Col. 6, Lines 15 - 42);
a setup protocol client in communication with the application and responsive to one or more requests from the application to establish a real-time streaming connection using a real-time streaming protocol with another device over a computer network (The premises management application 113 and/or the user device 106 may be configured to cause a connection to be established between the premises management application 113 and the one or more cameras 112 based on determining the first event. The connection may immediately be established based on determining the first event … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 5 - 42), the one or more requests including data specifying one or more streaming media channels and one or more bidirectional data channels to be created within the real-time streaming connection, wherein the setup protocol client is configured to create the specified one or more streaming media channels and the one or more bidirectional data channels within the established real-time streaming connection (The premises management application 113 and/or the user device 106 may be configured to cause a connection to be established between the premises management application 113 and the one or more cameras 112 based on determining the first event. The connection may immediately be established based on determining the first event … The connection may comprise a video channel, a data channel, a combination thereof (e.g., combined as one channel, or via two separate channels) … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42); and
a real-time streaming client in communication with the application and configured to:
receive other data from the other device over the bidirectional data channel of the real-time streaming connection and to communicate the other data to the application (The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112. The data channel may be unidirectional or bidirectional (e.g., both the premises management application 113 and the one or more cameras 112 can send and receive data in the same data channel), See Col. 6, Lines 15 - 42);
receive the data from the application and to communicate the data to the other device over the one or more bidirectional data channels within the real-time streaming connection (The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112. The data channel may be unidirectional or bidirectional (e.g., both the premises management application 113 and the one or more cameras 112 can send and receive data in the same data channel), See Col. 6, Lines 15 - 42), and
receive the streaming media data from the other device over the one or more streaming media channels within the real-time streaming connection and transmit the streaming media data to the application (The video channel (e.g., or a streaming channel, a media channel, an audio channel, a combined video and audio channel) may be configured to exchange audio, video, and/or image data between the premises management application 113 and the one or more cameras 112, See Col. 6, Lines 15 - 42).
As per claim 10, the computer system of claim 9, wherein the real- time streaming client receives the streaming media data over the one or more streaming media channels while communicating data to and from the application and the other device over the one or more bidirectional data channels (The video channel (e.g., or a streaming channel, a media channel, an audio channel, a combined video and audio channel) may be configured to exchange audio, video, and/or image data between the premises management application 113 and the one or more cameras 112 … The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112 … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42).
As per claim 17, a computer system comprising:
a processing system including one or more processing devices and one or more computer storage devices, the processing system processing computer program instructions that configure the processing system to implement:
a real-time streaming server instance configured to:
establish a respective real-time streaming connection with for each device in a plurality of devices using a real-time streaming protocol over a computer network (The user device 106 may each be implemented as one or more computing devices. Any device disclosed herein may be implemented using one or more computing nodes, such as virtual machines, executed on a single device and/or multiple devices, See Col. 3, Lines 3 - 16 … At least one of the server(s) device 102 may be configured to cause the connection to be established between the premises management application 113 and the one or more cameras 112. For example, the server device 102 may be configured to cause the connection to be established between the premises management application 113 and the one or more cameras 112 based on receiving the data indicating the first event at the user device 106, See Col. 10, Line 59 – Col. 11, Line 15), wherein the respective real-time streaming connection with each device includes one or more streaming media channels and one or more bidirectional data channels within the respective real-time streaming connection (The connection may comprise a video channel, a data channel, a combination thereof (e.g., combined as one channel, or via two separate channels) … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42),
communicate data with each device over the one or more bidirectional data channels within the respective real-time streaming connection, and stream streaming media data to each device over the one or more streaming media channels within the respective real-time streaming connection (The video channel (e.g., or a streaming channel, a media channel, an audio channel, a combined video and audio channel) may be configured to exchange audio, video, and/or image data between the premises management application 113 and the one or more cameras 112 … The data channel may be configured to exchange any type of data, including data that is not audio data, video data, or image data between the premises management application 113 and the one or more cameras 112 … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42); and
a setup protocol server instance responsive to requests from the plurality of devices to establish a real-time streaming connection with the real-time streaming server instance (The user device 106 may each be implemented as one or more computing devices. Any device disclosed herein may be implemented using one or more computing nodes, such as virtual machines, executed on a single device and/or multiple devices, See Col. 3, Lines 3 - 16 … At least one of the server(s) device 102 may be configured to cause the connection to be established between the premises management application 113 and the one or more cameras 112. For example, the server device 102 may be configured to cause the connection to be established between the premises management application 113 and the one or more cameras 112 based on receiving the data indicating the first event at the user device 106, See Col. 10, Line 59 – Col. 11, Line 15), each request from a device including data specifying one or more streaming media channels and one or more bidirectional data channels within a requested real-time streaming connection (The connection may comprise a video channel, a data channel, a combination thereof (e.g., combined as one channel, or via two separate channels) … The connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42), wherein the setup protocol server instance, in response to a request from a device, issues a request to the real-time streaming server instance to establish the requested real-time streaming connection, and responds to the device with connection information for the device to complete the requested real-time streaming connection (Causing the connection to be established between the premises management application 113 and the one or more cameras 112 may comprise sending data for establishing a connection between the one or more cameras 112 and the user device 106. The data may be sent, for example, to one or more of the gateway device 104, the one or more cameras 112, or the user device 106, See Col. 10, Line 59 – Col. 11, Line 15).
As per claim 21, the computer system of claim 1, wherein the streaming media data comprises live, real-time streaming video and audio data (For example, the video data may include a live video stream and/or video clips captured by the camera devices 112. The audio data may include a live audio stream and/or audio clips captured by the camera devices 112, such as a live audio stream and/or audio clips captured by one or more microphone(s) of the camera devices 112, See Col. 3, Line 66 – Col. 4, Line 16).
As per claim 22, the computer system of claim 9, wherein the streaming media data comprises live, real-time streaming video and audio data (For example, the video data may include a live video stream and/or video clips captured by the camera devices 112. The audio data may include a live audio stream and/or audio clips captured by the camera devices 112, such as a live audio stream and/or audio clips captured by one or more microphone(s) of the camera devices 112, See Col. 3, Line 66 – Col. 4, Line 16).
As per claim 23, the computer system of claim 17, wherein the streaming media data comprises live, real-time streaming video and audio data (For example, the video data may include a live video stream and/or video clips captured by the camera devices 112. The audio data may include a live audio stream and/or audio clips captured by the camera devices 112, such as a live audio stream and/or audio clips captured by one or more microphone(s) of the camera devices 112, See Col. 3, Line 66 – Col. 4, Line 16).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 3 – 6, 11 – 14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cardillo (US 12266249) and in view of Murillo (NPL / WebRTC-HTTP ingest protocol (WHIP) and WebRTC-HTTP Egress protocol (WHEP)).
As per claim 3, Cardillo discloses all limitations of claim 1.
Cardillo discloses:
the computer system of claim 1, wherein the setup protocol client comprises a WebRTC … client and the real-time streaming client comprises a WebRTC client supporting the WebRTC protocol (he connection may comprise, for example, a peer-to-peer connection, such as a Web Real-Time Communication (WebRTC) connection, See Col. 6, Lines 15 - 42).
Cardillo however does not expressly disclose:
a WebRTC for HTTP Ingest Protocol (WHIP) client.
Murillo discloses:
… a WebRTC for HTTP Ingest Protocol (WHIP) client (WHIP client, See Sections 2 and 3) …
It would have been obvious to an artisan of ordinary skill in the art before the Applicant's effective filing date of the claimed invention to combine Murillo’s teaching of a WHIP client, along with a setup protocol client comprising a WebRTC client to improve Cardillo’s system. Cardillo and Murillo both disclose systems WebRTC streaming protocols. Murillo’s system includes a WHIP client that encodes and delivers media to remote media servers. The combination is an improvement upon the existing system because a setup protocol client can comprise a WebRTC client, as taught by Cardillo, where the setup protocol client can be a WHIP client, as taught by Murillo, to allow a simpler protocol for streaming content in real-time using WebRTC.
As per claim 4, the computer system of claim 3, wherein the computer system comprises a WebRTC streaming server housing a WebRTC server instance and a WHIP/WHEP server instance (Murillo, WebRTC Media Server housing a server instance for the ingress and egress of streaming data to WHIP/WHEP clients, See Sections 2 and 3).
As per claim 5, the computer system of claim 3, wherein the computer system comprises a stream manager for a cluster of server computers comprising a plurality of WebRTC server computers, wherein each WebRTC server computer includes a respective WebRTC server instance, and wherein the stream manager includes a WHIP/WHEP server instance, and is configured to, in response to a request to the WHIP/WHEP server instance, select a WebRTC server computer for supplying the requested WebRTC connection and to communicate with the WebRTC server instance of the selected WebRTC server computer (Murillo, A plurality of WHIP/WHEP endpoints and WebRTC Media Servers on separate servers are provided and managed to deliver requests to each of said endpoint and server, See WHIP Section 4.3 / WHEP Section 4.6).
As per claim 6, the computer system of claim 3, wherein the computer system comprises a load balancer for a collection of server computers comprising a plurality of WebRTC server computers, each WebRTC server computer including a respective WebRTC server instance and an associated WHIP/WHEP server instance (Murillo, A load balancer is provided to balance the load between a plurality of WHIP/WHEP endpoints and WebRTC Media Servers, See WHIP Section 4.3 / WHEP Section 4.6), and wherein the load balancer is configured to, in response to a request, select a WebRTC server computer for supplying the requested WebRTC connection and to communicate with the WHIP/WHEP server instance of the selected WebRTC server computer, and wherein the WHIP/WHEP server instance of the selected WebRTC server computer is configured to respond to the setup protocol client (Murillo, The load balancer, in response to requests, selects a WebRTC Media Server to supply a WebRTC connection to wherein the WebRTC Media Server responds to the WHIP/WHEP client with the requested media, See WHIP Section 4.3 / WHEP Section 4.6).
As per claim 11, the computer system of claim 9, wherein the setup protocol client comprises a WebRTC for HTTP Egress Protocol (WHEP) client (Murillo, WHEP client, See Sections 2 and 3) and the real-time streaming client comprises a WebRTC client supporting the WebRTC protocol (Bergstrom, Client Device 200 comprises WebRTC 222 supporting the WebRTC protocol, See ¶101 – 102, Fig. 2).
As per claim 12, the computer system of claim 11, wherein the computer system comprises a WebRTC streaming server housing a WebRTC server instance and a WHIP/WHEP server instance (Murillo, WebRTC Media Server housing a server instance for the ingress and egress of streaming data to WHIP/WHEP clients, See Sections 2 and 3).
As per claim 13, the computer system of claim 11, wherein the computer system comprises a stream manager for a plurality of server computers comprising a plurality of WebRTC server computers, each WebRTC server computer including a respective WebRTC server instance, and wherein the stream manager includes a WHIP/WHEP server instance, and wherein the stream manager is configured to, in response to a request, select a WebRTC server computer for supplying the requested WebRTC connection and to communicate with the WebRTC server instance of the selected WebRTC server computer (Murillo, A plurality of WHIP/WHEP endpoints and WebRTC Media Servers on separate servers are provided and managed to deliver requests to each of said endpoint and server, See WHIP Section 4.3 / WHEP Section 4.6).
As per claim 14, the computer system of claim 11, wherein the computer system comprises a load balancer for a plurality of server computers including a plurality of WebRTC server computers, each WebRTC server computer including a respective WebRTC server instance and an associated WHIP/WHEP server instance (Murillo, A load balancer is provided to balance the load between a plurality of WHIP/WHEP endpoints and WebRTC Media Servers, See WHIP Section 4.3 / WHEP Section 4.6), and wherein the load balancer is configured to, in response to a request, select a WebRTC server computer for supplying the requested WebRTC connection and to communicate with the WHIP/WHEP server instance of the selected WebRTC server computer, and wherein the WHIP/WHEP server instance of the selected WebRTC server computer responds to the setup protocol client (Murillo, The load balancer, in response to requests, selects a WebRTC Media Server to supply a WebRTC connection to wherein the WebRTC Media Server responds to the WHIP/WHEP client with the requested media, See WHIP Section 4.3 / WHEP Section 4.6).
As per claim 19, the computer system of claim 18, wherein the setup protocol server instance comprises a WebRTC for HTTP Ingress or Egress Protocol (WHIP or WHEP) server instance and the real-time streaming server instance comprises a WebRTC server instance supporting the WebRTC protocol (Murillo, The setup protocol server instance comprises a WHIP or WHEP endpoint and the real-time streaming server instance supports WebRTC, See Sections 2 and 3).
Claim(s) 7, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cardillo (US 12266249) and in view of Kamaraju (US 12265750).
As per claim 7, Cardillo discloses all limitations of claim 1.
Cardillo however does not expressly disclose:
wherein the data communicated on the one or more bidirectional data channels within the real-time streaming connection includes metadata to be synchronized with the streaming media data transmitted on the one or more streaming media channels within the real-time streaming connection.
Kamaraju discloses:
the computer system of claim 1, wherein the data communicated on the one or more bidirectional data channels within the real-time streaming connection includes metadata to be synchronized with the streaming media data transmitted on the one or more streaming media channels within the real-time streaming connection (The shared experience may include … bidirectional video stream transfers where the service provider device provides a video stream and AR data back to the consumer device, and/or multi-peer exchanges where various endpoints to a communication session exchange video streams and/or AR data to other peers … WebRTC provides a framework for engaging in communication along with data sharing, such as over data channels established between the endpoints (peers) of a communication session. WebRTC may therefore be used to establish and provide an AR-based video conference call between two peers, e.g., a customer and a technical support agent, as well as to provide data channels for transferring data between peers. As discussed above, a desired number (all, or a selected subset) of frames in a video stream may be encoded with essential meta-data 302, 352, and be cross-referenced to synchronize with out-of-band data associated with the video stream, See Col. 14, Lines 35 - 53).
It would have been obvious to an artisan of ordinary skill in the art before the Applicant's effective filing date of the claimed invention to combine Kamaraju’s teaching of synchronizing metadata with streaming media, along with a transmitting and receiving via a streaming media channel and a bidirectional data channel to improve Cardillo’s system. Cardillo and Kamaraju both disclose systems WebRTC streaming protocols. Kamaraju’s system includes a WebRTC framework that includes media sharing along with data channels to provide synchronization of metadata with video streams. The combination is an improvement upon the existing system because transmitting and receiving via a streaming media channel and a bidirectional data channel can be achieved, as taught by Cardillo, where the streaming media can be synchronized with metadata via the channels, as taught by Kamaraju, to allow an improved system for transmitting streaming media over a media channel while simultaneously transmitting application data via a data channel to a client device.
As per claim 15, the computer system of claim 10, wherein the other data received on the one or more bidirectional data channels includes metadata to be synchronized with the streaming media data receives on the one or more streaming media channels, and wherein the application is configured to synchronize the metadata with the streaming media data during playback (Kamaraju, The shared experience may include … bidirectional video stream transfers where the service provider device provides a video stream and AR data back to the consumer device, and/or multi-peer exchanges where various endpoints to a communication session exchange video streams and/or AR data to other peers … WebRTC provides a framework for engaging in communication along with data sharing, such as over data channels established between the endpoints (peers) of a communication session. WebRTC may therefore be used to establish and provide an AR-based video conference call between two peers, e.g., a customer and a technical support agent, as well as to provide data channels for transferring data between peers. As discussed above, a desired number (all, or a selected subset) of frames in a video stream may be encoded with essential meta-data 302, 352, and be cross-referenced to synchronize with out-of-band data associated with the video stream, See Col. 14, Lines 35 - 53).
As per claim 20, the computer system of claim 17, wherein the data on the bidirectional data channel includes metadata to be synchronized with the live real-time streaming media data (Kamaraju, The shared experience may include … bidirectional video stream transfers where the service provider device provides a video stream and AR data back to the consumer device, and/or multi-peer exchanges where various endpoints to a communication session exchange video streams and/or AR data to other peers … WebRTC provides a framework for engaging in communication along with data sharing, such as over data channels established between the endpoints (peers) of a communication session. WebRTC may therefore be used to establish and provide an AR-based video conference call between two peers, e.g., a customer and a technical support agent, as well as to provide data channels for transferring data between peers. As discussed above, a desired number (all, or a selected subset) of frames in a video stream may be encoded with essential meta-data 302, 352, and be cross-referenced to synchronize with out-of-band data associated with the video stream, See Col. 14, Lines 35 - 53).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAZIA NAOREEN whose telephone number is (571)270-7282. The examiner can normally be reached M-F: 9:00 - 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Umar Cheema can be reached at 571-270-3037. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAZIA NAOREEN/Primary Examiner, Art Unit 2458
Prosecution Insights