DETAILED ACTION
This final Office action is responsive to Applicant’s amendment filed March 19, 2026. Claims 1, 2, 10, 12, and 13-20 have been amended. Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed March 19, 2026 have been considered but they are not fully persuasive.
Preliminarily, it is noted that Applicant’s claim amendments have overcome the previously-pending rejections under 35 U.S.C. § 112(b).
Additionally, Applicant’s claim amendments and arguments have been persuasive and overcome the rejection under 35 U.S.C. § 101.
Regarding the rejection under 35 U.S.C. § 103, Applicant argues that “Cha does not disclose or suggest using received reporting of packet-data activity at a panelist site as a basis to detect a user surfing through one or more content-selection menus at a panelist site.” (Pages 12-13 of Applicant’s response) The Examiner respectfully disagrees. As explained in the rejection, Gelman gathers information regarding user activity with various media sources and does so at the panelist site. As seen in ¶¶ 50, 60, 64 of Gelman, the agents used for monitoring can be executed on network devices which correspond to activity on a plurality of user devices. Since a user device is used by a user, it is presumed to reflect activity at the user, i.e., panelist, site. The Cha reference demonstrates the usefulness of monitoring user media-based activities, including in the area of accessing a menu and channel surfing. The Examiner maintains that Gelman presents technical approaches to media-based data gathering that may be adapted to Cha’s specific area of use to achieve the benefits described in the rejection. Furthermore, the Ranganathan reference has been introduced into the rejections in order to help address the details of the streaming meter being at a panelist site to detect data flow on a local area network at the panelist site.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gelman et al. (US 2024/0259320) in view of Cha et al. (Cha, Meeyoung, et al. “Watching Television Over an IP Network.” IMC '08: Proceedings of the 8th ACM SIGCOMM conference on Internet measurement (October 2008), Published 20 October 2008.) in view of Ranganathan et al. (US 2022/0385961).
[Claim 1] Gelman discloses a method for measuring media-surfing activity (Gelman evaluates user activity across various media-related services, which (based on a broadest reasonable interpretation) is an example of evaluating media-surfing activity, e.g., when media-surfing is viewed as using multiple media services (Gelman: ¶ 61 – “In certain embodiments, the type of service can indicate the specific type of application requested by a computing device (e.g., Netflix, Amazon, Zoom, etc.) and the category of the service can indicate the class of service (e.g., streaming, videoconference, web page access, etc.)“; ¶ 65 – “In certain embodiments, the processor can be further configured to identify the type of the one or more services, the category of the one or more services, or a combination thereof, based on examination of a packet payload of one or more packets associated with the network traffic.”), the method comprising:
receiving, by a computing system (¶¶ 43, 46, 64, 169 – server, processor, memory, software), based on the monitoring of the packet-data flow, reporting of packet-data activity at a panelist site (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶¶ 50, 60, 64 – The agents used for monitoring can be executed on network devices which correspond to activity on a plurality of user devices. Since a user device is used by a user, it is presumed to reflect activity at the user, i.e., panelist, site.);
using, by the computing system (¶¶ 43, 46, 64, 169 – server, processor, memory, software), at least the received reporting of packet-data activity at the panelist site as a basis to detect occurrence of media-surfing activity at the panelist site (¶ 47 – “In some embodiments, the service identification software may be configured to determine the different services used by computing devices at a particular location and to provide service usage information from the service ID agent to an ISP, for example. Suppose that an ISP may provide networking services to a particular network at a particular location via a particular router. At that location, a first person may be using an online conference software such as Zoom to participate in an online video conference meeting, a second person may be watching streaming channels such as Netflix and a third person may be downloading files, all on separate computing devices communicating with the particular router/For Zoom, high bandwidth and low latency is critical, downloading a file requires a lot of bandwidth but latency is not critical, and for streaming channels neither large bandwidth nor latency is critical since the data packets may be buffered during streaming. However, this scenario may still result in a critical application such as Zoom for remote workers, not getting enough bandwidth at a given location. The ISP, in having this service usage information, may be able to prioritize services supplied to its customers. For example, the ISP may prioritize the data pipelines to Zoom at the expense of the less critical applications. However, this data may need to first be obtained at a particular location to prioritize the customer's needs.”; ¶¶ 50, 60, 64 – The agents used for monitoring can be executed on network devices which correspond to activity on a plurality of user devices. Since a user device is used by a user, it is presumed to reflect activity at the user, i.e., panelist, site.); and
recording, by the computing system (¶¶ 43, 46, 64, 169 – server, processor, memory, software), the detected occurrence of media-surfing activity at the panelist site as media-exposure data (¶ 47 – “In some embodiments, the service identification software may be configured to determine the different services used by computing devices at a particular location and to provide service usage information from the service ID agent to an ISP, for example.”; ¶ 82 – “In certain embodiments, the fingerprinting/services processing signature database 930 can be configured to interact with a device profile/services database 940, which can store information associated with the devices and services being provided to the devices.”; ¶ 91 – “In some embodiments, the backend server may process the received data and may store it for retrieval. The retrieval may support aggregation by device_id, OS, Service_id, service_category, vendor, device_type, router_id, product_name, measure_name, and time (timestamp) with measure_value of RX and TX. The retrieved data may be stored in a database on the backend server.”; ¶ 165 – “In some embodiments, with regard to an agent implementation for test case and automation, the agent may execute a few services concurrently and may verify that the bytes sniffed match the byte count that the agent sent to the cloud per service. The database may be queried for the results that may be compared to the expected results. This embodiment represents how either a lab setup may be used for large scale automated testing, or in some cases, test and validate the system when it is live with users. An agent may generate simulated traffic and then validate that it was identified correctly with the right service ID, time, packets, etc.”).
Gelman evaluates user activity across various media-related services, which (based on a broadest reasonable interpretation) is an example of evaluating media-surfing activity, e.g., when media-surfing is viewed as using multiple media services (Gelman: ¶ 61 – “In certain embodiments, the type of service can indicate the specific type of application requested by a computing device (e.g., Netflix, Amazon, Zoom, etc.) and the category of the service can indicate the class of service (e.g., streaming, videoconference, web page access, etc.)“; ¶ 65 – “In certain embodiments, the processor can be further configured to identify the type of the one or more services, the category of the one or more services, or a combination thereof, based on examination of a packet payload of one or more packets associated with the network traffic.”). Gelman does not explicitly disclose wherein detecting the occurrence of media-surfing activity comprises detecting a user surfing through one or more content-selection menus. However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman wherein detecting the occurrence of media-surfing activity comprises detecting a user surfing through one or more content-selection menus in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
Gelman discloses monitoring, by a streaming meter at a panelist site, packet-data flow at the panelist site (Gelman: ¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶¶ 50, 60, 64 – The agents used for monitoring can be executed on network devices which correspond to activity on a plurality of user devices. Since a user device is used by a user, it is presumed to reflect activity at the user, i.e., panelist, site.; NOTE: The streaming meter is described in Applicant’s Specification as monitoring packet-data traffic data and it is sometimes described as being interchangeably used with the computing system (e.g., see Spec: ¶¶ 24-25).).
Gelman fails to explicitly disclose monitoring, by a streaming meter at a panelist site, packet-data flow on a local area network at the panelist site, and receiving the reporting of packet-data activity based on the monitoring of the packet-data flow on the local area network at the panelist site. However, Ranganathan discloses that a streaming meter may be provided to a panelist for installation by the panelist at the panelist site so that panelist activity across multiple media may be tracked via the LAN of the panelist site (Ranganathan: ¶¶ 50-56). High bandwidth consumption is among the various pieces of information that may be gathered (Ranganathan: ¶ 54). Audio or video components may incorporate information related to tuning (such as a packet identifying header) (Ranganathan: ¶ 17).
As discussed above, Gelman monitors packet-data flow (Gelman: (¶¶ 43, 46, 64, 169 – server, processor, memory, software), reporting of packet-data activity at a panelist site (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”).
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman to perform the steps of monitoring, by a streaming meter at a panelist site, packet-data flow on a local area network at the panelist site and receiving the reporting of packet-data activity based on the monitoring of the packet-data flow on the local area network at the panelist site in order to facilitate less invasive activity monitoring (as suggested in ¶ 52 of Ranganathan) while helping to improve media asset qualification (as suggested in ¶ 57 of Ranganathan).
[Claim 2] Gelman discloses wherein receiving the reporting of packet-data activity at the panelist comprises receiving the reporting from the streaming meter at the panelist site (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶¶ 50, 60, 64 – The agents used for monitoring can be executed on network devices which correspond to activity on a plurality of user devices. Since a user device is used by a user, it is presumed to reflect activity at the user, i.e., panelist, site.; NOTE: The streaming meter is described in Applicant’s Specification as monitoring packet-data traffic data and it is sometimes described as being interchangeably used with the computing system (e.g., see Spec: ¶¶ 24-25).).
[Claim 3] Gelman discloses wherein using at least the received reporting of packet-data activity at the panelist site as a basis to detect occurrence of media-surfing activity at the panelist site comprises:
determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”).
[Claim 4] Gelman discloses wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises:
referring to pre-established surfing-profile data that indicates one or more patterns of packet-data activity that are characteristic of surfing activity, as a basis to determine that the reported packet-data activity defines at least one of the one or more patterns of packet-data activity (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶ 67 – “In certain embodiments, the method can include generating one or more images, one or more strings, and/or one or more converted data converted from one or more encrypted data payloads associated with one or more data packets of the network traffic. In certain embodiments, the method can include determining the type of the one or more services, the category of the one or more services, or a combination thereof, based on a first pattern associated with the one or more images, the one or more strings, and/or the one or more converted data matching a second pattern classified for the type, the category, or a combination thereof.”).
[Claim 5] Gelman discloses wherein using at least the received reporting of packet-data activity at the panelist site as a basis to detect occurrence of media-surfing activity at the panelist site comprises:
determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity as to a particular streaming-media provider (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶ 67 – “In certain embodiments, the method can include generating one or more images, one or more strings, and/or one or more converted data converted from one or more encrypted data payloads associated with one or more data packets of the network traffic. In certain embodiments, the method can include determining the type of the one or more services, the category of the one or more services, or a combination thereof, based on a first pattern associated with the one or more images, the one or more strings, and/or the one or more converted data matching a second pattern classified for the type, the category, or a combination thereof.”),
wherein recording the detected occurrence of media-surfing activity at the panelist site as media-exposure data comprises recording the detected occurrence of media-surfing activity as to the particular streaming-media provider as media-exposure data (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶ 67 – “In certain embodiments, the method can include generating one or more images, one or more strings, and/or one or more converted data converted from one or more encrypted data payloads associated with one or more data packets of the network traffic. In certain embodiments, the method can include determining the type of the one or more services, the category of the one or more services, or a combination thereof, based on a first pattern associated with the one or more images, the one or more strings, and/or the one or more converted data matching a second pattern classified for the type, the category, or a combination thereof.”; ¶ 47 – “In some embodiments, the service identification software may be configured to determine the different services used by computing devices at a particular location and to provide service usage information from the service ID agent to an ISP, for example.”; ¶ 82 – “In certain embodiments, the fingerprinting/services processing signature database 930 can be configured to interact with a device profile/services database 940, which can store information associated with the devices and services being provided to the devices.).
[Claim 6] Gelman discloses wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity as to the particular streaming media provider comprises:
referring to pre-established surfing-profile data that indicates, separately for each of a plurality of streaming-media providers, one or more patterns of packet-data activity each characteristic of surfing activity as to the streaming-media provider, as a basis to determine that the reported packet-data activity defines at least one of the one or more patterns of packet-data activity as to the particular streaming-media provider (¶ 186 – “In this case, the data packets in the received data traffic by the network device may be encrypted using different protocols such as TCP, UDP and/or QUIC. The encryption of data packets from each application does not produce patternless data. In the encrypted data, the data packets may include application-specific patterns based on the applications such as Netflix, Disney Plus, etc. that may be used by the CNN model. Thus, the CNN model may be trained to learn the encrypting features without decrypting the payloads.”; ¶ 188 – “In certain embodiments, the data flow 2300 can include, at 2301, analyzing network traffic for various computing devices being monitored by the system of the present disclosure. Based on analyzing the network traffic, device fingerprinting can be conducted at 2302, which can identify the type of computing devices associated with the network traffic (e.g., initiating DNS requests, requesting services, using network services, etc.). At 2304, the flow 2300 can include conducting service identification for URLs that are encrypted (e.g., DNS requests made by a computing device), and, at 2306, the flow 2300 can include conducting service identification for non-encrypted URLs. In certain embodiments, the various service identifications and information associated with the network traffic and fingerprinting can be utilized as inputs, at 2308, to a machine learning model, which can use the data for training sets, validation sets, and/or testing sets. At 2310, the flow 2300 can include conducting, such as by utilizing the machine learning model, service IP mapping. For example, while URLs are available, a mapping between a service URL and its server IP address can be maintained. Thus, even when the URL is not available, the service can be quickly identified by the IP address. At 2312, the flow 2300 can include conducting packet-based identification. In certain embodiments, certain applications can use unique types of packets for data transfer. For example, VOIP applications can run on STUN. Thus, the machine learning model can recognize the packet type, which can indicate the service class (e.g., the type of service) associated with the packet type. At 2314, a deep learning model can be utilized. For example, per session, several data application payloads can be sent to a deep neural network for classification. The machine learning model can classify the network traffic associated with the computing devices based on traffic type (e.g., browsing, streaming, VoIP, etc.) or by application (e.g., Netflix, Facebook, etc.). In certain embodiments, the information from 2310, 2312, and 2314 can be utilized to train, at 2316, a service identification model (i.e., machine learning model). At 2318, an updated service model can be provided that can identify services from network traffic without having to examiner URLs. At 2320, the trended-updated service identification model can identify services for various network traffic associated with (e.g., initiated, delivered, and/or requested) computing devices.”; ¶ 67 – “In certain embodiments, the method can include generating one or more images, one or more strings, and/or one or more converted data converted from one or more encrypted data payloads associated with one or more data packets of the network traffic. In certain embodiments, the method can include determining the type of the one or more services, the category of the one or more services, or a combination thereof, based on a first pattern associated with the one or more images, the one or more strings, and/or the one or more converted data matching a second pattern classified for the type, the category, or a combination thereof.”).
[Claim 7] Gelman does not explicitly disclose:
wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises:
determining, based at least on timing of message communication with a streaming-media provider, that the packet-data activity defines a pattern that is characteristic of media-surfing activity.
However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). Cha’s comparison of detected viewing activity profiles to pre-identified patterns of activity as they relate to aspects of viewing (such as the three modes of surfing, viewing, and away) describes an example of a comparison of query signature data representing media content to digital reference signature data. Similarly, Gelman evaluates packet data to match actual network activity data to known patterns of data services and service categories to identify data services and service categories likely being used (Gelman: ¶¶ 69, 152, 188).
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman:
wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises:
determining, based at least on timing of message communication with a streaming-media provider, that the packet-data activity defines a pattern that is characteristic of media-surfing activity
in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
[Claim 8] Gelman does not explicitly disclose:
wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises:
determining, based at least on duration of streaming media transmission to the panelist site, that the packet-data activity defines a pattern that is characteristic of media-surfing activity.
However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). Cha’s comparison of detected viewing activity profiles to pre-identified patterns of activity as they relate to aspects of viewing (such as the three modes of surfing, viewing, and away) describes an example of a comparison of query signature data representing media content to digital reference signature data. Similarly, Gelman evaluates packet data to match actual network activity data to known patterns of data services and service categories to identify data services and service categories likely being used (Gelman: ¶¶ 69, 152, 188).
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman:
wherein determining that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises:
determining, based at least on duration of streaming media transmission to the panelist site, that the packet-data activity defines a pattern that is characteristic of media-surfing activity
in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
[Claim 9] Gelman does not explicitly disclose wherein determining based at least on the duration of streaming media transmission to the panelist site that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises determining that the duration of streaming media transmission to the panelist site is characteristic of program-preview streaming rather than full program streaming.
However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). Cha’s comparison of detected viewing activity profiles to pre-identified patterns of activity as they relate to aspects of viewing (such as the three modes of surfing, viewing, and away) describes an example of a comparison of query signature data representing media content to digital reference signature data. Similarly, Gelman evaluates packet data to match actual network activity data to known patterns of data services and service categories to identify data services and service categories likely being used (Gelman: ¶¶ 69, 152, 188). Cha’s short-intervaled channel surfing is an example of program-preview streaming, as opposed to “viewing,” which is characterized by certain time and other parameters that are more likely to be indicative of full program streaming.
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman wherein determining based at least on the duration of streaming media transmission to the panelist site that the packet-data activity defines a pattern that is characteristic of media-surfing activity comprises determining that the duration of streaming media transmission to the panelist site is characteristic of program-preview streaming rather than full program streaming in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
[Claim 10] Gelman does not explicitly disclose:
receiving, by the computing system, digital query signature data representing media content presented at the panelist site; and
matching, by the computing system, the digital query signature data with digital reference signature data, as a basis to identify streaming-media content presented at the panelist site,
wherein using, by the computing system, at least the received reporting of packet-data activity at the panelist site as a basis to detect occurrence of media-surfing activity at the panelist site comprises using, by the computing system, at least the received reporting of packet-data activity and the matching of the digital query signature data with the digital reference signature data, as a basis detect occurrence of media-surfing activity at the panelist site.
However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). Cha’s comparison of detected viewing activity profiles to pre-identified patterns of activity as they relate to aspects of viewing (such as the three modes of surfing, viewing, and away) describes an example of a comparison of query signature data representing media content to digital reference signature data. Similarly, Gelman evaluates packet data to match actual network activity data to known patterns of data services and service categories to identify data services and service categories likely being used (Gelman: ¶¶ 69, 152, 188).
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman:
receiving, by the computing system, digital query signature data representing media content presented at the panelist site; and
matching, by the computing system, the digital query signature data with digital reference signature data, as a basis to identify streaming-media content presented at the panelist site,
wherein using, by the computing system, at least the received reporting of packet-data activity at the panelist site as a basis to detect occurrence of media-surfing activity at the panelist site comprises using, by the computing system, at least the received reporting of packet-data activity and the matching of the digital query signature data with the digital reference signature data, as a basis detect occurrence of media-surfing activity at the panelist site
in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
[Claim 11] Gelman does not explicitly disclose wherein using by the computing system the matching of the digital query signature data with the digital reference signature data as a basis to detect occurrence of the media-surfing activity at the panelist site comprises finding by the computing system, based on the matching, presentation of program content at the panelist site for just a program-preview duration.
However, Cha evaluates activity related to television watching over an IP network (Cha: title) and identifies three modes of user behavior, including “surfing, viewing, and away” and Cha uses various patterns of timing in regard to changing channels to identify media surfing activity vs. viewing activity vs. away activity (Cha: p. 73: Section 3.2 --
PNG
media_image1.png
808
362
media_image1.png
Greyscale
).
Cha provides users with guides (i.e., menus) for additional information about selectable content (Cha: p. 81: Section 6 – “Channel selection process: Most channel changes are related to channel surfing and are short-intervaled. Therefore assisting users in channel selection is crucial to the quality of their viewing experience. In current IPTV and Internet video systems, electronic program guides (EPG) and metadata about channels such as tags and descriptions provide users with useful additional information about content.”) As discussed above, Gelman monitors packet-data activity. Similarly, Cha mentions the use of packet traces to make observations regarding Internet TV viewing habits (Cha: p. 82: Section 7). Cha’s comparison of detected viewing activity profiles to pre-identified patterns of activity as they relate to aspects of viewing (such as the three modes of surfing, viewing, and away) describes an example of a comparison of query signature data representing media content to digital reference signature data. Similarly, Gelman evaluates packet data to match actual network activity data to known patterns of data services and service categories to identify data services and service categories likely being used (Gelman: ¶¶ 69, 152, 188). Cha’s short-intervaled channel surfing is an example of program-preview streaming, as opposed to “viewing,” which is characterized by certain time and other parameters that are more likely to be indicative of full program streaming.
The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Gelman wherein using by the computing system the matching of the digital query signature data with the digital reference signature data as a basis to detect occurrence of the media-surfing activity at the panelist site comprises finding by the computing system, based on the matching, presentation of program content at the panelist site for just a program-preview duration in order to offer content providers useful insight into viewer behavior to facilitate more convenient and accurate content targeting, which may additionally allow for the preservation of network resources (as suggested in Cha: Sections 5.4, 6).
[Claims 12-18] Claims 12-18 recite limitations already addressed by the rejections of claims 1-2, 5-7, and 10-11 above; therefore, the same rejections apply.
Furthermore, Gelman discloses a computing system comprising: at least one processor; at least one non-transitory data storage; program instructions stored in the at least one non-transitory data storage and executable by the at least one processor to carry out the recited operations for measuring media-surfing activity (Gelman: ¶¶ 43, 46, 64, 169 – server, processor, memory, software).
[Claims 19-20] Claims 19-20 recite limitations already addressed by the rejections of claims 1 and 10 above; therefore, the same rejections apply.
Furthermore, Gelman discloses at least one non-transitory computer-readable medium having stored thereon instructions executable by at least one processor to cause a computing system to carry out the recited operations for measuring media-surfing activity (Gelman: ¶¶ 43, 46, 64, 169 – server, processor, memory, software).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUSANNA M DIAZ whose telephone number is (571)272-6733. The examiner can normally be reached M-F, 8 am-4:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached at (571) 270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SUSANNA M. DIAZ/
Primary Examiner
Art Unit 3625A
Prosecution Insights