DETAILED ACTION
This action is responsive to amended claims filed on 16 January 2026. Claims 1-20 are pending examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see remarks on page 12, filed 16 January 2026, with respect to the rejection(s) of claim(s) 1-20 under Yao (US 2025/0274893 A1) and Li (US 2022/0070923 A1) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Wangler et al. (US 20230300648 A1) (hereinafter Wang).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 7-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Wangler et al. (US 20230300648 A1) (hereinafter Wang).
Regarding Claim 1, Wang teaches a method comprising:
providing, by a network performance evaluation system for evaluating network performance of a telecommunication service, a User Equipment (UE) with instructions to perform an indication of one or more first network performance tests associated with communication between the UE and a first testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]:[0078] [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly.);
providing, by the network performance evaluation system, the UE with instructions to perform an indication of one or more second network performance tests associated with communication between the UE and a second testing computer, wherein the second testing computer is an external testing computer outside an internal network of the telecommunication service (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly. Similarly, remote devices performing test may be programmed to automatically log in to certain applications like warehouse control systems or sales management systems to test availability of access to them and possibly perform a test query to record overall service availability and response times. Measurement of service to external cloud 130 resources 131, 132, 133 is equally important and testing connectivity and resource availability from external dedicated targets 131, discretionary targets 133, perhaps a government printer or software as service systems that the company might use. The ability to reach a wide number of high volume web server web pages 132, for example FACEBOOK™, SKYPE™, CHROME™ and GOOGLE™ to list a few known to those skilled in the art, as well as customer critical web server web pages is also a good indicator of wireless network service health as it relates to traversing firewalls and gateways, and the health of web servers of specific concern to the customer, if included.);
receiving, by the network performance evaluation system, one or more first test results of the one or more first network performance tests that the UE was instructed to perform (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0127]-[0129]: [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly. Similarly, remote devices performing test may be programmed to automatically log in to certain applications like warehouse control systems or sales management systems to test availability of access to them and possibly perform a test query to record overall service availability and response times. Measurement of service to external cloud 130 resources 131, 132, 133 is equally important and testing connectivity and resource availability from external dedicated targets 131, discretionary targets 133, perhaps a government printer or software as service systems that the company might use. The ability to reach a wide number of high volume web server web pages 132, for example FACEBOOK™, SKYPE™, CHROME™ and GOOGLE™ to list a few known to those skilled in the art, as well as customer critical web server web pages is also a good indicator of wireless network service health as it relates to traversing firewalls and gateways, and the health of web servers of specific concern to the customer, if included.);
receiving, by the network performance evaluation system, one or more second test results of the one or more second network performance tests that the UE was instructed to perform (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0119] FIG. 9 is a diagram illustrating methods by which users of wireless networks being tested might submit their perceived experience to the wireless network service quality test system according to a preferred embodiment of the invention 900. It is often helpful when executing equipment performance tests, wireless network service quality tests in this case, to also know how a particular service level is perceived by the equipment users. Correlation between very positive performance test readouts and a similar user perception of service quality solidifies the validity test. Similarly, understanding the effect of what is considered mediocre, or unacceptable performance test readouts on user perception is equally important, if the test results are poor but the user response is positive at the same time, the tests employed may be inaccurate or may not be calibrated correctly. The invention may allow the collection of user experience feedback as part of wireless network service quality testing. In the current embodiment of the invention, user wireless network performance experience is collected using an applet that is part of the wireless network testing framework on the wireless network test device 910. A part of the applet displays a multiple choice request for user wireless network performance experience where the user can rate current network performance from one of five choices: “Excellent”, “Good”, “Sufficient”, “Not Sufficient” and “Very Poor” 911. The second portion of the applet instructs the user to describe any issues she is having on the wireless network in freeform text 912 and then gives space to do that 913. Response text is interpreted and mapped to standardized terms by the central analytics engine as part of overall testing 914. The choice from the first portion 911 of the applet are similarly used to measure general wireless network performance.);
determining, by the network performance evaluation system and based upon the one or more first test results, one or more first network performance metrics associated with the communication between the UE and the first testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0111] FIG. 2 is a flow diagram illustrating an exemplary method 200 for measuring and reporting wireless network service quality using remote devices, according to a preferred embodiment of the invention. To perform reproducible testing at least one, probably more than one, of remote wireless network testing devices 115 as depicted in FIG. 1 must be programmed with the suite of tests to be run 201 as depicted in FIG. 1 116, 111, 115. These remote test devices 115 then run the tests specified in test suite 202, which are attached to a unique test suite specific ID, such as but not limited to: wireless signal strength at test device location; wireless network BSSID; ability and latency logging onto the wireless network; negotiated wireless standard; signal level, wireless signal band or bands offered; ping round trip times; ability to upload and download data to LAN resident and cloud based dedicated targets, if available, measuring latency, data bandwidth, voice quality, video quality, packet loss, jitter; ability to access specified web pages and retrieve resources; ability to access the company's wireless phone carrier, if applicable; and ability to contact dedicated or critical phone targets and upload or download either voice or other data based on the needs of the customer. ) ;
determining, by the network performance evaluation system and based upon the one or more second test results, one or more second network performance metrics associated with the communication between the UE and the second testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0120] FIG. 10 is a diagram illustrating both passive and active testing of more than one wireless network by wireless network service quality test devices according to a preferred embodiment of the invention 1000. The system 1001 has the ability to run tests on two separate networks within a single test suite. In the case of testing the corporation's wireless phone provider network connection 1003, this testing may occur actively, attempts to connect to designated phone exchanges, to download designated data or to connect to designated servers and web pages to list a few illustrative examples, or passively, measurements of signal strength at test location, signal to noise ratios, radio and connect acknowledgement latency. Results may be reported to the phone service provider's quality management as well as client corporate analysts once connection with the primary wireless network 1004 is reestablished at the conclusion of mobile network testing 1003. When two separate wireless networks 1004 that require use of the RF radio is planned, this may be accomplished in two ways. First, the remote test device may have simultaneous dual RF radio capability, in which case the second wireless network not depicted for simplicity may be probed and the resultant data immediately sent to the central analysis and control server 410 for inclusion in the analysis results made available to the end user 480. Alternatively, a remote test device with only single RF capability may disconnect from the first network 1004, connect to the second network to be tested and while probing the network, store the resultant data internally. Upon completion of inspection of the second network, the remote test device may disconnect from the second network, re-connect to the first network 1004 ad send all of the stored result data from the second network to the central analysis and control server 410 for inclusion in the test results of the current test suite and presentation to the end users 480. A second network service quality test device 1002 with both wireless 1004 network and wired network 1005 connection capabilities may be used to execute interactive and background network performance tests 1006, 1008, either active such as data throughput, retry number, and ping return time 1007, 1009 or passive such as deep packet inspection, packet capture and protocol analysis, packet drop rates and wireless radio channel usage is manners similar to those described above. Again, data would be sent to the central analysis and control server 410 as soon as a connection is reestablished.); and
determining, by the network performance evaluation system, a network performance issue based upon a comparison of one or more reference metrics with at least one of .i the one or more first network performance metrics associated with the communication between the UE and the first testing computer or _ij the one or more second network performance metrics associated with the communication between the UE and the second testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0111] FIG. 2 is a flow diagram illustrating an exemplary method 200 for measuring and reporting wireless network service quality using remote devices, according to a preferred embodiment of the invention. To perform reproducible testing at least one, probably more than one, of remote wireless network testing devices 115 as depicted in FIG. 1 must be programmed with the suite of tests to be run 201 as depicted in FIG. 1 116, 111, 115. These remote test devices 115 then run the tests specified in test suite 202, which are attached to a unique test suite specific ID, such as but not limited to: wireless signal strength at test device location; wireless network BSSID; ability and latency logging onto the wireless network; negotiated wireless standard; signal level, wireless signal band or bands offered; ping round trip times; ability to upload and download data to LAN resident and cloud based dedicated targets, if available, measuring latency, data bandwidth, voice quality, video quality, packet loss, jitter; ability to access specified web pages and retrieve resources; ability to access the company's wireless phone carrier, if applicable; and ability to contact dedicated or critical phone targets and upload or download either voice or other data based on the needs of the customer.).
Regarding Claim 2, Wang teaches a method of claim 1:
Wherein at least one of:
the one or more first network performance tests are performed over one or more wireless connections between the UE and a node of a network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0135] In an example of the hidden node problem, access point 2 2703 and access point 3 2704 are out of range of one another (i.e., then cannot see each other to determine whether one of them is transmitting on a given channel to avoid co-channel interference). The client device 2701 can receive and transmit signals to and from both access points 2703, 2704, which could result in a situation where two or more of the three devices (the client device 2701 and the two access points 2703, 2704) are transmitting at the same time which can cause interference. The hidden node problem is perhaps the best use case of the client device view or perspective. In the hidden node problem, since the access points AP 2 2703 and AP 3 2704 cannot receive signals from one another, the client device view or perspective is the only means of obtaining information about possible interference between those two access points 2703, 2704. The client device can help detect interference at its location because, from its location, it is receiving signals on channel 1 from AP 3 2704 and on channel 4 from AP 2 2703, which it can report to the network.); or
the one or more second network performance tests are performed over one or more wireless connections between the UE and the node.
Regarding Claim 3, Wang teaches a method of claim 2 wherein:
The node comprises a wireless communication site of a telecommunication service provider (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0120] FIG. 10 is a diagram illustrating both passive and active testing of more than one wireless network by wireless network service quality test devices according to a preferred embodiment of the invention 1000. The system 1001 has the ability to run tests on two separate networks within a single test suite. In the case of testing the corporation's wireless phone provider network connection 1003, this testing may occur actively, attempts to connect to designated phone exchanges, to download designated data or to connect to designated servers and web pages to list a few illustrative examples, or passively, measurements of signal strength at test location, signal to noise ratios, radio and connect acknowledgement latency. Results may be reported to the phone service provider's quality management as well as client corporate analysts once connection with the primary wireless network 1004 is reestablished at the conclusion of mobile network testing 1003.).
Regarding Claim 4, Wang teaches a method of claim 3 wherein:
The network corresponds to an internal network, of the telecommunication service provider, comprising (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See above for paragraph [0120].):
the wireless communication site (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]: [0111] To perform reproducible testing at least one, probably more than one, of remote wireless network testing devices 115 as depicted in FIG. 1 must be programmed with the suite of tests to be run 201 as depicted in FIG. 1 116, 111, 115. These remote test devices 115 then run the tests specified in test suite 202, which are attached to a unique test suite specific ID, such as but not limited to: wireless signal strength at test device location; wireless network BSSID; ability and latency logging onto the wireless network; negotiated wireless standard; signal level, wireless signal band or bands offered; ping round trip times; ability to upload and download data to LAN resident and cloud based dedicated targets, if available, measuring latency, data bandwidth, voice quality, video quality, packet loss, jitter; ability to access specified web pages and retrieve resources; ability to access the company's wireless phone carrier, if applicable; and ability to contact dedicated or critical phone targets and upload or download either voice or other data based on the needs of the customer.);
the first testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0076] FIG. 1 is a block diagram illustrating an exemplary system architecture 100 for measuring and reporting wireless network service quality using remote test devices 115, according to a preferred embodiment of the invention. Under the embodiment, data concerning factors that affect the service quality levels of a wireless network of interest 120 which may include but are not limited to steady state signal strength, variability in signal strength, signal to noise ratio, packet retry rate, used data rate, current percentage of network traffic versus network capacity, radio attachment latency and success rate, resource request latency, web page load time and success rate, access to certain applications over network connection, voice quality, video quality, packet loss, jitter, location of the test device, and ping response time, among others known to those skilled in the art, are collected by wireless network connected remote test device 115 which most often will be an end-user's mobile device such as a smart phone or tablet running a pre-programmed network test framework application. Remote network test device 115, may also, under some circumstances, be a desktop workstation, a laptop computer, a kiosk, an Internet of Things (IoT) device, a wireless access point, modems with wireless (“Wi-Fi”) or optical (“Li-Fi”) capability such as asynchronous digital subscriber line (“ADSL”) or cable modems, or dedicated sensor devices for this purpose. Other test device possibilities exist; those put forth here are meant only as selected examples. A plurality of remote network test devices 115 may be used in each network service quality test.); and
a core network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0077] In this embodiment all of these parameters, with the programming functions to execute the desired network evaluation are stored in a cloud-based test code and configuration server 116 and, prior to testing, information, including a unique test suite ID, is transferred to a site's central analysis and control server 111 from which remote network test devices 115 to be used are programmed and configured. All data collected as part of the test suite will have the unique test suite ID when sent to central analysis and control server 111 and may be stored in a data store 113 long-term as such including the unique ID for retrieval and future transformation by central analysis and control server 111. Central analysis and control server 111 also provides an API 112 that allows it to share output from test analysis to third party network equipment 124 such as a network management server, to name one of a plurality of examples, and to retrieve important information from wireless network equipment such as maps with access point locations, wireless network firmware and software versions, network configuration information, traffic volume and quality parameters from network equipment, wireless access point 121 parameters where such data as wireless standards being supported);
the one or more first network performance tests are performed over the core network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0090] It is important to note that a “network” is used to denote any grouping of client devices 2004a-n that access a common service 2002. For example, it can be used to denote a local group of commonly-owned access points, client devices, and associated hardware (for example, a WiFi network in the headquarters building of a company) or can be used in a broader sense to denote a group of individually-owned client devices having a subscription to an online platform and accessing the online platform through a combination of privately-owned and publicly-accessible access points or a group of computing devices connected through a virtual private network (VPN). For the sake of clarity, only a single service 2002 is shown in the diagram, representing one “network” of client devices. However, the control server (or multiple control servers) may manage any number of such networks for any number of companies or groups.);
the second testing computer is connected to a public network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See above for paragraph [0090]); and
the one or more second network performance tests are performed over the core network and the public network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See above for paragraph [0090]).
Regarding Claim 5, Wang teaches a method of claim 4 comprising:
Isolating a problem area associated with the network performance issue to a first region of interest associated with the internal network based upon a first network performance metric of the one or more first network performance metrics not meeting a first network performance metric threshold of the one or more reference metrics (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0130] FIG. 27 is a diagram showing an example of network performance problem detection using multi-factor event correlation. Multi-factor event correlation is a technique whereby one or more client devices 2701 are used to infer actual or potential problems with network performance where such information is not otherwise available to the network. In effect, the client device(s) 2701 are used as external sensors to the network to detect events and conditions external to the network, such as coverage problems, co-channel and adjacent-channel interference, hidden node problems, roaming problems, latency, jitter, throughput problems, and connectivity problems. In multi-factor event correlation, the “view” or “perspective” of the network from the client device provides key information about the wireless network's performance at the location of the client device. Since the client device is at a location where no other network device is located (in most cases), the client device is the only device that can provide information to the network about wireless network performance at that location.); or
isolating the problem area associated with the network performance issue to a second region of interest associated with the public network based upon:
the first network performance metric of the one or more first network performance metrics meeting the first network performance metric threshold; and
a second network performance metric of the one or more second network performance metrics not meeting a second network performance threshold of the one or more reference metrics.
Regarding Claim 7, Wang teaches a method of claim 2 wherein:
The one or more first network performance tests comprise at least one of (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See above for paragraph [0090]):
an uplink (UL) network performance test in which the UE transmits first data to the first testing computer, wherein a test result of the UL network performance test is based upon a measure of data, of the first data transmitted by the UE, received by the first testing computer within a first time period associated with the UL network performance test (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0115] These test may recur automatically at set time intervals indefinitely, recur a predetermined number of times either regularly or pseudo-randomly, or occur one time only. Mobile network test devices then carry out those tests and send the wireless network performance data back to the centralized analysis and control server 502. The centralized server then applies a plurality of customer specified preprogrammed analytical functions on that data 504 and may also employ data obtained directly from third party wireless network equipment such as, but not limited to: wireless access points and signal amplifiers and repeaters. [0130] FIG. 27 is a diagram showing an example of network performance problem detection using multi-factor event correlation. Multi-factor event correlation is a technique whereby one or more client devices 2701 are used to infer actual or potential problems with network performance where such information is not otherwise available to the network. In effect, the client device(s) 2701 are used as external sensors to the network to detect events and conditions external to the network, such as coverage problems, co-channel and adjacent-channel interference, hidden node problems, roaming problems, latency, jitter, throughput problems, and connectivity problems. In multi-factor event correlation, the “view” or “perspective” of the network from the client device provides key information about the wireless network's performance at the location of the client device. Since the client device is at a location where no other network device is located (in most cases), the client device is the only device that can provide information to the network about wireless network performance at that location.); or
a downlink (DL) network performance test in which the first testing computer transmits second data to the UE, wherein a test result of the DL network performance test is based upon a measure of data, of the second data transmitted by the first testing computer, received by the UE within a second time period associated with the DL network performance test.
Regarding Claim 8, Wang teaches a method of claim 7 comprising:
Determining a UL network performance metric based upon the test result of the UL network performance test (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0077] While simple spot tests of a network are possible using only the remote network test device with the un-programmed network test framework application, the data collected in this fashion is not associated with data collected from concurrent testing run by other mobile devices and the data are not stored in such a way as to be later used to diagnose network issues or render time based status results of a network. To run continuous or recurring ongoing tests that may be used in further analysis and conclusion generation, the client must have an account which holds, among other information the specific network tests to run, the duration of the data collection for each test, where appropriate, the periodicity at which each test should be run, the number of recurrences for each test, the transformations to be run on each returned data set and potentially, the manner in which results are to be presented, to name a few examples of parameters that may comprise a test from the many known within the field. In this embodiment all of these parameters, with the programming functions to execute the desired network evaluation are stored in a cloud-based test code and configuration server 116 and, prior to testing, information, including a unique test suite ID, is transferred to a site's central analysis and control server 111 from which remote network test devices 115 to be used are programmed and configured. All data collected as part of the test suite will have the unique test suite ID when sent to central analysis and control server 111 and may be stored in a data store 113 long-term as such including the unique ID for retrieval and future transformation by central analysis and control server 111. Central analysis and control server 111 also provides an API 112 that allows it to share output from test analysis to third party network equipment 124 such as a network management server, to name one of a plurality of examples, and to retrieve important information from wireless network equipment such as maps with access point locations, wireless network firmware and software versions, network configuration information, traffic volume and quality parameters from network equipment, wireless access point 121 parameters where such data as wireless standards being supported (examples: b, g, a, n, ac), security protocols enforced (examples: WPA, WPA2, WPA Enterprise), access point RF radio transmitted signal strength, broadcast bands enabled (examples: 2.4 GHz, 5 GHz), collisions, retransmissions and percent capacity in use, to list a few members of a larger set of attributes, may be important to diagnosing wireless network service quality issues, proposing remedies, and sending notifications, capabilities for which the analytics engine of central analysis and control server 111 is programmed should network service levels fall below customer service level expectations.); and
isolating a problem area of the network performance issue to UL congestion associated with a wireless connection between the UE and the node based upon the UL network performance metric not meeting a UL network performance metric threshold of the one or more reference metrics (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: [0130] FIG. 27 is a diagram showing an example of network performance problem detection using multi-factor event correlation. Multi-factor event correlation is a technique whereby one or more client devices 2701 are used to infer actual or potential problems with network performance where such information is not otherwise available to the network. In effect, the client device(s) 2701 are used as external sensors to the network to detect events and conditions external to the network, such as coverage problems, co-channel and adjacent-channel interference, hidden node problems, roaming problems, latency, jitter, throughput problems, and connectivity problems. In multi-factor event correlation, the “view” or “perspective” of the network from the client device provides key information about the wireless network's performance at the location of the client device. Since the client device is at a location where no other network device is located (in most cases), the client device is the only device that can provide information to the network about wireless network performance at that location.).
Regarding Claim 9, Wang teaches a method of claim 7 comprising:
Determining a DL network performance metric based upon the test result of the DL network performance test (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127], [0130]-[0142]: See below for paragraph [0131]); and
isolating a problem area of the network performance issue to DL congestion associated with a wireless connection between the UE and the node based upon the DL network performance metric not meeting a DL network performance metric threshold of the one or more reference metrics (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See below for paragraph [0088]).
Regarding Claim 10, Wang teaches a method of claim 9 comprising:
Determining a second DL network performance metric associated with operation of the node when a congestion level of the node does not meet a threshold congestion level (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: [0088]); and
determining the DL network performance metric threshold based upon the second DL network performance metric (Wang, fig. 2, fig. 3 and fig. 10, [0099]-[0108], [0111]-[0121], [0122]-[0127]: [0100] The software application 2212 may perform network performance analyses on the data such as signal strength, nominal connection speed, throughput speed, and the like, and may perform calculations such as average, mean, and mode, or percentile, quartile, etc., of certain metrics. Changes to metrics over time may be recorded and correlated with network performance impacts. In some embodiments, the software application 2212 may be configured).
Regarding Claim 11, Wang teaches a method of claim 7:
wherein at least one of:
the UL network performance test comprises a UL User Datagram Protocol (UDP)network performance test (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: [0088]); or
the DL network performance test comprises a DL UDP network performance test.
Regarding Claim 12, Wang teach a method of claim 2 wherein:
the one or more first network performance tests comprise an uplink (UL) User Datagram Protocol (UDP) network performance test in which the UE transmits first data to the first testing computer, wherein a test result of the UL UDP network performance test is based upon a measure of data, of the first data transmitted by the UE, received by the first testing computer within a first time period associated with the UL UDP network performance test (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127], [0130]-[0142]: [0131] Multi-factor event correlation may use both active and passive testing, and may correlate the results of those tests against other known or measured network event information to infer problems with network performance. Active testing includes testing and measurement initiated by the client device such as initiation of connection attempts, initiation of data transmission and receipt (e.g., bandwidth, and speed/data rate testing), initiation of audio/voice calls, throughput testing, and server pings to determine connectivity. Passive testing includes testing and measurement of events to which the client device is exposed, but which have not necessarily been initiated by the client device, such as existing connections, available signals and their strength, data packet capture analysis, and the geolocation of the client device.) and
the one or more first network performance tests comprise a download Transmission Control Protocol (TCP) network performance test in which the first testing computer transmits second data to the UE, wherein a test result of the download TCP network performance test is based upon at least one of (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: [0139] Active testing 2902 includes testing and measurement initiated by the client device such as initiation of connection attempts, initiation of data transmission and receipt (e.g., bandwidth, and speed/data rate testing), initiation of audio/voice calls, throughput testing, and server pings to determine connectivity. Passive testing 2903 includes testing and measurement of events to which the client device is exposed, but which have not necessarily been initiated by the client device, such as existing connections, available signals and their strength, data packet capture analysis, and the geolocation of the client device. Station connectivity state testing 2904 may include information as to whether a connection exists, to which access point the client device is connected, the signal strength of the connection, etc.):
a duration of time between (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See above for paragraph [0088].):
a transmission of data by the first testing computer to the UE during the download TCP network performance test; and reception of an acknowledgment, by the first testing computer, indicating reception of the transmission of data by the UE (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See above for paragraph [0088].); or
a measure of data received by the UE from the first testing computer within a second time period associated with the download TCP network performance test.
Regarding Claim 13, Wang teaches a method of claim 12 wherein:
determining the one or more first network performance metrics comprises determining at least one of (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See below for paragraph [0094]):
a UL UDP network performance metric based upon the test result of the UL UDP network performance test (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See paragraph below [0088].);
a round trip time metric based upon the test result of the download TCP network performance test; or
a TCP download network performance metric based upon the test result of the download TCP network performance test.
Regarding Claim 14, Wang teaches a method of claim 13 comprising:
isolating a problem area of the network performance issue to a UL bandwidth condition associated with a wireless connection between the UE and the node based upon at least one of (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: [0094] The testing configurations and operational modes assigned by the control server 2001 may be further organized by population groups (e.g. a particular customer organization) to optimize the network performance management for the client device 2004a-n type mix (e.g., high/low bandwidth connections, high/low spare processing power, frequently connected/disconnected, etc.) enabling optimal client device and network coverage while minimizing the number of connected clients devices needed to achieve representative network performance data. [0112] Nor may the results be saved for later specific retrieval 303, and are therefore, under these conditions only and isolated snapshot of network function. If a user wants to take full advantage of the capabilities of the system 304, she must sign up for service, in this embodiment the user signs up for 100 remote test device service 305.) :
the UL UDP network performance metric not meeting a UL UDP network performance metric threshold of the one or more reference metrics (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See paragraph below [0088].);
the round trip time metric not meeting a round trip time metric threshold of the one or more reference metrics (Wang, fig. 2, fig. 3 and fig. 10, [0099]-[0108], [0111]-[0121], [0122]-[0127]: [0100] The software application 2212 may perform network performance analyses on the data such as signal strength, nominal connection speed, throughput speed, and the like, and may perform calculations such as average, mean, and mode, or percentile, quartile, etc., of certain metrics. Changes to metrics over time may be recorded and correlated with network performance impacts. In some embodiments, the software application 2212 may be configured with a wireless performance monitor that uses packet and frame filtering to extract and infer network performance data that are either not otherwise available or are not supported by any other components or functionality of the client device 2210. [0104] Further, the wireless performance monitor 2410 can be used to infer information not supported by a client device or not provided by the client device with sufficient accuracy. In some embodiments, for example, the wireless performance monitor 2410 may be configured to infer the modulation and coding scheme index (MCS index) for a given wireless connection. Each MCS index represents a set of frequency, streams, modulation, and coding that establish the maximum data rate/bandwidth that can be achieved using that index, and is an important piece of information in evaluating certain wireless network performance metrics.); or
the TCP download network performance metric not meeting a TCP download network performance metric threshold of the one or more reference metrics.
Regarding Claim 15, Wang teaches a method of claim 13 comprising:
isolating a problem area of the network performance issue to congestion associated with a core network based upon at least one of (Wang, fig. 2 and fig. 10, [0111]-[0121], [0122]-[0127]: See below for paragraph [0111]):
the UL UDP network performance metric not meeting a UL UDP network performance metric threshold of the one or more reference metrics (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: [0088] For example, the control server 2001 may configure client devices via remote instruction regarding which tests to run, how frequently such tests should be run, and thresholds for testing and reporting, and schedules and other conditions for reporting data. The application on the client devices 2004a-n may then run in the background, independently conducting system testing as configured by the control server 2001, analyzing results, storing test results locally, and reporting them as scheduled. In this case, the client devices 2004a-n act as independent agents, gathering, analyzing, storing, and reporting data. Alternately, the control server 2001 may take a more active role, directly instructing client devices 2004a-n to run certain tests and report the results, with the control server 2001 storing the data and conducting the analyses.);
the round trip time metric meeting a round trip time metric threshold of the one or more reference metrics; or
the TCP download network performance metric meeting a TCP download network performance metric threshold of the one or more reference metrics,wherein: the node comprises a wireless communication site of a telecommunication service provider;
the network corresponds to an internal network, of the telecommunication service provider, comprising (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See above for paragraph [0120].):
the wireless communication site (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]: See above for paragraph [0111].);
the first testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]: See above for paragraph [0078].); and
the core network (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0130]-[0138]: See paragraph above [0077].); and
at least one of:
the UL UDP network performance test is performed over the core network (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: See above for paragraph [0088].); or
the download TCP network performance test is performed over the core network.
Regarding Claim 17, Wang teaches a method of claim 16 comprising:
at least one of:
determining an external upload TCP network performance metric threshold of the one or more reference metrics based upon the internal upload TCP network performance metric (Wang, fig. 3 and fig. 8, [0122]-[0139], [0145]-[0157]: [0125] A fourth example is a set of graphs illustrating test results from a subset of the network parameters from a plurality of possible parameters which include ping response time 1230; web page download time 1240; network packet throughput 1260; voice over IP voice quality 1270; TCP download throughput 1250; and TCP upload throughput 1280 from a specific network device. All graphs measure service parameter quality as a function of time 1232, 1242, 1252, 1262, 1272, 1282. The graphs for ping response time 1230, web page download time 1240, network packet throughput 1260 and voice over IP voice quality 1270 report those parameters with respect to customer service level agreement (SLA) requirements 1231, 1241, 1261, 1271 with service level plotted as the percentage of that SLA 1233, 1243, 1263, 1273. Each graph shows a percentage of customer SLA that would lead to warning notifications 1234, 1244, 1264, 1274 being sent as illustrated in FIG. 13. Some of the graphs plot parameters on which the customer relies to have functioning at or very near 100% of their SLA level 1240, 1260 1270 whereas the embodiment is set to warn those designated only after ping response drops below approximately 80% of the customer's SLA 1234. All four graphs also have a critical service level set 1235, 1245, 1265, 1275. Service levels below these critical set points may cause more extreme notifications to be sent, possibly to a larger group of designated customer representatives and alarms may also be activated 1300. As an example of such a situation, it can be seen in the ping response time graph 1230 that service level not only falls below the level where warnings may be sent 1234, but falls below 1236 the customer's “critical” service level 1235. This may lead to alarms being activated at that time, possibly per the process illustrated in FIG. 13. The last two graphs 1250 and 1280 are plotted using time 1252, 1282, but use the less derived measure of Mbit/s for the y-axis 1251, 1281 with the extent of the y-axis fitted closely to the maximal throughput 1253, 1283 in both of these situations, warnings 1254, 1284 and critical notifications 1255, 1285 are both set such that a network failure would be required, or were not set by the customer. SLA and MBits/s are two examples of measurements that may be graphed by the invention but are not exhaustive. The invention may be used to display any measurement known to those skilled in the art and appropriate to the parameter being displayed.); or
determining an external download TCP network performance metric threshold of the one or more reference metrics based upon the internal download TCP network performance metric; and
isolating a problem area of the network performance issue to congestion associated with the public network based upon at least one of (Wang, fig. 2 and fig. 10, [0111]-[0121], [0122]-[0127]: [0111] Within central analysis and control server 111, data received from previously mentioned sources and possibly other sources specific to the needs or the test suite customer, are transformed using pre-programmed analysis functions to produce a representation of all tested aspects of network function. While these representations may be purely numerical, such a display is very difficult to quickly analyze and results are thus usually depicted as a proportion of 100% function with customer-decided minimum service levels as line graphs over a time period of interest 206. Given possession of floorplans and wireless access point maps, the system may also display service quality, coverage, congestion, failures, errors, user feedback and similar factors as colored topographical like maps to make interpreting the data and possible problem areas as easy as possible. Drops in service quality below certain limits may have profound consequences on customer operations and therefore central analysis and control server 111 has APIs to communicate with third party alarm and control systems 207.):
the external upload TCP network performance metric not meeting the external upload TCP network performance metric threshold (Wang, fig. 2, fig. 3 and fig. 10, [0111]-[0121], [0122]-[0127]: For example, the control server 2001 may configure client devices via remote instruction regarding which tests to run, how frequently such tests should be run, and thresholds for testing and reporting, and schedules and other conditions for reporting data. The application on the client devices 2004a-n may then run in the background, independently conducting system testing as configured by the control server 2001, analyzing results, storing test results locally, and reporting them as scheduled. In this case, the client devices 2004a-n act as independent agents, gathering, analyzing, storing, and reporting data. Alternately, the control server 2001 may take a more active role, directly instructing client devices 2004a-n to run certain tests and report the results, with the control server 2001 storing the data and conducting the analyses.); or
the external download TCP network performance metric not meeting the external download TCP network performance metric threshold.
Regarding Claim 18, Wang teaches a method:
receiving, by a User Equipment (UE) and from a network performance evaluation system for evaluating network performance of a telecommunication service, a set of instructions;
executing, by the UE, the set of instructions to (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]: [0085] Thus, the locus of control of the system and the location of processing and analysis may be shifted up or down the hierarchy of system components as needed. This may be done by causing the control server to issue a set of configuration instructions comprising a level of autonomy authorized for each client device. The level of autonomy may range from minimal (e.g., reporting and testing based on a tests pre-selected in the configuration instructions and schedules pre-defined in the configuration instructions) to maximal (e.g., wherein the client device itself determines, based on an application running on the device, what tests to select and perform, when to perform the tests, what test result data to store, whether and how often to report to the control server, and how to dynamically change the frequency (i.e., periodicity of testing) and granularity (i.e. level of detail captured during testing) of testing based on changing network conditions). Upon receipt of the configuration instructions, each client device will establish a network testing regime consistent with the level of authority set forth in the configuration instructions. In some configurations, the configuration instructions may include instructions for a given client device to act as an intermediary control server for a group of client devices, wherein that client device issues configuration instructions to the group in a manner similar to the control server, and the group reports back to that client device (acting as an intermediary control server). The client device acting as an intermediary control server will, as part of its network testing regime, determine whether and how often to report its group's data back to the control server, based on its level of autonomy.):
perform first communication with a first testing computer and derive a first network performance test result based upon the first communication(Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0127]-[0129]: [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly. Similarly, remote devices performing test may be programmed to automatically log in to certain applications like warehouse control systems or sales management systems to test availability of access to them and possibly perform a test query to record overall service availability and response times. Measurement of service to external cloud 130 resources 131, 132, 133 is equally important and testing connectivity and resource availability from external dedicated targets 131, discretionary targets 133, perhaps a government printer or software as service systems that the company might use. The ability to reach a wide number of high volume web server web pages 132, for example FACEBOOK™, SKYPE™, CHROME™ and GOOGLE™ to list a few known to those skilled in the art, as well as customer critical web server web pages is also a good indicator of wireless network service health as it relates to traversing firewalls and gateways, and the health of web servers of specific concern to the customer, if included.); and
perform second communication with a second testing computer and derive a second network performance test result based upon the second communication. wherein the second testing computer is an external testing computer outside an internal network of the telecommunication service (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: See above for paragraph [0078]); and
transmitting the first network performance test result and the second network performance test result to the network performance evaluation system (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0120] FIG. 10 is a diagram illustrating both passive and active testing of more than one wireless network by wireless network service quality test devices according to a preferred embodiment of the invention 1000. The system 1001 has the ability to run tests on two separate networks within a single test suite. In the case of testing the corporation's wireless phone provider network connection 1003, this testing may occur actively, attempts to connect to designated phone exchanges, to download designated data or to connect to designated servers and web pages to list a few illustrative examples, or passively, measurements of signal strength at test location, signal to noise ratios, radio and connect acknowledgement latency. Results may be reported to the phone service provider's quality management as well as client corporate analysts once connection with the primary wireless network 1004 is reestablished at the conclusion of mobile network testing 1003. When two separate wireless networks 1004 that require use of the RF radio is planned, this may be accomplished in two ways. First, the remote test device may have simultaneous dual RF radio capability, in which case the second wireless network not depicted for simplicity may be probed and the resultant data immediately sent to the central analysis and control server 410 for inclusion in the analysis results made available to the end user 480. Alternatively, a remote test device with only single RF capability may disconnect from the first network 1004, connect to the second network to be tested and while probing the network, store the resultant data internally. Upon completion of inspection of the second network, the remote test device may disconnect from the second network, re-connect to the first network 1004 ad send all of the stored result data from the second network to the central analysis and control server 410 for inclusion in the test results of the current test suite and presentation to the end users 480. A second network service quality test device 1002 with both wireless 1004 network and wired network 1005 connection capabilities may be used to execute interactive and background network performance tests 1006, 1008, either active such as data throughput, retry number, and ping return time 1007, 1009 or passive such as deep packet inspection, packet capture and protocol analysis, packet drop rates and wireless radio channel usage is manners similar to those described above. Again, data would be sent to the central analysis and control server 410 as soon as a connection is reestablished.).
Regarding Claim 19, Wang teaches a method of claim 18 wherein:
At least one of the first network performance test result or the second network performance test result is used to determine a network performance issue (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]:[0078] [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly.).
Regarding Claim 20, Wang teaches a computer comprising:
a processor, the processor configured to execute instructions to perform operations comprising:
providing a User Equipment (UE) with an indication of one or more first network performance tests associated with communication between the UE and a first testing computer Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112]:[0078] [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly.);
providing the UE with an indication of one or more second network performance tests associated with communication between the UE and a second testing computer, wherein the second testing computer is an external testing computer outside an internal network of a telecommunication service (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly. Similarly, remote devices performing test may be programmed to automatically log in to certain applications like warehouse control systems or sales management systems to test availability of access to them and possibly perform a test query to record overall service availability and response times. Measurement of service to external cloud 130 resources 131, 132, 133 is equally important and testing connectivity and resource availability from external dedicated targets 131, discretionary targets 133, perhaps a government printer or software as service systems that the company might use. The ability to reach a wide number of high volume web server web pages 132, for example FACEBOOK™, SKYPE™, CHROME™ and GOOGLE™ to list a few known to those skilled in the art, as well as customer critical web server web pages is also a good indicator of wireless network service health as it relates to traversing firewalls and gateways, and the health of web servers of specific concern to the customer, if included.);
receiving one or more first test results of the one or more first network performance tests (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126], [0127]-[0129]: [0078] Based upon programming and test system setup, the remote network test devices may attempt to connect to and download or upload resources to company internal dedicated target test devices 122. These devices may be connected to the same networked by cable or connected wirelessly. They may also be connected to another segment or network within the company to allow testing of a wider range of infrastructure components. There may also be other devices present on a company's network infrastructure that are critical to operations that are therefore tested 123. An example may be wired or wirelessly connected printers that the company uses which, if not available for jobs, may cause process delays or work stoppages; some may print manufacturing orders, others invoices and still others general duty jobs. Another example may be wirelessly connected or wired scanners. Service requirements for these resources are expected to be high and action in event of significant reduction or loss of function needed swiftly. Similarly, remote devices performing test may be programmed to automatically log in to certain applications like warehouse control systems or sales management systems to test availability of access to them and possibly perform a test query to record overall service availability and response times. Measurement of service to external cloud 130 resources 131, 132, 133 is equally important and testing connectivity and resource availability from external dedicated targets 131, discretionary targets 133, perhaps a government printer or software as service systems that the company might use. The ability to reach a wide number of high volume web server web pages 132, for example FACEBOOK™, SKYPE™, CHROME™ and GOOGLE™ to list a few known to those skilled in the art, as well as customer critical web server web pages is also a good indicator of wireless network service health as it relates to traversing firewalls and gateways, and the health of web servers of specific concern to the customer, if included.);
receiving one or more second test results of the one or more second network performance tests (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0119] FIG. 9 is a diagram illustrating methods by which users of wireless networks being tested might submit their perceived experience to the wireless network service quality test system according to a preferred embodiment of the invention 900. It is often helpful when executing equipment performance tests, wireless network service quality tests in this case, to also know how a particular service level is perceived by the equipment users. Correlation between very positive performance test readouts and a similar user perception of service quality solidifies the validity test. Similarly, understanding the effect of what is considered mediocre, or unacceptable performance test readouts on user perception is equally important, if the test results are poor but the user response is positive at the same time, the tests employed may be inaccurate or may not be calibrated correctly. The invention may allow the collection of user experience feedback as part of wireless network service quality testing. In the current embodiment of the invention, user wireless network performance experience is collected using an applet that is part of the wireless network testing framework on the wireless network test device 910. A part of the applet displays a multiple choice request for user wireless network performance experience where the user can rate current network performance from one of five choices: “Excellent”, “Good”, “Sufficient”, “Not Sufficient” and “Very Poor” 911. The second portion of the applet instructs the user to describe any issues she is having on the wireless network in freeform text 912 and then gives space to do that 913. Response text is interpreted and mapped to standardized terms by the central analytics engine as part of overall testing 914. The choice from the first portion 911 of the applet are similarly used to measure general wireless network performance.);
determining, based upon the one or more first test results, one or more first network performance metrics associated with the communication between the UE and the first testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0111] FIG. 2 is a flow diagram illustrating an exemplary method 200 for measuring and reporting wireless network service quality using remote devices, according to a preferred embodiment of the invention. To perform reproducible testing at least one, probably more than one, of remote wireless network testing devices 115 as depicted in FIG. 1 must be programmed with the suite of tests to be run 201 as depicted in FIG. 1 116, 111, 115. These remote test devices 115 then run the tests specified in test suite 202, which are attached to a unique test suite specific ID, such as but not limited to: wireless signal strength at test device location; wireless network BSSID; ability and latency logging onto the wireless network; negotiated wireless standard; signal level, wireless signal band or bands offered; ping round trip times; ability to upload and download data to LAN resident and cloud based dedicated targets, if available, measuring latency, data bandwidth, voice quality, video quality, packet loss, jitter; ability to access specified web pages and retrieve resources; ability to access the company's wireless phone carrier, if applicable; and ability to contact dedicated or critical phone targets and upload or download either voice or other data based on the needs of the customer);
determining, based upon the one or more second test results, one or more second network performance metrics associated with the communication between the UE and the second testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0120] FIG. 10 is a diagram illustrating both passive and active testing of more than one wireless network by wireless network service quality test devices according to a preferred embodiment of the invention 1000. The system 1001 has the ability to run tests on two separate networks within a single test suite. In the case of testing the corporation's wireless phone provider network connection 1003, this testing may occur actively, attempts to connect to designated phone exchanges, to download designated data or to connect to designated servers and web pages to list a few illustrative examples, or passively, measurements of signal strength at test location, signal to noise ratios, radio and connect acknowledgement latency. Results may be reported to the phone service provider's quality management as well as client corporate analysts once connection with the primary wireless network 1004 is reestablished at the conclusion of mobile network testing 1003. When two separate wireless networks 1004 that require use of the RF radio is planned, this may be accomplished in two ways. First, the remote test device may have simultaneous dual RF radio capability, in which case the second wireless network not depicted for simplicity may be probed and the resultant data immediately sent to the central analysis and control server 410 for inclusion in the analysis results made available to the end user 480. Alternatively, a remote test device with only single RF capability may disconnect from the first network 1004, connect to the second network to be tested and while probing the network, store the resultant data internally. Upon completion of inspection of the second network, the remote test device may disconnect from the second network, re-connect to the first network 1004 ad send all of the stored result data from the second network to the central analysis and control server 410 for inclusion in the test results of the current test suite and presentation to the end users 480. A second network service quality test device 1002 with both wireless 1004 network and wired network 1005 connection capabilities may be used to execute interactive and background network performance tests 1006, 1008, either active such as data throughput, retry number, and ping return time 1007, 1009 or passive such as deep packet inspection, packet capture and protocol analysis, packet drop rates and wireless radio channel usage is manners similar to those described above. Again, data would be sent to the central analysis and control server 410 as soon as a connection is reestablished.); and
determining a network performance issue based upon a comparison of one or more reference metrics with at least one of i the one or more first network performance metrics associated with the communication between the UE and the first testing computer or ii the one or more second network performance metrics associated with the communication between the UE and the second testing computer (Wang, fig. 1-fig.3, [0076]-[0084], [0085]-[0107], [0111]-[0112], [0118]-[0126]: [0111] FIG. 2 is a flow diagram illustrating an exemplary method 200 for measuring and reporting wireless network service quality using remote devices, according to a preferred embodiment of the invention. To perform reproducible testing at least one, probably more than one, of remote wireless network testing devices 115 as depicted in FIG. 1 must be programmed with the suite of tests to be run 201 as depicted in FIG. 1 116, 111, 115. These remote test devices 115 then run the tests specified in test suite 202, which are attached to a unique test suite specific ID, such as but not limited to: wireless signal strength at test device location; wireless network BSSID; ability and latency logging onto the wireless network; negotiated wireless standard; signal level, wireless signal band or bands offered; ping round trip times; ability to upload and download data to LAN resident and cloud based dedicated targets, if available, measuring latency, data bandwidth, voice quality, video quality, packet loss, jitter; ability to access specified web pages and retrieve resources; ability to access the company's wireless phone carrier, if applicable; and ability to contact dedicated or critical phone targets and upload or download either voice or other data based on the needs of the customer.).
Allowable Subject Matter
Claim 6 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wang et al. (US 20250071034 A1) abstract discloses A distributed network performance management system and method that distributes a large portion of the network performance management to wireless client devices connected to the network. Rather than rely on a central server to perform the bulk of network performance management, a distributed network performance management system offloads much of the work of service quality testing, reporting, and troubleshooting to wireless client devices that are connected to the network. It utilizes spare computing power and storage space on the wireless client devices to reduce the cloud operation costs of the system including such things as bandwidth requirements, data storage requirements, and data processing requirements (See fig. 1C and fig. 3-5).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Francesca Lima Santos whose telephone number is (571)272-6521. The examiner can normally be reached Monday thru Friday 7:30am-5pm, ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marcus R Smith can be reached at (571) 270-1096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANCESCA LIMA SANTOS/Examiner, Art Unit 2468
/MARCUS SMITH/Supervisory Patent Examiner, Art Unit 2468
Prosecution Insights