DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments and arguments filed on 2/26/2026 have been considered. Claims 1-20 are pending in the application. Applicant’s amendments to the specifications have overcome each and every objection previously set forth in the Non-Final Office Action mailed on 12/19/2025.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Please see 35 U.S.C 103 Rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 8-11, 12, 14, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Min et al (US 20170061566 A1) and Huang et al (US 10497168 B2), hereinafter Min and Huang respectively.
Regarding claim 1, Min teaches a method comprising: obtaining one or more performance statistics associated with processing first content data of an application using one or more first processors and one or more second processors;
“In some embodiments, additional and/or alternative analysis may be performed to determine whether to run the application on the GPU 210 or the CPU 202, such as a performance history of running like applications on the GPU 210” – Par 37, Lines 6-10
NOTE: Performance history as described by Min is used to determine which tasks should be performed on the CPU or offloaded to the GPU. This performance history implies the existence of performance metrics associated with a previous content. Network device 106, see Col , Lines 2-65, contains a CPU and a GPU and will decide which of the processors to use based on some previous first content data. This performance metric is obtained from performance monitoring module 320 and application performance module 324, see Par 28 and 29.
generating, based at least on the one or more performance statistics, telemetry data associated with processing second content data for the application,
“In other words, the network device 106 estimates how the CPU might perform if the processing were to be performed on the CPU and how the GPU might perform if the processing were to be performed on the GPU. Based on the performance metric estimations, the network device 106 can then determine whether to perform the service on the CPU or the GPU.” – Par 14, Lines 13-18
NOTE: Second content data can be understood as the next instance of the application being run by the system. When using the performance history obtained from the first content data, generating some form of telemetry data must be inherit/obvious in order to guide the decision on what data transport processes should go to the CPU or GPU.
the telemetry data indicating to execute one or more first data transport processes using the one or more first processors
“Accordingly, in such an embodiment, if the estimated latency does not meet the maximum latency requirement of the application (i.e., is not less than), the network device may determine the application should be processed by the CPU 202 rather than being offloaded to the GPU 210 for execution.” – Par 52, Lines 11-15
NOTE: Min teaches that the decision to offload tasks to the GPU or processed by the CPU is based on a threshold which can be understood as telemetry data that is used to guide the decision. The CPU performs the network packet processing if the performance requirement is not met.
and to execute one or more second data transport processes using the one or more second processors;
“Otherwise, if the network device 106 determines to offload the application to the GPU 210, the method 500 advances to block 538, wherein the network device schedules the application for the GPU 210 (i.e., provides an indication to a scheduler of the GPU 210 to schedule the application for processing by the GPU 210).” – Par 54, Lines 6-11
NOTE: Min shows that when the decision to offload the tasks to the GPU is made, the GPU performs the network packet processing tasks assigned to it.
generating, based at least on the telemetry data and using the one or more first processors to process the second content data by executing the one or more first transport processes and the one or more second processors to process the second content data by executing the one or more second transport processes, processed content data associated with the application;
“Each VM may perform the relevant processing of the network packets based on the service for which they were configured using a graphics processing unit (GPU) of the network device 106 (see, e.g., the GPU 210 of FIG. 2) or a central processing unit (CPU) of the network device 106 (see, e.g., the CPU 202 of FIG. 2)” – Par 14, Lines 1-5
NOTE: Min teaches that the processors perform tasks for network packet processing such as encryption or video encoding, see Par 1 and Par 20. The device 106 which contains the CPU and GPU process these network packet data which can be understood as second content data that is processed to create processed content data.
Min does not teach generating, using the one or more first processors, the second content data for the application; and sending the processed content data to one or more client devices. However, Huang teaches generating, using the one or more first processors, the second content data for the application;
“The CPU 212 is configured to cooperate with the GPUs 213, 215, 217, to generate the data sets for the virtual cameras” – Col 7, Lines 47-49
NOTE: Huang teaches a CPU that generates new content data in the form of virtual views images to then be processed into a virtual view stream for the client. After the combination, the concept of using the one or more first processors to generate the content data for the application can be added to Min’s system for processing network packet tasks.
and sending the processed content data to one or more client devices.
In one embodiment, the virtual view broadcaster 210 is a cloud-based device, such as a cloud server (or servers), that generates content in the cloud to be streamed to the client computing device 230.” – Col 6, Lines 62-65
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings of Huang to use the one or more first processors to generate second content data for the application and send processed content data to client devices. One would be motivated to make this combination in order to process the new instance of the application (second content data) and process it using telemetry data indicating which tasks should go to the CPU or GPU in order to create processed content data to be delivered.
Regarding claim 10, the claim describes a system that performs the steps of a method as described in claim 1. Therefore, the system in claim 10 corresponds to the method disclosed in claim 1 and is rejected for the same reasons of obviousness above.
NOTE: Claim 10 refers specifically to a central processing unit and parallel processing unit as the respective one or more first and second processors. Please see rejection of claim 1 which shows a CPU and a GPU being used as a first and second processor.
Regarding claim 2, Min in view of Huang teaches the method of claim 1. Min further teaches wherein: the telemetry data represents a profile associated with the application, the profile being associated with the one or more performance statistics according to one or more processing metrics;
“For example, the system resource performance metrics may include such performance metrics as a CPU utilization, a GPU utilization, a memory utilization, cache hits/misses, a GPU thread occupancy, a cache miss rate, a translation lookaside buffer (TLB) miss, a page fault, etc.” – Par 28, Lines 14-28
NOTE: The systems resource performance metrics are obtained in network device 106, see Par 47. This collection of data indicative of utilization or other statistics of the components in the network device can be interpreted as a profile associated with the application.
and the method further comprises: determining the one or more first data transport processes for the one or more first processors to execute based at least on the profile indicating that the one or more first processors are to execute the one or more first data transport processes;
“Accordingly, in such an embodiment, if the estimated latency does not meet the maximum latency requirement of the application (i.e., is not less than), the network device may determine the application should be processed by the CPU 202 rather than being offloaded to the GPU 210 for execution.” – Par 52, Lines 11-15
NOTE: Min teaches using telemetry data, which is collected by network device 106, a decision is made to determine whether the network packet workload is given to the CPU or the GPU. The collection of the telemetry data guides in this decision to either let the CPU process the task or offload it to the GPU. A profile that suggests that the performance requirement is not met will lead to the CPU handing the first data transport processes.
and determining the one or more second data transport processes for the one or more second processors to execute based at least on the profile indicating that the one or more second processors are to execute the one or more second data transport processes.
“Otherwise, if the network device 106 determines to offload the application to the GPU 210, the method 500 advances to block 538, wherein the network device schedules the application for the GPU 210 (i.e., provides an indication to a scheduler of the GPU 210 to schedule the application for processing by the GPU 210).” – Par 54, Lines 6-11
NOTE: If the collected telemetry data determines that the GPU would be more efficient to handle a task, then the tasks is offloaded from the CPU to the GPU instead.
Regarding claim 11, the claim describes a system that performs the steps of a method as described in claim 2. Therefore, the system in claim 11 corresponds to the method disclosed in claim 2 and is rejected for the same reasons of obviousness above.
NOTE: The limitations of claim 11 differ from claim 2 in that the profile is associated with the central processing unit and parallel processing unit. Please see rejection claim 2 which shows a profile associated with the use of a CPU and GPU
Regarding claim 3, Min in view of Huang teaches the method of claim 2. Min further teaches wherein: the one or more performance statistics are associated with one or more previous sessions associated with the application;
“In some embodiments, additional and/or alternative analysis may be performed to determine whether to run the application on the GPU 210 or the CPU 202, such as a performance history of running like applications on the GPU 210” – Par 37, Lines 6-10.
NOTE: Min’s use of performance history of the applications ran on the GPU implies that a previous session is analyzed to obtain performance statistics regarding that previously executed instance.
and the generating of the profile is for a current session associated with the application.
“For example, the system resource performance metrics may include such performance metrics as a CPU utilization, a GPU utilization, a memory utilization, cache hits/misses, a GPU thread occupancy, a cache miss rate, a translation lookaside buffer (TLB) miss, a page fault, etc.” – Par 28, Lines 14-28
NOTE: Min teaches the use of device 106 to obtain a collection of telemetry data which can be understood as a profile. This profile would need to be relevant towards the current execution of the application since the telemetry data obtained is used to determine how the data transport tasks should be distributed for the current session.
Regarding claim 5, Min in view of Huang teaches the method of claim 1. Min further teaches determining, based at least on second telemetry data associated with processing third content data for a second application, to execute the one or more second data transport processes using the one or more first processors; generating, based at least on using the one or more first processors to execute the one or more second data transport processes for third content data, second processed content data associated with the second application;
“Each VM may perform the relevant processing of the network packets based on the service for which they were configured using a graphics processing unit (GPU) of the network device 106 (see, e.g., the GPU 210 of FIG. 2) or a central processing unit (CPU) of the network device 106 (see, e.g., the CPU 202 of FIG. 2)” – Par 14, Lines 1-6
NOTE: Min teaches that the methods for processing network packet content can be executed multiple times which can be understood as separate sessions of the application. The limitations of claim 5 can be interpreted as another instance occurring. Processing statistics that make up the telemetry data associated with a new “third” content data that guides the processing of the network packets can be obtain for a new session. The CPU and GPU in device 106 are assigned their respective data transport tasks again to process this new content data to create processed content data.
Min does not teach sending the second processed content data to one or more second client devices. sending the second processed content data to one or more second client devices. However, Huang teaches sending the second processed content data to one or more second client devices
In one embodiment, the virtual view broadcaster 210 is a cloud-based device, such as a cloud server (or servers), that generates content in the cloud to be streamed to the client computing device 230.” – Col 6, Lines 62-65
After the combination, the concept of sending processed content data to a client device as taught by Huang can be used be Min in order to perform the necessary packet processing tasks to create a processed network packet and be delivered to a client device.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings of Huang to send the second processed content data to a client device. One would be motivated to make this combination to ensure that the new processed content data from the new application session is sent to the viewer so that the content they see on their device is updated. For example, in the case of video streaming, multiple instances of the application are performed so that the viewer can see new content on their device.
Regarding claim 14, the claim describes a system that performs the steps of a method as described in claim 5. Therefore, the system in claim 14 corresponds to the method disclosed in claim 5 and is rejected for the same reasons of obviousness above.
Regarding claim 8, Min in view of Huang teaches the method of claim 1 Min teaches wherein the generating the processed content data comprises: causing the third content data to be provided to the one or more second processors;
“The method 500 begins with block 502, in which the network device 106 determines whether a GPU offload request was received. In other words, the network device 106 determines whether a network packet processing application is to be offloaded to the GPU 210 (i.e., processed by the GPU 210 rather than the CPU 202).” – Par 45, Lines 4-9
NOTE: Min teaches that the network device 106 determines when a network processing packet processing application should be offloaded from a first processor (CPU) to another processor (GPU). Content data regarding the packet processing application must be present in order to make the decision. Third content data can be interpreted as the next context data generated from the next application session after the second content data was processed from the previous session.
and generating, based at least the telemetry data and using the one or more second processors to process the third content data by executing the one or more second transport processes, the processed content data associated with the application.
“Additionally or alternatively, the GPU 210 is generally capable of parallelizing network packet processing (e.g., internet protocol (IP) forwarding, hashing, pattern matching, etc.) via the processor cores of the GPU 210. Accordingly, the GPU 210 can be an alternative to the CPU 202 (i.e., the CPU) for performing at least a portion of the processing of the network packet.” – Par 20, Lines 10-16
NOTE: Min teaches that the second processor (GPU) can be used to handle network packet processing tasks based off the telemetry data indicating what tasks should be offloaded to the GPU. This would then create processed network packet content data to be delivered to client devices.
Min does not teach wherein the generating the processed content data comprises: generating, based at least on the telemetry data and using the one or more first processors to process the second content data by executing the one or more first transport processes, third content data. However, Huang teaches wherein the generating the processed content data comprises: generating, based at least on the telemetry data and using the one or more first processors to process the second content data by executing the one or more first transport processes, third content data.
“The CPU 212 is configured to cooperate with the GPUs 213, 215, 217, to generate the data sets for the virtual cameras” – Col 7, Lines 47-49
NOTE: Huang teaches a CPU that generates new content data in the form of virtual views images to then be processed into a virtual view stream for the client. This process can be repeated in the next instance of the application to create a “third content data” After the combination, the concept of using the one or more first processors to generate the content data for the application can be added to Min’s system for determining when to offload processing network packet tasks to the GPU or process in the CPU.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings of Huang to generate third content data using the first processors (CPU) by executing first data transport tasks. One would be motivated to make this combination to update the content of what the viewer can see since the third content’s generation is based on the processing of the content data that came before it.
Regarding claim 9, Min in view of Huang teaches the method of claim 1. Min further teaches wherein: the one or more first transport processes comprise one or more of: a first packetization; a first forward error correction; a first encoding; or a first encryption;
Similar to the GPU scheduler module 342, the CPU scheduler module 344 is configured to schedule processes (e.g., the network processing application) for execution by the CPU 202
Min teaches a first processor that handles tasks related to network packet processing which can be understood as packetization.
and the one or more second transport processes comprise one or more of: a second packetization; a second forward error correction; a second encoding; or a second encryption.
For example, in some embodiments, the processor cores of the GPU 210 may be configured to individually handle 3D rendering tasks, blitter (e.g., 2D graphics), video, and video encoding/decoding tasks, by providing electronic circuitry that can perform mathematical operations rapidly using extensive parallelism and many concurrent threads. Additionally or alternatively, the GPU 210 is generally capable of parallelizing network packet processing (e.g., internet protocol (IP) forwarding, hashing, pattern matching, etc.) via the processor cores of the GPU 210.” – Par 20
Min teaches that the second transport process performed by the second processor (GPU) can be encoding/decoding tasks and packetization processing tasks.
Regarding claim 12, Min in view of Huang taches the system of claim 10. Min further teaches wherein the one or more performance statistics indicate at least: one or more first performance values associated with the one or more central processing units processing second content data by executing the one or more first data transport processes;
“The system resource performance metrics may include any data indicative of a utilization or other statistic of one or more physical or virtual components or available resources of the network device 106. For example, the system resource performance metrics may include such performance metrics as a CPU utilization, a GPU utilization, a memory utilization, cache hits/misses, a GPU thread occupancy, a cache miss rate, a translation lookaside buffer (TLB) miss, a page fault, etc.” – Par 28, Lines 10-18
NOTE: Min teaches that one of the system resource performance metrics may include CPU utilization. This can be interpreted as a performance value associated with the one or more central processing units processing content data. CPU utilization would indicate what resources are being used to process the first content data transport processes.
and one or more second performance values associated with the one or more parallel processing units processing the second content data by executing the one or more second data transport processes.
“The system resource performance metrics may include any data indicative of a utilization or other statistic of one or more physical or virtual components or available resources of the network device 106. For example, the system resource performance metrics may include such performance metrics as a CPU utilization, a GPU utilization, a memory utilization, cache hits/misses, a GPU thread occupancy, a cache miss rate, a translation lookaside buffer (TLB) miss, a page fault, etc.” – Par 28, Lines 10-18
NOTE: Min teaches that one of the system resource performance metrics may include GPU utilization. This can be interpreted as a performance value associated with the one or more parallel processing units processing content data. GPU utilization would indicate what resources are being used to process the second content data transport processes.
Regarding claim 18, Min in view of Huang teaches the system of claim 10. Min does not teach wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system implementing one or more large language models (LLMs);a system for generating synthetic data; a system incorporating one or more virtual machines (VMs);a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. However, Huang teaches wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system implementing one or more large language models (LLMs);a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
“The virtual view broadcaster 210 can be implemented on one or more network (e.g., cloud) servers to render content in a real time (or semi-real time) content engine and to create virtual view images that can be stitched together into a 360 degree video for streaming to virtual viewers in real-time or semi-real time.” – Par 27
NOTE: Huang discloses the use of cloud servers for rendering content in real time which obviously use the cloud computing resources
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min to incorporate the teachings of Huang to have a system implemented, at least partially, using cloud computing resources. One would be motivated to make this combination since cloud computing would allow for real-time update of processed content and secure measures for delivering content through encryption.
Claim(s) 4, 7, and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Min, Huang, and Zhang et al (US 20250001312 A1), hereinafter Zhang.
Regarding claim 4, Min in view of Huang teaches the method of claim 1. Min does not teach determining, based at least on the one or more performance statistics, that the one or more second data transport processes being executed by the one or more first processors caused one or more errors associated with streaming the first content data; and determining, based at least on the one or more errors associated with streaming the second content data, to execute the one or more second data transport processes using the one or more second processors, wherein the generating the telemetry data is based at least on the determining to execute the one or more second data transport processes using the one or more second processors. However, Zhang teaches determining, based at least on the one or more performance statistics,
“In one implementation, performance for a host device 108, especially for a determination whether the host device 108 is suitable for hosting a requested application session, is determined based at least in part on a set of application-specific performance metrics. – Par 24
that the one or more second data transport processes being executed by the one or more first processors caused one or more errors associated with streaming the first content data;
“In some examples, these metrics may include, frame rate, frame rate consistency, input latency (e.g., delay between local user input and screen rendition), streaming metrics, including network outbound bandwidth and network latency, encoding capabilities such as encoded frames per second, percentage of frame drops, etc. These metrics can be affected by the capability of hardware components, e.g., CPU, GPU, memory, storage I/O, network adapter, and the like for the host device 108.” – Par 24
NOTE: Zhang teaches the performance metrics regarding frame rate/frame dropping, which can be understood as errors associated with processing first content data, can be monitored from hardware components such as the CPU (first processor).
and determining, based at least on the one or more errors associated with streaming the second content data, to execute the one or more second data transport processes using the one or more second processors, wherein the generating the telemetry data is based at least on the determining to execute the one or more second data transport processes using the one or more second processors.
“To do so, the system resource management module 340 may be capable of instantiating (i.e., creating) VMs, suspending VMs, shutting down (i.e., closing) VMs, and redirecting network traffic to either the GPU 210 or the CPU 202 for more efficient processing (e.g., is faster, is an improvement, a more efficient level of resource usage, etc.). In some embodiments, the “more efficient” processor may be determined by a GPU performance metric that is higher or lower than a CPU performance metric, depending on the particular performance metric being compared.” – Min Par 38, Lines 7-16
NOTE: After the combination, the performance statistics which may indicate an error from the CPU as taught by Zhang can be added to Min’s method of offloading the network packet processing tasks from the CPU to the GPU. This would allow the performance metrics obtained by Min to determine that the CPU may have some error associated with performing the processing tasks in order to produce processed content and would then indicate to the system to offload the task from the CPU to the GPU.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to modify Min by incorporating the teachings of Zhang to determine that the first data transport processes cause the first processor to have an error and determine, based off the errors, to execute the send data transport processes on the second processor. One would be motivated to make this combination as it would yield a predictable result of being able to cover for the downtime caused by the error by offloading the task from one processor to another. This would allow the content data to continue processing efficiently.
Regarding claim 13, the claim describes a system that performs the steps of a method as described in claim 4. Therefore, the system in claim 13 corresponds to the method disclosed in claim 4 and is rejected for the same reasons of obviousness above.
NOTE: the limitations of claim 13 refer to errors associated with processing content data due to executing first data transport tasks on the central processing unit and executing second data transport processes on the parallel processing unit. Please see rejection of claim 4 which shows the use of a CPU and GPU respectively.
Regarding claim 7, Min in view of Huang teaches the method of claim 1. Min does not teach wherein the one or more performance statistics comprise one or more of: one or more first errors that occur based at least on the one or more first processors executing at least one of the one or more first data transport processes or the one or more second data transport processes; one or more second errors that occur based at least on the one or more second processors executing at least one of the one or more first data transport processes or the one or more second data transport processes; one or more bitrates associated with the application; or one or more frame rates associated with the application. However, Zhang teaches wherein the one or more performance statistics comprise one or more of: one or more first errors that occur based at least on the one or more first processors executing at least one of the one or more first data transport processes or the one or more second data transport processes; one or more second errors that occur based at least on the one or more second processors executing at least one of the one or more first data transport processes or the one or more second data transport processes; one or more bitrates associated with the application; or one or more frame rates associated with the application.
“In some examples, these metrics may include, frame rate, frame rate consistency, input latency (e.g., delay between local user input and screen rendition), streaming metrics, including network outbound bandwidth and network latency, encoding capabilities such as encoded frames per second, percentage of frame drops, etc. These metrics can be affected by the capability of hardware components, e.g., CPU, GPU, memory, storage I/O, network adapter, and the like for the host device 108.” – Par 24
Zhang teaches that one of the performance statistics comprises frame drops associated with the executing the application on the CPU or GPU. After the combination, the identification of frame drops when executing an application on the processors as taught by Zhang can be added to Min’s system for processing network packet tasks to identify frame dropping during execution of these data transport tasks which may slow down the generation of processed content.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to modify Min by incorporating the teachings of Zhang to include the measurement of frame rates associated with the application. It is beneficial to measure the frame rate over time when the processors are performing in order to see any frame dropping. This error is a sign that that one (or both) of the processing units are being overloaded. The frame rate measurement is crucial to know in order to prevent issues from bottle necking or system resource utilization.
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Min, Huang, and Statton (US 12348834 B1), hereinafter Statton.
Regarding claim 6, Min in view of Huang teaches the method of claim 1. Min does not teach receiving, from the one or more client devices, a request to begin a session associated with the application, wherein generating the telemetry data is further based at least on the request. However, Statton teaches receiving, from the one or more client devices, a request to begin a session associated with the application,
“Specifically, the service provider can encode content to be provided to requesting client devices as a set of encoded content segments transmitted in accordance with a streaming protocol.” – Col 1, Lines 42-45.
NOTE: For the content to be transmitted after a request from the client, there obviously would be a need for the client to request a session associated with the application.
wherein generating the telemetry data is further based at least on the request
“In other words, the network device 106 estimates how the CPU might perform if the processing were to be performed on the CPU and how the GPU might perform if the processing were to be performed on the GPU. Based on the performance metric estimations, the network device 106 can then determine whether to perform the service on the CPU or the GPU.” – Par 14, Lines 13-18
NOTE: Min teaches obtaining telemetry data based on parameter statistics during the execution of an application’s session. After the combination, the concept of requesting a session of the application from a client device as taught by Statton can be added to Min who teaches analyzing performance statistics to generate telemetry data during the current instance of the application.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to modify Min by incorporating the teachings of Statton to generate telemetry data based on the client’s request to start a session of the application. One would be motivated to make this combination to provide control to the client on when the application should be executed. The request from the client would then allow for the processing decisions to be specific to the needs of that particular session. It also prevents and unnecessary monitoring of performance since the telemetry data is obtained only when the request to start a session is made.
Regarding claim 15, the claim describes a system that performs the steps of a method as described in claim 6. Therefore, the system in claim 15 corresponds to the method disclosed in claim 6 and is rejected for the same reasons of obviousness above.
Claim(s) 16 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Min, Huang, and Cuervo et al (“MAUI: Making Smartphones Last Longer with Code Offload”), hereinafter Cuervo.
Regarding claim 16, Min in view of Huang teaches the system of claim 10. Min teaches wherein the determination of the telemetry data comprises: determining, based at least on the one or more performance statistics, the one or more central processing units include a first performance when executing the one or more first transport processes and the one or more second transport processes;
“Accordingly, in such an embodiment, if the estimated latency does not meet the maximum latency requirement of the application (i.e., is not less than), the network device may determine the application should be processed by the CPU 202 rather than being offloaded to the GPU 210 for execution.” – Par 52, Lines 11-15
NOTE: Min shows that network packet processing tasks can be executed solely on the CPU. Performance monitor module 320 monitors the data indicative of utilization of the components of the network device 106 which contains the CPU and GPU, see par 28 and 29. This can be interpreted that when the CPU executes the data transport processes without offloading, the performance statistics of the CPU is obtained to get the “first performance”.
determining, based at least on the one or more performance statistics, the one or more central processing units include a second performance when executing the one or more first transport processes while refraining from executing the one or more second transport processes;
“In other words, the network device 106 determines whether a network packet processing application is to be offloaded to the GPU 210 (i.e., processed by the GPU 210 rather than the CPU 202).”
Min shows that some network packet processing tasks may be offloaded to the GPU instead of processing on the CPU. The performance monitor module can then monitor the data indicative of the utilization of the components in the network device 106 which contains the CPU and GPU. This can be interpreted that when the CPU offloads data processing tasks to the GPU it no stops executing the tasks on the CPU and the performance statistic of the CPU can be obtained to get the “second performance”.
Min does not teach determining, based at least on the first performance and the second performance, the telemetry data associated with the application. However, Cuervo teaches determining, based at least on the first performance and the second performance, the telemetry data associated with the application.
“MAUI instruments each method of an application to determine the cost of offloading it, such as the amount of state that needs to be transferred for remote execution, and the benefit of offloading it, such as the number of CPU cycles saved due to the offload. Additionally, MAUI continuously measures the network connectivity to the infrastructure, estimating its bandwidth and latency. All these variables are used to formulate an optimization problem whose solution dictates which methods should be offloaded to the infrastructure and which should continue to execute locally on the smartphone.” – Section 3, Par 4
NOTE: Cuervo teaches a profiler that collects performance statistics of the CPU executing the tasks alone locally and if it were to offload its tasks remotely, (“a profiler, which instruments the program and collects measurements of the program’s energy and data transfer requirements.” – Section 3, Par 5, Lines 5-7). Both of these performance statistics are considered to decide whether the system will benefit from continuing with executing the tasks on the CPU or offload it to another processor. After the combination, the concept of taking a first and second performance metric to make a decision on how to process a task as taught by Cuervo can be incorporated by Min to generate telemetry data that considers the first and second performance metric to decide whether to offload the network packet processing tasks to a GPU or continue executing in the CPU.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings of Cuervo to determine the telemetry data by considering the first and second performance of the central processing units. One would be motivated to make this combination to allow the teachings of Min to compare the benefits of either letting the CPU handle the processing of the network packet processing tasks or offloading it to the GPU. This would allow the system to make a better decision as to when to offload the tasks so that the content data may be processed efficiently for delivery to a client device.
Regarding claim 17, Min in view of Huang and Cuervo teach the system of claim 16. Min further teaches wherein the one or more processors are further to: determine one or more performance requirements associated with the application, wherein the telemetry data associated with the application is further determined based at least on the one or more performance requirements.
“ In block 536, the network device 106 determines whether to offload the application to the GPU 210 based on at least a portion of the estimated GPU performance metrics, the estimated CPU performance metrics, and the system resource performance metrics. For example, in an embodiment wherein the application has a threshold application performance requirement, such as a maximum latency requirement, one of the estimated GPU performance metrics may be an estimated latency associated with the GPU 210 processing the application. Accordingly, in such an embodiment, if the estimated latency does not meet the maximum latency requirement of the application (i.e., is not less than), the network device may determine the application should be processed by the CPU 202 rather than being offloaded to the GPU 210 for execution.” – Par 52
NOTE: Min teaches that to determine whether or not to offload the application to the GOU, a threshold application performance requirement must be met. This can be interpreted to show that the telemetry data which indicates where the data transport process should execute must meet a performance requirement.
Claim(s) 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Min, Huang, and Schmit et al (US 20100166055 A1), hereinafter Schmit.
Regarding claim 19, Min teaches a processor comprising: one or more graphics processing units to: generate content data associated with an application;
“Each VM may perform the relevant processing of the network packets based on the service for which they were configured using a graphics processing unit (GPU) of the network device 106 (see, e.g., the GPU 210 of FIG. 2) or a central processing unit (CPU) of the network device 106 (see, e.g., the CPU 202 of FIG. 2)” – Par 14, Lines 1-5
NOTE: Min teaches that network packet processing, which can be interpreted as content data associated with an application, can be processed using a GPU.
and process the content data using one or more first data transport processes to generate second content data, wherein the one or more data first transport processes are determined based at least on telemetry data associated with the application;
“Accordingly, in such an embodiment, if the estimated latency does not meet the maximum latency requirement of the application (i.e., is not less than), the network device may determine the application should be processed by the CPU 202 rather than being offloaded to the GPU 210 for execution.” – Par 52, Lines 11-15
Min teaches that telemetry data associated with the application in the form of a latency threshold may determine that the first data transport processes for processing network packets should process the content data in the CPU (one or more first processors). The first data transport processes executed then generates second content data based on the first content data
Min does not teach one or more central processing units to: obtain the second content data from the one or more parallel processing units; process the second content data using one or more second data transport processes to generate processed content data. However, Schmit teaches one or more central processing units to: obtain the second content data from the one or more parallel processing units;
“In a GPU-based system, the GPU may be programmed to offload some or all of the video encoding tasks to the CPU in order to achieve additional performance benefits.” – Par 37
NOTE: Schmit teaches that the GPU may offload the video encoding tasks (content data associated with the application) to the CPU. This can be interpreted as the central processing unit obtaining the second content data from the parallel processor unit. After the combination, the method of obtaining second content data from the GPU to the CPU can be incorporated with Min’s system for processing network packets. This would allow offloading data transport tasks from the CPU to the GPU and vice versa which would improve the efficiency of processing processed content data.
process the second content data using one or more second data transport processes to generate processed content data
“Each VM may perform the relevant processing of the network packets based on the service for which they were configured using a graphics processing unit (GPU) of the network device 106 (see, e.g., the GPU 210 of FIG. 2) or a central processing unit (CPU) of the network device 106 (see, e.g., the CPU 202 of FIG. 2)” – Min Par 14, Lines 1-5
NOTE: Content data is processes using the second data transport processes executed on the GPU to create processed content data to be delivered. After the combination, the concept of obtaining content data from the GPU to the CPU as taught by Schmit can be used in Min’s system form processing network packets in order to generate the processed content data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings of Schmit to have the central processing unit obtain content data from the parallel processor unit. One would be motivated to make this combination to create system that performs more efficiently by offloading tasks from one processor to another. This would prevent the generation of processed content data from stalling or slowing down.
Min in view of Schmit still does not teach and cause the processed content data to be streamed to the one or more client devices. However, Huang teaches and cause the processed content data to be streamed to the one or more client devices.
“The virtual view broadcaster 210 is configured to generate a virtual view stream for the client computing device 230 from one or more virtual cameras positioned in a computer application based on application data” – Col 6, Lines 59-62.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min by incorporating the teachings if Huang to stream the processed content data to one or more client devices. One would be motivated to make this combination since it would yield the predicted result of allowing the client device to receive updated content data.
Regarding claim 20, Min in view of Huang and Schmidt teaches the processor of claim 19. Min does not teach wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system implementing one or more large language models (LLMs);a system for generating synthetic data; a system incorporating one or more virtual machines (VMs);a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. However, Huang teaches wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system implementing one or more large language models (LLMs);a system for generating synthetic data; a system incorporating one or more virtual machines (VMs);a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
“The virtual view broadcaster 210 can be implemented on one or more network (e.g., cloud) servers to render content in a real time (or semi-real time) content engine and to create virtual view images that can be stitched together into a 360 degree video for streaming to virtual viewers in real-time or semi-real time.” – Par 27
NOTE: Huang discloses the use of cloud servers for rendering content in real time which obviously use the cloud computing resources.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Min to incorporate the teachings of Huang to have a system implemented, at least partially, using cloud computing resources. One would be motivated to make this combination since cloud computing would allow for real-time update of processed content and secure measures for delivering content through encryption.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID V. NGUYEN whose telephone number is (571)272-6111. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID VAN NGUYEN/
Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights