DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax, Regular postal mail, or EFS Web (PTO/SB/439).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter.
During examination, the claims must be interpreted as broadly as their terms reasonably allow. In re American Academy of Science Tech Center, 367 F.3d 1359, 1369, 70 U.S.P.Q.2d 1827, 1834 (Fed. Cir. 2004). Independent claim 16 recites a “computer-readable medium,” which is not comprehensively defined by the specification. The broadest reasonable interpretation of a claim drawn to a computer readable medium covers forms of transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. Transitory propagating signals are non-statutory subject matter. In re Nuijten, 500 F.3d 1346, 1356-57, 84 U.S.P.Q.2d 1495, 1502 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). See also Subject Matter Eligibility of Computer Readable Media, 1351 Off. Gaz. Pat. Office 212 (Feb. 23, 2010). Examiner suggests adding the word “non-transitory.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 5-11, 13-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Palermo et al. (US 20210075730) in view of Wang et al. (US 20210141676).
As per claim 1, Palermo teaches the invention substantially as claimed including an apparatus ([0112], the DLB 1800 of FIG. 18 is an example implementation of the DLB 202 of FIG. 2, the DLB 304 of FIG. 3, the DLB 410 of FIGS. 4-6, the DLB 808 of FIG. 8, the DLB 1008 of FIG. 10, and/or the DLB 1404 of FIG. 14. The DLB 1800 includes an example configuration controller 1810, an example event controller 1820, an example queue controller 1830, an example reorder controller 1840, an example aggregation controller 1850, an example arbitration controller 1860, an example telemetry controller 1870, and example storage 1880) comprising:
an interface and circuitry, coupled to the interface ([0116], the event controller 1820 implements front-end logic (e.g., front-end logic circuitry) of the DLB 1800 that can interface with a NIC, a producer core, etc.), the circuitry to perform load balancing of requests ([0080], application 212 directs the data flow 214 from the input 216 to the producer core 204 via a first one of the NICs 210; [0113], the configuration controller 1810 can invoke the DLB 1800 to execute, perform, and/or otherwise effectuate load balance operations associated with a multi-core computing system (e.g., the multi-core computing system 200, 302, 402 of FIGS. 2-6); and [0116], event controller 1820 can identify an incoming elephant flow from a NIC and invoke the queue controller 1830 and/or, more generally, the DLB 1800, to begin processing computing tasks associated with the incoming elephant flow, such as identifying queues to store data pointers, identifying available one(s) of worker cores, etc., and/or a combination thereof) received from one or more cores in a central processing unit (CPU) ([0062], the HQM enables pipelined processing of data (e.g., data packets in a cellular or other wireless network) between multiple producers (e.g., producer cores) and multiple consumers (e.g., consumer cores). A producer core can offload scheduling of computing tasks to the example HQM to allocate a workload by the producer core to an available consumer core of a plurality of consumer cores; and [0136], the producers 1904 are cores (e.g., producer cores) of a multi-core processor, such as ones of the producer core 408 of the processor 406 of FIGS. 4-6. In this example, the consumers 1906 are cores (e.g., consumer cores) of a multi-core processor, such as ones of the consumer core 414 of the processor 406 of FIGS. 4-6) wherein:
the circuitry comprises:
first circuitry to selectively perform ordering of requests from the one or more cores ([0094], The DLB 410 dynamically distributes packets to available one(s) of the worker cores 412. For example, the DLB 410 can distribute the enqueued data pointers to available one(s) of the worker cores 412 based on ordered scheduling; and [0095], the DLB 410 can use ordered queues. For example, the DLB 410 can use ordered queues when there are one or more producers (e.g., one or more producer cores) queueing up to communicate to multiple consumers (e.g., consumer cores) with a requirement to dynamically balance the workload across the multiple consumers and then to restore the original enqueue order),
second circuitry to allocate the requests into queue elements prior to allocation to one or more receiver cores of the one or more cores to process the requests ([0081], one or more of the DLBs 202 can enqueue data (e.g., add and/or otherwise place an element, such as a queue element, onto a queue) from the producer core 204 and dequeue (e.g., remove an element, such as a queue element, from a queue) the enqueued data to one(s) of the worker cores 208, such as a first worker core (W1), a second worker core (W2), and/or a third worker core (W3) of the worker cores 208), and
third circuitry to perform:
adjust a number of queues associated with a core of the one or more cores by changing a number of consumer queues (CQs) allocated to a single domain (Wang, 20210141676, [0027], Multiple data queues can mapped to the same CPU core, and this can be configured by a user; and [0042], detecting the overloaded CPU core by monitoring the receive queue length, and re-configuring the RSS indirection table such that the congested CPU core is mapped to fewer queues) and
adjust a number of target cores in a group of target cores to be load balanced (Palermo, [0063], the DLB can scale (e.g., dynamically scale) up a quantity of consumer cores used to facilitate a distribution, transmission, and/or processing of an elephant flow to optimize and/or otherwise improve a throughput, a line rate, a bandwidth, etc., associated with the elephant flow).
Palermo fails to specifically teach, third circuitry to perform: adjust a number of queues associated with a core of the one or more cores by changing a number of consumer queues (CQs) allocated to a single domain.
However, Wang teaches, third circuitry to perform:
adjust a number of queues associated with a core of the one or more cores by changing a number of consumer queues (CQs) allocated to a single domain (Wang, 20210141676, [0027], Multiple data queues can mapped to the same CPU core, and this can be configured by a user; and [0042], detecting the overloaded CPU core by monitoring the receive queue length, and re-configuring the RSS indirection table such that the congested CPU core is mapped to fewer queues).
Palermo and Wang are analogous because they are each related to load balancing. Palermo teaches a method of load balancing in a distributed environment using queues for packet transmission. ([0063], the HQM implements a load balancer (e.g., a DLB) to improve load balancing and workload distribution in computer network architectures. In such disclosed examples, the DLB can scale (e.g., dynamically scale) up a quantity of consumer cores used to facilitate a distribution, transmission, and/or processing of an elephant flow to optimize and/or otherwise improve a throughput, a line rate, a bandwidth, etc., associated with the elephant flow; and [0095], the DLB 410 can use ordered queues. For example, the DLB 410 can use ordered queues when there are one or more producers (e.g., one or more producer cores) queueing up to communicate to multiple consumers (e.g., consumer cores) with a requirement to dynamically balance the workload across the multiple consumers and then to restore the original enqueue order). Wang teaches a method of load balancing among cores using a NIC including dynamically assigning queues to a plurality of cores. (Abstract, NIC can also be configured to determine whether the CPU core is overloaded based on the receive queue length. The NIC can also be configured to redirect data packets that were targeted from the first receive queue to the CPU core to another CPU core responsive to a determination that the CPU core is overloaded; [0027], Multiple data queues can mapped to the same CPU core, and this can be configured by a user; and [0042], detecting the overloaded CPU core by monitoring the receive queue length, and re-configuring the RSS indirection table such that the congested CPU core is mapped to fewer queues). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the teachings of Palermo would be modified with the queue redistribution mechanism taught by Wang resulting in a system that provides load balancing among redistributable queues. Therefore, it would have been obvious to combine the teachings of Palermo and Wang.
As per claim 2, Palermo teaches, wherein the requests comprise one or more of: a combined ATOMIC and ORDERED flow type ([0063], the DLB can distribute the elephant flow based on a scheduling type (e.g., atomic scheduling, ordered scheduling, etc.) to one or more consumer cores), a load balancer descriptor, or a memory write request.
As per claim 5, Palermo teaches, wherein the third circuitry is to order memory space writes from multiple caching agents (CAs) prior to output to a consumer core ([0058], A producer (e.g., a data producer) can refer to an agent (e.g., a hardware agent, a software agent, etc.) that places a type of message onto a queue (e.g., a buffer, a computing queue, a computing task queue, etc.); and [0095], the DLB 410 can use ordered queues) and load balance memory write requests from multiple home agents (HAs) ([0058], consumer (e.g., a data consumer) can refer to the same agent or a different agent that can remove the message from the queue for processing. In some instances, the message can refer to machine-readable data representative of one or more pointers (e.g., one or more identifiers) that correspond to data in memory (e.g., non-volatile memory, volatile memory, etc.) or other indications of a computing task to be executed; and [0095], the DLB 410 can use ordered queues. For example, the DLB 410 can use ordered queues when there are one or more producers (e.g., one or more producer cores) queueing up to communicate to multiple consumers (e.g., consumer cores) with a requirement to dynamically balance the workload across the multiple consumers and then to restore the original enqueue order).
As per claim 6, Palermo teaches, wherein the third circuitry is to process a load balancer descriptor associated with a packet transmission or packet receipt ([0056], the HQM can enable pipelined packet processing and support hundreds of millions of queue management and load balancing operations per second for run-to-completion and pipelined network processing approaches; and [0113], the configuration controller 1810 can invoke the DLB 1800 to execute, perform, and/or otherwise effectuate load balance operations associated with a multi-core computing system (e.g., the multi-core computing system 200, 302, 402 of FIGS. 2-6).
As per claim 7, Palermo teaches, wherein the third circuitry is to manage buffer allocation ([0127], the reorder controller 1840 generates, maintains, and/or otherwise operates a buffer (e.g., a reorder buffer) to store enqueued data prior to moving the enqueued data to a different queue. For example, the reorder controller 1840 can reorder data packets that have been processed separately and/or out-of-order into a single stream for a subsequent task (e.g., a reception or transmission of the stream). In such examples, the reorder controller 1840 can reorder the data packets by reordering the corresponding data pointers. In some examples, the reorder controller 1840 implements reorder logic, such as reorder logic circuitry).
As per claim 8, Palermo teaches, comprising the CPU communicatively coupled to the circuitry to perform load balancing of requests ([0136], system 1900 includes the DLB 1902, example producers 1904, and example consumers 1906. In this example, the producers 1904 are cores (e.g., producer cores) of a multi-core processor, such as ones of the producer core 408 of the processor 406 of FIGS. 4-6. In this example, the consumers 1906 are cores (e.g., consumer cores) of a multi-core processor, such as ones of the consumer core 414 of the processor 406 of FIGS. 4-6).
As per claim 9, Palermo teaches, comprising a server comprising the CPU, the circuitry to perform load balancing of requests, and a network interface device ([0079], multi-core computing system 200 including an example processor 201 including example dynamic load balancers (DLBs) 202. For example, the multi-core computing system 200 can implement one of the DUs 122, the CUs 124, the core devices 126, etc., of FIG. 1. The multi-core computing system 200 includes an example producer core 204, an example consumer core 206, example worker cores 208, example NICs 210, and an example application (e.g., a firmware and/or software application) 212), wherein the circuitry to perform load balancing of requests is to load balance operations of the network interface device ([0098], the producer core 408 can obtain example data flows (FLOW 1, 2, & 3) 602 from a producer (e.g., the NIC 316 of FIG. 3) and the DLB 410 can assign a first flow identifier (FLOW1), a second flow identifier (FLOW2), and a third flow identifier (FLOW3) to a respective one of the data flows 602. In such examples, the DLB 410 can distribute the data flows 602, cause the data flows 602 to be processed (e.g., by one(s) of the worker cores 412), re-order the data flows 602, and/or aggregate the data flows 602 based on the first through third flow identifiers).
As per claim 10, this is the “method claim” corresponding to claim 1 and is rejected for the same reasons. The same motivation used in the rejection of claim 10 is applicable to the instant claim.
As per claim 11, this claim is similar to claim 2 and is rejected for the same reasons.
As per claim 13, this claim is similar to claim 5 and is rejected for the same reasons.
As per claim 14, this claim is similar to claim 6 and is rejected for the same reasons.
As per claim 15, this claim is similar to claim 7 and is rejected for the same reasons.
As per claim 16, this is the “computer-readable medium claim” corresponding to claim 1 and is rejected for the same reasons. The same motivation used in the rejection of claim 1 is applicable to the instant claim.
As per claim 17, this claim is similar to claim 2 and is rejected for the same reasons.
As per claim 20, this claim is similar to claim 5 and is rejected for the same reasons.
Claims 3, 12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Palermo-Wang as applied to independent claims 1, 10, and 16 and in further view of Tsirkin et al. (EP 4250106).
As per claim 3, Tsirkin teaches, wherein the adjust a number of queues associated with a core by changing a number of CQs allocated to a single domain comprises adjust a number of queue identifiers (QIDs) associated with the core (Abstract, the first I/O queue handling thread is associated with a first set of one or more queue identifiers, and each queue identifier identifies a queue being handled by the first I/O queue handling thread, and, responsive to determining that the preemption flag is equal to the first value, transferring the first set of one or more queue identifiers to a second I/O queue handling thread executing on a second processor. Transferring the first set of queue identifiers may include removing the one or more queue identifiers from the first set).
The combination of Palermo-Wang and Tsirkin are analogous because they are each related to load balancing. Palermo teaches a method of load balancing in a distributed environment using queues for packet transmission. Wang teaches a method of load balancing among cores using a NIC including dynamically assigning queues to a plurality of cores. Tsirkin teaches a method of load balancing for packet processing using queues and re-assignable queue identifiers. (Abstract, the first I/O queue handling thread is associated with a first set of one or more queue identifiers, and each queue identifier identifies a queue being handled by the first I/O queue handling thread, and, responsive to determining that the preemption flag is equal to the first value, transferring the first set of one or more queue identifiers to a second I/O queue handling thread executing on a second processor. Transferring the first set of queue identifiers may include removing the one or more queue identifiers from the first set). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the teachings of the combination of Palermo-Wang would be modified with the queue redistribution mechanism taught by Tsirkin resulting in a system that provides load balancing among redistributable queues including queue identifiers. Therefore, it would have been obvious to combine the teachings of Palermo-Wang and Tsirkin.
As per claim 12, this claim is similar to claim 3 and is rejected for the same reasons. The same motivation used in the rejection of claim 3 is applicable to the instant claim.
As per claim 18, this claim is similar to claim 3 and is rejected for the same reasons. The same motivation used in the rejection of claim 3 is applicable to the instant claim.
Claims 4 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Palermo-Wang as applied to independent claims 1, 10, and 16 and in further view of Gupta et al. (US 20210064426).
As per claim 4, the combination of Palermo-Wang fails to specifically teach, wherein based on reduction of workload to a core removed from the group of cores), reduce power to the core removed from the group of cores.
However, Gupta teaches, wherein based on reduction of workload to a core removed from the group of cores ([0155], the power controller may scale the core domain power budget for different cores/core types based on one or more energy performance preference values. In an embodiment, these energy performance preference (EPP) values may be so-called EPP hints received by way of one or more configuration registers), reduce power to the core removed from the group of cores ([0047], a power controller may control the processor to be power managed by some form of dynamic voltage frequency scaling (DVFS) in which an operating voltage and/or operating frequency of one or more cores or other processor logic may be dynamically controlled to reduce power consumption in certain situations; and [0156], a power-to-performance table may be accessed based on the scaled core domain power budget for each of the different cores/core types to determine an operating point for each of the different cores/core types… it is possible for this operating point to be optimized or reduced to a lower level in light of physical or environmental constraints).
The combination of Palermo-Wang and Gupta are analogous because they are each related to load balancing. Palermo teaches a method of load balancing in a distributed environment using queues for packet transmission. Wang teaches a method of load balancing among cores using a NIC including dynamically assigning queues to a plurality of cores. Gupta teaches a power-efficient resource allocation. (Abstract, resource allocation circuit may: receive a power budget for a first core and at least one second core and scale the power budget based at least in part on at least one energy performance preference value to determine a scaled power budget; determine a first maximum operating point for the first core and a second maximum operating point for the at least one second core based at least in part on the scaled power budget; determine a first efficiency value for the first core based at least in part on the first maximum operating point for the first core and a second efficiency value for the at least one second core based at least in part on the second maximum operating point for the at least one second core; and report a hardware state change to an operating system scheduler based on the first efficiency value and the second efficiency value). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the teachings of the combination of Palermo-Wang would be modified with the power-based resource allocation mechanism taught by Gupta resulting in a system that provides load balancing among redistributable queues in a power efficient manner. Therefore, it would have been obvious to combine the teachings of Palermo-Wang and Gupta.
As per claim 19, this claim is similar to claim 4 and is rejected for the same reasons. The same motivation used in the rejection of claim 4 is applicable to the instant claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and is as follows:
Inventor
Application No.
Teaches
Kutch
US 20200348973
Performance based resource allocation:
the application configured to write application performance measurements to one or more telemetry registers associated with the core. In some examples, the one or more telemetry registers can be designated for the application to store performance measurements from the application. In some examples, an orchestrator can read the one or more telemetry registers associated with the core. In some examples, the orchestrator selectively causes modification of resource allocation to the application based on read contents of the one or more telemetry registers. Utilization of the core can be 100% whereas the performance measurements can indicate a level of busyness of the application
[0045], Orchestrator 206 can be configured to correlate an application's telemetry information to adjust resource allocation to improve performance or stability of the application or reduce power of the core
McDonnell et al.
US 20190042331
Power-aware resource allocation and load balancing:
[0035], If the consumer queue for the worker core is empty, then the worker core performs other tasks or goes to sleep at block 506
[0036], Each worker core returns accumulated credits to the credit pool when the number of accumulated credits reaches a predetermined maximum number for that worker core at block 602
[0044], For applications with varying traffic loads, embodiments of the present invention provide better power savings (e.g., a higher percentage of worker cores in a lower power state)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MELISSA A HEADLY whose telephone number is (571)272-1972. The examiner can normally be reached Monday- Friday 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at 571-272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MELISSA A HEADLY/Examiner, Art Unit 2197
Prosecution Insights