Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding applicants arguments directed at the 35 USC 103 rejection of claim 1 as being unpatentable under Hao et al. (US 20240267324 A1) in view of Thubert et al. (US 20200259680 A1).
Argument A:
Applicant first argues that Hao's architecture is a chassis-based switching architecture and not a Clos fabric (Remarks, pages 10-11).
This argument attacks Hao individually, whereas the rejection is based on the combination of Hao and Thubert. The rejection expressly acknowledged that Hao does not explicitly teach a Clos configured backend network and relied on Thubert for this feature. One cannot show nonobviousness by attacking references individually where the rejection is based on a combination of references. In re Keller, 642 F.2d 413, 425 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 1097 (Fed. Cir. 1986); MPEP 2145(IV).
Thubert unambiguously teaches the Clos configuration:
Thubert describes deployment of its techniques “in a fat tree topology, also referred to as a ‘CLOS’ topology” (0078), an example “optimized for the Clos/Fat tree fabric design” (0055), and a “2-planes canonical Clos” (0079).
Thubert's fabric is a multi-tier arrangement of independent switching devices (a Top-of-Fabric (superspine) layer, a Top-of-Pod (spine) layer, and a leaf/terminal layer (0035; Figs. 2-5, 9A; Fig. 11 showing spine, mid, and leaf layers)) which is precisely the “multi-tiered fabric of independent spine switches and leaf switches” that Applicant contends is missing from the combination.
Regarding the broadest reasonable interpretation (BRI) of the recitation itself, it is noted that “implemented within a Clos configured backend network of a web scale network” appears in the preamble of each independent claim as the environment in which the recited steps are performed. A preamble reciting only the intended use or environment of the invention is generally not accorded patentable weight (MPEP 2111.02). The rejection nevertheless treated the recitation as limiting and demonstrated that it is taught by the combination; the observation is made here only to underscore that the recitation cannot carry the structural weight Applicant assigns to it.
Further as to BRI, the claims do not define “web scale network” or “backend network.” Applicant's own specification describes the back-end network as the network that “connect[s] specialized endpoints to one another” and states that “[h]istorically the back-end network has been used for High Performance Compute (HPC) and storage applications” (specification, 0004, 0016).
Hao is directed to exactly such networks: Hao states that “[a] typical application scenario of embodiments of this application is high performance computing (HPC)” in which “[c]omputing nodes are usually interconnected through a chassis-shaped device in an HPC network” (0091-0092). Hao's networking scale supports interconnection of 289 chassis-shaped devices serving 83,232 400G server ports (0068), and the connected hosts expressly include web servers and search engine servers (0085).
Under BRI in light of Applicant's own specification, Hao operates within a backend (HPC) network of a web scale network; the Clos configuration of that backend network is supplied by Thubert, as set forth in the rejection.
Applicant next argues that Thubert is non-analogous art directed to an industrial smart grid substation network (Remarks, page 10). This mischaracterizes Thubert by elevating a single example embodiment to the whole of the reference. The smart grid substation of Thubert's Fig. 7 is introduced by Thubert as “a degenerate variation” (0079) — one application among several. Thubert's disclosure as a whole is directed to distribution of data packets across “a fat tree network topology . . . of physical network devices (e.g. in an Internet Protocol (IP) based network)” (0030), to “distributed link layer switching domains such as cloud-type computing domains utilizing fat tree topologies” (0007), to transmission “between different fat tree topologies (e.g., between different cloud-computing data centers)” (0028), and to “[a] cloud structure . . . deployed using one or more physical data centers,” where “LISP is used as mapper/resolver” and the overlay is controlled “using SDN or LISP, and/or VxLAN” (0087).
A reference must be considered for everything it teaches and is not limited to its preferred or exemplary embodiments. MPEP 2123.
Under the two-prong test of In re Bigio, 381 F.3d 1320, 1325 (Fed. Cir. 2004), and In re Clay, 966 F.2d 656, 658-59 (Fed. Cir. 1992), Thubert is analogous under both prongs:
First, Thubert is from the same field of endeavor as the claimed invention — packet distribution over Clos/fat-tree switching fabrics of data networks using an overlay (VxLAN) and a LISP map server/map resolver control function — which is the very field and protocol environment described in Applicant's specification (0011, 0023, 0035).
Second, Thubert is reasonably pertinent to the problem confronting the inventors, namely efficient, reliable, and scalable delivery of packets across a multi-stage fabric (0005-0007). Moreover, Hao itself directs the skilled artisan to fat-tree fabrics: “Currently, an interconnection topology used for the HPC network is mainly a fat-tree (fat tree) manner, and full mesh (full mesh) networking is performed by using two tiers of switches, namely, spine and leaf (spine-leaf) switches or more tiers of switches” (Hao, 0093).
A person of ordinary skill implementing Hao's congestion-aware forwarding in an HPC/data-center network was therefore already directed by Hao's own disclosure to the Clos/fat-tree topology taught by Thubert; consulting Thubert is not resort to a remote art.
Applicant's third argument — that Thubert's Clos topology “cannot simply be grafted onto Hao's chassis architecture” without restructuring Hao (Remarks, page 11) — applies an incorrect standard. The test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. In re Keller, 642 F.2d at 425; MPEP 2145(III).
The rejection proposes implementing Hao's congestion-aware, VOQ-based scheduling and control-plane distribution within a Clos-configured fabric as taught by Thubert — the application of a known fabric topology to a known forwarding system to yield the predictable result of congestion-aware scheduling within a Clos backend network.
This is the use of a known technique to improve similar devices in the same way, and/or the simple substitution of one known interconnection topology for another to obtain predictable results. KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 416-17 (2007); MPEP 2143(I)(B)-(D).
Hao itself frames direct-mesh interconnection and fat-tree (spine-leaf) fabrics as the two recognized, interchangeable design alternatives for such networks, with known trade-offs (0093-0094);
selecting the acknowledged mainstream alternative is a predictable variation, not a restructuring.
It is further noted that Thubert's Clos fabric is not limited to IED leaves or PRP/HSR constraints; Thubert's Figs. 2-6, 9A, and 11 disclose generic fat-tree fabrics of generic network devices with generic leaf devices.
Argument B:
Applicant argues that Hao's control plane is “a traditional IP/L3 routing control plane, not a service registration broker,” that Hao's congestion notification is “reactive” and “peer-to-peer” rather than “proactive endpoint registration,” and that forwarding tables are not “registration information” (Remarks, pages 11-13). These arguments are premised on limitations that do not appear in claim 1. Claim 1 recites “registering, with a service control plane, a plurality of egress endpoints and status of the egress endpoints” and “distributing, by the service control plane to a plurality of ingress virtual output queues (VOQs), registration information relating to the plurality of egress endpoints.”
The claim does not recite:
(a) any particular registration protocol;
(b) that the egress endpoints themselves initiate registration by announcing their “identity and availability,”
(c) any publication-subscription mechanism (recited only in dependent claims 4, 12, and 20),
(d) LISP or MSMRs (recited only in dependent claims 7-8 and 15-16), or
(e) any “on-demand” distribution.
During examination, claims are given their broadest reasonable interpretation consistent with the specification, and limitations from the specification's embodiments are not read into the claims. MPEP 2111, 2111.01. Applicants’ arguments import the specification's DPU-based LISP xTR client/agent embodiment (specification, 0024-0026) into claim 1.
Applicant's proposed construction of “service control plane” is also contradicted by Applicant's own specification.
The specification repeatedly identifies “BGP-EVPN with Route Reflector (RR)” — a Border Gateway Protocol based routing control plane — as an express example of the claimed service control plane: “a distributed protocol service control plane (e.g., LISP MSMR or BGP-EVPN with RR)” (specification, 0025; see also 0001, 0011, 0023, 0035, 0039).
A construction of “service control plane” that excludes BGP-based routing control planes is therefore not consistent with the specification and cannot be the broadest reasonable interpretation.
Hao's control plane is a BGP control plane (Hao, Fig. 5, “Main control board (BGP protocol)”; Fig. 10; 0080) whose functionality is distributed across the interconnected chassis-shaped devices and their cooperating chips, consistent with the recited service control plane, including the “distributed service control plane” of dependent claims 2, 10, and 18.
As mapped in the rejection, Hao teaches the registering step under BRI.
Hao's control plane “advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status” (0080).
In Hao's control-plane procedure (Fig. 10; 0193-0222), each destination chassis advertises, via BGP, route prefixes reachable through its outbound (egress) interfaces (S701-S702), and the receiving control plane records for each such egress interface its identity (TB ID and TP ID), its location (the forwarding chip and board on which it resides), its role (shortest path/non-shortest path), and its ECMP group membership (0161-0163; 0221-0222; Fig. 8).
The status of these egress interfaces is likewise registered and maintained:
congested states are recorded (0032; 0130-0131; 0139; 0255, “the forwarding chip 1 records a congested state of the interface 10 in a forwarding table”),
and faulty states are recorded (0228).
Notably, Applicant's own remarks state that “[r]egistration information in the claimed invention includes endpoint identity, location, and status” (Remarks, page 12).
That is exactly the content of the information registered and recorded in Hao: identity (TB ID/TP ID), location (chip/line card), and status (role, congested state, faulty state).
Hao likewise teaches the distributing step.
“The control plane delivers the generated forwarding table to the forwarding plane” (0080), and “the main control board sends the first forwarding entry to each of N forwarding chips” (0222) — including the uplink chips that maintain an ingress VOQ for each outbound interface (0154-0155; 0249).
The delivered information — egress interface identities, roles, and recorded congested/faulty states — is plainly “registration information relating to the plurality of egress endpoints,” and the uplink chips schedule service flows from the ingress VOQs to the egress based on that information and its updates (0249-0250). The claim requires no more.
Applicant's “proactive versus reactive” distinction finds no support in the claim, which recites no timing, trigger, or initiator for the registering step.
Regardless, Hao's registration is proactive:
the control-plane procedure of Fig. 10 “is performed before the method shown in FIG. 7 or the method shown in FIG. 9” (0193), such that the identities, locations, roles, and ECMP groupings of the egress interfaces are registered and distributed before any packet forwarding occurs, and status updates (congestion/fault notifications) thereafter update the recorded registrations — a sequence that directly parallels claims 1 and 3.
Applicant's characterization of the congestion notifications as “peer-to-peer” is likewise unavailing: the claims do not preclude control-plane functionality that is distributed across multiple cooperating processors; to the contrary, the claims and specification require and emphasize a control plane that is “distributed” across “distributed processes/CPUs, avoiding a single point of failure” (claims 2, 10, 18; specification, 0025). Hao further teaches the switching chip back-pressing congested state (0040; 0156) and the main control board provisioning the multicast notification distribution entries in advance (0253), confirming control-plane participation in status maintenance and distribution.
Finally, while the rejection relies on Hao for these steps (rendering Applicant's “Thubert does not remedy” argument moot), Thubert independently evidences endpoint registration with a fabric-managing service control plane: leaf devices generate and flood “registration message(s)” (0095-0096; Fig. 12A, items 88-90), root allocation is “reported to MSMR” and the MSMR is updated (Fig. 12B, items 110, 122), and “[t]he map server/resolver (e.g., LISP) 14 managing the fabric 100 can be updated of the status of the trees” (0065). In the combination, the LISP MSMR service control plane of Thubert manages the Clos fabric while Hao supplies the egress registration, status maintenance, distribution to VOQ-maintaining chips, and VOQ scheduling.
Argument C:
Applicant argues that the stated motivation — to improve data packet distribution, citing Thubert 0002-0007 — is insufficient (Remarks, pages 13-14).
The cited paragraphs are not a “generic statement”; they identify a recognized need in the field for “scalable and dynamically controllable distribution of . . . packets between distributed link layer switching domains such as cloud-type computing domains utilizing fat tree topologies” (Thubert, 0007), and Thubert teaches that its arrangement is “optimized for the Clos/Fat tree fabric design, and takes advantage of that particular design” to provide reliable delivery “in a cheap and efficient fashion” (0055). A benefit expressly taught by the secondary reference is a rational underpinning for the combination.
Moreover, under KSR, the obviousness analysis “need not seek out precise teachings directed to the specific subject matter of the challenged claim,” and “any need or problem known in the field of endeavor addressed by the patent and to which the claim relates can provide a reason for combining the elements in the manner claimed.” KSR, 550 U.S. at 418-21.
The combination is further supported by rationales the Supreme Court endorsed in KSR:
the use of a known technique (arranging forwarding devices in a Clos/fat-tree fabric) to improve a similar device (Hao's congestion-aware, VOQ-scheduled forwarding network) in the same way, and the simple substitution of one known, finite interconnection topology for another with predictable results. MPEP 2143(I)(B)-(D).
Critically, the bridge between the references is built by Hao itself: Hao acknowledges that the mainstream interconnection topology for HPC networks is the fat-tree, spine-leaf fabric (0093), and expressly compares its direct-interconnect approach against fat-tree networking (0093-0094). The skilled artisan thus faced a recognized, finite set of known topology options with well-understood trade-offs.
To the extent Hao expresses a preference for direct interconnection on delay/cost grounds, a reference's preference for its own solution, or its identification of trade-offs in a known alternative, does not teach away from that recognized alternative. In re Fulton, 391 F.3d 1195, 1201 (Fed. Cir. 2004.) MPEP 2143(X).
Applicant's assertion that the references “address fundamentally different problems in fundamentally different environments” is addressed in section A above:
Hao is directed to HPC/data-center scale networks (0068, 0085, 0091-0094) — the very networks Applicant's specification identifies as the historic backend network (specification, 0004, 0016) — and Thubert is directed to packet distribution across Clos/fat-tree fabrics in cloud-type computing domains and data centers (0007, 0028, 0030, 0087). Both references, and the claims, concern efficient packet delivery across large-scale switching fabrics.
As to hindsight, the rejection relies only on the express teachings of the references and knowledge within the level of ordinary skill — including Hao's own identification of fat-tree spine-leaf fabrics as the prevailing alternative — and not on knowledge gleaned solely from Applicant's disclosure. In re McLaughlin, 443 F.2d 1392, 1395 (CCPA 1971).
As to reasonable expectation of success, only a reasonable expectation is required, not absolute predictability. In re O'Farrell, 853 F.2d 894, 903-04 (Fed. Cir. 1988). MPEP 2145(X).
Both references describe conventional packet-switched IP networking hardware; fat-tree/Clos fabrics of such hardware were, per Hao 0093, the mainstream deployment for the relevant networks, and Thubert 0030 and 0055 describe Clos fabrics of ordinary IP network devices.
Applicant's assertion that the combination “would require extensive redesign of both systems” is unsupported argument, which cannot take the place of evidence in the record. In re Pearson, 494 F.2d 1399, 1405 (CCPA 1974); MPEP 2145(I).
Dependent Claims:
Claim 4: "publishing . . . to subscriber ingress VOQs"
Applicant argues that these claims require a publication-subscription model in which “recipients explicitly subscribe to topics, and publishers send messages only to subscribers, enabling on-demand, selective distribution” (Remarks, pages 14-15).
The claims recite no such requirements. Claims 4, 12, and 20 recite “publishing, by the service control plane, one or more messages related to the updated registration information to subscriber ingress VOQs of the plurality of ingress VOQs.”
They do not recite subscription requests, topics, on-demand semantics, or exclusion of non-subscribers;
Those features are drawn from the specification's LISP pub-sub example (specification, 0026) and are not in the claims. MPEP 2111.01.
Under BRI, Hao teaches the recited publishing.
When a congested state of an egress interface changes, the detecting forwarding chip generates a congestion notification message and sends it as a multicast message, where “a multicast group is the (N-1) other forwarding chips” (0139; 0145-0147) — a one-to-many publication of update messages to a defined group of recipients.
The recipients are the chips that maintain the ingress VOQs for the affected outbound interfaces (0154-0155; 0249) and that record the update and schedule accordingly (0250; 0255).
Membership in the notification multicast group — provisioned in advance by the main control board via the multicast forwarding entry (0147; 0253) — reasonably constitutes subscription under BRI: the VOQ-maintaining chips are enrolled recipients of published updates concerning the egress interfaces for which they maintain queues.
The published messages are “related to the updated registration information” because they carry the updated congested states — which is precisely what Applicant's specification defines the updated registration information to comprise (specification, 0026; see also claims 3, 11, 19, “the updated registration information comprises congestion information”).
Additionally, the control plane's delivery of updated forwarding tables recording those congested states (0080; 0255) constitutes messages related to the updated registration information distributed to the VOQ-maintaining chips.
In the combination, Thubert further supplies the LISP MSMR framework — the very demand-based map server/resolver arrangement the specification associates with pub-sub — which is “updated of the status” of the fabric and responds to resolution requests (Thubert, 0065).
Claims 7, 15 (LISP) and 8, 16 (MSMRs) (Remarks, pages 15-17)
Claims 7 and 15 recite that “the Clos configured backend network is configured in accordance with Locator ID Separation Protocol (LISP),” and claims 8 and 16 recite that the service control plane comprises “a distributed service control plane including one or more map resolvers (MRs) map servers (MSs) (MSMRs).”
Neither claim recites any additional function that the LISP configuration or the MSMRs must perform beyond the steps of the base claims, which are met by the combination as set forth above.
Thubert teaches a Clos/fat-tree fabric managed by a LISP map server/map resolver: “[t]he map server/resolver (e.g., LISP) 14 managing the fabric 100” (0065), a management device “executing a map server/map resolver (MSMR)” (0066), an MSMR implemented as one or more distributed management devices or as a separate device 136 (0065-0066; Fig. 11), and a cloud deployment in which “LISP is used as mapper/resolver” (0087).
Applicant's argument that Thubert's LISP/MSMR serves “a fundamentally different purpose” (multicast tree management rather than egress registration for VOQ scheduling) does not overcome the rejection.
A reference may be relied upon for all that it teaches, and the use to which the combined teachings are put need not be identical to the purpose for which the secondary reference employed them. In re Heck, 699 F.2d 1331, 1332-33 (Fed. Cir. 1983); MPEP 2123.
As KSR recognized, “[a] person of ordinary skill is also a person of ordinary creativity,” and familiar items may have obvious uses beyond their primary purposes. KSR, 550 U.S. at 420-21.
In the rejection's combination, it is Hao — not Thubert — that supplies the egress registration, status distribution, and VOQ scheduling functions of the base claims;
Thubert supplies the Clos fabric configuration and the LISP/MSMR control-plane infrastructure recited in claims 7-8 and 15-16.
It is further noted that Thubert's MSMR itself receives registrations and status: leaves flood “registration message(s)” (0095-0096), root allocations are “reported to MSMR” (Fig. 12B, item 110), and the MSMR “can be updated of the status of the trees” (0065).
Applicant's contention that “transposing Thubert's MSMR function onto Hao's chassis architecture does not yield the claimed invention” again attacks the references individually and applies the bodily-incorporation standard rejected in In re Keller, as addressed in section A above.
For at least the foregoing reasons, Applicant's arguments are not persuasive, and the rejection of claims 1-20 under 35 U.S.C. 103 as being unpatentable over Hao in view of Thubert is maintained
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hao et al. (US 20240267324 A1) in view of Thubert et al. (US 20200259680 A1).
Regarding claim 1, Hao teaches a method implemented within a backend network of a web scale network, ([0079; 0085] network system backplane, where the system includes hosting web based services.) the method comprising: (0006; packet forwarding method)
registering,(recording information about the output interface device equivalent to registering) with a service control plane,(control plane) a plurality of egress endpoints (output interface device) and status of the egress endpoints (status of the outbound (egress) interface devices;) (0080; The control plane advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status, and the like; 0032; receiving a second notification message from the second forwarding chip, where the second notification message indicates a congested state of the second outbound interface of the second forwarding chip; and recording the congested state of the second outbound interface based on the second notification message.)
distributing, by the service control plane to a plurality of ingress virtual output queues (VOQs) (the control plane generates and provides forwarding tables to the forwarding plane), registration information relating to the plurality of egress endpoints;(the control plane generates and provides forwarding tables to the forwarding plane) (0080; The forwarding plane includes each forwarding chip. For example, the forwarding chip 231 and the forwarding chip 232 are disposed on the line card 230, and a switching chip is disposed on the switching board 220. The control plane advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status, and the like. The control plane delivers the generated forwarding table to the forwarding plane. The forwarding chip 232 on the forwarding plane searches a table for and forwards, based on the forwarding table delivered by the control plane, a packet received by the network interface (not shown in the figure). The forwarding table delivered by the control plane is stored, for example, in a memory (not shown in the figure) on the line card. In some embodiments, the control plane and the forwarding plane may be deployed on different physical devices.)
based at least in part on the distributing, (forwarding table see mapping above) scheduling packets for transmission (scheduling flow from VOQ to engress) from the plurality of ingress VOQs to the plurality of egress endpoints; (0249; The VOQ buffers ingress traffic before the ingress traffic enters the switching board, to avoid head-of-line (HOL) blocking. In a direction to an egress, a scheduler schedules a service flow corresponding to an ingress VOQ, and sends permitted credit values (credits) of different bandwidths to all service flows flowing to the egress, to accurately allocate a bandwidth based on a user service level agreement (SLA) and ensure quality of service (QOS).)
and forwarding, by the plurality of ingress VOQs to the plurality of egress endpoints, packets (forwarding packets from the ingress VOQ to the egress). ([0164] The target outbound interface is an outbound interface used to forward the first data packet in the M outbound interfaces. There are a plurality of implementations of selecting the target outbound interface. [0074] The line card 230 is also referred to as an interface board (interface board), a line processing unit (LPU), or a service board. The line card 230 is configured to: provide various service interfaces and forward a packet; 0077; When forwarding chips on different line cards need to interact, the switching board 220 is configured to forward data between forwarding chips on different line cards, to implement mutual communication between the forwarding chips. [0084] The packet forwarding method provided in embodiments of this application is applied to a network that includes a plurality of chassis-shaped devices and a plurality of hosts. The plurality of chassis-shaped devices are connected to each other. For a specific connection manner between the plurality of chassis-shaped devices, refer to the architecture shown in FIG. 1, FIG. 2, or FIG. 3. Optionally, each chassis-shaped device has a hardware structure shown in FIG. 4 or FIG. 5. The plurality of chassis-shaped devices are configured to forward a data packet from a source host to a destination host, to support data exchange between different hosts. [0098] The method shown in FIG. 7 is described by using, as an example, a scenario in which a first host is connected to the first chassis-shaped device and the second host is connected to the second chassis-shaped device. In the process of forwarding the data packet, the first chassis-shaped device is responsible for receiving the data packet from the first host, and forwarding the data packet to the second chassis-shaped device. The second chassis-shaped device is responsible for forwarding, to the second host, a data packet whose destination is the second host. [0249] The uplink chip maintains a VOQ for each outbound interface. If an outbound interface of an inter-chassis interconnection link is congested, the switching chip back presses a congested state to the uplink chip. Consequently, packets in the VOQ queue are backlogged. The VOQ is an independent queue maintained by the chassis-shaped device for different outbound interfaces. The VOQ buffers ingress traffic before the ingress traffic enters the switching board, to avoid head-of-line (HOL) blocking. In a direction to an egress, a scheduler schedules a service flow corresponding to an ingress VOQ, and sends permitted credit values (credits) of different bandwidths to all service flows flowing to the egress, to accurately allocate a bandwidth based on a user service level agreement (SLA) and ensure quality of service (QOS). [0250] During route forwarding, the uplink chip determines a queue depth of the VOQ corresponding to the outbound interface. If the queue depth of the VOQ exceeds a configured threshold, the uplink chip determines that the outbound interface corresponding to the VOQ is congested. If there are a plurality of outbound interfaces of a shortest path, the uplink chip selects the remaining outbound interfaces of the shortest path for forwarding. If all outbound interfaces of the shortest path are congested, the uplink chip selects an outbound interface of a non-shortest path for forwarding.)
Hao does not explicitly teach the underlined a method implemented within a Clos configured backend network of a web scale network,
In an analogous art Thubert teaches a method ([0024] method) implemented within a Clos configured backend network (0079; a 2-planes canonical Clos is put together for the backbone and access layers) of a web scale network ([0077] Hence, the propagation of the multicast message throughout the redundant multicast trees 104 enables any network device in the fat tree network topology 100 to operate in operation 78 as a VxLAN ingress endpoint for traffic “(*,G”) destined for an overlay fabric VxLAN egress endpoint: the ingress endpoint can be selected by the management device 14 and/or auto-selected by the VxLAN egress endpoint, as appropriate. [0078] As apparent from the foregoing, the example embodiments enable deployment of multiple redundant multicast trees in a fat tree topology, also referred to as a “CLOS” topology, for reliable delivery of multicast traffic. [0079] FIG. 7 illustrates the management device 14 applying multiple multicast trees to a secondary smart grid substation. In that case, a degenerate variation is proposed whereby a 2-planes canonical Clos is put together for the backbone and access layers, while the IEDs form a third layer. That third layer acts as leaves in this example. The planes are illustrated below as a blue (dark) and a red (shaded) plane, and the planes only meet at the level of the IEDs, since they are suited for end-to-end redundancy protocols such as PRP and HSR. Hence, FIG. 7 illustrates the management device computing the trees in different planes, which makes the trees non congruent by definition.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing of the application to modify the teachings of [Hao] to include [a Clos topology] as is taught by [Thubert].
The suggestion/motivation for doing so is to improve data packet distribution [0002-0007].
Regarding claim 2, Hao in view of Thubert teach the method of claim 1, and is disclosed above, Hao further teaches wherein the service control plane comprises a distributed service control plane. (0193; Fig 7-9 showing control plane procedure between multiple devices and is therefore equivalent to distributed service control plane)
Regarding claim 3, Hao in view of Thubert teach the method of claim 1, and is disclosed above, Hao further teaches further comprising:
updating the registration information relating to the plurality of egress endpoints to provide updated registration information, (The specification defines the updated registration information as being congestion information [0026]; Hao teaches [0130] In some embodiments, the first chassis-shaped device determines the congested state of the outbound interface based on at least one of a queue depth, bandwidth utilization, a buffer (buffer) length, and a remaining bandwidth of the outbound interface. [0032] and recording the congested state of the second outbound interface based on the second notification message. [0036] In a possible implementation, the second notification message includes an identifier of the second forwarding chip, an identifier of the second outbound interface, and a congested state identifier. [0040] In the implementation, the switching chip back presses the congested state of the outbound interface, so that processing overheads caused when a forwarding chip on which the outbound interface is located notifies another forwarding chip of the congested state are reduced when the forwarding chip can be aware of the congested state of the interconnection link. [0228] For a forwarding chip i in the N forwarding chips, the forwarding chip i monitors whether each outbound interface of the forwarding chip i is faulty. If an outbound interface a of the forwarding chip i is faulty, the forwarding chip i generates a fault notification message, and sends the fault notification message to (N−1) other forwarding chips different from the forwarding chip i in the N forwarding chips. Each of the (N−1) other forwarding chips receives the fault notification message of the forwarding chip i, determines, based on the fault notification message, that the outbound interface a is faulty, and records a faulty state of the outbound interface a. The other forwarding chips query an ECMP group that includes the outbound interface a in a forwarding entry, and delete the outbound interface a from the ECMP group that include the outbound interface a. [0131] For example, the congested state is determined based on the queue depth. For example, the first chassis-shaped device obtains a queue depth of each of the M outbound interfaces. The first chassis-shaped device compares the queue depth of each outbound interface with a depth threshold. If a queue depth of an outbound interface exceeds the depth threshold, the first chassis-shaped device determines that the outbound interface is in a congested state. If a queue depth of an outbound interface is less than or equal to the depth threshold, the first chassis-shaped device determines that the outbound interface is in a non-congested state. Optionally, the congested state further includes a moderately congested state and a heavily congested state. Different depth thresholds are used to determine whether the congested state is a moderately congested state or a heavily congested state. For example, the depth threshold includes a first depth threshold and a second depth threshold, and the first depth threshold is greater than the second depth threshold. If a queue depth of an outbound interface exceeds the first depth threshold, the first chassis-shaped device determines that the outbound interface is in a heavily congested state. If a queue depth of an outbound interface does not exceed the first depth threshold but exceeds the second depth threshold, the first chassis-shaped device determines that the outbound interface is in a moderately congested state.)
wherein the updated registration information comprises congestion information related to the plurality of egress endpoints; (Mapping above + [0139] For a forwarding chip i in the N forwarding chips, the forwarding chip i determines a congested state of each outbound interface of the chip, and the forwarding chip i generates a congestion notification message based on the congested state of each outbound interface, and sends the congestion notification message to (N−1) other forwarding chips different from the forwarding chip i in the N forwarding chips. Each of the (N−1) other forwarding chips receives the congestion notification message of the forwarding chip i, and records the congested state of each outbound interface of the forwarding chip i based on the congestion notification message of the forwarding chip I; [0033] In the implementation, an intra-chassis inter-chip congestion awareness and notification mechanism is implemented, so that a current congested state of an interconnection link between a local chassis and another chassis can be aware of in a timely manner, and a forwarding path with better comprehensive performance is selected based on the congested state during packet forwarding.)
distributing, by the service control plane to a plurality of ingress VOQs, the updated registration information relating to the plurality of egress endpoints; (The control plane generates the forwarding table, and records updated congested states, and then delivers and provides the forwarding table mapping above + 0080; The control plane delivers the generated forwarding table to the forwarding plane. The forwarding chip 232 on the forwarding plane searches a table for and forwards, based on the forwarding table delivered by the control plane, a packet received by the network interface (not shown in the figure). The forwarding table delivered by the control plane is stored, for example, in a memory (not shown in the figure) on the line card. In some embodiments, the control plane and the forwarding plane may be deployed on different physical devices. [0255] The forwarding chip 2 monitors a congested state of the interface 10 and a congested state of the interface 11. When the interface 10 of the forwarding chip 2 is congested, the forwarding chip 2 generates a notification message, and sends the notification message to a forwarding chip 1. The notification message includes a TB ID of the forwarding chip 2, a TP ID of the interface 10, and a congestion degree. After receiving the notification message sent by the forwarding chip 2, the forwarding chip 1 records a congested state of the interface 10 in a forwarding table.
based at least in part on distributing the updated registration information (congestion data see mapping above), further scheduling packets (Ingress VOQ flow scheduler scheduling flow to the egress) for transmission from the plurality of ingress VOQs (VOQs ingress traffic)to the plurality of egress endpoints (flowing to the egress); ([0249] The uplink chip maintains a VOQ for each outbound interface. If an outbound interface of an inter-chassis interconnection link is congested, the switching chip back presses a congested state to the uplink chip. Consequently, packets in the VOQ queue are backlogged. The VOQ is an independent queue maintained by the chassis-shaped device for different outbound interfaces. The VOQ buffers ingress traffic before the ingress traffic enters the switching board, to avoid head-of-line (HOL) blocking. In a direction to an egress, a scheduler schedules a service flow corresponding to an ingress VOQ, and sends permitted credit values (credits) of different bandwidths to all service flows flowing to the egress, to accurately allocate a bandwidth based on a user service level agreement (SLA) and ensure quality of service (QOS). [0250] During route forwarding, the uplink chip determines a queue depth of the VOQ corresponding to the outbound interface. If the queue depth of the VOQ exceeds a configured threshold, the uplink chip determines that the outbound interface corresponding to the VOQ is congested. If there are a plurality of outbound interfaces of a shortest path, the uplink chip selects the remaining outbound interfaces of the shortest path for forwarding. If all outbound interfaces of the shortest path are congested, the uplink chip selects an outbound interface of a non-shortest path for forwarding.)
and based at least in part on the further scheduling, (flow scheduler) forwarding, by the plurality of ingress VOQs to the plurality of egress endpoints, packets. ([0249] The uplink chip maintains a VOQ for each outbound interface. If an outbound interface of an inter-chassis interconnection link is congested, the switching chip back presses a congested state to the uplink chip. Consequently, packets in the VOQ queue are backlogged. The VOQ is an independent queue maintained by the chassis-shaped device for different outbound interfaces. The VOQ buffers ingress traffic before the ingress traffic enters the switching board, to avoid head-of-line (HOL) blocking. In a direction to an egress, a scheduler schedules a service flow corresponding to an ingress VOQ, and sends permitted credit values (credits) of different bandwidths to all service flows flowing to the egress, to accurately allocate a bandwidth based on a user service level agreement (SLA) and ensure quality of service (QOS). [0250] During route forwarding, the uplink chip determines a queue depth of the VOQ corresponding to the outbound interface. If the queue depth of the VOQ exceeds a configured threshold, the uplink chip determines that the outbound interface corresponding to the VOQ is congested. If there are a plurality of outbound interfaces of a shortest path, the uplink chip selects the remaining outbound interfaces of the shortest path for forwarding. If all outbound interfaces of the shortest path are congested, the uplink chip selects an outbound interface of a non-shortest path for forwarding.)
Regarding claim 4, Hao in view of Thubert teach the method of claim 3, and is disclosed above, Hao further teaches wherein distributing the updated registration information comprises publishing, (delivering by the control the forwarding table including recorded congested state information) by the service control plane, (0080; The control plane advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status, and the like. The control plane delivers the generated forwarding table to the forwarding plane. The forwarding chip 232 on the forwarding plane searches a table for and forwards, based on the forwarding table delivered by the control plane, a packet received by the network interface (not shown in the figure). The forwarding table delivered by the control plane is stored, for example, in a memory (not shown in the figure) on the line card. In some embodiments, the control plane and the forwarding plane may be deployed on different physical devices. (0032; receiving a second notification message from the second forwarding chip, where the second notification message indicates a congested state of the second outbound interface of the second forwarding chip; and recording the congested state of the second outbound interface based on the second notification message.) [0255] The forwarding chip 2 monitors a congested state of the interface 10 and a congested state of the interface 11. When the interface 10 of the forwarding chip 2 is congested, the forwarding chip 2 generates a notification message, and sends the notification message to a forwarding chip 1. The notification message includes a TB ID of the forwarding chip 2, a TP ID of the interface 10, and a congestion degree. After receiving the notification message sent by the forwarding chip 2, the forwarding chip 1 records a congested state of the interface 10 in a forwarding table.))
one or more messages related to the updated registration information (0032; receiving a second notification message from the second forwarding chip, where the second notification message indicates a congested state of the second outbound interface of the second forwarding chip; and recording the congested state of the second outbound interface based on the second notification message.) [0255] The forwarding chip 2 monitors a congested state of the interface 10 and a congested state of the interface 11. When the interface 10 of the forwarding chip 2 is congested, the forwarding chip 2 generates a notification message, and sends the notification message to a forwarding chip 1. The notification message includes a TB ID of the forwarding chip 2, a TP ID of the interface 10, and a congestion degree. After receiving the notification message sent by the forwarding chip 2, the forwarding chip 1 records a congested state of the interface 10 in a forwarding table.)
to subscriber ingress VOQs of the plurality of ingress VOQs. (0080; The control plane advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status, and the like. The control plane delivers the generated forwarding table to the forwarding plane. The forwarding chip 232 on the forwarding plane searches a table for and forwards, based on the forwarding table delivered by the control plane, a packet received by the network interface (not shown in the figure). The forwarding table delivered by the control plane is stored, for example, in a memory (not shown in the figure) on the line card. In some embodiments, the control plane and the forwarding plane may be deployed on different physical devices.)
Regarding claim 5, Hao in view of Thubert teach the method of claim 4, and is disclosed above, Hao further teaches wherein the service control plane comprises a distributed service control plane. (0193; Fig 7-9 showing control plane procedure between multiple devices and is therefore equivalent to distributed service control plane)
Regarding claim 6, Hao in view of Thubert teach the method of claim 3, and is disclosed above, Hao further teaches wherein distributing (delivering the forwarding table) the updated registration information (recording congestion information and state in the forwarding table) comprises directly distributing, by the service control plane (delivering by the control plane) to the plurality of ingress VOQs,(VOQ forwarding plane) the updated registration information (congestion information). ((0080; The control plane advertises and learns of a route, generates a forwarding table, processes signaling and a protocol packet, configures and maintains a device status, and the like. The control plane delivers the generated forwarding table to the forwarding plane. The forwarding chip 232 on the forwarding plane searches a table for and forwards, based on the forwarding table delivered by the control plane, a packet received by the network interface (not shown in the figure). The forwarding table delivered by the control plane is stored, for example, in a memory (not shown in the figure) on the line card. In some embodiments, the control plane and the forwarding plane may be deployed on different physical devices. (0032; receiving a second notification message from the second forwarding chip, where the second notification message indicates a congested state of the second outbound interface of the second forwarding chip; and recording the congested state of the second outbound interface based on the second notification message.) [0255] The forwarding chip 2 monitors a congested state of the interface 10 and a congested state of the interface 11. When the interface 10 of the forwarding chip 2 is congested, the forwarding chip 2 generates a notification message, and sends the notification message to a forwarding chip 1. The notification message includes a TB ID of the forwarding chip 2, a TP ID of the interface 10, and a congestion degree. After receiving the notification message sent by the forwarding chip 2, the forwarding chip 1 records a congested state of the interface 10 in a forwarding table.) ([0249] The uplink chip maintains a VOQ for each outbound interface. If an outbound interface of an inter-chassis interconnection link is congested, the switching chip back presses a congested state to the uplink chip. Consequently, packets in the VOQ queue are backlogged. The VOQ is an independent queue maintained by the chassis-shaped device for different outbound interfaces. The VOQ buffers ingress traffic before the ingress traffic enters the switching board, to avoid head-of-line (HOL) blocking. In a direction to an egress, a scheduler schedules a service flow corresponding to an ingress VOQ, and sends permitted credit values (credits) of different bandwidths to all service flows flowing to the egress, to accurately allocate a bandwidth based on a user service level agreement (SLA) and ensure quality of service (QOS). [0250] During route forwarding, the uplink chip determines a queue depth of the VOQ corresponding to the outbound interface. If the queue depth of the VOQ exceeds a configured threshold, the uplink chip determines that the outbound interface corresponding to the VOQ is congested. If there are a plurality of outbound interfaces of a shortest path, the uplink chip selects the remaining outbound interfaces of the shortest path for forwarding. If all outbound interfaces of the shortest path are congested, the uplink chip selects an outbound interface of a non-shortest path for forwarding.)
Regarding claim 7, Hao in view of Thubert teach the method of claim 1, and is disclosed above, Hao does not explicitly teach wherein the Clos configured backend network is configured in accordance with Locator ID Separation Protocol (LISP).
In an analogous art Thubert wherein the Clos configured backend network is configured in accordance with Locator ID Separation Protocol (LISP). ([0055] The example of FIG. 4 is optimized for the Clos/Fat tree fabric design, and takes advantage of that particular design to provide reliable multicast in a cheap and efficient fashion. [0065] The map server/resolver (e.g., LISP) 14 managing the fabric 100 can be updated of the status of the trees 104; [0066] FIG. 5 illustrates a variation of FIG. 4, where a management device (14 of FIG. 4) (e.g., executing a map server/map resolver (MSMR)) selecting a pair of trees to be used in the fabric for a particular multicast flow. In particular, the map server/map resolver (MSMR) ; [0078] As apparent from the foregoing, the example embodiments enable deployment of multiple redundant multicast trees in a fat tree topology, also referred to as a “CLOS” topology, for reliable delivery of multicast traffic.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing of the application to modify the teachings of [Hao] to include [a Clos and Lisp configuration] as is taught by [Thubert].
The suggestion/motivation for doing so is to improve data packet distribution [0002-0007].
Regarding claim 8, Hao in view of Thubert teaches the method of claim 7, and is disclosed above, Hao does not explicitly teach but Thubert teaches wherein the service control plane comprises a distributed service control plane including one or more map resolvers (MRs) map servers (MSs) (MSMRs). ([0055] The example of FIG. 4 is optimized for the Clos/Fat tree fabric design, and takes advantage of that particular design to provide reliable multicast in a cheap and efficient fashion. [0065] The map server/resolver (e.g., LISP) 14 managing the fabric 100 can be updated of the status of the trees 104; [0066] FIG. 5 illustrates a variation of FIG. 4, where a management device (14 of FIG. 4) (e.g., executing a map server/map resolver (MSMR)) selecting a pair of trees to be used in the fabric for a particular multicast flow. In particular, the map server/map resolver (MSMR) ; [0078] As apparent from the foregoing, the example embodiments enable deployment of multiple redundant multicast trees in a fat tree topology, also referred to as a “CLOS” topology, for reliable delivery of multicast traffic.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing of the application to modify the teachings of [Hao] to include [a distributed control plane with MSMRs] as is taught by [Thubert].
The suggestion/motivation for doing so is to improve data packet distribution [0002-0007].
Regarding claims 9-16, the claims inherit the same rejection as claim 1-8 above for reciting similar limitations in the form system claim. (0064-0067; system; 0279; processors; )
Regarding claims 17-20, the claims inherit the same rejection as claim 1-8 above for reciting similar limitations in the form of a non-transitory computer readable media claim. (0049; 0278; 0284; computer readable storage media/memory storing instructions).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDERRAHMEN H CHOUAT whose telephone number is (571)431-0695. The examiner can normally be reached on Mon-Fri from 9AM to 5PM PST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Parry, can be reached at telephone number 571-272-8328. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center to authorized users only. Should you have questions about access to the USPTO patent electronic filing system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via a variety of formats. See MPEP § 713.01. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/InterviewPractice.
Abderrahmen Chouat
Examiner
Art Unit 2451
/Chris Parry/Supervisory Patent Examiner, Art Unit 2451
Prosecution Insights