DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending.
Response to Arguments
Regarding Objections:
Applicant’s amendments and arguments regarding the objection of claims 9, 10, 19, and 20 have been fully considered and are found to be persuasive. The objection of claims 9, 10, 19, and 20 are withdrawn.
Regarding 35 U.S.C. 112:
Applicant’s amendments and arguments regarding the rejection of claims 1-20 under 35 U.S.C. 112(b) have been fully considered and are found to be persuasive. The rejections of claims 1-20 under 35 U.S.C. 112(b) are withdrawn.
Regarding: Prior Art Rejections:
Applicant’s amendments and arguments regarding the rejection of claims 1-20 under 35 U.S.C. 103 have been fully considered and are moot due to new grounds of rejection necessitated by applicant’s amendments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 9-11, 12-16, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Henkel et al. US 20230106531 A1 in view of Sharma et al. US 20240422107 A1 in view of Sridhar et al. US 20240078123 A1.
Regarding claim 1, Henkel teaches the invention substantially as claimed including:
A method for configuring a logical network in a container orchestration system (in at least Figs 1, 2; [0009] a network controller for creating and managing virtual networks; [0064] network controller 24 may implement respective cluster masters for one or more Kubernetes clusters)):
that includes a plurality of custom resources defined according to respective custom resource definitions (CRDs) ([0136] Kubernetes provides two ways to add custom resources to a cluster: [0137] Custom Resource Definitions (CRDs) are simple and can be created without any programming), the method comprising:
at a network management system external to the container orchestration system (in at least Figs 1, 2; [0009] a network controller for creating and managing virtual networks; [0064] network controller 24 may implement respective cluster masters for one or more Kubernetes clusters)):
receiving a definition of a logical router for the logical network (in at least [0091] In the simplest form, VNRs 52 represent a logical abstraction of a router set in the context of Kubernetes; [0251] control nodes 232 may obtain custom resources defining VNRs 52), the logical router definition specifying a set of one or more layer 7 (L7) services to be performed on data messages processed by the logical router ([0142] Such custom resources may be alternately termed and referred to herein as “custom resources for SDN architecture configuration.” These may include VNs, VNRs, bgp-as-a-service (BGPaaS), subnet, virtual router, service instance, project, physical interface, logical interface, node, network ipam, floating ip, alarm, alias ip, access control list, firewall policy, firewall rule, network policy, route target, routing instance); and
via a control plane of the container orchestration system, defining for the logical router: (i) a first custom resource (CR) instance associated with a first CRD (Control in Fig. 1, 2, 3; in at least [0091] where VNRs 52 may be defined as a custom resource to facilitate interconnectivity between VNs 50); and
(ii) for each L7 service specified in the logical router definition, a separate second CR instance associated with a second CRD ([0146] In the case that API request 301 is a create request for a custom resource, reconciler 816 can act on the create event for the instance data for the custom resource. Reconciler 816 may create instance data for custom resources that the requested custom resource depends on. As an example, an edge node custom resource may depend on a virtual network custom resource, a virtual interface custom resource, and an IP address custom resource. In this example, when reconciler 816 receives a create event on an edge node custom resource, reconciler 816 can also create the custom resources that the edge node custom resource depends upon, e.g., a virtual network custom resource, a virtual interface custom resource, and an IP address custom resource; Examiner notes: the container management system creates custom resource instances for all dependencies of the requested custom resource),
wherein the first CR instance implements logical forwarding for the logical router ([0010] Instances of the Virtual Network Router custom resource facilitate the exchange (referring to asymmetric or symmetric import and export) of routing information using import and export route targets that are configured within routing instances for respective virtual networks; [0096] Kubernetes may be expanded, via a custom resource representative of VNR 52A, to translate VNR 52A into one or more import and export policies that are deployed with respect to VN 50A and VN 50N so as configure intercommunication via routing information distribution between VN 50A and VN 50N. Once configured, VN 50A may export routing information (e.g., representative of routes for VN 50A) to VN 50N and import routing information (e.g., representative of routes for VN 50N) to VN 50A. Likewise, VN 50N may export routing information (e.g., representative of routes for VN 50N) to VN 50A and import routing information (e.g., representative of routes for VN 50A) to VN 50N),
Henkel does not explicitly teach each second CR instance implements the respective L7 service specified in the logical router definition.
However, Sharma teaches each second CR instance implements the respective L7 service specified in the logical router definition ([0086] Virtual router custom resource 31 may include configuration elements conventionally exposed by a physical or virtual router, but the configuration elements may be consolidated along with Kubernetes native/built-in resources to support a unified intent model that is realized by Kubernetes controllers and by custom resource controller(s) that work to reconcile the actual state of the virtual router with the intended state. The application package 29 for virtual router 21A and provided to the orchestrator 23A, executing on the computing device, includes a deployer pod to create the virtual router custom resource definition (CRD) with the orchestrator 23A, which then (1) monitors and reconciles the virtual router custom resource instance on configuration changes, and (2) deploys (including redeploying/restarting) a pod for virtual router 21A that includes those containers used to implement virtual router 21A with the requested configurations. As a result, the techniques provide a technical improvement in that the administrator for the network or a distributed application, when performing virtual router 21A deployment and configuration, is able to rely on a cloud-native framework that leverages an orchestrator interface of orchestrator 23A to simplify configuring or reconfiguring virtual router 21A, even for configurations that must occur at virtual router 21A initialization; [0119] Virtual router 220 may perform bridging (e.g., E-VPN) and routing (e.g., L3VPN, IP-VPNs) for virtual networks. Virtual router 220 may perform networking services such as applying security policies, NAT, multicast, mirroring, and load balancing; [0142] API server 320 validates and configures data for objects, such as virtual computing instances (e.g., pods of containers), services, and replication controllers, for instance. A service may be an abstraction that defines a logical set of pods and the policy used to access the pods. The set of pods implementing a service are selected based on the service definition. A service may be implemented in part as, or otherwise include, a load balancer; [0144] custom resource controllers; Examiner notes: container orchestrator creates a virtual router pod based on a custom resource definition with container instances which carry out the functions of the virtual router pod i.e., applies security policies, load balancing, etc.. The orchestrator monitors the custom resource definition for changes and updates the virtual router pod to reflect updates in virtual router configuration).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined Sharma’s virtual router custom resource reconciliation with the existing system. A person of ordinary skill in the art would have been motivated to make this combination to provide the resulting system with the advantage of simple and accessible cloud configuration and reconfiguration of virtual routing resources (see Sharma [0012] he techniques provide a technical improvement in that the administrator for the network or a distributed application, when performing virtual router deployment and configuration, is able to rely on a cloud-native framework that leverages an orchestrator interface to simplify configuring or reconfiguring a virtual router, even for configurations that must occur at virtual router initialization).
Henkel and Sharma do not explicitly teach the logical router definition specifying a set of one or more layer 7 (L7) services to be performed on data messages processed by the logical router.
However, Sridhar teaches the logical router definition specifying a set of one or more layer 7 (L7) services (Fig. 2, 3; [0061] Network services of services 233 may include security services (e.g., firewall), policy enforcement, proxy, load balancing, or other L4-L7 services).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined Sridhar’s L7 services with the existing system of Henkel. A person of ordinary skill in the art would have been motivated to make this combination to provide the resulting system with the advantage of implementing sharing of common layer 7 services within the cluster environment of Henkel (see Sridhar [0002] A service mesh provides an infrastructure layer for modern distributed applications to exchange information between various microservices in a secure and observable way. For example, a simple e-commerce application can be divided into microservices, which may include a product-view service to show product information, a database service to maintain inventory of products, and a cart service to track products selected by a user. Other examples of distributed applications can have hundreds or even thousands of different microservices. The service mesh layer may control and manage inter-service communication. These microservices may interact with other services through a service proxy. These service proxies are configured and managed by service mesh controllers).
Regarding claim 2, Henkel, Sharma, and Sridhar teach the method of claim 1.
Henkel further teaches the logical router is a first logical router defined for the network; the logical router provides a connection between the logical network and external networks (at least [0010] Instances of the Virtual Network Router custom resource facilitate the exchange (referring to asymmetric or symmetric import and export) of routing information using import and export route targets that are configured within routing instances for respective virtual networks; [0317] FIGS. 13A and 13B are diagrams illustrating a fourth instance in which virtual network routers may be configured to enable mesh interconnectivity between virtual networks in accordance with various aspects of the techniques described in this disclosure. In the example of FIG. 13A, the general network structure is similar to that shown in the example of FIG. 12A except that VNs 1500A and 1500B, pods 1502A and 1502B, and VNR 1506A are in a first namespace 1504A (shown as “namespace-1”) while VNs 1500C and 1500D, pods 1502C and 1502D, and VNR 1506B are in a second, different namespace 1504B (shown as “namespace-2”)); and
in response to the definition of the first CR instance and the separate CR instances, the control plane instantiates (i) at least one Pod for the first CR instance to perform logical forwarding for data messages sent between the logical network and the external networks and (ii) at least one Pod for each separate CR instance to perform the associated L7 service (Fig 13 Pods; ([0043] Any server of servers 12 may be configured with virtual execution elements, such as pods or virtual machines, by virtualizing resources of the server to provide some measure of isolation among one or more processes (applications) executing on the server)).
Regarding claim 3, Henkel, Sharma, and Sridhar teach the method of claim 2.
Henkel further teaches configuring at least one interface on the Pod for the first CR instance to communicate with an external router, wherein a configuration of the interface is based on the definition of the logical router (Fig. 13-15 Intra VNR communications; [0092] That is, rather that resort to defining how routing is to occur between two or more VNs 50, the administrator may define one or more VNRs 52 to interconnect VNs 50 without having to manually develop and deploy extensive policies and/or routing instance configurations to enable the exchange of routing information between such VNs 50. Instead, the administrator (which may have little understanding of routing protocols) may define a custom resource (e.g., one or more of VNRs 52) using familiar Kubernetes syntax/semantics (or even just by dragging graphical elements and specifying interconnections between this graphical element representative of, as an example, VNR 52A, and graphical elements representative of, again as an example, VNs 50A and 50N)).
Regarding claim 4, Henkel, Sharma, and Sridhar teach the method of claim 2.
Henkel further teaches configuring connectivity between the Pod for the first CR instance and the Pods for each of the separate CR instances (Fig 12; [0315] All Pods of VNs connected to both VNR 1506A (“VNR-web”) and VNR 1506B (“VNR-db”) can, by causing VNR 1506A, 1506B to import/export each others' routes, in this way be enabled to communicate with each other using VNRs and labels; Fig 14/15; [0328] In this instance, VNs 1500A-1500C can only interconnect with a VNR 1516A and 1516B in the same namespace, and only spoke VNR 1516B and hub VNR 1516A can communicate across namespaces 1504A and 1504B for the reasons noted above. Assuming authorization is given by the administrators of both namespaces 1504A and 1504B, the hub-and-spoke connectivity shown at the bottom of FIG. 15 may be enabled in a similar if not substantially similar manner to that described above with respect to the example of FIG. 14).
Regarding claim 5, Henkel, Sharma, and Sridhar teach the method of claim 2.
Henkel further teaches the set of L7 services is a first set of L7 services, the method further comprising: receiving a definition of a second logical router for the logical network that specifies a second set of L7 services to be performed on data messages processed by the second logical router; and defining additional separate CR instances associated with the second CRD for each L7 service specified for the second logical router (Fig 15 VNR spoke in namespace-1 routing Pod 1/VN1 and Pod 2/VN2 and VNR hub in namespace-2 routing Pod 3/VN3; Examiner notes: Kubernetes is able to spin up additional routers and associated pods for different services).
Regarding claim 7, Henkel, Sharma, and Sridhar teach the method of claim 5.
Henkel further teaches the first logical router is a first tier of logical router for interfacing with the external networks (Fig 15 VNR spoke interfaces with VNR hub) and the second logical router is a second tier of logical router for handling data traffic to a subset of logical network endpoints (Fig 15 VNR hub handles traffic for endpoints in VN3; [0173] Virtual routers may be processes or threads, or a component thereof, executed by the physical servers, e.g., servers 12 of FIG. 1, that dynamically create and manage one or more virtual networks usable for communication between virtual network endpoints. In one example, virtual routers implement each virtual network using an overlay network, which provides the capability to decouple an endpoint's virtual address from a physical address (e.g., IP address) of the server on which the endpoint is executing).
Regarding claim 9, Henkel, Sharma, and Sridhar teach the method of claim 1.
Henkel further teaches the network management system specifies to the control plane initial numbers of Pods for implementing the first CR instance and each of the separate CR instances ([0209] A service may be an abstraction that defines a logical set of pods and the policy used to access the pods. The set of pods implementing a service are selected based on the service definition. A service may be implemented in part as, or otherwise include, a load balancer. API server 300A and custom API server 301A may implement a Representational State Transfer (REST) interface to process REST operations and provide the frontend, as part of the configuration plane for an SDN architecture, to a corresponding cluster's shared state stored to configuration store 1328. API server 300A may represent a Kubernetes API server).
Regarding claim 10, Henkel, Sharma, and Sridhar teach the method of claim 9.
Henkel further teaches the network management system specifies a physical connectivity requirement for each of the Pods that implement the first CR instance indicating that said Pods are required to have physical connectivity to a set of one or more external routers outside of the cluster (Table 2; [0226] This metadata information may be copied to each pod replica created by the controller manager 1326. When the SDN controller manager 1325 is notified of these pods, SDN controller manager 1325 may create virtual networks as listed in the annotations (“red-network”, “blue-network”, and “default/extns-network” in the above example) and create, for each of the virtual networks, a virtual network interface per-pod replica (e.g., pod 202A) with a unique private virtual network address from a cluster-wide address block (e.g. 10.0/16) for the virtual network).
Regarding claim 11, Henkel, Sharma, and Sridhar teach the method of claim 1.
Henkel further teaches defining an orchestrator for monitoring Pods that implement the first CR instance and the separate CR instances ([0064] orchestrator 23 controls the deployment, scaling, and operations of containers across clusters of servers 12 and providing computing infrastructure, which may include container-centric computing infrastructure. Orchestrator 23 and, in some cases, network controller 24 may implement respective cluster masters for one or more Kubernetes clusters; [0068] The orchestrator 23 and container platform 19 use CNI 17 to manage networking for pods, including pod 22; [0193] Orchestration agent 592 is an agent of an orchestrator, e.g., orchestrator 23 of FIG. 1, that receives container specification data for containers and ensures the containers execute by computing device 500. Container specification data may be in the form of a manifest file sent to orchestration agent 592 from orchestrator 23 or indirectly received via a command line interface, HTTP endpoint, or HTTP server. Container specification data may be a pod specification (e.g., a PodSpec—a YAML (Yet Another Markup Language) or JSON object that describes a pod) for one of pods 502 of containers. Based on the container specification data, orchestration agent 592 directs container engine 590 to obtain and instantiate the container images for containers 529, for execution of containers 529 by computing device 500).
Regarding claim 12, it is the non-transitory machine-readable medium of claim 1. Therefore, it is rejected for the same reasons as claim 1.
Regarding claims 13-16, they are the non-transitory machine-readable media of claims 2-5respectively. Therefore, they are rejected for the same reasons as claims 2-5 respectively.
Regarding claims 19 and 20, they are the non-transitory machine-readable media of claims 9 and 10 respectively. Therefore, they are rejected for the same reasons as claims 9 and 10 respectively.
Allowable Subject Matter
Claims 6, 8, 17, and 18 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to HARRISON LI whose telephone number is (703) 756-1469. The
examiner can normally be reached Monday-Friday 9:00am-5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Aimee Li can be reached on (571) 272-4169. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.L./
Examiner, Art Unit 2195
/Aimee Li/Supervisory Patent Examiner, Art Unit 2195
Prosecution Insights