Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5-10, 12-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over ASVEREN (Publication 2021/0328858) in view of BARNARD (Publication 2018/0324260).
As to claim 1, ASVEREN teaches a method for removing redundant channels between a client and pods of a service, the method comprising:
sending, from the client to the service, a first request for an application exposed by the service, the first request corresponding to a request for a first channel (EN: via CNI Add);
receiving, at the client from a first pod associated with the service, a response indicating acceptance of the first request, establishment of the first channel between the first pod and the client, and an identifier of the first pod;
storing, by the client and in a data structure associating channels with pods of the service, an association between the first channel and a name of the first pod ([0057] In the exemplary system 100, kubemetes worker node 2 108 includes a Standby Instance 212, e.g., a Standby Session Border Controller Pod 212 which includes a set of containers or workloads that collectively form a software application that provides session border controller services when activated, a plurality of interface cleanup service Pods (Interface Cleanup Service Pod 1 214, . . . , Interface Cleanup Service Pod X 215) which provides interface management services in response to Pod requests for network interfaces unknown to kubelet 219, i.e., interfaces which are not visible to the kubelet 219, a CNI service Plugin 216 which includes, among other things, a CNI Add API 217 and a CNI Delete API 218, a kubelet 219 which is a software node management application, a Kube-proxy 220 which is a network proxy which reflects services as defined in the Kubemetes API on each node and can do simple TCP, UDP, and SCTP stream forwarding or round robin TCP, UDP, and SCTP forwarding across a set of backends, and a CNI Plug-in Daemon 221 which provides network connectivity between nodes and which monitor's the health or status of other nodes and Pods on the node upon which it is executing and upon detection of a failed Pod will take corrective action e.g., to delete the failed Pod. CNI service 216 adds network interfaces in response to requests made via the CNI Add API 217. For example, a Pod may directly request an additional network interface by invoking or calling the CNI service 216 via the CNI Add API 217 exposed by the CNI service Plugin 216. Similarly, the CNI service 216 deletes network in response to requests made via the CNI Delete API 218. The CNI Plug-In Daemon 221 runs continuously and checks on other nodes health as well as whether any Pods on the node on which it is executing have failed.
[0058] In some embodiments, the CNI Plug-in application exposes/handles both the CNI ADD API/CNI DELETE APIs. These APIs are called to add/delete network interfaces. The Kubelet calls CNI ADD during Pod initialization and CNI DELETE when Pod terminates. Application running on a Pod also calls them for network interfaces, which it adds/deletes independent of the Kubernetes system standard components, e.g., independent of the Kubelet managing the node. A Kubernetes node corresponds to a host (physical or virtual). There is a single Kubelet and CNI plug-in application on each worker node. On each node, there is also a CNI Plug-in Daemon. The CNI Plug-in Daemon provides network connectivity between nodes. When a Pod fails, but the node on which the Pod is located does not fail that is it remains operational, the Kubelet on that node calls or invokes the CNI DELETE API for that Pod so that network interface it added during initialization is cleaned up. As part of this process, CNI Plug-in Daemon on that node informs other Plug-in Daemons on other nodes of the system so that inter-node network connectivity is updated properly. When the node fails, the Kubelet and CNI entities on that node are also down. They cease to exist. In such a case, CNI plug-in Daemons on other nodes of the system detect the node's failure, e.g. they don't receive health-check messages from the CNI plug-in daemon on the failed node, and CNI plug-in Daemons on the other nodes update inter-node network connectivity properly to reflect the failed node status. For interfaces added by an application executing on a Pod independent of the Kubernetes system, the CNI DELETE API needs to be called when the Pod has failed or is down but the node remains operational. This is similar to the Kubelet calling CNI DELETE for the network interface it created during Pod initialization and this operation is achieved through the use of a request to an Internet Cleanup Service Pod located on the failed node as discussed in connection with the methods and signaling diagrams discussed in the specification.
[0059] In some embodiments, the invocation of a request for an additional interface, for example a CNI ADD request is an API call via the CNI ADD API. The request, invocation or call including the IP Address of the interface being a parameter of this API call. Similarly, in some embodiments, the invocation of a request to delete an interface, for example a CNI DEL request is an API call via the CNI DEL API. The request, invocation or call including the IP Address of the interface to be deleted.
[0060] The CNI ADD API request/call also has other parameters beyond the IP address. Some of the pertinent parameters are discussed/shown in the context of the following exemplary CNI ADD request/call: CNI ADD (network interface name, IP Address, container-id, network namespace path). The network interface name parameter is the name of the desired network interface, e.g. net0. The IP Address parameter is IP Address to be bound for the network interface, e.g. 172.192.156.32. The container-id parameter is the container-id of the Pod making the CNI ADD request, e.g. 571c3a115fcf. The network namespace path parameter is the path to the network namespace to be added, i.e. /proc/[pid]/ns/net or a bind-mount/link to it.
[0061] With respect to the IP address the CNI ADD (IP-X, . . . ) request/call is a request that the Pod making the request wants a network interface to be created with the address being IP-X. The IP Address to use for the interface is determined by the Pod. The Pod has the information for the interface request, e.g., the IP address to request, through configuration and/or by getting it from another entity of the Kubernetes system such as for example an IP Address Allocation Manager. Some of the pertinent parameters of CNI DEL request/call are discussed/shown in the context of the following exemplary CNI DEL request/call: CNI DEL(network interface name, container-id, network namespace path). The network interface name parameter is name of the network interface to be deleted, e.g. net0. The container-id parameter is the container-id of the Pod making the CNI DEL request, e.g. 571c3a115fcf. The network namespace path is the path to the network namespace to be deleted, i.e. /proc/[pid]/ns/net or a bind-mount/link to it.
[0062] When there is an interface with IP-X, a new CNI ADD (IP-X, . . . ) should not be called as it would create confusion in terms of IP packet routing, which is handled by CNI plug-in daemon. Prior to adding the interface with the IP-X address using the CNI ADD (IP-X, . . . ) call/request, the CNI DEL needs to be called, e.g., CNI DEL (IP-X, . . . ). The Interface Cleanup Service Pod on the node with the failed Pod to which the interface with the IP-X address is allocated makes this CNI DEL call/request in various embodiments of the invention.
[0063] In Kubernetes, a Service is an abstraction which defines a logical set of Pods and a policy which defines how the set of Pods is accessed. In the present example, the Kubernetes service includes an Active Pod and a Standby Pod.)
However, ASVERAN does not teach shutting down a channel in response to the first pod being associated with the first channel and one or more other channels.
BARNARD teaches in response to the first pod being associated with the first channel and one or more other channels, shutting down, by the client, the one or more other channels (abstract; figure 5; 0039] As previously noted, a number of sessions established for an account, for a user, and/or for a group of users may be limited. This provides those managing an account with an ability to set the maximum number of concurrent interactive sessions a user can have. The number (e.g., 1) may be set by an administrator and may vary from user to user and/or between roles of users.
[0048] FIG. 5 illustrates a process 420 that may be performed by the platform 104. At least part of the functions may be embodied using hardware or software in the load balancer 404, the nodes 408, 414, or 418, application server 107, databases 108, and/or other connected devices. The platform 104 receives a request to access an instance (block 422). This request may be received from any connected device (e.g., the client 102) using a web browser, a dedication application program, and/or another suitable mechanism for sending a request to the platform 104. The request may include valid credentials (e.g., username and password). If the credentials are invalid, the platform 104 may not recognize the request as a valid request.
[0049] In response to a valid request, the platform 104 then determines whether the number of sessions that would result from granted access would surpass a threshold (block 424). For example, the platform 104 may determine whether a threshold limit is set based on a user, a group of users, roles of users, total number of users for the instance, and/or other limitations that may be placed on the instance.
[0050] When the number of sessions will surpass the threshold, the platform 104 determines (e.g., via the first node 408) whether an existing session to be terminated exists on a common node with the session to be established (block 426). When only a single node exists, the session to be established may be automatically considered as being on a common node with the existing session. When the session to be created and the session to be terminated are on a common node, the node may terminate the existing session directly (block 428). However, if the nodes are not common for the session to be established and the session to be terminated, the node where the session is to be established may send a message to the node where the session is to be terminated to terminate the session (block 430). Regardless of which node terminates the existing session, terminating the session may include indicating that the existing session is now locked out in the database 108 and returning the session to a login page. Returning the session to the login page may occur when the now-terminated session is reasserted.
[0051] In addition to terminating the existing session, the session to be established is established (block 432). Establishing the session may include incrementing a number of active sessions in the database 108 (unless the established session is only replacing an existing session). In the illustrated embodiment, the message is sent before establishing the session. However, in some embodiments, this order may be reversed. Regardless of the order of the termination of the old session and establishment of the new session, in some embodiments, the sending of the message to close the existing session may only be generated and/or sent after the credentials are validated that are to be used in establishing the new session. In such embodiments, this order reduces likelihood of closing of the session by unauthorized users, such as a denial of service attack.; EN: It would be obvious to one of ordinary skill in the art that the threshold could be set to only allow one session / connection between nodes.)
.
PNG
media_image1.png
878
776
media_image1.png
Greyscale
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing of the claimed invention to combine the teachings of BARNARD to the teachings of ASVEREN in order to reduce the likelihood of closing of the session by unauthorized users ([0051]).
As to claim 2, BARNARD teaches in response to shutting down the one or more other channels, sending, from the client to the service, one or more additional requests for one or more additional channels to replace the one or more other channels (0039, 0048-0051; FIGURE 5, ITEMS 428, 432 AND 434).
PNG
media_image1.png
878
776
media_image1.png
Greyscale
As to claim 3, BARNARD teaches determining whether an environment is associated with multiple channels in the data structure is performed periodically by the client (19. The method of claim 18, wherein invalidating the first session comprises limiting transmission of the cluster message to a threshold periodicity, wherein the cluster message identifies all sessions to be terminated by any nodes connected to the second node.; [0037] The management service 310 may include one or more servers providing access to and managing the database108. The management service 310 may allocate or provision resources, such as application instances in the resources 306 or 308, from a respective environment 302 or 304. Further, the management service 310 may create, modify, or remove information in the database 108. For example, the management service 310 may manage a catalogue of resources in more than a single environment (even if the environments may not directly communicate with each other). Using this catalogue, the management service 310 may discover new resources, provision resources, allocate resources, modify, and/or remove resources from the catalogue across a single environment or multiple environments. In some embodiments, these actions may be initiated using the client 102, scheduled for periodic occasions, or a combination thereof. For example, a client 102 may receive a request, via its input structures, to query an identity of an application program interface (API) used by a resource to access a particular vendor/provider for the environment 302 that is passed to the management service 310 to query the database 108. As another example, the client 102 may receive a request, via its input structures, to query an identity of a user authorized to access a particular resource that is passed to the management service 310. [0040] When a user (or group of users) already has a maximum number of concurrent active sessions active. When a user logs in, the current sessions are identified. If the new session will result in the total number of sessions exceeding the maximum number, at least one session is closed. For example, for each excess session, an oldest sessions may be closed based on its creation date by sending out a cluster message across all nodes to set the locked_out attribute on the session(s) that is to be terminated. When the user tries to access the closed instance through a session that is locked_out, the client 102 will be redirected to the login page. In some embodiments, a single cluster message may correspond to multiple sessions to terminate. In other words, cluster messages may be bundled. For instance, cluster messages may be bundled to reduce likelihood of congestion of communications due to the cluster messages. In such embodiments, transmission of the cluster messages may be limited to a specific periodicity (e.g., once per minute) so that the transmitted cluster message bundles may identify all sessions to be closed that have been identified since the last cluster message was transmitted.). ASVEREN teaches the environment is associated with a pod ([0057-0061]).
As to claim 5, ASVEREN teaches the first pod includes a plurality of containers running workloads of the application ([0006] Pods support multiple cooperating containers (or processes) that form a cohesive unit of service. The containers in a Pod are co-located on the same physical or virtual machine in the cluster and share resources including networking and storage resources. Each Pod is assigned a unique IP address and every container in a Pod shares the network namespace, including the IP address and network ports. A Pod can specify a set of shared storage Volumes that are accessible to the containers in the Pod.).
As to claim 6, ASVEREN teaches the first pod runs in a virtual machine ([0006] Pods support multiple cooperating containers (or processes) that form a cohesive unit of service. The containers in a Pod are co-located on the same physical or virtual machine in the cluster and share resources including networking and storage resources. Each Pod is assigned a unique IP address and every container in a Pod shares the network namespace, including the IP address and network ports. A Pod can specify a set of shared storage Volumes that are accessible to the containers in the Pod.) .
As to claim 7, ASVEREN teaches the service is container orchestration service ([0002] Kubernetes is an open source project hosted by the Cloud Native Computing Foundation (CNCF). It provides an open source container orchestration engine for automating deployment, scaling, and management of containerized applications. The Kubernetes orchestration platform or system can be used to deploy or distribute applications sometimes referred to as workloads, e.g., session border controller applications or other program applications on nodes on hosts. The definition of various Kubernetes elements will now be discussed. Some of the definitions discussed herein have been taken or derived from information at the Kubernetes website https://kubernetes.io/docs/ and its glossary. [0010] A Kubernetes service is an abstract way to expose an application running on a set of Pods as a network service. The Service makes sure that network traffic can be directed to the current set of Pods for the workload. [0014] For example, it may be determined that there is a need for five instances of session borders controllers and the Kubernetes orchestrates or decides which hosts of a cluster of hosts or nodes (e.g., 10 hosts or nodes) that the five session border controllers should be deployed or instantiated on. For example, SBC 1 may be deployed or implemented on the second node or host; SBC 2 may be deployed on the first node or host, SBC 3 may be deployed on the fifth node or host, SBC 4 may be deployed on the eighth node or host, and SBC 4 may be deployed on the tenth node or host. Each instance of an SBC being a Pod. That is each instance of an SBC being a set of containers or workloads that make up a SBC. Upon the instantiation and initialization of the SBC instance or Pod, the kubelet management software application for the node invokes the container network interface (CNI) service and establishes the IP address to be used by the SBC instance or Pod for communications. With respect to the SBC instances, the standard Kubernetes interface created by the kubelet management software application for the node does not meet the needs, i.e., does not satisfy the networking interface requirements of the SBC instance or Pod due to IP Address failover semantics needed for SBC (as elaborated further later in this document). For example, the standard Kubernetes networking interface for communicating with external entities from a Pod does not satisfy the requirements for an SBC instance or Pod. As a result the SBC instance or Pod will then directly call or invoke the container network interface (CNI) service application to obtain additional interfaces for communicating with external entities. The kubelet's management software for the node is unaware of the assignment of these additional interfaces to the SBC instance or Pod.)
As to claims 8-10 and 12-14, reference is made to an apparatus that corresponds to the method of claims 1-3 and 5-7 and is therefore met by the rejection of claims 1-3 and 5-7 above. Claim 8 further details the system comprising: at least one processor; and at least one memory (See ASVEREN [0158-0159]).
As to claims 15-17 and 19, reference is made to a non-transitory computer readable medium that corresponds to the method of claims 1-3 and 5 and is therefore met by the rejection of claims 1-3 and 5 above.
As to claim 20, ASVEREN teaches the first pod runs in a virtual machine and the service is a Kubernetes service ([0005] A Pod is the basic execution unit of a Kubernetes application. It represents a single instance of an application in Kubernetes. A Pod encapsulates an application's container(s), storage resources, a unique network Internet Protocol (IP) address, and options which govern how the container(s) should execute. Each Pod runs a single instance of a given application. If you want to scale your application you use multiple Pods, one for each instance. [0006] Pods support multiple cooperating containers (or processes) that form a cohesive unit of service. The containers in a Pod are co-located on the same physical or virtual machine in the cluster and share resources including networking and storage resources. Each Pod is assigned a unique IP address and every container in a Pod shares the network namespace, including the IP address and network ports. A Pod can specify a set of shared storage Volumes that are accessible to the containers in the Pod. [0010] A Kubernetes service is an abstract way to expose an application running on a set of Pods as a network service. The Service makes sure that network traffic can be directed to the current set of Pods for the workload.).
Claims 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over ASVEREN in view of BARNARD as applied to claim 1 above, and further in view of SHEN (Publication 20220038501).
As to claim 4, ASVEREN and BARNARD does not explicitly teach that the request is via an ingress.
SHEN teaches sending, from the client to the service, the first request is performed via an ingress (figure 3, 4, [0006] When an agent receives a request for flow entries that relate to a particular Kubernetes concept (e.g., to a specific network policy), the agent identifies flow entries realized by the forwarding element executing on its container host that match the request. For example, for specific network policies or network policy rules, flow entries include a specific identifier in one of the match or action fields (e.g., a conjunction identifier, for conjunctive flow entries). Specific pods can be identified by network addresses (or data link addresses) used in flow entries (e.g., as match conditions). For each identified flow entry that matches the request, the agent generates mapping data that maps elements of the flow entry to specific Kubernetes concepts (e.g., pods, network policies, rules, etc.). For instance, matches over table identifiers, network addresses, and other conditions may be indicative of specific network policies and/or network policy rules, pods, nodes, etc. Raw flow entry data may be difficult for a network administrator or application developer to understand, so the generated mapping data is provided along with each flow entry for presentation to the requesting user. In different embodiments, this data is provided to the controller or directly to a user interface (e.g., a command line interface) from which the request was received.
[0050] As shown, the process 200 begins by receiving (at 205) a request for information about flow entries associated with a particular Kubernetes concept in a cluster. The request may relate to a particular network policy (i.e., a declared Kubernetes network policy), or a specific entity in the cluster (e.g., a particular pod, node, or service). In addition, some embodiments allow more complex requests, such as a request for all flow entries relating to any network policy that are applied to a specific pod. In different embodiments, this request may be received at the CNI agent directly from the command line interface tool associated with the CNI (or a different interface with which a developer or administrator interacts) or via the centralized CNI controller (e.g., based on a request to the controller from the CLI tool).
[0057] As mentioned, FIG. 7 illustrates an example of such a report provided by the CLI tool of some embodiments based on data returned by a CNI agent. Before discussing this report, an example network policy will be discussed. FIG. 5 illustrates an example network policy 500 of some embodiments for web servers in a cluster. Specifically, this network policy 500 is referred to as web-app-policy and is applied to pods that match the label app=web-server. This network policy 500 is an ingress policy that, for the web-server pods, only allows http ingress traffic (TCP port 80) from pods that match the label app=web-client. That is, the web server pods are only allowed to receive http traffic from the web client pods.
[0065] Instead, the CNI of some embodiments provides an efficient way to export ongoing connections correlated to Kubernetes concepts and associated with network policy information, so that the consumers (e.g., a policy analytics engine, visualization solution, or direct user observation) can more easily identify the patterns of the connections within the cluster as well as the network policies and specific network policy rules that impact the different connections. Thus, some embodiments collect connection information from the data plane, append Kubernetes context to the connection information, and export the connection data (with the appended context) using, e.g., IPFIX. The context added to a connection may include source pod, source node, destination pod, destination node, destination service (if the connection is between a pod and a service in the cluster), and ingress and/or egress network policy and policy rules.).
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing of the claimed invention to apply SHEN to the combination of ASVEREN and BARNARD in order to manage connection information, including ingress and egress policies and data.
As to claim 11, reference is made to an apparatus that corresponds to the method of claim 4 and is therefore met by the rejection of claim 4 above. Claim 11 which depends from 8 further details the system comprising: at least one processor; and at least one memory (See ASVEREN [0158-0159]).
As to claim 18, reference is made to a non-transitory computer readable medium that corresponds to the method of claim 4 and is therefore met by the rejection of claim 4 above.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEWIS ALEXANDER BULLOCK JR whose telephone number is (571)272-3759. The examiner can normally be reached Monday-Friday, 9:00-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cordelia Zecher can be reached at 571-272-7771. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199