DETAILED ACTION
1. Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
2. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2023-0138219, filed on October 17, 2023.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on February 14, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on October 15, 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-3, 5-6, 9-14, 16-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. and Baggerman et al. US 10176550 B1.
5. Kim et al. US 20220179661 A1 was cited in IDS filed on 10/15/2025 and Baggerman et al. US 10176550 B1 was cited in IDS filed on 2/14/2024.
6. With regard to claim 1, Kim teaches:
A method for executing Artificial Intelligence (AI) services based on virtual infrastructures using a cluster of computing servers ([0011] Meanwhile, in the container-based artificial intelligence service providing environment, a plurality of containers may be provided for one artificial intelligence service, for high availability of the container, that is, for constantly keeping the container available.), comprising:
configuring a sharing type of a computational processing unit ([0012] The container provided by the artificial intelligence service client may provide the artificial intelligence service to the client by using a CPU or a GPU of the artificial intelligence service server. For this, the artificial intelligence service client may provision resources such as a CPU, a GPU, a memory, and the like of the artificial intelligence service server necessary for maintenance and management of the container on the container.);
executing an AI service based on a virtual infrastructure using requirements for the AI service and information about the sharing type of the computational processing unit ([0021] The identifying the container in which the artificial intelligence model is to be loaded may include identifying the container in which the artificial intelligence model is to be loaded based on resources provisioned to the plurality of containers and resources required to the plurality of artificial intelligence models; [0022] The identifying the container in which the artificial intelligence model is to be loaded may include identifying a resource usable in each container based on the resources provisioned respectively to the plurality of containers and the resources required to the artificial intelligence model loaded in each container, and identifying the container in which the artificial intelligence model is to be loaded based on the identified resource.); and
Kim teaches of provisioning necessary resources to containers for AI services to use and that the processor may identify a container in which the artificial intelligence model is to be loaded based on resources provisioned on the plurality of containers and resources required to the plurality of artificial intelligence models. The plurality of containers may execute the function corresponding to the artificial intelligence service requested from the client device based on a graphic processing unit (GPU) or a central processing unit (CPU). The execution of the function corresponding to the artificial intelligence service based on a GPU or a CPU may refer to execution of a function corresponding to the artificial intelligence service using resource such as the GPU or the CPU ([0092]). Although this method helps efficiently allocate resources, which is a form of optimization, Kim does not explicitly use the term “optimization”. However, according to the Meriam webster dictionary, optimize is “as in to improve.” Therefore, improving resource allocation is optimization. Baggerman was brought in to help further define and emphasize that Kim’s teachings are optimization.
However, in analogous art, Baggerman teaches:
performing optimization for the service (Col. 4, lines 34-37, The present disclosure provides an improved approach for optimizing the allocation of GPU resources to virtual machines supported by nodes in a networked virtualization system; Col. 2, lines 60 – Col. 3, lines 7, A vGPU profile indicates how virtualized resources of a physical GPU may be allocated to virtual machines supported by a node in which the physical GPU is located. Based on a comparison of the GPU resources used by the virtual machines and the GPU resources allocated to the virtual machines, the vGPU profile mechanism may reassign a virtual machine to a different vGPU profile and reallocate GPU resources to the virtual machine according to the vGPU profile to which the virtual machine is reassigned. The virtual machine is reassigned to a different vGPU profile if reassignment is likely to achieve a more efficient allocation of GPU resources to the virtual machine (i.e., GPU resources are reallocated to more closely match the GPU resource requirements of the workload of the virtual machine).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Baggerman of performing optimization for the service. Kim teaches of provisioning necessary resources to containers for AI models to be loaded. AI models are loaded into containers according to required resources, which include GPU and CPU. This process optimizes resource usage, although the word “optimization” is not explicitly used. According to the Meriam webster dictionary, optimize is “as in to improve.” Therefore, improving resource allocation is optimization. However, Baggerman is used to further strengthen and clarify this definition. Baggerman teaches of an allocation process that mirrors Kim’s. Moreover, Baggerman admits that the process of reallocating resources to achieve efficient allocation of GPU resources to the virtual machine is optimizing the allocation of GPU resources to the VMs. Therefore, together, Kim and Baggerman teach a method of configuring a sharing type of a computational processing unit and executing an AI service based on the virtual infrastructure using requirements for the AI service and information about the sharing type of the computation processing unit, which performs optimization for the AI service.
7. With regard to claim 2, Kim further teaches:
wherein the sharing type of the computational processing unit includes a type in which a resource of the computational processing unit is shared by multiple virtual infrastructures and a type in which virtual computational processing units are generated by partitioning the resource of the computational processing unit ([0011] Meanwhile, in the container-based artificial intelligence service providing environment, a plurality of containers may be provided for one artificial intelligence service, for high availability of the container, that is, for constantly keeping the container available; [0012] The container provided by the artificial intelligence service client may provide the artificial intelligence service to the client by using a CPU or a GPU of the artificial intelligence service server. For this, the artificial intelligence service client may provision resources such as a CPU, a GPU, a memory, and the like of the artificial intelligence service server necessary for maintenance and management of the container on the container; [0021] The identifying the container in which the artificial intelligence model is to be loaded may include identifying the container in which the artificial intelligence model is to be loaded based on resources provisioned to the plurality of containers and resources required to the plurality of artificial intelligence models; [0028] The plurality of containers may execute a function corresponding to an artificial intelligence service requested from the client based on a graphic processing unit (GPU) or a central processing unit (CPU).).
8. With regard to claim 3, Kim further teaches:
wherein the sharing type of the computational processing unit includes a first type in which the entire computational processing unit supports AI services of multiple virtual infrastructures ([0011] Meanwhile, in the container-based artificial intelligence service providing environment, a plurality of containers may be provided for one artificial intelligence service, for high availability of the container, that is, for constantly keeping the container available.),
a second type in which the entire computational processing unit supports AI services of multiple virtual infrastructures but the AI services of the multiple virtual infrastructures are integrated into a single context ([0020] The method includes identifying a container in which an artificial intelligence model is to be loaded, based on attributes of a plurality of containers in which a library for artificial intelligence models is loaded and attributes of a plurality of artificial intelligence models registered in a model store, loading the artificial intelligence model in the container based on the library loaded in the containers, based on a request for an artificial intelligence service being received from a client, obtaining a function corresponding to the requested artificial intelligence service from a database including a plurality of functions, identifying a container in which an artificial intelligence model corresponding to the artificial intelligence service is loaded among the plurality of containers in which artificial intelligence models are loaded, and obtaining data for the request from the artificial intelligence model loaded in the identified container by executing, in the container, the obtained function based on the library loaded in the container, and transmitting the obtained data to the client.),
a third type in which multiple virtual computational processing units are generated by partitioning memory of the computational processing unit ([0012] The container provided by the artificial intelligence service client may provide the artificial intelligence service to the client by using a CPU or a GPU of the artificial intelligence service server. For this, the artificial intelligence service client may provision resources such as a CPU, a GPU, a memory, and the like of the artificial intelligence service server necessary for maintenance and management of the container on the container.), and
a fourth type in which multiple virtual computational processing units are generated by partitioning the memory and cores of the computational processing unit ([0012] The container provided by the artificial intelligence service client may provide the artificial intelligence service to the client by using a CPU or a GPU of the artificial intelligence service server. For this, the artificial intelligence service client may provision resources such as a CPU, a GPU, a memory, and the like of the artificial intelligence service server necessary for maintenance and management of the container on the container; Examiner’s Note: The provisioning of CPUs is the partitioning of cores.).
9. With regard to claim 5, Kim further teaches:
wherein a type of the virtual infrastructure includes a virtual machine or a container ([0009] Meanwhile, in the container-based environment, the artificial intelligence service client may generate a container for providing the artificial intelligence service and provide this to the artificial intelligence service server (artificial intelligence server (AI server)).).
10. With regard to claim 6, Kim further teaches:
wherein performing the optimization comprises performing optimization for partitioning of the computational processing unit, a batch size, a combination of AI models to be simultaneously executed, and the sharing type of the computational processing unit ([0021] The identifying the container in which the artificial intelligence model is to be loaded may include identifying the container in which the artificial intelligence model is to be loaded based on resources provisioned to the plurality of containers and resources required to the plurality of artificial intelligence models; [0167] The reinforcement training module 590 may receive information on the time when the function is executed in each container, and determine, if the same function is executed in the container later, whether it is to be executed in a GPU-based container (e.g., container 530) or in the CPU-based container 540. For example, if the function execution time of the GPU-based container is shorter than predetermined time, the reinforcement training module 570 may determine that it is efficient to execute the corresponding function in the CPU-based container, if the corresponding function is called again. In addition, if the function execution time of the CPU-based container is equal to or longer than the predetermined time, the reinforcement training module 590 may determine that it is efficient to execute the corresponding function in the GPU-based container, if the corresponding function is called again.).
Kim teaches of provisioning necessary resources to containers for AI services to use and that the processor may identify a container in which the artificial intelligence model is to be loaded based on resources provisioned on the plurality of containers and resources required to the plurality of artificial intelligence models. The plurality of containers may execute the function corresponding to the artificial intelligence service requested from the client device based on a graphic processing unit (GPU) or a central processing unit (CPU). The execution of the function corresponding to the artificial intelligence service based on a GPU or a CPU may refer to execution of a function corresponding to the artificial intelligence service using resource such as the GPU or the CPU ([0092]). Although this method helps efficiently allocate resources, which is a form of optimization, Kim does not explicitly say that this process performs optimization for the service. According to the Meriam webster dictionary, optimize is “as in to improve.” Therefore, improving resource allocation is optimization. Baggerman was brought in to help further define and emphasize that Kim’s teachings are optimization.
However, in analogous art, Baggerman teaches:
performing optimization for the service (Col. 4, lines 34-37, The present disclosure provides an improved approach for optimizing the allocation of GPU resources to virtual machines supported by nodes in a networked virtualization system; Col. 2, lines 60 – Col. 3, lines 7, A vGPU profile indicates how virtualized resources of a physical GPU may be allocated to virtual machines supported by a node in which the physical GPU is located. Based on a comparison of the GPU resources used by the virtual machines and the GPU resources allocated to the virtual machines, the vGPU profile mechanism may reassign a virtual machine to a different vGPU profile and reallocate GPU resources to the virtual machine according to the vGPU profile to which the virtual machine is reassigned. The virtual machine is reassigned to a different vGPU profile if reassignment is likely to achieve a more efficient allocation of GPU resources to the virtual machine (i.e., GPU resources are reallocated to more closely match the GPU resource requirements of the workload of the virtual machine).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Baggerman of performing optimization for the service. Kim teaches of provisioning necessary resources to containers for AI models to be loaded. AI models are loaded into containers according to required resources, which include GPU and CPU. This process optimizes resource usage, although the word “optimization” is not explicitly used. According to the Meriam webster dictionary, optimize is “as in to improve.” Therefore, improving resource allocation is optimization. However, Baggerman is used to further strengthen and clarify this definition. Baggerman teaches of an allocation process that mirrors Kim’s. Moreover, Baggerman admits that the process of reallocating resources to achieve efficient allocation of GPU resources to the virtual machine is optimizing the allocation of GPU resources to the VMs. Therefore, together, Kim and Baggerman teach a method of configuring a sharing type of a computational processing unit and executing an AI service based on the virtual infrastructure using requirements for the AI service and information about the sharing type of the computation processing unit, which performs optimization for the AI service.
11. With regard to claim 9, Kim teaches the method of claim 6 and AI services but fails to explicitly teach wherein performing the optimization comprises performing service migration and, when necessary, changing the sharing type of the computational processing unit.
However, in analogous art, Baggerman teaches:
wherein performing the optimization comprises performing service migration and, when necessary, changing the sharing type of the computational processing unit (Col. 8, lines 53-67, In some embodiments, once a virtual machine 110, 112 is reassigned from an initial vGPU profile 108 to a different vGPU profile 108, the workload of the virtual machine 110, 112 may be migrated. For example, the workload of a virtual machine 110, 112 may be migrated from a physical GPU 107 associated with the vGPU profile 108 initially assigned to the virtual machine 110, 112 to a different physical GPU 107 associated with the vGPU profile 108 to which the virtual machine 110, 112 was reassigned. The workload of a virtual machine 110, 112 may be migrated from a first physical GPU 107 to a second physical GPU 107, in which the first physical GPU 107 and the second physical GPU 107 are on the same graphics board 104, on different graphics boards 104 on the same node 105, or on different graphics boards on different nodes 105.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Baggerman wherein performing the optimization comprises performing AI service migration and, when necessary, changing the sharing type of the computational processing unit. Kim teaches of provisioning necessary resources to containers for AI models to be loaded. AI models are loaded into containers according to required resources, which include GPU and CPU. Similarly, Baggerman teaches of an allocation process that mirrors Kim’s. Moreover, Baggerman admits that the process of reallocating resources to achieve efficient allocation of GPU resources to the virtual machine is optimizing the allocation of GPU resources to the VMs. Therefore, together, Kim and Baggerman teach a method of configuring a sharing type of a computational processing unit and executing an AI service based on the virtual infrastructure using requirements for the AI service and information about the sharing type of the computation processing unit, which performs optimization for the AI service. Additionally, migrating/reassigning the VMs describes a reassignment of a virtual machine 110, 112 from one vGPU profile 108 to a different vGPU profile 108 that will likely result in a more efficient allocation of GPU resources. In other words, GPU resources that are allocated to a virtual machine 110, 112 based on the vGPU profile 108 to which it is reassigned will support the anticipated GPU resource requirements of the virtual machine 110, 112 while minimizing the likelihood that the GPU resources will be wasted. Information describing the reassignment(s) 140 may be presented to a system administrator or other user (e.g., via management console 170), as discussed in Baggerman (Col. 7, lines 51-61).
12. With regard to claim 10, Kim teaches the method of claim 7 and the idea of AI services but fails to explicitly teach wherein performing the optimization comprises, when a utilization rate of the computational processing unit is greater than a first threshold value, redeploying an AI service being executed on the computational processing unit on another computational processing unit.
However, in analogous art, Baggerman further teaches:
wherein performing the optimization comprises, when a utilization rate of the computational processing unit is greater than a first threshold value, redeploying an service being executed on the computational processing unit on another computational processing unit (Col. 20, lines 27-47, As shown in FIG. 4C, each alert is associated with a vGPU gauge 452, as well as in information below the gauge 454 describing the nature of the alert (e.g., usage of GPU resources by a virtual machine 110, 112 that has exceeded a threshold percentage of the GPU resources allocated to the virtual machine 110, 112) and the time at which the alert was activated. Furthermore, information describing an alert may include one or more interactive elements that allow a user to provide an input responsive to the alert. For example, portion 454 of the user interface 405 includes different buttons that allow a user to respond to the alert by selecting an action to reassign the virtual machine 110, 112 to a different vGPU profile 108 automatically (e.g., by the profile reassignment module 130) or manually. In this example, in response to receiving an input indicating a request to manually reassign the virtual machine 110, 112 to a different vGPU profile 108, the drop-down menus in portion 454 may also allow a system administrator or other user to reassign the virtual machine 110, 112 to a different vGPU profile 108 via the management console 170 on a different node 105 or host).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Baggerman wherein performing the optimization comprises, when a utilization rate of the computational processing unit is greater than a first threshold value, redeploying an service being executed on the computational processing unit on another computational processing unit. Kim teaches of provisioning necessary resources to containers for AI models to be loaded. AI models are loaded into containers according to required resources, which include GPU and CPU. Similarly, Baggerman teaches of an allocation process that mirrors Kim’s. Moreover, Baggerman admits that the process of reallocating resources to achieve efficient allocation of GPU resources to the virtual machine is optimizing the allocation of GPU resources to the VMs. Therefore, together, Kim and Baggerman teach a method of configuring a sharing type of a computational processing unit and executing an AI service based on the virtual infrastructure using requirements for the AI service and information about the sharing type of the computation processing unit, which performs optimization for the AI service. Additionally, Baggerman teaches that when the usage of GPU resources by a VM has exceeded a threshold percentage of the GPU resources, the VM can be reassigned. The process of reassignment allows the vGPU profile mechanism to evaluate each virtual machine to determine whether reassignment of a virtual machine to a different vGPU profile is likely to result in a more efficient utilization of GPU resources, as dicussed in Baggerman (Col. 14, lines 59-63).
13. With regard to claim 11, Kim teaches the method of claim 7 and the idea of AI services but fails wherein performing the optimization comprises, when throughput of services simultaneously executed on the computational processing unit is less than a second threshold value, performing migration of the service.
However, in analogous art, Baggerman further teaches:
wherein performing the optimization comprises, when throughput of services simultaneously executed on the computational processing unit is less than a second threshold value, performing migration of the service (Col. 6, lines 27-34, For example, a workload profile 128 for a virtual machine 110, 112 running a database management system may be characterized by a high read/write ratio and usage of a high amount of CPU and RAM. Workload profiles 128 may be described in terms of ranges (e.g., between 200 to 300 MB of memory), thresholds (e.g., less than 10% CPU), ratios or percentages (e.g., 75% of available GPU used), patterns (e.g., I/O patterns), etc; Col. 7, lines 22-42, For example, a profile reassignment rule may specify that a virtual machine 110, 112 that is assigned to a particular vGPU profile 108 should be reassigned to a different vGPU profile 108 if a difference between an amount of GPU resources utilized by the virtual machine 110, 112 and an amount of GPU resources allocated to the virtual machine 110, 112 is at least a threshold amount. In this example, an additional profile reassignment rule may specify that the virtual machine 110, 112 should be reassigned to a particular vGPU profile 108 based on a type of workload being processed by the virtual machine 110, 112. In some embodiments, a profile reassignment rule may specify a vGPU profile 108 to which a virtual machine 110, 112 should be reassigned based on a use case corresponding to one or more types of workloads being processed by the virtual machine 110, 112. For example, a profile reassignment rule may specify that a particular vGPU profile 108 should be assigned to virtual machines 110, 112 having one or more types of workloads that are associated with a use case corresponding to a high end designer (e.g., a graphic designer, a video editor, etc.); Col. 16, lines 10-28, [...] while a virtual machine user who requires a moderate amount of GPU resources may correspond to a “designer” use case, and a virtual machine user who requires the fewest GPU resources may correspond to a “power user” use case. Alternatively, the use case predicted 318 by the use case predictor 138 may describe more a specific category of virtual machine users (e.g., “gamer,” “graphic designer,” etc.). The profile reassignment module 130 may reallocate 320 GPU resources to the virtual machine 110, 112 based on the information describing the classified workload(s) and/or the use case associated with the virtual machine 110, 112. For example, the profile reassignment module 130 may reassign the virtual machine 110, 112 to a different vGPU profile 108 associated with the classified workload(s) and/or the user case associated with the virtual machine 110, 112 and reallocate 320 GPU resources to the virtual machine 110, 112 that correspond to the vGPU profile 108 to which the virtual machine 110, 112 was reassigned.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Baggerman wherein performing the optimization comprises, when throughput of services simultaneously executed on the computational processing unit is less than a second threshold value, performing migration of the service. Kim teaches of provisioning necessary resources to containers for AI models to be loaded. AI models are loaded into containers according to required resources, which include GPU and CPU. Similarly, Baggerman teaches of an allocation process that mirrors Kim’s. Moreover, Baggerman admits that the process of reallocating resources to achieve efficient allocation of GPU resources to the virtual machine is optimizing the allocation of GPU resources to the VMs. Therefore, together, Kim and Baggerman teach a method of configuring a sharing type of a computational processing unit and executing an AI service based on the virtual infrastructure using requirements for the AI service and information about the sharing type of the computation processing unit, which performs optimization for the AI service. Additionally, Baggerman teaches that when the usage of GPU resources is determined to be less than a threshold percentage of the GPU resources, the VM can be reassigned to a GPU that is less resource intensive. The process of reassignment allows the vGPU profile mechanism to evaluate each virtual machine to determine whether reassignment of a virtual machine to a different vGPU profile is likely to result in a more efficient utilization of GPU resources, as dicussed in Baggerman (Col. 14, lines 59-63).
14. Regarding claim 12, it is rejected under the same reasoning as claim 1 above. Therefore, it is rejected under the same rationale.
15. Regarding claim 13, it is rejected under the same reasoning as claim 2 above. Therefore, it is rejected under the same rationale.
16. Regarding claim 14, it is rejected under the same reasoning as claim 3 above. Therefore, it is rejected under the same rationale.
17. Regarding claim 16, it is rejected under the same reasoning as claim 5 above. Therefore, it is rejected under the same rationale.
18. Regarding claim 17, it is rejected under the same reasoning as claim 6 above. Therefore, it is rejected under the same rationale.
19. With regard to claim 20, Kim teaches:
A method for executing Artificial Intelligence (AI) services based on virtual infrastructures using a cluster of computing servers, comprising:
executing AI services based on virtual infrastructures using requirements for the AI services and information about a combination of AI models to be simultaneously executed ([0020] The method includes identifying a container in which an artificial intelligence model is to be loaded, based on attributes of a plurality of containers in which a library for artificial intelligence models is loaded and attributes of a plurality of artificial intelligence models registered in a model store, loading the artificial intelligence model in the container based on the library loaded in the containers, based on a request for an artificial intelligence service being received from a client, obtaining a function corresponding to the requested artificial intelligence service from a database including a plurality of functions, identifying a container in which an artificial intelligence model corresponding to the artificial intelligence service is loaded among the plurality of containers in which artificial intelligence models are loaded, and obtaining data for the request from the artificial intelligence model loaded in the identified container by executing, in the container, the obtained function based on the library loaded in the container, and transmitting the obtained data to the client.); and
performing optimization for the AI services, wherein:
the information about the combination of the AI models to be simultaneously executed includes information about throughput of each of the AI models when the multiple AI models are simultaneously executed ([0071] The trained artificial intelligence model file may be a file including a hidden layer of the trained artificial intelligence model and labeling information output as a result value by the artificial intelligence model, the model prediction code may indicate an assembly of codes (or program) necessary for obtaining a result value for an input value of the trained model, and the operation information on the artificial intelligence model may include resource information such as a CPU, a GPU, a memory, and the like minimally necessary to use the artificial intelligence model.).
20. Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. and Baggerman et al. US 10176550 B1, as applied in claim 1, in further view of Hsiao et al. US 20230092808 A1.
21. With regard to claim 4, Kim further teaches:
wherein the requirements for the AI service include information about a resource of a computational processing unit, information about a model of the AI service, information about a type of a virtual infrastructure, and whether isolated execution is required ([0071] Meanwhile, the artificial intelligence model client 200 may register a trained artificial intelligence model file, a model prediction code, and operation information on the artificial intelligence model in the model store (not illustrated). The trained artificial intelligence model file may be a file including a hidden layer of the trained artificial intelligence model and labeling information output as a result value by the artificial intelligence model, the model prediction code may indicate an assembly of codes (or program) necessary for obtaining a result value for an input value of the trained model, and the operation information on the artificial intelligence model may include resource information such as a CPU, a GPU, a memory, and the like minimally necessary to use the artificial intelligence model; [0104] The function corresponding to the artificial intelligence service is generated by the artificial intelligence service client 300 and the artificial intelligence service client 300 selects the trained artificial intelligence model registered in the model store and generates an artificial intelligence service function including the model prediction code of the selected artificial intelligence model, and accordingly, the function corresponding to the artificial intelligence service may include information on the artificial intelligence model; [0105] The processor 130 may identify the container on which the artificial intelligence model corresponding to the artificial intelligence service is loaded, among the plurality of containers based on the information on the artificial intelligence model included in the obtained function.).
Although Kim teaches operation information on the artificial intelligence model that includes requirements for the AI service regarding a resource of a computational processing unit, information about a model of the AI service, information about a type of a virtual infrastructure, Kim fails to explicitly teach that the information also includes whether isolated execution is required.
However, in analogous art, Hsiao teaches:
and whether isolated execution is required ([0002] For artificial intelligence (AI) field, it is important to develop a protection scheme for protecting AI model being attacked. Conventionally, the AI model maybe injected to a kernel (e.g. a Linux kernel) of an operating system (OS; e.g. an Android system) for inference, and may be driven in the kernel. However, since all of the AI models of different applications (APPs) running on the OS are driven in the kernel, the AI models will be exposed when injected to the kernel or driven in the kernel. A crypted AI model may be decrypted and executed in an isolated execution environment (e.g. a trusted execution environment, TEE); [0029] FIG. 6 is a diagram illustrating relationship between a system 60 for model protection and an isolated execution environment for model decryption according to an embodiment of the present invention. The system 60 may include a processor (e.g. a CPU 650 that acts as the processor 12 shown in FIG. 1) , a transmission interface 630, and a DMA circuit (e.g. an APU 640) . The processor may be arranged to execute software modules, including a guest VM 600, a primary VM 610, and a hypervisor 620, wherein a command hub 621 is a software module integrated in the hypervisor 620. An Android system with a Linux kernel may run on the guest VM 600, and the guest VM 600 may include a crypted model (e.g. a crypted AI model) 602 in the Android system, wherein at least one client 601 in the Linux kernel may send a model protection command MPC to the primary VM 610 through the command hub 621. In this embodiment, the crypted model 602 may be injected from the Android system to the Linux kernel for inference. The processor may be further arranged to execute an isolated execution environment, such as a trusted execution environment (TEE) 604, and the TEE 604 may be arranged to perform decryption on the crypted model 602 to generate a protected model 603.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim with the teachings of Hsiao of whether isolated execution is required. Similarly to Kim, Hsiao teaches of a system that can execute a guest VM, primary VM, and hypervisor. The guest VM may include a model (i.e. AI model) and configure a protection setting for a protected model that is derived from the model (Abstract; [0004]; [0022]). However, since all of the AI models of different applications (APPs) running on the OS are driven in the kernel, the AI models will be exposed when injected to the kernel or driven in the kernel. To combat this, the processer may be arranged to execute an isolated execution environment, such as a trusted execution environment (TEE) 604, and the TEE 604 may be arranged to perform decryption on the crypted model 602 to generate a protected model 603, as discussed in Hsiao ([0002]; [0029]).
22. Regarding claim 15, it is rejected under the same reasoning as claim 4 above. Therefore, it is rejected under the same rationale.
23. Claims 7-8 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. and Baggerman et al. US 10176550 B1, as applied in claim 1, in further view of Moon US 20220318655 A1.
24. With regard to claim 7, Kim and Baggerman teach the method of claim 1 but fail to explicitly teach wherein executing the AI service comprises inserting the AI service into a ready queue when a resource of a computational processing unit satisfying the requirements for the AI service is not present.
However, in analogous art, Moon teaches:
wherein executing the AI service comprises inserting the AI service into a ready queue when a resource of a computational processing unit satisfying the requirements for the AI service is not present ([0055] In the case where the artificial intelligence service request is received from the user device when all GPU resources are not available, the master server 220 may store the artificial intelligence service request in a priority queue, and may process the GPU resources in the order of priority when the GPU resources are available.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim and Baggerman with the teachings of Moon wherein executing the AI service comprises inserting the AI service into a ready queue when a resource of a computational processing unit satisfying the requirements for the AI service is not present. Similarly to Kim and Baggerman, Moon teaches of a method of providing an artificial intelligence service. Moreover, Moon teaches that when GPU resources are not available, the master server can store the AI request in a priority queue that processes GPU resources in the order of priority when the GPU resources are available ([0055]). This ensures that tasks are eventually completed and that the worker server 320 may perform the artificial intelligence processing according to the request of the user device 350. The project created according to the request of the user device 350 may be performed by being allocated the GPU of at least one worker server 320, as discussed in Moon ([0059]).
25. With regard to claim 8, Moon further teaches:
wherein performing the optimization comprises determining whether to perform optimization based on a utilization rate of the computational processing unit and whether an AI service waiting in the ready queue is present ([0052] Therefore, it is necessary to efficiently utilize GPU resources by monitoring active GPUs that are operating and inactive GPUs that are not operating; [0055] In the case where the artificial intelligence service request is received from the user device when all GPU resources are not available, the master server 220 may store the artificial intelligence service request in a priority queue, and may process the GPU resources in the order of priority when the GPU resources are available; Examiner’s Note: Efficiently utilizing GPU resources is optimizing. GPU resource availability is analogous with utilization rate of the computational processing unit, and the AI service is stored on the priority queue.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Kim and Baggerman with the teachings of Moon wherein performing the optimization comprises determining whether to perform optimization based on a utilization rate of the computational processing unit and whether an AI service waiting in the ready queue is present. Similarly to Kim and Baggerman, Moon teaches of a method of providing an artificial intelligence service. Moreover, Moon teaches that when GPU resources are not available, the master server can store the AI request in a priority queue that processes GPU resources in the order of priority when the GPU resources are available ([0055]). This ensures that tasks are eventually completed and that the worker server 320 may perform the artificial intelligence processing according to the request of the user device 350. The project created according to the request of the user device 350 may be performed by being allocated the GPU of at least one worker server 320. Moreover, it is necessary to efficiently utilize GPU resources by monitoring active and inactive GPUs, as discussed in Moon ([0052]; [0059]).
26. Regarding claim 18, it is rejected under the same reasoning as claim 7 above. Therefore, it is rejected under the same rationale.
27. Regarding claim 19, it is rejected under the same reasoning as claim 8 above. Therefore, it is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AN-AN N NGUYEN whose telephone number is (571)272-6147. The examiner can normally be reached Monday-Friday 8:00-5:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AIMEE LI can be reached at (571) 272-4169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AN-AN NGOC NGUYEN/Examiner, Art Unit 2195
/Aimee Li/Supervisory Patent Examiner, Art Unit 2195
Prosecution Insights