DETAILED ACTION
This office action is in response to the application filed on March 8, 2024.
Claims 1-15 are pending and have been examined. Claims 1-15 are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicants’ claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. The present application claims foreign priority based on Korean Patent Application No. 10-2021-0121180 filed September 10, 2021. The examiner notes that a certified copy (in Korean) of the above-noted application was received on March 8, 2024.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statements filed March 8, 2024, which complies with 37 CFR 1.97. As such, the information disclosure statement has been placed in the application file and the information referred to therein has been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
Step 1: The claim does not fall within one of the four statutory categories of invention (process, machine, manufacture, or composition of matter) – see MPEP 2106.03, or,
Step 2: The claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis – see MPEP 2106.04:
Step 2A, Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? - see MPEP 2106.05
MPEP 2106.04(a)(2)(I) states: “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
MPEP 2106.04(a)(2)(III) states: “Accordingly, the “mental processes” abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgements, and opinions.
Further, the MPEP states: “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g. pen and paper or a slide run) to perform the claim limitation.
Using the two-step inquiry, it is clear that Claims 1-15 are each directed to non-statutory subject matter as shown below:
With respect to Claim 1:
Step 1: Claim 1 is directed to a method, also known as a process, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: A judicial exception is recited in the claim as it recites mental processes, which are abstract ideas:
“A method of providing, by a device, an artificial intelligence service, the method comprising:
identifying neural network requirements related to a purpose of the artificial intelligence service and an execution environment of the device;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
“selecting at least one neural network model satisfying the neural network requirements, based on neural network model information about a plurality of preregistered neural network models;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“obtaining a neural network model for providing the artificial intelligence service, by using the selected at least one neural network model;” (Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and providing the artificial intelligence service through the obtained neural network model.” (Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
With respect to Claim 2:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“obtaining the neural network model information about the plurality of preregistered neural network models stored in at least one memory in the device or in an external server;” (Obtaining neural network model information is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and registering the plurality of preregistered neural network models by storing the neural network model information in the at least one memory” (Regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining neural network model information is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Registering the plurality of preregistered neural network models by storing the neural network model information in at least one memory is regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claim 3:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in the claim as it recites a mental process, which is an abstract idea:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on a recognition target object to be recognized by using the obtained neural network model, at a position and time at which the device provides the artificial intelligence service.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 4:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in the claim as it recites a mental process, which is an abstract idea:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on at least one of execution environment information about the device, information about a recognition target object to be recognized according to the purpose of the artificial intelligence service, and hardware resource feature information about the device providing the artificial intelligence service.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 5:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in the claim as it recites a mental process, which is an abstract idea:
“wherein the selecting the at least one neural network model comprises selecting the at least one neural network model based on performance information comprising information about recognition accuracy and latency of each of the plurality of preregistered neural network models.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 6:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“downloading the plurality of preregistered neural network models from an external server or an external database and storing the plurality of preregistered neural network models in at least one memory of the device.” (Regarded as a generic computer function of receiving and storing data. Receiving and storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Downloading the plurality of preregistered neural network models from an external server or external database and storing the plurality of preregistered neural network models in at least one memory of the device is regarded as a generic computer function of receiving and storing data. Receiving and storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claim 7:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“wherein the providing the artificial intelligence service through the obtained neural network model comprises: obtaining image data by photographing a surrounding environment of the device;” (Obtaining image data is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and recognizing an object corresponding to the purpose of the artificial intelligence service, by applying the image data to the obtained neural network model.” (Applying image data to an obtained neural network model in order to recognize an object corresponding to the purpose of an artificial intelligence service only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining image data is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Applying image data to an obtained neural network model in order to recognize an object corresponding to the purpose of an artificial intelligence service only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
With respect to Claim 8:
Step 1: Claim 8 is directed to an apparatus, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: A judicial exception is recited in the claim as it recites mental processes, which are abstract ideas:
“A device for providing an artificial intelligence service, the device comprising:
at least one memory storing at least one instruction; and
at least one processor configured to execute the at least one instruction to:
identify neural network requirements related to a purpose of the artificial intelligence service and an execution environment of the device;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
“select, based on neural network model information about a plurality of preregistered neural network models, at least one neural network model satisfying the neural network requirements among the plurality of preregistered neural network models;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“obtain a neural network model for providing the artificial intelligence service, by using the selected at least one neural network model;” (Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and provide the artificial intelligence service through the obtained neural network model” (Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
With respect to Claim 9:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“further comprising a communication interface, wherein the at least one processor is further configured to execute the at least one instruction to:
obtain the neural network model information from an external server by using the communication interface or obtain the neural network model information from the plurality of preregistered neural network models stored in a neural network model storage in the device;“ (Obtaining neural network model information is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and register the plurality of preregistered neural network models by storing the obtained neural network model information in the at least one memory” (Regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining neural network model information is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Registering the plurality of preregistered neural network models by storing the neural network model information in at least one memory is regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claim 10:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“wherein the at least one processor is further configured to execute the at least one instruction to identify the neural network requirements based on at least one of execution environment information about the device, information about a recognition target object to be recognized according to the purpose of the artificial intelligence service, and hardware resource feature information about the device providing the artificial intelligence service.” (At least one processor configured to execute at least one instruction to identify neural network requirements based on at least one execution of environment information about the device, information about a recognition target object to be recognized according to the purpose of the artificial intelligence service, and hardware resource feature information about the device providing the artificial intelligence service generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. At least one processor configured to execute at least one instruction to identify neural network requirements based on at least one execution of environment information about the device, information about a recognition target object to be recognized according to the purpose of the artificial intelligence service, and hardware resource feature information about the device providing the artificial intelligence service generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claim 11:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“wherein the at least one processor is further configured to execute the at least one instruction to select the at least one neural network model based on performance information comprising information about recognition accuracy and latency of each of the plurality of preregistered neural network models.” (At least one processor further configured to execute at least one instruction to select at least one neural network model based on performance information comprising information about recognition accuracy and latency of each of the plurality of preregistered neural network models generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. At least one processor further configured to execute at least one instruction to select at least one neural network model based on performance information comprising information about recognition accuracy and latency of each of the plurality of preregistered neural network models generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claim 12:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“further comprising a communication interface, wherein the at least one processor is further configured to execute the at least one instruction to:
control the communication interface to download the plurality of preregistered neural network models from an external server or an external database, (Regarded as a generic computer function of downloading (receiving) data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
and store the plurality of preregistered neural network models in the at least one memory.” (Regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Downloading a plurality of preregistered neural network models from an external server or external database is regarded as a generic computer function of downloading (receiving) data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Storing a plurality of preregistered neural network models in at least one memory Regarded as a generic computer function of storing data. Storing data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claim 13:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8. An additional judicial exception is recited in the claim as it recites mental processes, which are abstract ideas:
“wherein the at least one processor is further configured to execute the at least one instruction to:
select a plurality of neural network models satisfying the neural network requirements, (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
and construct the obtained neural network model by combining the selected plurality of neural network models in any one of a sequential structure, a parallel structure, or a hybrid structure that is a combination of the sequential structure and the parallel structure.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 14:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 8.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“The device of claim 8, further comprising: a camera, wherein the at least one processor is further configured to execute the at least one instruction to:
obtain image data by photographing a surrounding environment thereof by using the camera, (Obtaining image data is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
and recognize an object corresponding to the purpose of the artificial intelligence service, by
applying the image data to the obtained neural network model.” (Applying image data to an obtained neural network model in order to recognize an object corresponding to the purpose of an artificial intelligence service only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining image data is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Applying image data to an obtained neural network model in order to recognize an object corresponding to the purpose of an artificial intelligence service only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
With respect to Claim 15:
Step 1: Claim 15 is directed to an apparatus, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: A judicial exception is recited in the claim as it recites mental processes, which are abstract ideas:
“A computer program product comprising a non-transitory computer-readable storage medium, wherein the computer-readable storage medium comprises instructions for a method of providing, by a device, an artificial intelligence service, the method comprising:
identifying neural network requirements related to a purpose of the artificial intelligence service and an execution environment of the device;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
“selecting at least one neural network model satisfying the neural network requirements, based on neural network model information about a plurality of preregistered neural network models;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“obtaining a neural network model for providing the artificial intelligence service, by using the selected at least one neural network model;” (Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“and providing the artificial intelligence service through the obtained neural network model.” (Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Obtaining a neural network model is regarded as a generic computer function of receiving data. Receiving data is considered insignificant extra-solution activity – see MPEP 2106.05(g). Providing an artificial intelligence service through an obtained neural network model is only indicating the particular function of providing the artificial intelligence service is performed by and offered through a computer. This only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e. changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-12, and 14-15 are rejected under 35 U.S.C. 102 as being unpatentable over Shen et al., (US Patent Application Number US20230077258A1 filed on August 10, 2021, hereinafter “Shen”).
With respect to Claim 1:
Shen teaches:
“A method of providing, by a device, an artificial intelligence service, the method comprising:
identifying neural network requirements related to a purpose of the artificial intelligence service and an execution environment of the device;” (Paragraph 0368 recites a device comprising one or more deployment pipelines depending on information desired from data generated by the device, such as an MM machine requiring detection of anomaly or an MRI machine requiring image enhancement (identifying neural network requirements related to a purpose of the artificial intelligence service). Paragraph 0365 further recites output or pre-trained models part of the deployment pipeline can include neural networks.)
“selecting at least one neural network model satisfying the neural network requirements, based on neural network model information about a plurality of preregistered neural network models;” (Paragraph 0370 recites an application including a processing task involving a machine learning model, where the user may desire to select a machine learning model from a model registry containing pre-trained models (plurality of preregistered neural network models) or implement their own machine learning model (selecting at least one neural network model to satisfy neural network requirements based on neural network model information). Paragraph 0365 further recites output or pre-trained models part of the deployment pipeline can include neural networks.)
“obtaining a neural network model for providing the artificial intelligence service, by using the selected at least one neural network model;” (Paragraph 0376 recites a machine learning model located from a model registry, where is the model is not already in a cache, a validation step occurs to ensure the appropriate machine learning model is loaded into a cache.)
“and providing the artificial intelligence service through the obtained neural network model.” (Paragraph 0375 recites artificial intelligence (AI) services may leverage an AI system to execute machine learning models, such as neural networks, for a specific inferencing task.)
With respect to Claim 2:
Shen teaches:
“obtaining the neural network model information about the plurality of preregistered neural network models stored in at least one memory in the device or in an external server;” (Paragraph 0349 recites the machine learning model registry may be backed by object storage, such as cloud storage (stored in at least one memory in the device or in an external server), where the model registry may store pre-trained machine learning models that are then selected to perform a specific task (obtaining the neural network model information about the plurality of preregistered neural network models).)
“and registering the plurality of preregistered neural network models by storing the neural network model information in the at least one memory” (Paragraph 0351 recites an existence of machine learning models, where once they are partially or fully trained at one location, they can then be added (registered) to a model registry containing multiple machine learning models trained to a perform a variety of different inference tasks (plurality of preregistered neural network models). Paragraph 0376 further recites a copy of a model may be saved to a cache (storing the neural network model information in the at least one memory).)
With respect to Claim 4:
Shen teaches:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on at least one of execution environment information about the device, information about a recognition target object to be recognized according to the purpose of the artificial intelligence service, and hardware resource feature information about the device providing the artificial intelligence service.” (Paragraph 0360 recites hardware including GPUs and CPUs in at least one embodiment, where the software may be optimized for GPU processing with respect to machine learning (neural network) tasks (execution environment information about the device). Paragraph 0369 recites applications can be used for performing processing tasks on imaging data or other data from devices, where each application may be responsible for different inferencing tasks like image enhancement, segmentation, feature detection, etc. For example, there may be more than one deployment pipeline depending on information desired from data generated by the device, where detection of anomalies may be required by an MM machine, akin to information about a recognition target object being recognized according to the purpose of the artificial intelligence service. Paragraph 0373 further recites a scheduler may allocate and distribute resources between different artificial intelligence applications in view of requirements and availability of the system (hardware resource feature information about the device providing the artificial intelligence service).)
With respect to Claim 5:
Shen teaches:
“wherein the selecting the at least one neural network model comprises selecting the at least one neural network model based on performance information comprising information about recognition accuracy and latency of each of the plurality of preregistered neural network models.” (Paragraph 0370 recites a user may select a machine learning model from a model registry for inclusion in an application for performing a processing task. Paragraph 0371 recites the system may contain a user interface that the user can use for selecting models that have varying performance values (selecting at least one neural network model based on performance information). Paragraph 0047 recites a pruning process that can take into account a predicted performance value or impact of a neural network model, such as overall performance of the network. Paragraphs 0050 and 0065 recite hardware-aware latency pruning (HALP) that can be used to formulate pruning as a resource allocation optimization problem, where a final trained model is created that is pruned to produce maximum accuracy within a target performance budget. Paragraph 0063 also recites neurons can be grouped and an importance and latency reduction can be calculated for each group, where the pruning algorithm can then be executed to select portions of a neural network to be retained for a current latency target or budget (based on performance information comprising information and recognition accuracy and latency of preregistered neural network models).
With respect to Claim 6:
Shen teaches:
“downloading the plurality of preregistered neural network models from an external server or an external database and storing the plurality of preregistered neural network models in at least one memory of the device.” (Paragraph 0086 recites the involvement of a data center which includes tools, services, software, or other resources to train one or more machine learning models. Paragraph 0092 further clarifies that a computer system used to perform marching learning model training could be a server system (downloading a plurality of preregistered neural network models from an external server). Paragraph 0093 recites the computer system also containing a process that may include single or multiple levels of cache memory (storing the plurality of preregistered neural network models in at least one memory of the device).)
With respect to Claim 7:
Shen teaches:
“wherein the providing the artificial intelligence service through the obtained neural network model comprises: obtaining image data by photographing a surrounding environment of the device;” ( Paragraph 0368 recites a deployment system may execute deployment pipelines that include any number of applications that may be applied to imaging data generated by an imaging device (obtaining image data). Paragraph 0103 further recites one embodiment of the invention may include a component such as a camera (by photographing a surrounding environment of the device).)
“and recognizing an object corresponding to the purpose of the artificial intelligence service, by applying the image data to the obtained neural network model.” (Paragraph 0375 recites artificial intelligence (AI) services may leverage the AI system to execute machine learning models, like neural networks, for object detection, feature detection, and other inference tasks (recognizing an object corresponding to the purpose of the artificial intelligence service). Paragraph 0368 further clarifies a deployment system may execute deployment pipelines that include any number of applications that may be applied to imaging data (by applying the image data to the obtained neural network model).)
With respect to Claim 8:
Due to containing similar claim language as Claim 1, refer to the 102 rejection for Claim 1 above.
With respect to Claim 9:
Due to containing similar claim language as Claim 2, refer to the 102 rejection for Claim 2 above.
With respect to Claim 10:
Due to containing similar claim language as Claim 4, refer to the 102 rejection for Claim 4 above.
With respect to Claim 11:
Due to containing similar claim language as Claim 5, refer to the 102 rejection for Claim 5 above.
With respect to Claim 12:
Due to containing similar claim language as Claim 6, refer to the 102 rejection for Claim 6 above.
With respect to Claim 14:
Shen teaches:
“The device of claim 8, further comprising: a camera, wherein the at least one processor is further configured to execute the at least one instruction to:
obtain image data by photographing a surrounding environment thereof by using the camera, (Paragraph 0368 recites a deployment system may execute deployment pipelines that include any number of applications that may be applied to imaging data generated by an imaging device (obtaining image data). Paragraph 0103 further recites one embodiment of the invention may include a component such as a camera (by photographing a surrounding environment of the device using the camera).)
and recognize an object corresponding to the purpose of the artificial intelligence service, by
applying the image data to the obtained neural network model.” (Paragraph 0375 recites artificial intelligence (AI) services may leverage the AI system to execute machine learning models, like neural networks, for object detection, feature detection, and other inference tasks (recognizing an object corresponding to the purpose of the artificial intelligence service). Paragraph 0368 further clarifies a deployment system may execute deployment pipelines that include any number of applications that may be applied to imaging data (by applying the image data to the obtained neural network model).)
With respect to Claim 15:
Due to containing similar claim language as Claim 1, refer to the 102 rejection for Claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Shen et al., (US Patent Application Number US20230077258A1 filed on August 10, 2021, hereinafter “Shen”), in view of Busch et al., (US Patent Application Number US20200242471A1 filed on January 30, 2020, hereinafter “Busch”).
With respect to Claim 3:
Shen teaches:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on a recognition target object to be recognized by using the obtained neural network model, at a position and time at which the device provides the artificial intelligence service.” (Paragraph 0368 recites a device comprising one or more deployment pipelines depending on information desired from data generated by the device, such as an MM machine requiring detection of anomaly or an MRI machine requiring image enhancement (identifying neural network requirements related to a purpose of the artificial intelligence service). Paragraph 0365 further recites output or pre-trained models part of the deployment pipeline can include neural networks.)
Shen does not appear to explicitly disclose:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on a recognition target object to be recognized by using the obtained neural network model, at a position and time at which the device provides the artificial intelligence service.”
However, Busch teaches:
“wherein the identifying the neural network requirements comprises identifying the neural network requirements based on a recognition target object to be recognized by using the obtained neural network model, at a position and time at which the device provides the artificial intelligence service.” (Paragraph 0069 recites an observer device that periodically records an image from an imaging sensor and then passes the image through a deep neural network to obtain a result, wherein the device then makes a decision on whether the device should cause a transition in a state machine model of an observed machine. Paragraph 0343 further recites the decision is based on the object in view of the camera, as determined by the current approximate camera elevation and image recognition. Paragraph 0359 also recites how the observer device could be different autonomous cars being an observer for a location-based state machine at different times (recognition target object to be recognized by using the obtained neural network model at a position and time at which the device provides the artificial intelligence service).)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) to combine the teachings of Shen with the teachings of Busch, which are both in the same field of invention. A PHOSITA would have been motivated to apply Busch’s techniques consisting of adapting neural network processing according to a recognition target and object and operational context of the device (such as the time and location of operation) into Shen’s artificial intelligence service system in order to improve image recognition accuracy and allow the device to choose neural network models based on a real-world operating context in which the artificial intelligence service is being provided.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Shen et al., (US Patent Application Number US20230077258A1 filed on August 10, 2021, hereinafter “Shen”), in view of Dong et al., (WIPO Application Number WO2021051268A1 filed on September 17, 2019, hereinafter “Dong”).
With respect to Claim 13:
Shen teaches:
“wherein the at least one processor is further configured to execute the at least one instruction to: select a plurality of neural network models satisfying the neural network requirements, (Paragraph 0370 recites an application including a processing task involving a machine learning model, where the user may desire to select a machine learning model from a model registry containing pre-trained models (plurality of preregistered neural network models) or implement their own machine learning model (selecting at least one neural network model to satisfy neural network requirements based on neural network model information). Paragraph 0365 further recites output or pre-trained models part of the deployment pipeline can include neural networks.)
and construct the obtained neural network model by combining the selected plurality of neural network models in any one of a sequential structure, a parallel structure, or a hybrid structure that is a combination of the sequential structure and the parallel structure.”
Shen does not appear to explicitly disclose:
“wherein the at least one processor is further configured to execute the at least one instruction to: select a plurality of neural network models satisfying the neural network requirements,
and construct the obtained neural network model by combining the selected plurality of neural network models in any one of a sequential structure, a parallel structure, or a hybrid structure that is a combination of the sequential structure and the parallel structure.”
However, Dong teaches:
“wherein the at least one processor is further configured to execute the at least one instruction to: select a plurality of neural network models satisfying the neural network requirements,
and construct the obtained neural network model by combining the selected plurality of neural network models in any one of a sequential structure, a parallel structure, or a hybrid structure that is a combination of the sequential structure and the parallel structure.” (Page 6, Paragraph 4 recites a neural network model consisting of a first preset neural network model and two second preset neural network models, where the first and second preset models are connected (combined) in series and the two second preset neural network models are connected in a parallel structure.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) to combine the teachings of Shen with the teachings of Dong, which are both in the same field of invention. A PHOSITA would have been motivated to combine Shen’s method of identifying neural network requirements and selecting suitable models based off those requirements in order to provide an artificial intelligence service with Dong’s method of constructing a neural network model from a plurality of neural network components using parallel structures. Applying the multimodal neural network structure from Dong onto the models used in Shen would improve computational efficiency and recognition accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vibha Bhat whose telephone number is (571)-272-7091. The examiner can normally be reached on Monday – Thursday from 8:00 AM to 5:00 PM EST and every other Friday from 8:00 AM to 4:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. See MPEP § 713.01. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes, can be reached at telephone number (571)-270-1006. The fax phone number for the organization where this application or proceeding is assigned is (571)-273-8300. Information regarding the status of an application may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or (572)-272-1000.
/Vibha Bhat/Examiner
Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights