DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the Application filed on 04/26/2024. Claims 1-21 are pending in the case. Claims 1, 10, and 16 are independent claims.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-21 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter. During examination, the claims must be interpreted as broadly as their terms reasonably allow. In re American Academy of Science Tech Center, 367 F.3d 1359, 1369, 70 U.S.P.Q.2d 1827, 1834 (Fed. Cir. 2004). Independent claim 16 recites a “computer-readable media,” which is not comprehensively defined by the specification. The broadest reasonable interpretation of a claim drawn to a computer readable medium covers forms of transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. Transitory propagating signals are non-statutory subject matter. In re Nuijten, 500 F.3d 1346, 1356-57, 84 U.S.P.Q.2d 1495, 1502 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). See also Subject Matter Eligibility of Computer Readable Media, 1351 Off. Gaz. Pat. Office 212 (Feb. 23, 2010). Examiner suggests adding the word “non-transitory.”
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1, 2, 9-11, 16, and 17 are rejected under 35 U.S.C. § 103 as being unpatentable over Liu et al. (U.S. Pat. App. Pub. No. 2023/0289616, hereinafter Liu) in view of Simon (Simon, Julien. 2019. “Amazon SageMaker Debugger – Debug Your Machine Learning Models | Amazon Web Services.” Amazon Web Services. December 3, 2019. https://aws.amazon.com/blogs/aws/amazon-sagemaker-debugger-debug-your-machine-learning-models/, hereinafter Simon 2019).
As to independent claim 1, Liu teaches:
A system for inserting instrumentation hooks into a machine learning model, the system comprising:
a computing device configured to (Figures 2-4. Paragraphs 38 and 42-45):
receive an indication of a machine learning model (Paragraph 45, "At block 205 (Profiling), the processor may perform a model profiling execution of the machine learning model." Figure 2, block 205);
identify one or more aspects of the machine learning model, wherein the one or more aspects of the machine learning model include one or more of: a tensor; a layer; or a node (Paragraph 23, "tensors may refer to model data related to states of a model such as model parameters, gradients, and/or optimizer states, etc., that are related by the model structure definition." Paragraph 26. Paragraph 53, "the gathered or obtained information or data in a profiling phase may include a relationship between the tensors and their corresponding module(s) in the model");…
based on the received indication, alter the machine learning model by inserting at least one instrumentation hook into the machine learning model, the instrumentation hook being configured to collect data regarding the at least one aspect (Paragraph 27, "a hook may be registered, installed, arranged, or otherwise associated with or on a tensor or a module of the model that contains the tensor." Figure 3, blocks 320 and 330).
Liu uses hooks to obtain information about specified tensors and modules (see Liu at paragraphs 26, 27, and 47-55), but does not appear to expressly teach receive an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model.
Simon 2019 teaches receive an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model (Page 2, "in order to select the tensor collections you want to save, the frequency at which you want to save them, and whether you want to save the values themselves or a reduction (mean, average,etc.)." Page 4, "A SessionHook object attached to the TensorFlow session, putting everything together and saving requiredtensors during training").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Simon 2019 to explore the evolution of tensors over time, confirm hypotheses and fix the root cause of issues (see Simon 2019 at page 2).
As to dependent claim 2, Liu further teaches the instrumentation hook includes a condition and wherein the instrumentation hook is configured to (Paragraph 27, "a “hook” is a term of art and may refer to a function, operation, or algorithm that is executed or triggered when e.g., a condition is met"): collect data regarding the at least one aspect based on a determination that the condition has been met (Paragraph 27, "a hook may include a handler (a function, operation, or algorithm in the hook), which may be used to perform a desired or predetermined action, function, or operation (e.g., monitoring and/or recording the status, execution sequence/timestamp, execution phase, etc. of a tensor and/or its module, etc.)").
As to dependent claim 9, Liu further teaches aspects of the machine learning model selected based on the indication of the machine learning model (Paragraph 5, "performing a model profiling execution before a model normal execution, allocating or assigning tensors of the model into a plurality of chunks based on profiling results from the model profiling execution." Paragraph 103, "determine a module of the model for the tensors"); and one or more performance-based heuristics associated with each aspect of the one or more aspects of the machine learning model (Paragraph 50, "the gathered or obtained information or data in a profiling phase may include an amount of the GPU memory usage (e.g., increases, etc.) during the forward operation phase and/or in the backward operation phase")
Liu does not appear to expressly teach the indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model is based on one or more of: user input identifying second one or more aspects of the machine learning model.
Simon 2019 teaches the indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model is based on one or more of: user input identifying second one or more aspects of the machine learning model (Page 2, "you can decide which ones you want to save during training").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Simon 2019 to explore the evolution of tensors over time, confirm hypotheses and fix the root cause of issues (see Simon 2019 at page 2).
As to independent claim 10, Liu teaches:
A method comprising:
receiving an indication of a machine learning model (Paragraph 45, "At block 205 (Profiling), the processor may perform a model profiling execution of the machine learning model." Figure 2, block 205);
identifying one or more aspects of the machine learning model (Paragraph 23, "tensors may refer to model data related to states of a model such as model parameters, gradients, and/or optimizer states, etc., that are related by the model structure definition." Paragraph 26. Paragraph 53, "the gathered or obtained information or data in a profiling phase may include a relationship between the tensors and their corresponding module(s) in the model");…
aspects of the machine learning model selected based on the indication of the machine learning mode (Paragraph 5, "performing a model profiling execution before a model normal execution, allocating or assigning tensors of the model into a plurality of chunks based on profiling results from the model profiling execution." Paragraph 103, "determine a module of the model for the tensors")l; or
one or more performance-based heuristics associated with each aspect of the one or more aspects of the machine learning model (Paragraph 50, "the gathered or obtained information or data in a profiling phase may include an amount of the GPU memory usage (e.g., increases, etc.) during the forward operation phase and/or in the backward operation phase");
based on the received indication, inserting at least one instrumentation hook into the machine learning model (Paragraph 27, "a hook may be registered, installed, arranged, or otherwise associated with or on a tensor or a module of the model that contains the tensor." Figure 3, blocks 320 and 330); and
collecting data via the instrumentation hook (Paragraph 73, "The handler of the post-backward hook, when being triggered or executed (e.g., immediately after the backward operation), may (1) record or determine the execution status of a tensor (e.g., whether a tensor is executed or not), and/or (2) collect or obtain the generated gradients in the backward operation phase").
Liu uses hooks to obtain information about specified tensors and modules (see Liu at paragraphs 26, 27, and 47-55), but does not appear to expressly teach receiving an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model, the indication that the instrumentation hook is to be used being based on one or more of: user input identifying second one or more aspects of the machine learning model.
Simon 2019 teaches receiving an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model, the indication that the instrumentation hook is to be used being based on one or more of (Page 2, "in order to select the tensor collections you want to save, the frequency at which you want to save them, and whether you want to save the values themselves or a reduction (mean, average,etc.)." Page 4, "A SessionHook object attached to the TensorFlow session, putting everything together and saving requiredtensors during training"): user input identifying second one or more aspects of the machine learning model (Page 2, "you can decide which ones you want to save during training").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Simon 2019 to explore the evolution of tensors over time, confirm hypotheses and fix the root cause of issues (see Simon 2019 at page 2).
As to dependent claim 11, Liu further teaches collecting data via the at least one instrumentation hook further comprises: determining whether a condition associated with the instrumentation hook has been met (Paragraph 27, "a “hook” is a term of art and may refer to a function, operation, or algorithm that is executed or triggered when e.g., a condition is met"); and based on the determining, collecting the data regarding the at least one aspect via the at least one instrumentation hook (Paragraph 27, "a hook may include a handler (a function, operation, or algorithm in the hook), which may be used to perform a desired or predetermined action, function, or operation (e.g., monitoring and/or recording the status, execution sequence/timestamp, execution phase, etc. of a tensor and/or its module, etc.)").
As to independent claim 16, Liu teaches:
One or more instances of computer-readable media collectively having contents configured to cause a computing device to perform a method for inserting instrumentation hooks into a machine learning model, the method comprising (Paragraph 7, "a non-transitory computer-readable medium having computer-executable instructions stored thereon is provided. The instructions, upon execution, cause one or more processors to perform operations"):
receiving an indication of a machine learning model (Paragraph 45, "At block 205 (Profiling), the processor may perform a model profiling execution of the machine learning model." Figure 2, block 205);
identifying one or more aspects of the machine learning model, wherein the one or more aspects of the machine learning model include one or more of: a tensor; a layer; or a node (Paragraph 23, "tensors may refer to model data related to states of a model such as model parameters, gradients, and/or optimizer states, etc., that are related by the model structure definition." Paragraph 26. Paragraph 53, "the gathered or obtained information or data in a profiling phase may include a relationship between the tensors and their corresponding module(s) in the model");…
based on the received indication, inserting at least one instrumentation hook into the machine learning model (Paragraph 27, "a hook may be registered, installed, arranged, or otherwise associated with or on a tensor or a module of the model that contains the tensor." Figure 3, blocks 320 and 330); and
collecting data via the instrumentation hook (Paragraph 73, "The handler of the post-backward hook, when being triggered or executed (e.g., immediately after the backward operation), may (1) record or determine the execution status of a tensor (e.g., whether a tensor is executed or not), and/or (2) collect or obtain the generated gradients in the backward operation phase").
Liu uses hooks to obtain information about specified tensors and modules (see Liu at paragraphs 26, 27, and 47-55), but does not appear to expressly teach receiving an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model.
Simon 2019 teaches receiving an indication that an instrumentation hook is to be used to collect data for at least one aspect of the one or more aspects of the machine learning model (Page 2, "in order to select the tensor collections you want to save, the frequency at which you want to save them, and whether you want to save the values themselves or a reduction (mean, average,etc.)." Page 4, "A SessionHook object attached to the TensorFlow session, putting everything together and saving requiredtensors during training").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Simon 2019 to explore the evolution of tensors over time, confirm hypotheses and fix the root cause of issues (see Simon 2019 at page 2).
As to dependent claim 17, Liu further teaches collecting data via the at least one instrumentation hook further comprises: determining whether a condition associated with the instrumentation hook has been met (Paragraph 27, "a “hook” is a term of art and may refer to a function, operation, or algorithm that is executed or triggered when e.g., a condition is met"); and based on the determining, collecting the data regarding the at least one aspect via the at least one instrumentation hook (Paragraph 27, "a hook may include a handler (a function, operation, or algorithm in the hook), which may be used to perform a desired or predetermined action, function, or operation (e.g., monitoring and/or recording the status, execution sequence/timestamp, execution phase, etc. of a tensor and/or its module, etc.)").
Claim 3 is rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Simon 2019 and Rauschmayr et al. (Detect Adversarial Inputs Using Amazon SageMaker Model Monitor and Amazon SageMaker Debugger | Amazon Web Services.” 2022. Amazon Web Services. April 5, 2022. https://aws.amazon.com/blogs/machine-learning/detect-adversarial-inputs-using-amazon-sagemaker-model-monitor-and-amazon-sagemaker-debugger/, hereinafter Rauschmayr).
As to dependent claim 3, the rejection of claim 1 is incorporated.
Liu does not appear to expressly teach the instrumentation hook is configured to: collect data regarding the at least one aspect while the machine learning model is generating an inference.
Rauschmayr teaches the instrumentation hook is configured to: collect data regarding the at least one aspect while the machine learning model is generating an inference (Page 5, "#capture layers for every inference request" and SaveConfig).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Rauschmayr to detect adversarial inputs (see Rauschmayr at page 1).
Claim 4 is rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Simon 2019 and Bhaskhar (Bhaskhar, Nandita. 2020. “Intermediate Activations — the Forward Hook.” Nandita Bhaskhar. August 17, 2020. https://web.stanford.edu/~nanbhas/blog/forward-hooks-pytorch/, hereinafter Bhaskhar).
As to dependent claim 4, the rejection of claim 1 is incorporated.
While Liu includes a handler (see Liu at paragraph 27, "a hook may include a handler (a function, operation, or algorithm in the hook)"), but does not appear to expressly teach the instrumentation hook is configured to: receive software code related to the collection of data regarding the at least one aspect; and cause the software code to be executed.
Bhaskhar teaches the instrumentation hook is configured to: receive software code related to the collection of data regarding the at least one aspect (Page 8, "hook (module, input, output) -> None or modified output". Page 9, "register_forward_hook"); and cause the software code to be executed (Page 9, "register_forward_hook").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning hook techniques of Bhaskhar to debug the model (see Bhaskhar at page 2).
Claims 5, 12, and 18 are rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Simon 2019 and Simon (Simon, Julien. 2020. “New – Profile Your Machine Learning Training Jobs With Amazon SageMaker Debugger | Amazon Web Services.” Amazon Web Services. December 8, 2020. https://aws.amazon.com/blogs/aws/profile-your-machine-learning-training-jobs-with-amazon-sagemaker-debugger/, hereinafter Simon 2020).
As to dependent claim 5, the rejection of claim 1 is incorporated.
Liu does not appear to expressly teach generate one or more graphs based on at least the data collected via the at least one instrumentation hook.
Simon 2020 teaches generate one or more graphs based on at least the data collected via the at least one instrumentation hook (Page 2, "While your training job is running, you can easily visualize these metrics in Amazon SageMaker Studio, our web-based integrated development environment for ML. As you would expect, all data is also available through the SageMaker Debugger API, and you can retrieve it to build your own graphs").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning debugging techniques of Simon 2020 to make it much easier to identify and fix training issues caused by hardware resource usage (see Simon 2020 at page 1).
As to dependent claim 12, the rejection of claim 10 is incorporated.
Liu does not appear to expressly teach generating one or more graphs based on at least the data collected via the at least one instrumentation hook.
Simon 2020 teaches generating one or more graphs based on at least the data collected via the at least one instrumentation hook (Page 2, "While your training job is running, you can easily visualize these metrics in Amazon SageMaker Studio, our web-based integrated development environment for ML. As you would expect, all data is also available through the SageMaker Debugger API, and you can retrieve it to build your own graphs").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning debugging techniques of Simon 2020 to make it much easier to identify and fix training issues caused by hardware resource usage (see Simon 2020 at page 1).
As to dependent claim 18, the rejection of claim 16 is incorporated.
Liu does not appear to expressly teach generating one or more graphs based on at least the data collected via the at least one instrumentation hook.
Simon 2020 teaches generating one or more graphs based on at least the data collected via the at least one instrumentation hook (Page 2, "While your training job is running, you can easily visualize these metrics in Amazon SageMaker Studio, our web-based integrated development environment for ML. As you would expect, all data is also available through the SageMaker Debugger API, and you can retrieve it to build your own graphs").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning debugging techniques of Simon 2020 to make it much easier to identify and fix training issues caused by hardware resource usage (see Simon 2020 at page 1).
Claims 6-8, 13-15, 19, and 21 are rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Simon 2019 and Uppalapati (U.S. Pat. App. Pub. No. 2023/0195590, hereinafter Uppalapati).
As to dependent claim 6, the rejection of claim 1 is incorporated.
Liu does not appear to expressly teach identify an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches identify an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 131-139 and 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 7, Liu does not appear to expressly teach cause the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches cause the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 8, the rejection of claim 1 is incorporated.
Liu further teaches identify at least one insertion point based on the at least one aspect (Paragraph 47, "the gathered or obtained information or data in a profiling phase may include a hook-able attribute of each tensor." Paragraph 84).
Liu does not appear to expressly teach determine the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model; and based on the determination, insert the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point.
Uppalapati teaches determine the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model (Paragraphs 84 and 85); and based on the determination, insert the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point (Paragraphs 84 and 85).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 13, the rejection of claim 10 is incorporated.
Liu does not appear to expressly teach identifying an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches identifying an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 131-139 and 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 14, Liu does not appear to expressly teach causing the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches causing the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 15, the rejection of claim 10 is incorporated.
Liu further teaches identifying at least one insertion point based on the at least one aspect (Paragraph 47, "the gathered or obtained information or data in a profiling phase may include a hook-able attribute of each tensor." Paragraph 84).
Liu does not appear to expressly teach determining the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model; and based on the determination, inserting the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point.
Uppalapati teaches determining the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model (Paragraphs 84 and 85); and based on the determination, inserting the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point (Paragraphs 84 and 85).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 19, the rejection of claim 16 is incorporated.
Liu does not appear to expressly teach identifying an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches identifying an aspect of the one or more aspects of the machine learning model to optimize based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 131-139 and 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
As to dependent claim 21, the rejection of claim 16 is incorporated.
Liu further teaches identifying at least one insertion point based on the at least one aspect (Paragraph 47, "the gathered or obtained information or data in a profiling phase may include a hook-able attribute of each tensor." Paragraph 84).
Liu does not appear to expressly teach determining the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model; and based on the determination, inserting the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point.
Uppalapati teaches determining the extent to which the insertion of the at least one instrumentation hook at the at least one insertion point will impact the operation of the machine learning model (Paragraphs 84 and 85); and based on the determination, inserting the at least one instrumentation hook within the machine learning model at one or more insertion points of the at least one insertion point (Paragraphs 84 and 85).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
Claim 20 is rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Simon 2019, Simon 2020, and Uppalapati.
As to dependent claim 20, the rejection of claim 18 is incorporated.
Liu does not appear to expressly teach causing the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook.
Uppalapati teaches causing the identified aspect to be optimized based on at least the data collected via the at least one instrumentation hook (Figure 6. Paragraphs 146-149).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the machine learning hooks of Liu to include the machine learning profiling techniques of Uppalapati to generate expected and runtime performance metrics at various levels of granularity (see Uppalapati at paragraph 17).
Conclusion
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Casey R. Garner whose telephone number is 571-272-2467. The examiner can normally be reached Monday to Friday, 8am to 5pm, Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Casey R. Garner/Primary Examiner, Art Unit 2123
