DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that does not use the word “means,” and are interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are, where the generic place holder has been underlined and the functional language italicize:
“a bytecode analyzer configured to analyze a computational structure of a model based on bytecode of an AI model including dynamic control flow;
a bytecode tracer configured to generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result from the bytecode analyzer” in claim 1.
“a bytecode converter configured to convert a function or method of the model into bytecode and an operational structure table generator configured to generate an operational structure table (OP-CODE table) by arranging the converted bytecode” in claim 2.
“an operational structure table analyzer configured to receive the operational structure table from the bytecode analyzer and analyze it
and a graph tree generator configured to generate a graph tree for the model using the analyzed operational structure table” in claim 3.
“an algorithm decision unit configured to receive an image and an XAI algorithm as inputs and compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm” in claim 7.
“a visualizer configured to visualize a decision-making process of the XAI algorithm for an image” in claim 8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Drawings
Color photographs and color drawings are not accepted in utility applications unless a petition filed under 37 CFR 1.84(a)(2) is granted. Any such petition must be accompanied by the appropriate fee set forth in 37 CFR 1.17(h), one set of color drawings or color photographs, as appropriate, if submitted via the USPTO patent electronic filing system or three sets of color drawings or color photographs, as appropriate, if not submitted via the via USPTO patent electronic filing system, and, unless already present, an amendment to include the following language as the first paragraph of the brief description of the drawings section of the specification:
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
Color photographs will be accepted if the conditions for accepting color drawings and black and white photographs have been satisfied. See 37 CFR 1.84(b)(2).
Claim Objections
Claims 1 and 9 are objected to because of the following informalities:
In claim 1 and claim 9, “based on bytecode of an AI model including dynamic control flow” should read as “based on bytecode of an AI (artificial intelligence) model including dynamic control flow.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-8 are further rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Claims 1-3 and 7-8 recite claim limitations
“a bytecode analyzer configured to analyze a computational structure of a model based on bytecode of an AI model including dynamic control flow;
a bytecode tracer configured to generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result from the bytecode analyzer” in claim 1;
“a bytecode converter configured to convert a function or method of the model into bytecode and an operational structure table generator configured to generate an operational structure table (OP-CODE table) by arranging the converted bytecode” in claim 2;
“an operational structure table analyzer configured to receive the operational structure table from the bytecode analyzer and analyze it
and a graph tree generator configured to generate a graph tree for the model using the analyzed operational structure table” in claim 3;
“an algorithm decision unit configured to receive an image and an XAI algorithm as inputs and compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm” in claim 7;
“a visualizer configured to visualize a decision-making process of the XAI algorithm for an image” in claim 8
interpreted under 35 U.S.C. 112(f) wherein the means lack written description of the invention in full, clear, concise and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
In light of the specification of the pre-grant publication of the instant application
PNG
media_image1.png
336
298
media_image1.png
Greyscale
wherein the XAI apparatus is disclosed to be a computer program and there is no recitation of a “processor,” “hardware,” or “circuit” in the specification or shown in any figures, (“[0062] FIG. 5 is a diagram illustrating the configuration of an XAI apparatus according to an embodiment of the present invention. [0063] Referring to FIG. 5 , an XAI apparatus 5 includes a model analyzer 51, an algorithm decision unit 53, and a visualizer 55. [0064] The XAI apparatus 5 may be implemented in the form of a computer program. The computer program may include one or more instructions, on which the methods/operations according to various embodiments of the present invention may be implemented.”) Thus, sufficient structure for the means of the XAI apparatus is not provided. See MPEP § 2163.03, subsection VI.
Further, there is insufficient written description for the specific algorithms for achieving the means of the limitations below:
“a bytecode analyzer configured to analyze a computational structure of a model based on bytecode of an AI model including dynamic control flow”
PNG
media_image2.png
360
500
media_image2.png
Greyscale
Para. [0078] does not provide a clear algorithm of analyzing a computational structure and merely recites “[0078] Referring to FIGS. 5 and 6 , the bytecode analyzer 510 analyzes bytecode of a forward method of a ResNet model shown in (a), one of the Torchvision models. The forward method shown in (a) of the ResNet model may call self.forward_impl.” Wherein fig. 6 merely shows the forward function (a) and bytecode of the forward function (b).
“a bytecode tracer configured to generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result from the bytecode analyzer”
Para. [0007] does not provide a clear algorithm of using the analysis result to generate a model graph and merely recites “a bytecode tracer configured to generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result from the bytecode analyzer.”
“an operational structure table analyzer configured to receive the operational structure table from the bytecode analyzer and analyze it”
Para. [0075] does not provide a clear algorithm of analyzing the operational structure table and merely recites “[0075] The operational structure table analyzer 521 analyzes the operational structure table.”
Claims 2-8 are further rejected on virtue of their dependency to claim 1.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 and analogous claim 9 recite “analyzing a computational structure of a model based on bytecode of an AI model including dynamic control flow;” it is unclear if:
the analyzing step is performed on the computational structure of a model and bytecode of an AI model
or the analyzing step is merely performed on a computational structure of a model wherein the model is based on bytecode of an AI model.
For examination purposes, Examiner interprets the BRI of the limitation as 2.
Claims 2-8 and 10-12 are further rejected on virtue of their dependencies to the base claim.
Claim 4 recites the limitation "the trace table." There is insufficient antecedent basis for this limitation in the claim.
Claim 4 further recites “generate graph information for the model through the variable tree;” it is unclear if the graph information is generated:
for the model and the variable tree (both intended use)
for the model based on the variable tree
For examination purposes, Examiner interprets the BRI of the limitation as 1.
Claim 6 is further rejected on virtue of its dependency to claim 4.
Claim 5 contains the trademark/trade name “PyTorch.” Where a trademark or trade name is used in a claim as a limitation to identify or describe a particular material or product, the claim does not comply with the requirements of 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 1982). The claim scope is uncertain since the trademark or trade name cannot be used properly to identify any particular material or product. A trademark or trade name is used to identify a source of goods, and not the goods themselves. Thus, a trademark or trade name does not identify or describe the goods associated with the trademark or trade name. In the present case, the trademark/trade name is used to identify/describe "invoked data" and, accordingly, the identification/description is indefinite.
Further, the term “normally” in claim 5 is a relative term which renders the claim indefinite. The term “normally” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It is unclear to what metes and bounds the XAI “normally” performs bytecode tracing and what would be considered abnormal.
Claim 6 recites “wherein the bytecode tracer is configured to generate a variable tree and a graph tree.” There is insufficient antecedent basis for “a variable tree” and “a graph tree” in the claim as it is unclear if the variable tree and graph tree are the same as the variable tree and graph tree recited in parent claim 4.
Claim 7 recites the limitation "the generated graph.” There is insufficient antecedent basis for this limitation in the claim.
Claim 8 is further rejected on virtue of its dependency to claim 7.
Claim limitations a bytecode analyzer configured to analyze a computational structure of a model based on bytecode of an AI model including dynamic control flow;
a bytecode tracer configured to generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result from the bytecode analyzer” in claim 1;
“a bytecode converter configured to convert a function or method of the model into bytecode and an operational structure table generator configured to generate an operational structure table (OP-CODE table) by arranging the converted bytecode” in claim 2;
“an operational structure table analyzer configured to receive the operational structure table from the bytecode analyzer and analyze it
and a graph tree generator configured to generate a graph tree for the model using the analyzed operational structure table” in claim 3;
“an algorithm decision unit configured to receive an image and an XAI algorithm as inputs and compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm” in claim 7;
“a visualizer configured to visualize a decision-making process of the XAI algorithm for an image” in claim 8 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. See 112(a) rejection. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claims 2-8 are further rejected on virtue of their dependency to claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In regards to claim 1,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A – Prong 1: Judicial Exception Recited?
MPEP 2106.04(a)(2)(I) “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
Yes, the claim recites a mental process, specifically:
analyze a computational structure of a model based on bytecode of an AI model including dynamic control flow
This limitation encompasses an evaluation of a computational structure of a model and providing a judgement.
generate a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result
This limitation encompasses providing an opinion in the form of a model graph after an evaluation of an analysis result.
Therefore, the claim recites a mental process.
Step 2A – Prong 2: Integrated into a Practical Solution?
MPEP 2106.05(f) Mere Instructions To Apply An Exception has found simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. The following steps are mere instructions to apply:
An explainable artificial intelligence (XAI) apparatus comprising:
a bytecode analyzer configured to
and a bytecode tracer configured to
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering to be insignificant extra-solution activity. The following steps are insignificant extra-solution activities:
Mere data gathering:
for providing visualization of decision-making process of an XAI algorithm
The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed on practicing the abstract idea.
The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitation is mere data gathering (Insignificant Extra-Solution Activity) and Mere Instructions To Apply An Exception and a generic device do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.
The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Further, the claim recites presenting data by generic device.
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to
gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary
computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607,
610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP
Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)
(sending messages over a network); buy SAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112
USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);
but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106
(Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how
interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides
the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
(emphasis added)).
The additional elements have been considered both individually and as an ordered
combination in the significantly more consideration.
The claim is ineligible.
In regards to claim 2,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
convert a function or method of the model into bytecode
This limitation directs to a mathematical relationship. See MPEP 2106.04(a)(2)(I)(A.) ii. a conversion between binary coded decimal and pure binary, Benson, 409 U.S. at 64, 175 USPQ at 674;
generate an operational structure table (OP-CODE table) by arranging the converted bytecode
This limitation directs to a mathematical relationship. See MPEP 2106.04(a)(2)(I)(A.) iv. organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721.
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the bytecode analyzer comprises a bytecode converter configured to… an operational structure table generator configured to…
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the bytecode analyzer comprises a bytecode converter configured to… an operational structure table generator configured to…
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
In regards to claim 3,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
receive the operational structure table from the bytecode analyzer and analyze it and… generate a graph tree for the model using the analyzed operational structure table
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses an evaluation of the generated operational structure table to provide an opinion in the form of a graph tree. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the bytecode tracer comprises an operational structure table analyzer configured to… and a graph tree generator configured to..
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
receive the operational structure table from the bytecode analyzer
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the bytecode tracer comprises an operational structure table analyzer configured to… and a graph tree generator configured to..
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
receive the operational structure table from the bytecode analyzer
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
In regards to claim 5,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
the XAI apparatus normally performs bytecode tracing of the bytecode tracer, and when the invoked data is a layer not provided by PyTorch, the XAI apparatus performs bytecode tracing of the bytecode tracer after performing a bytecode analysis on an unsupported code through the bytecode analyzer
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
wherein when data is invoked through the bytecode tracer and the invoked data is a layer provided by PyTorch
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
the XAI apparatus normally performs bytecode tracing of the bytecode tracer, and when the invoked data is a layer not provided by PyTorch, the XAI apparatus performs bytecode tracing of the bytecode tracer after performing a bytecode analysis on an unsupported code through the bytecode analyzer
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
wherein when data is invoked through the bytecode tracer and the invoked data is a layer provided by PyTorch
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
In regards to claim 6,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
generate a variable tree and a graph tree including slicing and operation processes when image shape is greater than or equal to a predetermined value, and may generate a variable tree and a graph tree including slicing and operation processes after transposing axes of an image through a transpose process when the image shape is smaller than the predetermined value.
Examiner’s note: The recitation of “may generate” does not necessitate the limitation of generating a variable tree and a graph tree; thus, the BRI of the claim encompasses merely “wherein the bytecode tracer is configured to generate a variable tree and a graph tree including slicing and operation processes when image shape is greater than or equal to a predetermined value.”
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses an evaluation of image shape and providing an opinion of a variable tree and a graph tree. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the bytecode tracer is configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the bytecode tracer is configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
In regards to claim 7,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses providing an opinion of the contribution of each image pixel through the XAI algorithm after an evaluation of the generated graph. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
further comprising an algorithm decision unit configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
receive an image and an XAI algorithm as inputs
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
further comprising an algorithm decision unit configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
receive an image and an XAI algorithm as inputs
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
In regards to claim 8,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
further comprising a visualizer configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
visualize a decision-making process of the XAI algorithm for an image
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
further comprising a visualizer configured to
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f)
visualize a decision-making process of the XAI algorithm for an image
This limitation directs to mere data gathering of insignificant extra-solution activity. See MPEP § 2106.05(g)
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
In regards to claim 9,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A – Prong 1: Judicial Exception Recited?
MPEP 2106.04(a)(2)(I) “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
Yes, the claim recites a mental process, specifically:
analyzing a computational structure of a model based on bytecode of an AI model including dynamic control flow
This limitation encompasses an evaluation of a computational structure of a model and providing a judgement.
generating a model graph for providing visualization of decision-making process of an XAI algorithm using an analysis result
This limitation encompasses providing an opinion in the form of a model graph after an evaluation of an analysis result.
Therefore, the claim recites a mental process.
Step 2A – Prong 2: Integrated into a Practical Solution?
MPEP 2106.05(f) Mere Instructions To Apply An Exception has found simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. The following steps are mere instructions to apply:
A method for analyzing a model using an explainable artificial intelligence (XAI) apparatus (generic computer component to apply the abstract idea)
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering to be insignificant extra-solution activity. The following steps are insignificant extra-solution activities:
Mere data gathering:
for providing visualization of decision-making process of an XAI algorithm
The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed on practicing the abstract idea.
The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitation is mere data gathering (Insignificant Extra-Solution Activity) and Mere Instructions To Apply An Exception and a generic device do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.
The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Further, the claim recites presenting data by generic device.
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to
gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary
computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607,
610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP
Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)
(sending messages over a network); buy SAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112
USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);
but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106
(Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how
interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides
the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
(emphasis added)).
The additional elements have been considered both individually and as an ordered
combination in the significantly more consideration.
The claim is ineligible.
In regards to claim 10,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein the analyzing of the computational structure of the model comprises converting a function or method of the model into bytecode
This limitation directs to a mathematical relationship. See MPEP 2106.04(a)(2)(I)(A.) ii. a conversion between binary coded decimal and pure binary, Benson, 409 U.S. at 64, 175 USPQ at 674;
and generating an operational structure table by arranging the converted bytecode
This limitation directs to a mathematical relationship. See MPEP 2106.04(a)(2)(I)(A.) iv. organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721.
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
In regards to claim 11,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – process.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
wherein the generating of the model graph comprises analyzing the generated operational structure table and generating a graph tree using the analyzed operational structure table
This limitation directs to a mental process that can be performed in the human mind, by a human using pen and paper, or using a computer as a tool to perform the concept and encompasses an evaluation of the generated operational structure table to provide an opinion in the form of a graph tree. See MPEP 2106.04(a)(2)(III)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 5 and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub no. US20240095052A1 (“Ansel”) in view of Taylor, JohnMark, and Nikolaus Kriegeskorte. "Extracting and visualizing hidden activations and computational graphs of PyTorch models with TorchLens." Scientific Reports 13.1 (2023): 14375. (“Taylor”).
In regards to claim 1 and analogous claim 9,
Ansel teaches An explainable artificial intelligence (XAI) apparatus comprising: a bytecode analyzer configured to analyze a computational structure of a model based on bytecode [of an AI model including dynamic control flow];
Examiner’s note: Examiner interprets the bytecode analyzer as software.
(Ansel, “[0029] FIG. 2 illustrates another example process 200 of converting an object 202 to bytecode. In particular embodiments, the object 202 may be embodied as a file containing code. The code may be converted to bytecode. In particular embodiments, the object 202 may be translated into bytecode during the process 200. In particular embodiments, the object 202 may be translated to a PyFrame object 204 and a PyCode object 206. The PyFrame Object 204 may reference the PyCode Object 206. While this disclosure describes the process 200 with respect to translating an object 202 to PyFrame Object 204 and PyCode Object 206, this disclosure contemplates translating the object 202 to other objects. In particular embodiments, PyCode Object 206 may be embodied as bytecode. In particular embodiments, a dynamic bytecode analysis and transformation process 208 may be performed on the PyCode Object 206 [a bytecode analyzer configured to analyze a computational structure of a model based on bytecode]. In particular embodiments, the dynamic analysis and transformation process 208 may be performed by a just-in-time compiler. As an example and not by way of limitation, the process 208 may be performed by TorchDynamo which may be configured to perform analysis on bytecode to modify the bytecode prior to executing the bytecode.”
PNG
media_image3.png
582
701
media_image3.png
Greyscale
)
Ansel teaches and a bytecode tracer configured to generate a model graph [for providing visualization of decision-making process of an XAI algorithm] using an analysis result from the bytecode analyzer.
Examiner’s note: Examiner interprets the bytecode tracer as software.
(Ansel, [0029], “In particular embodiments, the just-in-time compiler may extract one or more operations from PyCode Object 206 to generate one or more FX Graphs 210. [generate a model graph ie FX graph… using an analysis result (see 208 and 210 in fig. 2 for using the analysis result to generate the FX graph) from the bytecode analyzer]”)
However, Ansel does not explicitly teach of an AI model including dynamic control flow; for providing visualization of decision-making process of an XAI algorithm
Rather, Ansel discloses artificial neural networks (Ansel, “[0052] FIG. 10 illustrates an example artificial neural network (“ANN”) 1000. In particular embodiments, an ANN may refer to a computational model comprising one or more nodes.”) and TorchDynamo as an improvement over PyTorch (Ansel, “[0004] In particular embodiments, TorchDynamo may be a Python-level just-in-time compiler that is designed to make unmodified PyTorch programs faster by modifying the Python bytecode before the bytecode is executed.”)
Taylor teaches of an AI model including dynamic control flow;
(Taylor, pg, 6 para. 4, “In cases where a network contains conditional control flow [of an AI model including dynamic control flow], where an if–then statement is evaluated on a tensor to yield a single Boolean value (e.g., to perform different computations based on the value of the model’s input), TorchLens marks the node yielding this single Boolean value in yellow, infers the beginning of the set of computations involved in computing this Boolean value, and marks the beginning of the branch of the computational graph involved in computing this Boolean value with a bold “IF” label (Fig. 2F).”)
Taylor teaches for providing visualization of decision-making process of an XAI algorithm
(Taylor, pg. 5 -pg. 6 Visualization, “In addition to returning the ModelHistory data structure with information about the forward pass, TorchLens can also automatically produce a visualization of the model’s computational graph (Fig. 2). The visualization enables the user to easily understand the structure of the model and a given layer’s place within that structure, aiding in both the overall understanding of a model, and in the selection of particular layers from which to extract activations [for providing visualization of decision-making process of an XAI algorithm].”)
Ansel and Taylor are both considered to be analogous to the claimed invention because they are in the same field of determining computational graphs from PyTorch models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ansel to incorporate the teachings of Taylor in order to provide TorchLens as doing so enables the framework of Ansel to extract results of all intermediate operations, provide an intuitive visualization of the computational graph, and be applied to any PyTorch models, including models with conditional logic (Taylor, Abstract, “Many models are implemented in PyTorch, the leading framework for building DNN models. Here we introduce TorchLens, a new open-source Python package for extracting and characterizing hidden-layer activations in PyTorch models. Uniquely among existing approaches to this problem, TorchLens has the following features: (1) it exhaustively extracts the results of all intermediate operations, not just those associated with PyTorch module objects, yielding a full record of every step in the model’s computational graph, (2) it provides an intuitive visualization of the model’s complete computational graph along with metadata about each computational step in a model’s forward pass for further analysis, (3) it contains a built-in validation procedure to algorithmically verify the accuracy of all saved hidden-layer activations, and (4) the approach it uses can be automatically applied to any PyTorch model with no modifications, including models with conditional (if–then) logic in their forward pass, recurrent models, branching models where layer outputs are fed into multiple subsequent layers in parallel, and models with internally generated tensors (e.g., injections of noise). Furthermore, using TorchLens requires minimal additional code, making it easy to incorporate into existing pipelines for model development and analysis, and useful as a pedagogical aid when teaching deep learning concepts.”)
In regards to claim 2 and analogous claim 10,
Ansel and Taylor teach The XAI apparatus of claim 1,
Ansel teaches wherein the bytecode analyzer comprises a bytecode converter configured to convert a function or method of the model into bytecode
(Ansel, “[0029] FIG. 2 illustrates another example process 200 of converting an object 202 to bytecode. In particular embodiments, the object 202 may be embodied as a file containing code. The code may be converted to bytecode [convert a function or method of the model into bytecode]. In particular embodiments, the object 202 may be translated into bytecode during the process 200.”)
Ansel teaches and an operational structure table generator configured to generate an operational structure table (OP-CODE table) by arranging the converted bytecode.
(Ansel, [0032], “In particular embodiments, the just-in-time compiler may generate FX graphs that are then compiled into functions using a user-defined compiler, such as the compiled functions shown in FIG. 6. The bytecode 400 may be transformed to include calls to the compiled functions generated from the FX graphs. In particular embodiments, the compiled functions as shown in FIG. 6 may include opcodes [and an operational structure table generator configured to generate an operational structure table (OP-CODE table) by arranging the converted bytecode; wherein Ansel discloses compiling opcodes in an opcode table as seen in fig. 6], names, targets, args, and kwargs. In particular embodiments, the just-in-time compiler may generate one or more guards corresponding to the bytecode 500.”
PNG
media_image4.png
336
619
media_image4.png
Greyscale
Fig. 6)
In regards to claim 3 and analogous claim 11,
Ansel and Taylor teach The XAI apparatus of claim 2,
Ansel teaches wherein the bytecode tracer comprises an operational structure table analyzer configured to receive the operational structure table from the bytecode analyzer and analyze it
(Ansel, [0032], “In particular embodiments, the just-in-time compiler may generate FX graphs that are then compiled into functions using a user-defined compiler, such as the compiled functions shown in FIG. 6 . The bytecode 400 may be transformed to include calls to the compiled functions generated from the FX graphs. In particular embodiments, the compiled functions as shown in FIG. 6 may include opcodes, names, targets, args, and kwargs [receive the operational structure table from the bytecode analyzer and analyze it; wherein Examiner interprets compiling the opcode to be “analyzing” the opcode].”)
Taylor teaches and a graph tree generator configured to generate a graph tree for the model using the analyzed operational structure table.
(Taylor, pg. 4, “Figure 2. TorchLens visualization features. (A) Code and visualization for a simple example model. Graph nodes correspond to tensor operations, edges correspond to parent–child relationships between operations (i.e., that the output of one operation is the input to another). Input operations are shown in green and output operations are shown in red. Operations in a module are shown as a box; other operations are shown as ellipses [using the analyzed operational structure table; wherein operations (provided from the table of Ansel) are pulled to generate the graph tree in the context of the applied references]. Nodes for operations that create a new tensor inside of the model (e.g., from torch.rand) are rendered with dashed lines. The visualization also shows the name of the model in the top left, the number of tensors computed in the model and their total file size, and the number of trainable parameters in the model along with their total file size…. (F) For models containing dynamic control flow with if–then branching, the operation returning the final Boolean value in evaluating the “if” statement is marked in yellow, and the full set of operations involved in evaluating the “if” statement is labeled with a boldfaced “IF”. [generate a graph tree for the model; see fig. 2(F) for the graph tree of the conditional model ]
PNG
media_image5.png
421
598
media_image5.png
Greyscale
”)
In regards to claim 5,
Ansel and Taylor teach The XAI apparatus of claim 1,
Ansel teaches wherein when data is invoked through the bytecode tracer and the invoked data is a layer provided by PyTorch,
(Ansel, [0004], “TorchDynamo may dynamically rewrite Python bytecode in order to extract sequences of PyTorch operations [wherein when data is invoked through the bytecode tracer and the invoked data is a layer provided by PyTorch] into an FX Graph which is then just-in-time compiled with a user-defined compiler.”)
Ansel teaches the XAI apparatus normally performs bytecode tracing of the bytecode tracer,
(Ansel, [0029], “In particular embodiments, a dynamic bytecode analysis and transformation process 208 [the XAI apparatus normally performs bytecode tracing of the bytecode tracer] may be performed on the PyCode Object 206. In particular embodiments, the dynamic analysis and transformation process 208 may be performed by a just-in-time compiler. As an example and not by way of limitation, the process 208 may be performed by TorchDynamo which may be configured to perform analysis on bytecode to modify the bytecode prior to executing the bytecode.”)
Ansel teaches and when the invoked data is a layer not provided by PyTorch, the XAI apparatus performs bytecode tracing of the bytecode tracer after performing a bytecode analysis on an unsupported code through the bytecode analyzer.
(Ansel, [0029], “In particular embodiments, the object 202 may be embodied as a file containing code. The code may be converted to bytecode [after performing a bytecode analysis on an unsupported code through the bytecode analyzer ]… In particular embodiments, a dynamic bytecode analysis and transformation process 208 [the XAI apparatus performs bytecode tracing of the bytecode tracer] may be performed on the PyCode Object 206. In particular embodiments, the dynamic analysis and transformation process 208 may be performed by a just-in-time compiler. As an example and not by way of limitation, the process 208 may be performed by TorchDynamo which may be configured to perform analysis on bytecode to modify the bytecode prior to executing the bytecode.”)
Claim(s) 4 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ansel in view of Taylor in further view of Pan, Zhixin, and Prabhat Mishra. "Hardware acceleration of explainable machine learning." 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2022. (“Pan”)
In regards to claim 4 and analogous claim 12,
Ansel and Taylor teach The XAI apparatus of claim 3,
Ansel teaches wherein the graph tree generator is configured to store a calculation result of an operation process [in the trace table] while analyzing the operational structure table received from the bytecode analyzer,
(Ansel, “[0027] In particular embodiments, the computing system may execute the plurality of bytecode based at least on the compiled function [wherein the graph tree generator is configured to store a calculation result of an operation process]”)
(Ansel, [0032], “In particular embodiments, the just-in-time compiler may generate FX graphs that are then compiled into functions using a user-defined compiler, such as the compiled functions shown in FIG. 6 . The bytecode 400 may be transformed to include calls to the compiled functions generated from the FX graphs. In particular embodiments, the compiled functions as shown in FIG. 6 may include opcodes [while analyzing the operational structure table received from the bytecode analyzer], names, targets, args, and kwargs.”)
Taylor teaches generate graph information for the model through the variable tree, and store the generated graph information in the graph tree.
(Taylor, pg. 1, Abstract, “…TorchLens has the following features: (1) it exhaustively extracts the results of all intermediate operations, not just those associated with PyTorch module objects, yielding a full record of every step in the model’s computational graph…”)
(Taylor, pg. 4, “Figure 2. TorchLens visualization features. (A) Code and visualization for a simple example model. Graph nodes correspond to tensor operations, edges correspond to parent–child relationships between operations (i.e., that the output of one operation is the input to another). Input operations are shown in green and output operations are shown in red. Operations in a module are shown as a box; other operations are shown as ellipses [generate graph information for the model through the variable tree; wherein operations (provided from the table of Ansel) are pulled to generate the graph tree in the context of the applied references]. Nodes for operations that create a new tensor inside of the model (e.g., from torch.rand) are rendered with dashed lines. The visualization also shows the name of the model in the top left, the number of tensors computed in the model and their total file size, and the number of trainable parameters in the model along with their total file size…. (F) For models containing dynamic control flow with if–then branching, the operation returning the final Boolean value in evaluating the “if” statement is marked in yellow, and the full set of operations involved in evaluating the “if” statement is labeled with a boldfaced “IF”. [store the generated graph information in the graph tree; see fig. 2(F) for the graph tree of the conditional model ]
PNG
media_image5.png
421
598
media_image5.png
Greyscale
”)
PNG
media_image6.png
264
207
media_image6.png
Greyscale
However, Ansel and Taylor do not explicitly teach [wherein the graph tree generator is configured to store a calculation result of an operation process] in the trace table [while analyzing the operational structure table received from the bytecode analyzer], generate a variable tree by tracing a process of variable changes according to a calculation sequence of the operation process of the trace table,
Pan teaches [wherein the graph tree generator is configured to store a calculation result of an operation process] in the trace table [while analyzing the operational structure table received from the bytecode analyzer], generate a variable tree by tracing a process of variable changes according to a calculation sequence of the operation process of the trace table
Examiner’s note: Examiner interprets the variable tree in light of fig. 11(b) of the instant application.
(Pan, Section IV. E. pg. 10 col. 2-pg. 11 col.1, “To understand how model distillation gathers insights, let us consider an example on malware detection from ResNet50. The ML-based detector receives running data of MIRAI malware [33] as input in the format of a trace table [in the trace table], where each row represents the hex values in a register in specific clock cycles (each column represents a specific clock cycle). Figure 12 shows a snapshot of the trace table.
PNG
media_image7.png
313
488
media_image7.png
Greyscale
Our proposed method computed the corresponding contribution factor of each clock cycle towards the output using model distillation. Contribution factors are shown as weights in the last (colored) row. Clearly we can see that the weight of C2 is significantly larger than the others. By tracing the execution, it has been shown that C2 corresponds to the timestamp of assigning value to the variable “ATTACK VECTOR” in Mirai. This variable records the identity of attack modes, based on which the bot takes relative actions to perform either a UDP attack or DNS attack [generate a variable tree by tracing a process of variable changes according to a calculation sequence of the operation process of the trace table; wherein Pan discloses tracing the execution and noting variable changes correlated with timestamps (clock cycle)].”)
Pan is considered to be analogous to the claimed invention because they are reasonably pertinent to the problem the inventor faced of tracing computations of programs and in the same field of explainable AI. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ansel and Taylor to incorporate the teachings of Pan in order to provide a trace table as a method of recording computations as doing so provides a technique to illustrate reasons for classification and provides confidence in the prediction of the model. (Pan, Section IV. E. pg. 10 col. 2-pg. 11 col.1, “This attack-mode flag is the most important feature of a majority of malware bot programs, and our proposed method successfully extracted it from the traces to illustrate the reason for classifying it as a malware. This interpretation not only provides confidence in malware detection but also helps in malware localization.”)
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Ansel in view of Taylor in further view of Pan in further view of Sousa, Rafael, et al. "Tensor slicing and optimization for multicore NPUs." Journal of Parallel and Distributed Computing 175 (2023): 66-79. (“Sousa”)
In regards to claim 6,
Ansel and Taylor and Pan teach The XAI apparatus of claim 4,
Pan teaches wherein the bytecode tracer is configured to generate a variable tree
(Pan, Section IV. E. pg. 10 col. 2-pg. 11 col.1, “Our proposed method computed the corresponding contribution factor of each clock cycle towards the output using model distillation. Contribution factors are shown as weights in the last (colored) row. Clearly we can see that the weight of C2 is significantly larger than the others. By tracing the execution, it has been shown that C2 corresponds to the timestamp of assigning value to the variable “ATTACK VECTOR” in Mirai. This variable records the identity of attack modes, based on which the bot takes relative actions to perform either a UDP attack or DNS attack [wherein the bytecode tracer is configured to generate a variable tree; wherein Pan discloses tracing the execution and noting variable changes correlated with timestamps (clock cycle)].”)
Taylor teaches and a graph tree including [slicing] and operation processes [when image shape is greater than or equal to a predetermined value], and may generate a variable tree and a graph tree including slicing and operation processes after transposing axes of an image through a transpose process when the image shape is smaller than the predetermined value.
Examiner’s note: The recitation of “may generate” does not necessitate the limitation of generating a variable tree and a graph tree; thus, the BRI of the claim encompasses merely “wherein the bytecode tracer is configured to generate a variable tree and a graph tree including slicing and operation processes when image shape is greater than or equal to a predetermined value.”
(Taylor, pg. 9 Validation, “The approach used by TorchLens (described in the “Implementation” section below) can in principle be used for any PyTorch model. The space of possible DNN models is infinite because the number of ways that tensor operations can be assembled into a computational graph is unlimited [and a graph tree including [slicing] and operation processes]; wherein Taylor discloses TorchLens is capable of being used for any PyTorch model (see also Table 1 on pg. 1) for improvements of coverage over existing methods
PNG
media_image8.png
280
1029
media_image8.png
Greyscale
)
However, Ansel and Taylor and Pan do not explicitly teach [a graph tree including] slicing [and operation processes] when image shape is greater than or equal to a predetermined value
Sousa teaches [a graph tree including] slicing [and operation processes] when image shape is greater than or equal to a predetermined value
(Sousa, pg. 1, section 1 col. 2, “It is well-known that convolution is the most expensive operation of a CNN, accounting for the largest share of a CNN execution. Given the size of its tensor inputs [when image shape is equal to a predetermined value] and the wide variety of configuration parameters (e.g., kernel size, stride, etc), selecting the best data mapping which maximizes convolution parallelism while minimizing memory transactions is a key factor to the performance of any AI accelerator. This is particularly critical for multicore Neural Processing Units (NPUs), which have stringent (on-chip) memory constraints and need to achieve large inference throughput. To achieve that, Convolution input tensors and weights need to be divided into slices [slicing] that fit into NPU on-chip memories.”)
Sousa is considered to be analogous to the claimed invention because they are reasonably pertinent to the problem the inventors faced of determining tensor operations for particular models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ansel, Taylor and Pan to incorporate the teachings of Sousa in order to provide tensor slicing as doing so minimizes memory transactions and maximizes parallelism, particularly for CNN (Sousa, Abstract, “Although code generation for Convolution Neural Network (CNN) models has been extensively studied, performing efficient data slicing and parallelization for highly-constrained Multicore Neural Processor Units (NPUs) is still a challenging problem. Given the size of convolutions’ input/output tensors and the small footprint of NPU on-chip memories, minimizing memory transactions while maximizing parallelism and MAC utilization are central to any effective solution. This paper proposes a TensorFlow XLA/LLVM compiler optimization pass for Multicore NPUs, called Tensor Slicing Optimization (TSO), which: (a) maximizes convolution parallelism and memory usage across NPU cores; and (b) reduces data transfers between host and NPU on-chip memories by using DRAM memory burst time estimates to guide tensor slicing. To evaluate the proposed approach, a set of experiments was performed using the NeuroMorphic Processor (NMP), a multicore NPU containing 32 RISC-V cores extended with novel CNN instructions. Experimental results show that TSO is capable of identifying the best tensor slicing that minimizes execution time for a set of CNN models. Speed-ups of up to 21.7% result when comparing the TSO burst based technique to a no-burst data slicing approach. To validate the generality of the TSO approach, the algorithm was also ported to the Glow Machine Learning framework. The performance of the models were measured on both Glow and TensorFlow XLA/LLVM compilers, revealing similar results.”)
Claim(s) 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Ansel in view of Taylor in further view of Selvaraju, Ramprasaath R., et al. "Grad-cam: Visual explanations from deep networks via gradient-based localization." arXiv:1610.02391v4 [cs.CV] 3 Dec 2019 (“Selvaraju”)
In regards to claim 7,
Ansel and Taylor teach The XAI apparatus of claim 1,
However, Ansel and Taylor do not explicitly teach further comprising an algorithm decision unit configured to receive an image and an XAI algorithm as inputs and compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm
Selvaraju teaches further comprising an algorithm decision unit configured to receive an image and an XAI algorithm as inputs and compute the generated graph to determine contribution of each gradient or image pixel to each layer through the XAI algorithm.
(Selvaraju, pg. 2 col. 2-pg. 3 col. 1, “(4) We present Grad-CAM visualizations for ResNets [24] applied to image classification and VQA (Sec. 8.2)
(5) We use neuron importance from Grad-CAM and neuron names from [4] and obtain textual explanations for model decisions [receive an image and an XAI algorithm as inputs and compute the generated graph (ie executing the computational graph of Ansel and Taylor) to determine contribution ie importance of each gradient or image pixel to each layer through the XAI algorithm] (Sec. 7).”; see section 7 for details on how the contribution is determined)
Selvaraju considered to be analogous to the claimed invention because they are in the same field of explainable AI. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ansel and Taylor to incorporate the teachings of Selvaraju in order to a technique for providing visual explanations of CNNs as doing so makes them more transparent and explainable (Selvaraju, Abstract, “We propose a technique for producing ‘visual ex planations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable.”)
In regards to claim 8,
Ansel and Taylor and Selvaraju teach The XAI apparatus of claim 7,
Selvaraju teaches further comprising a visualizer configured to visualize a decision-making process of the XAI algorithm for an image.
(Selvaraju, pg. 3 fig. 1, “Fig. 1: (a) Original image with a cat and a dog. (b-f) Support for the cat category according to various visualizations [a visualizer configured to visualize a decision-making process of the XAI algorithm for an image] for VGG-16 and ResNet. (b) Guided Backpropagation [53]: highlights all contributing features. (c, f) Grad-CAM (Ours): localizes class-discriminative regions, (d) Combining (b) and (c) gives Guided Grad-CAM, which gives high-resolution class-discriminative visualizations. Interestingly, the localizations achieved by our Grad-CAM technique, (c) are very similar to results from occlusion sensitivity (e), while being orders of magnitude cheaper to compute. (f, l) are Grad-CAM visualizations for ResNet-18 layer. Note that in (c, f, i, l), red regions corresponds to high score for class, while in (e, k), blue corresponds to evidence for the class. Figure best viewed in color.
PNG
media_image9.png
303
814
media_image9.png
Greyscale
”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
CN Pub no. CN112070202A Anhui Cambricon Information Technology Co Ltd (“Anhui”) teaches of an AI model including dynamic control flow; (See Anhui, Background)
NPL: R. Pan et al., "Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks," arXiv:2212.10774v1 [cs.HC] 21 Dec 2022
NPL: Trace Table. Wikipedia. Dec. 23rd, 2023. https://web.archive.org/web/20231223082055/https://en.wikipedia.org/wiki/Trace_table
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights