DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed 7/13/2026.
Priority
As indicated in the non-final Office Action mailed 3/11/2026 (hereinafter “the previous Office Action”), applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. As noted in the previous Office Action, the present application is a national stage application under 35 U.S.C. 371 of International Application No. PCT/CN2020/138906, filed on December 24, 2020.
Response to Amendment
Applicant’s amendment filed 7/13/2026 has been entered. In the amendment, claims 1-12, 15, 18 and 20 were amended, and no claims were cancelled or added. Thus, claims 1-20 are pending.
Some of the objections to the specification and drawings set forth in the previous Office Action are withdrawn in view of the 7/13/2026 amendments to the specification and drawings (i.e., the filing of replacement drawings for FIGs. 1 and 4-9). However, as documented below, objections to the drawings and specification remain.
The previous objections to claims 1-16, 18 and 20 are withdrawn in view of the amendments to the claims. However, as discussed below, objections to claims 3-6, 12-13 and 17-20 remain. These objections were necessitated by applicant’s amendments.
Claims 2, 4, 12, 13 and 18 are no longer being interpreted under 35 U.S.C. 112(f) in view of the amendments to the claims and remarks.
The rejections of claims 3-6, 12-13 and 18 under 35 U.S.C. 112(a) set forth in the previous office action are withdrawn in view of the amendments to the claims and remarks.
The previous rejections of claims 3-6, 12-13 and 18 under 35 U.S.C. 112(b) are withdrawn in view of the amendments to the claims and remarks.
Response to Arguments
Applicant's arguments with respect to the objections to the specification and drawings have been fully considered and are persuasive in part. In particular, as discussed below, some previously-identified objections to the specification and drawings are maintained.
With respect to the previous objections to the specification and drawings, applicant’s entire argument consists of stating “The Office Action objects to the specification for the informalities. Applicant has amended several paragraphs and/or figures.” and “The Office Action objects to Figures 1 and 3-9 because of the informalities. Applicant has amended several figures.” (applicant’s remarks, page 10). While the examiner acknowledges that the amendments to the specification and drawings (i.e., the filing of replacement drawings for FIGs. 1 and 4-9) addressed most of the previously-identified objections to the specification and drawings, some of the previously-identified objections were not addressed by the amendments.
Applicant's arguments with respect to the objections claims 1-16, 18 and 20 have been fully considered and are persuasive. However, as discussed below, objections to claims 3-6, 12-13 and 17-20 remain. These objections were necessitated by applicant’s amendments.
Claims are no longer being interpreted under 35 U.S.C. 112(f) in view of the amendments to the claims and remarks.
Applicant's arguments with respect to the invocation of 35 U.S.C. 112(f) in the previous office action have been fully considered and are persuasive. As indicated above, claims 2, 4, 12, 13 and 18 are no longer being interpreted under 35 U.S.C. 112(f) in view of the amendments to the claims and applicant’s remarks.
Applicant's arguments with respect to the rejections of claims 3-6, 12-13 and 18 under 35 U.S.C. 112(a) in the previous office action have been fully considered and are persuasive.
Applicant's arguments with respect to the rejections of claims 3-6, 12-13 and 18 under 35 U.S.C. 112(b) in the previous office action have been fully considered and are persuasive.
Applicant's arguments with respect to the rejections of claims 1-20 under 35 U.S.C. 101 have been fully considered, but are not persuasive.
With apparent reference to independent claims 1, 10 and 17, applicant asserts, which examiner does not concede, that “under Step 2A, Prong One of the Alice/Mayo framework, the claimed convolutional neural network (CNN) pooling operations are specific computational processes that cannot practically be performed in the human mind and do not preempt a mathematical formula.” (applicant’s remarks, page 10).
With continued, apparent reference to the independent claims, applicant asserts “Operating on input samples through convolutional layers and pooling stages involves processing complex, high-dimensional data tensors. The human mind is simply not equipped to perform these high-volume neural network processing steps. Furthermore, while the claims may involve mathematical concepts to achieve a specific result (dynamic neural network pooling), they do not explicitly recite a mathematical formula or equation to be preempted.” (applicant’s remarks, page 11).
Examiner’s response:
Regarding applicant’s above-noted arguments and assertions regarding the rejections of claims 1-20 under 35 U.S.C. §101 as being directed to an abstract idea, the Examiner respectfully disagrees with applicant’s assertions and allegations, and points applicant to the discussion below.
First, regarding the assertion that “the claimed convolutional neural network (CNN) pooling operations are specific computational processes that cannot practically be performed in the human mind and do not preempt a mathematical formula.” (applicant’s remarks, page 10), the claims are not rejected on the basis that all the limitations can “be performed in the human mind”, nor are the claims rejected on the basis that they “preempt a mathematical formula”. Rather, the claims are rejected as reciting mental processes that are concepts performed in the human mind (including an observation, evaluation, judgment, opinion) combined with a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations).
In particular, as detailed in the previous Office Action and below, the limitations recited in claims 1, 10 and 17, using respective similar language:
generating a plurality of soft weights1 based on the input sample;
performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value; and
performing conditional normalization on the aggregated value to generate an output - as drafted, under their broadest reasonable interpretation (BRI), in view of the specification, cover concepts performed in the human mind (evaluation, judgement, or opinion to generate weights based on observing received input, using the weights to generate an aggregated value by aggregating/grouping the input, combined with a mathematical concept - i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations – to normalize the aggregated value to generate an output value). The above limitations in the context of these claims encompass, inter alia, generating weights based on input, aggregating/grouping the input based on the weights to generate an aggregated value, and normalizing the aggregated value to generate an output (can be performed as mental processes, evaluation/judgement/opinion to decide on weights based on the input sample/data, aggregating/grouping the observed input data and weights and normalizing the aggregated value - corresponding to mental processes which can be done mentally or by pen and paper).
Second, contrary to applicant’s assertion regarding the purported eligibility of the claims because “Operating on input samples through convolutional layers and pooling stages involves processing complex, high-dimensional data tensors.” and because “The human mind is simply not equipped to perform these high-volume neural network processing steps.” (applicant’s remarks, page 11), none of applicant’s claims recite any “tensors” or processing thereof, much less “processing complex, high-dimensional data tensors” or performing such “high-volume neural network processing steps” relied upon by applicant.
Applicant next alleges, which examiner does not acquiesce to, that “The Examiner evaluates the claims at an excessively high level of generality and fails to recognize the specific technological improvement to the functioning of a CNN. As established in Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB September 26, 2025, Appeals Review Panel Decision), "Examiners and panels should not evaluate claims at such a high level of generality" when meaningful technical limitations are dismissed without adequate explanation. Desjardins strictly prohibits ignoring specific claim limitations that integrate an abstract idea into a practical application.” (applicant’s remarks, page 11).
Applicant further alleges that the “Examiner erroneously characterizes the CNN and data processing limitations as mere "data gathering" or "well-understood, routine, conventional activities." However, the claims recite a specific logical structure for a new CNN pooling stage: using the input sample to generate soft weights, which then dynamically drive both "conditional aggregation" and "conditional normalization."” and points to unclaimed embodiments in the specification, which are not reflected in the claims, before concluding “this specific arrangement creates a "distribution-adaptive" and "sample-aware" pooling module that preserves task-related information better than conventional pooling methods, such as static max or average pooling. This constitutes a non-abstract improvement to the internal operation and technological capability of deep convolutional neural networks.” (applicant’s remarks, pages 11-12).
With continued reference to Desjardins, applicant states “In Desjardins, the Appeals Review Panel found claims eligible because they recited a specific method of training that produced a technological improvement, such as reduced storage or prevention of forgetting.” and asserts that “Applicant's claims recite the specific steps of dynamic conditional pooling utilizing generated soft weights that achieve the technical benefit of adaptively weighting individual feature pixels regarding a local map region to enrich the expressiveness of aggregated features and dynamically adjusting the affine transform in normalization.” before concluding that “the claims … are meaningfully integrated into a practical application and are patent eligible under Step 2A, Prong Two. For at least these reasons, claims 1-20 recite patent-eligible subject” [sic – subject matter] (applicant’s remarks, page 12).
Examiner’s response:
First, regarding applicant’s general assertions that “The Examiner evaluates the claims at an excessively high level of generality and fails to recognize the specific technological improvement to the functioning of a CNN.” and “the Examiner erroneously characterizes the CNN and data processing limitations as mere "data gathering" or "well-understood, routine, conventional activities." (applicant’s remarks, page 11), Examiner disagrees with Applicant’s characterization of the 101 analysis and rejections in the previous Office Action.
Contrary to applicant’s assertions, as detailed in the previous Office Action and below, the limitations recited in claims 1, 10 and 17, using respective similar language, of: receiving an input at a convolutional layer of a convolutional neural network (CNN); receiving an input sample at a pooling stage of the convolutional layer; and generate an output for the convolutional layer - are insignificant extra-solution activities that are not integrated into the claims as a whole and do not add a meaningful limitation to the above-noted mental processes and mathematical concept specified in these claims. That is, as detailed in the previous Office Action and below, “receiving an input” at a generically-recited convolutional layer and pooling stage of a generically-recited convolutional neural network (CNN) and “generate an output for the” generically-recited convolutional layer amounts to mere data gathering (i.e., receiving provided/transmitted input data) and necessary data outputting (See MPEP § 2106.05(g)).
In contrast to applicant’s assertions, as noted in the previous Office Action and in the section 101 rejections below, receiving, transmitting and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Thus, contrary to applicant’s assertions, the recitations of “receiving an input at a convolutional layer of a convolutional neural network (CNN); receiving an input sample at a pooling stage of the convolutional layer” and “generate an output for the convolutional layer” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Second, regarding applicant’s reliance on the decision of the Appeals Review Panel in Ex parte Desjardins, No. 2024-000567 (P.T.A.B. Sept. 26, 2025), hereinafter Desjardins, in Desjardins, unlike in the claims at issue here, the appellants specifically argued that the claimed invention “address[es] challenges in continual learning and model efficiency by reducing storage requirements and preserving task performance across sequential training”. Desjardins, op. at 7. That is, the appellant in Desjardins specifically alleged that the claimed subject matter improves machine learning itself. In contrast, Applicant in the instant case does not point to any specific claim language that characterizes an improvement, and does not point to any claim language that is analogous to the claims at issue in Desjardins. Regarding the decision in Desjardins relied upon by applicant, that particular eligibility determination was grounded in a finding that the claims improved machine learning technology, rather than merely applying mathematical calculations. In contrast, the present claims do not recite the functioning of a neural network or to machine learning technology itself. The claims merely recite using “a convolutional layer of a convolutional neural network (CNN)”, without specifying any improvement to how the neural network is used a tool to perform mathematical processing, rather than being the subject of a technological improvement.
Applicant in the instant case repeatedly characterizes the improvement as pertaining to a purported “improvement to the internal operation and technological capability of deep convolutional neural networks” (applicant’s remarks, page 12, characterizing, but not citing any claim language). The claims at issue here are distinguishable from Desjardins because, as noted by Applicant, Desjardins includes an improvement related to addressing catastrophic forgetting (Id.) while the instant claims only use a generically-recited “convolutional neural network” (CNN) at a high level to perform an abstract idea and Desjardins provided a specific training strategy that allowed its model to preserve performance on earlier tasks even as it learns new ones. In contrast applicant’s instant claims merely employ a generically-recited CNN without reciting any details of its training (see, e.g., the above-noted, relied upon claim language reciting "receiving an input at a convolutional layer of a convolutional neural network (CNN)" and "receiving an input sample at a pooling stage of the convolutional layer," (applicant’s remarks, page 11, citing claim 1) and use normalization on data values at a high level merely “to generate an output for the convolutional layer”.
Desjardins is distinguishable from the present claims. In Desjardins, unlike the claims at issue here, the appellants specifically identified claim language that reflected a technological improvement. In particular, the appellants argued that, when evaluating the claim as a whole, independent claim 1 included the following limitation evidencing an improvement: "adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task". The board was persuaded that this limitation constituted an improvement to the operation of the machine learning model itself, rather than merely an abstract mathematical calculation (Desjardins, op. at 9.). Thus, the appellant in Desjardins expressly alleged and supported that the claimed subject matter improved machine learning technology itself. By contrast, in the instant application, Applicant does not identify any specific claim language characterizing a comparable technological improvement, nor does Applicant point to claim limitations analogous to those at issue in Desjardins.
Third, it is unclear how generating an aggregated value by aggregating received input samples using generated weights based on the input, and then normalizing the “aggregated value to generate “an output for a convolutional layer” achieves the purported technological improvement or advantage.
Moreover, as detailed below, the claimed operations and steps recited in claims 1, 10 and 17, under their broadest reasonable interpretation (BRI), in light of the specification, encompass mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) combined with a mathematical concept but for the recitation of generic computer components, mere instructions to apply an exception language and insignificant extra-solution activity language.
Lastly, as detailed in the section 101 rejections below, independent claims 1, 10 and 17 mainly recite generating weights based on observing received input, using the weights to generate an aggregated value by aggregating/grouping the input, combined with a mathematical concept to normalize the aggregated value to generate an output value). The above-referenced limitations in the context of these claims encompass, inter alia, generating weights based on input, aggregating/grouping the input based on the weights to generate an aggregated value, and normalizing the aggregated value to generate an output (can be performed as mental processes, evaluation/judgement/opinion to decide on weights based on the input sample/data, aggregating/grouping the observed input data and weights and normalizing the aggregated value - corresponding to mental processes which can be done mentally or by pen and paper).
If this particular arrangement provides a technological advantage, this advantage is not sufficiently reflected in the claims. At the level of detail provided, the limitations are seen to encompass a mental process combined with a mathematical concept. Thus, the claims do not recite a practical application or otherwise overcome the rejections under 101.
For the reasons stated above, the rejections of claims 1-20 under 35 U.S.C. 101 remain and are being maintained.
Applicant's arguments with respect to the rejections of claims 1-6, 9-13 and 16-18 under 35 U.S.C. 103 have been fully considered, but are not persuasive.
Applicant states “Independent claim 1 requires "generating a plurality of soft weights based on the input sample" and then "performing conditional aggregation on the input sample utilizing the plurality of soft weights." The claim clearly requires that the same generated "soft weights" be utilized to perform the conditional aggregation. In the Office Action, the Examiner relies on Luo to teach generating soft weights, pointing to Luo's importance weights (e.g., wk)” (applicant’s remarks, page 14, citing claim 1 and page 44 of the previous Office Action). Applicant then asserts, to which examiner does not acquiesce, that “The Examiner has improperly mixed two completely different, unrelated variables from two different references to satisfy a single claim element. If Luo and Wang were combined as proposed, the resulting system would use Luo's wk to combine normalization statistics and Wang's s_ri * c_ri to pool region proposals. The combination provides no teaching or suggestion to utilize Luo's normalization weights for Wang's spatial aggregation, nor does it teach utilizing a single generated set of soft weights to perform conditional aggregation.” before concluding that “the combination fails to teach generating and utilizing the same "plurality of soft weights" for conditional aggregation as claimed.” Id.
With continued reference to claim 1, applicant states “the claim recites "receiving an input sample at a pooling stage of the convolutional layer," performing conditional aggregation to generate an aggregated value, and then "performing conditional normalization on the aggregated value to generate an output for the convolutional layer."” and alleges that “Luo's Switchable Normalization normalizes standard CNN hidden feature maps, not Wang's extra-network aggregated confidence maps.” before concluding that “The proposed combination fails to teach the claimed sequence of pooling, aggregation, and normalization within a convolutional layer to generate an output for that layer. (applicant’s remarks, pages 14-15).
Examiner’s response:
The art rejections are being maintained.
As a preliminary matter, while applicant’s arguments are directly solely to the purported deficiencies of Luo and Wang, the examiner notes that the section 103 rejection of claim 1 in the previous Office Action was based on a combination of Luo, Wang and Nemlekar (i.e., Luo in view of Wang, and further in view of Nemlekar).
Applicant's arguments regarding the 35 U.S.C. 103 rejections of the claims are primarily based on the combination of Luo and Wang allegedly failing to teach "generating a plurality of soft weights based on the input sample; performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value; and performing conditional normalization on the aggregated value to generate an output for the convolutional layer." and purported inadequacies in the motivation to combine the references.
The examiner respectfully disagrees with applicant’s assertions vis-à-vis claim 1and the characterizations of Luo and Wang, and points applicant to the discussion of Luo and Wang below.
First, regarding the above-noted generating and performing operations recited in claim 1, contrary to applicant’s assertions, and as detailed in the previous Office Action and below, these limitations are taught by Luo in view of Wang. In particular, Luo discloses generating a plurality of soft weights2 based on the input sample (see, e.g., pages 5, Sect. 4.1.2, right column, last paragraph, “using only 16 and 32 samples, such that their batch sizes are the same as (8; 2) and (8; 4).” [i.e., input sample has batch sizes] and 6, Sect 4.1.3, left column, paragraph 4 “Fig.1 (a) and Fig.4 plot histograms to compare the importance weights of SN with respect to different tasks and batch sizes. … SN adapts to various scenarios by changing its importance weights. For example, SN prefers BN when the minibatch is sufficiently large [i.e., based on the size of the input sample], while it selects LN instead when small minibatch is presented” [i.e., generate changeable/updateable soft weights based on the input sample size]); … and
performing conditional normalization on the aggregated value to generate an output for the convolutional layer (see, e.g., pages 2, left column, paragraph 1 and right column, paragraphs 2 and 4, “We introduce Switchable Normalization (SN), which is applicable in … CNNs … enabling each normalization layer in a deep network to have its own operation”, “Dynamic Normalization (DN) to learn arbitrary normalization for different convolutional layers. … we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN”, 5, Sect. 4.1.1, “For each setting, the gradients are aggregated … and the means and variances of the normalization methods are computed” and 16, Appendix A, “Let h^ be the output of the SN layer” [i.e., conditional normalization on an aggregated value to generate the CNN’s convolutional layer output]).
While Luo discloses input “samples per GPU). For each setting, the gradients are aggregated over all GPUs” and “spatial pyramid pooling (i.e. one type of multi-scale global pooling)” (see, pages 5, Sect. 4.1.1 and 13, Sect 4.3), Luo is not relied on for explicitly disclosing performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value. However, as discussed in the previous Office Action and below, Wang teaches this limitation (see, e.g., page 6, paragraph 1, which explicitly discloses that “We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals to compute a confidence map Ct… The resulted confidence map Ct aggregates not only the region-wise score but also their spatial extent. … the weighted spatial average pooling is shown in Fig. 2” [reproduced below] [i.e., perform average pooling and conditional aggregation using the weights to generate an aggregated value/score]).
PNG
media_image1.png
200
400
media_image1.png
Greyscale
Wang, FIG. 2
Further, regarding “performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value”, Wang discloses a method of using spatial average pooling to aggregate input data images into pooled feature vectors (Wang, page 6, paragraph 1, which explicitly discloses “We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent.”). Paragraph 31 of applicant’s specification clearly indicates that spatial average pooling is a viable implementation of the claimed conditional aggregation operation as recited in claim 1 (and similarly recited in independent claims 10 and 17).
Second, regarding applicant’s general assertion that the “Examiner has improperly mixed two completely different, unrelated variables from two different references to satisfy a single claim element” (applicant’s remarks, page 14), and applicant’s apparent argument that a person of skill in the art would not be motivated to combine the Luo reference with the Wang reference, or that the references are not analogous art based on the above-noted, purported deficiencies in the motivation to combine Luo and Wang in the previous office action, a review of KSR and MPEP § 2141.01(a) suggests that applicant is mistaken: “Under the correct analysis, any need or problem known in the field and addressed by the patent [or application at issue] can provide a reason for combining the elements in the manner claimed.” KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 420, 82 USPQ2d 1385, 1397 (2007). This does not require that the reference be from the same field of endeavor as the claimed invention, in light of the Supreme Court's instruction that "[w]hen a work is available in one field of endeavor, design incentives and other market forces can prompt variations of it, either in the same field or a different one." Id. at 417, 82 USPQ2d 1396. Rather, a reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See Bigio, 381 F.3d at 1325, 72 USPQ2d at 1212.”
Further, even assuming arguendo that Applicant’s assertion that “If Luo and Wang were combined as proposed, the resulting system would use Luo's wk to combine normalization statistics and Wang's s_ri * c_ri to pool region proposals” is correct, which examiner does not concede, Luo’s wk value or weight is not cited, mentioned or relied upon in the section 103 rejections of claim 1, or in the rejections of independent claims 10 or 17.
Moreover, even assuming for the sake of argument that applicant’s assertions that “The Examiner has improperly mixed two completely different, unrelated variables from two different references to satisfy a single claim element” and “The combination provides no teaching or suggestion to utilize Luo's normalization weights for Wang's spatial aggregation, nor does it teach utilizing a single generated set of soft weights to perform conditional aggregation.” (applicant’s remarks, page 14) amount to an argument that a person of skill in the art would not be motivated to combine the Luo and Wang references, or that the references are not analogous art, as noted in the previous office action and below, Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. Luo teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling at a pooling stage on CNN data. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Luo and Wang by pooling Luo’s input data before further processing. Doing so would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with Luo’s normalization and Wang’s pooling performing the same together as they did separately. (See, MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Regarding independent claims 10 and 17 and the dependent claims, applicant generally asserts “Independent claims 10 and 17 recite similar limitations and are patentable over the cited art for the same reasons. The dependent claims are patentable at least by virtue of their dependence on an allowable base claim. Therefore, Applicant respectfully requests the withdrawal of the § 103 rejection of claims 1-6, 9-13, and 16-18.” (applicant’s remarks, page 15).
Examiner’s response:
The examiner respectfully disagrees with applicant’s general assertions regarding the purported novelty of independent claims 10 and 17 and “the dependent claims” (i.e., claims 2-6, 9, 11-13, and 17-18).
As discussed below, the combination of Luo in view of Wang and further in view of Nemlekar teaches all the recited limitations of independent claims 1, 10 and 17.
As further discussed in detail below, the applied combination of references (i.e., Luo in view of Wang and further in view of Nemlekar) teaches all the limitations of dependent claims 2-6, 9, 11-13, and 17-18.
For the reasons stated above, the rejections of claims 1-6, 9-13 and 16-18 under 35 U.S.C. 103 remain and are being maintained.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference characters not mentioned in the description:
Reference characters 116 shown in Figure 1 are not found in the detailed description (see, paragraph 22 describing FIG. 1).
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Reference characters 116 shown in Figure 1 are not found in the detailed description (see, e.g., paragraph 22 describing FIG. 1). Appropriate correction is required.
Claim Objections
Claims 3-6, 12-13 and 17-20 are objected to because of the following informalities:
Claims 3 and 12, which depend directly from claims 2 and 11, respectively, both recite “wherein the at least one soft agent is to perform:” (see, lines 1-2 of claims 3 and 12). Amended intervening claims 2 and 11 previously introduced “at least one soft agent module stored in memory to be executed by the one or more processors”. As such, for clarity and consistency, the subsequent recitations of “the at least one soft agent” in claims 3 and 12 should recite “the at least one soft agent module stored in the memory to be executed by the one or more processors” (see, e.g., lines 1-2 of claim 18).
Lines 1-2 of claim 18, which depends directly from claim 17, recites “wherein the at least one soft agent module stored in the memory to be executed by the one or more processors is to perform:”; however, claim 17 recites “one or more processors;” in line 2, “a memory” in line 4 and “at least one soft agent;” in line 11. As such, for clarity and to provide proper antecedent basis for the subsequently-recited “at least one soft agent module stored in the memory to be executed by the one or more processors” in dependent claim 18, “at least one soft agent;” in line 11 of independent claim 17 should recite “at least one soft agent module stored in the memory to be executed by the one or more processors;”. Appropriate correction is required.
Also, claims 4-6, 13 and 18-20 which each depend directly or indirectly from claims 3, 12 and 17, respectively, are objected to based on their respective dependencies from claims 3, 12 and 17.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding claims 1, 10 and 17, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to one or more non-transitory computer-readable storage media, corresponding to an article of manufacture, claim 10 is directed to an apparatus, and claim 17 is directed to a system, corresponding to machines, which are each one of the statutory categories.
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including an observation, evaluation, judgment, opinion) combined with a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations).
The limitations recited in claims 1, 10 and 17, using respective similar language:
generating a plurality of soft weights3 based on the input sample;
performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value; and
performing conditional normalization on the aggregated value to generate an output - as drafted, under their broadest reasonable interpretation (BRI), in view of the specification, cover concepts performed in the human mind (evaluation, judgement, or opinion to generate weights based on observing received input, using the weights to generate an aggregated value by aggregating/grouping the input, combined with a mathematical concept - i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations – to normalize the aggregated value to generate an output value). The above limitations in the context of these claims encompass, inter alia, generating weights based on input, aggregating/grouping the input based on the weights to generate an aggregated value, and normalizing the aggregated value to generate an output (can be performed as mental processes, evaluation/judgement/opinion to decide on weights based on the input sample/data, aggregating/grouping the observed input data and weights and normalizing the aggregated value - corresponding to mental processes which can be done mentally or by pen and paper).
The claim limitations, under their broadest reasonable interpretations (BRIs), cover performance of the limitations in the mind but for the recitation of generic computer components (e.g., “One or more non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” (claim 1), “An apparatus comprising: one or more processors; and a memory to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers, wherein the one or more processors are to:” <perform operations> (claim 10) and “A computing system comprising: one or more processors; a data storage to store data including instructions for the one or more processors; and a memory including random access memory (RAM) to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers, wherein the computing system is to:” <perform operations> “at least one soft agent” (claim 17)) combined with a mathematical concept - mathematical relationships, mathematical formulas or equations, or mathematical calculations (e.g., “performing conditional normalization on the aggregated value”).
Therefore, the claims are directed to an abstract idea - mental processes combined with a mathematical concept.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite, using respective similar language, the additional elements: receiving an input at a convolutional layer of a convolutional neural network (CNN);
receiving an input sample at a pooling stage of the convolutional layer; and
generate an output for the convolutional layer.
These are insignificant extra-solution activities that are not integrated into the claims as a whole and do not add a meaningful limitation to the above-noted mental processes and mathematical concept specified in these claims. That is, “receiving an input” at a generically-recited convolutional layer and pooling stage of a generically-recited convolutional neural network and “generate an output for the” generically-recited convolutional layer amounts to mere data gathering (i.e., receiving provided/transmitted input data) and necessary data outputting (See MPEP § 2106.05(g)).
The claims also recite the additional elements: “One or more non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” (claim 1), “An apparatus comprising: one or more processors; and a memory to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers, wherein the one or more processors are to:” <perform operations> (claim 10) and “A computing system comprising: one or more processors; a data storage to store data including instructions for the one or more processors; and a memory including random access memory (RAM) to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers, wherein the computing system is to:” <perform operations> “wherein the plurality of soft weights are generated by at least one soft agent” (claim 17). The computer-readable medium storing instructions, apparatus and system comprising one or more processors and memory, convolutional neural network (CNN) and soft agent are recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., a computer including generically-recited one or more processors, computer-readable medium and memory storing instructions and the generically-recited CNN) and soft agent amount to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Regarding the “convolutional neural network (CNN)”, aside from stating “the CNN having a plurality of layers including one or more convolutional layers”, no details of the neural network are recited, and the neural network is recited at a high level of generality and the network can be constructed by hand with pen and paper. Thus, the claimed “neural network”, under the BRI, in light of the specification, could be any neural network “having a plurality of layers including one or more convolutional layers”, which could be constructed and updated/retrained by hand with pen and paper. That is, the “neural network” limitations give the indication that the neural network can be constructed by hand with pen and paper.
The neural network is recited at a high level of generality and therefore is being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Receiving, transmitting and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitations of “receiving an input at a convolutional layer of a convolutional neural network (CNN); receiving an input sample at a pooling stage of the convolutional layer” and “generate an output for the convolutional layer” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited “non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of claim 1, and apparatus and system comprising “one or more processors” and a memory and a data storage of claims 10 and 17) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 2 and 11, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a non-transitory computer-readable storage medium as depending from claim 1, and claim 11 is directed to an apparatus as depending from claim 10, thus the analysis for patent eligibilities of claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims both recite “wherein the plurality of soft weights are generated by at least one soft agent”.
This limitation does nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what generating the soft weights includes, i.e., using a generically-recited “soft agent.”
Dependent claims 2 and 11, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims covers a mental process of evaluation, judgement, or opinion to generate weights based on observing received input. This can be performed as a mental process. A person can merely decide on weights based on observing the input data.
Thus, this limitation does nothing to alter the analysis of claims 1 and 10.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite the additional element: “the plurality of soft weights are generated by at least one soft agent module stored in memory to be executed by the one or more processors.”
The “by at least one soft agent module stored in memory to be executed by the one or more processors” amounts to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited “soft agent module stored in memory to be executed by the one or more processors”, the “non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1, and the apparatus comprising “one or more processors” and memory of base claim 10) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of these dependent claims is not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 3, 12 and 18, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a non-transitory computer-readable storage medium as depending from claim 2, claim 12 is directed to an apparatus as depending from claim 11, and claim 18 is directed to a system as depending from claim 17, thus the analysis for patent eligibilities of claims 2, 11 and 18, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims each recite “global aggregation of the input sample to aggregate the input sample along all but one input dimension;
mapping of the aggregated input sample to generate a mapped input sample; and
scaling of the mapped input sample to generate the plurality of soft weights.”
These limitations do nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept. This is because the additional limitations merely limit the invention to a narrower abstract idea by further narrowing what generating the soft weights includes, i.e., using a generically-recited “soft agent” to aggregate the observed input sample/data, map/correlate the input sample to create/generate a mapped input sample, and then a mathematical concept to scale the mapped input to generate weights.
Dependent claims 3, 12 and 18, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The additional limitations added by these claims cover a mental process of evaluation, judgement, or opinion to generate weights based on observing received input by aggregating the observed input sample/data and to map/correlate the input sample (corresponding to mental processes which can be done mentally or by pen and paper), combined with a mathematical concept - scaling the mapped input (corresponding to a mathematical concept - mathematical relationships, mathematical formulas or equations, or mathematical calculations).
Thus, these limitations do nothing to alter the analysis of claims 2, 11 and 17.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite the additional element: “wherein the at least one soft agent4 is to perform:” <the above-noted operations/mental processes combined with a mathematical concept>
The “the at least one soft agent is to perform:” <the operations> amounts to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to implement the abstract idea electronically (i.e., with the recited “soft agent”, the “non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1, and the apparatus and system comprising “one or more processors” and a memory and a data storage of base claims 10 and 17) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 4 and 13, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a non-transitory computer-readable storage medium as depending from claim 3 and claim 13 is directed to an apparatus as depending from claim 12, thus the analysis for patent eligibilities of claims 2-3 and 11-12, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims both recite “wherein the at least one soft agent includes a first soft agent to support the conditional aggregation and a second soft agent5 to support the conditional normalization.”
These limitations do nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept. This is because the additional limitations merely limit the invention to a narrower abstract idea by further narrowing what the conditional aggregation and normalization include, i.e., using a generically-recited 1st and 2nd “soft agent”.
Dependent claims 4 and 13, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The additional limitations added by these claims cover a mental process of evaluation, judgement, or opinion to conditionally aggregate a data value based on observing received input sample/data (corresponding to mental processes which can be done mentally or by pen and paper), and then conditionally normalize the aggregated value (corresponding to a mathematical concept - mathematical relationships, mathematical formulas or equations, or mathematical calculations).
Thus, these limitations do nothing to alter the analysis of claims 1-3 and 10-12.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite the additional elements: “wherein the at least one soft agent includes a first soft agent to support” <the above-noted mental process> “and a second soft agent6 to support” <the above-noted mathematical concept>.
The 1st and 2nd soft agent “to support” <the above-noted mental process and mathematical concept> amount to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited 1st and 2nd “soft agent”, the “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1, and the apparatus comprising “one or more processors” and memory of base claim 10) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claim 5, this claims is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a non-transitory computer-readable storage medium as depending from claim 4, thus the analysis for patent eligibilities of claims 2-4, and of base claim 1 are incorporated herein.
Step 2A Prong 1: The claim recites “wherein the first soft agent includes a fully connected layer for mapping and a layer for scaling.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the 1st soft agent includes, i.e., “a fully connected layer for mapping and a layer for scaling.”
Dependent claim 5, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of evaluation, judgement, or opinion to map data based on observing received input sample/data (corresponding to mental processes which can be done mentally or by pen and paper), and scaling data (corresponding to a mathematical concept - mathematical relationships, mathematical formulas or equations, or mathematical calculations). Also, the limitation includes intended use language with no patentable weight (e.g., “layer for mapping and a layer for scaling.”).
Thus, this limitation does nothing to alter the analysis of claims 1-4.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claim recites the additional elements: “a fully connected layer for” <the above-noted mental process> “and a layer for” <the above-noted mathematical concept>.
The generically-recited layers “for” performing <the above-noted mental process and mathematical concept> amount to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B Analysis: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited layers and the “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
This claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of this dependent claim is not sufficient to amount to significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 6, this claims is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a non-transitory computer-readable storage medium as depending from claim 4, thus the analysis for patent eligibilities of claims 2-4, and of base claim 1 are incorporated herein.
Step 2A Prong 1: The claim recites “wherein the second soft agent includes a long short-term memory (LSTM) block to provide mapping and scaling.”
This limitation does nothing to alter the fundamental nature of the claim as a mental process combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what the 2nd soft agent includes, i.e., “a long short-term memory (LSTM) block to provide mapping and scaling.”
Dependent claim 6, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. The additional limitation added by this claim covers a mental process of evaluation, judgement, or opinion to map data based on observing received input sample/data (corresponding to mental processes which can be done mentally or by pen and paper), and scaling data (corresponding to a mathematical concept - mathematical relationships, mathematical formulas or equations, or mathematical calculations). Also, the limitation includes intended use language with no patentable weight (e.g., “block to provide mapping and scaling.”).
Thus, this limitation does nothing to alter the analysis of claims 1-4.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claim recites the additional elements: “a long short-term memory (LSTM) block to provide” <the above-noted mental process and mathematical concept>.
The generically-recited LSTM memory block “to provide” performing <the above-noted mental process and mathematical concept> amount to the recitation of the words “apply it” (or an equivalent) or amount to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B Analysis: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited LSTM memory block and the “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
This claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of this dependent claim is not sufficient to amount to significantly more than the abstract idea. This claim is not patent eligible.
Regarding claims 7, 14 and 19, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a non-transitory computer-readable storage medium as depending from claim 1, claim 14 is directed to an apparatus as depending from claim 10, and claim 19 is directed to a system as depending from claim 17, thus the analysis for patent eligibilities of claims 1, 10 and 17 are incorporated herein.
Step 2A Prong 1: The claims each recite “wherein performing the conditional aggregation includes: … weighting an output of each of the convolutional filters with a respective soft weight of the plurality of soft weights.”
This limitation does nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept. This is because the additional limitation merely limits the invention to a narrower abstract idea by further narrowing what “performing the conditional aggregation includes”, i.e., “weighting an output of each of the convolutional filters with a respective soft weight of the plurality of soft weights.”
Dependent claims 7, 14 and 19, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims covers a mental process of evaluation, judgement, or opinion to weight an observed output of each of the generically-recited convolutional filters with a respective soft weight of an observed set of soft weights (corresponding to mental processes which can be done mentally or by pen and paper).
Thus, this limitation does nothing to alter the analysis of claims 1, 10 and 17.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite the additional element: “receiving the input sample at a plurality of convolutional kernels for a plurality of convolutional filters.”
This is an insignificant extra-solution activity that is not integrated into the claims as a whole and does not add a meaningful limitation to the above-noted mental processes and mathematical concept specified in these claims. That is, “receiving the input sample” at a generically-recited “plurality of convolutional kernels for a plurality of convolutional filters” amounts to mere data gathering (i.e., receiving provided/transmitted input data) (See MPEP § 2106.05(g)).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Receiving, transmitting and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitations of “receiving the input sample at a plurality of convolutional kernels for a plurality of convolutional filters” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the recited “convolutional kernels for a plurality of convolutional filters”, “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of claim 1, and apparatus and system comprising “one or more processors” and a memory and a data storage of claims 10 and 17) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 8, 15 and 20, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a non-transitory computer-readable storage medium as depending from claim 1, claim 15 is directed to an apparatus as depending from claim 10, and claim 20 is directed to a system as depending from claim 17, thus the analysis for patent eligibilities of claims 1, 10 and 17 are incorporated herein.
Step 2A Prong 1: The claims each recite “wherein performing the conditional normalization includes:
performing standardization to generate a standardized representation of a feature map; and
performing an affine transform to re-scale and re-shift the standardized feature map7.”
These limitations do nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept. This is because the additional limitations merely limit the invention to a narrower abstract idea by further narrowing what “performing the conditional normalization includes”, i.e., “performing standardization to generate a standardized representation of a feature map; and performing an affine transform to re-scale and re-shift the standardized feature map.”
Dependent claims 8, 15 and 20, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The additional limitations added by these claims cover mathematical concepts - generating a standardized representation of a feature map via standardization and then performing an affine transform to re-scale and re-shift the standardized feature map (corresponding to mathematical concepts - mathematical relationships, mathematical formulas or equations, or mathematical calculations).
Thus, these limitations do nothing to alter the analysis of claims 1, 10 and 17.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Mere instructions to apply the mathematical concept electronically do not meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Mere instructions to apply the mathematical concepts electronically (i.e., with the “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of claim 1, and apparatus and system comprising “one or more processors” and a memory and a data storage of claims 10 and 17) do not amount to significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements of these dependent claims are not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Regarding claims 9 and 16, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a non-transitory computer-readable storage medium as depending from claim 1, and claim 16 is directed to an apparatus as depending from claim 10, thus the analysis for patent eligibilities of claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, “performing convolution and detection to generate the input sample from the input received”.
This limitation does nothing to alter the fundamental nature of the claims as a mental process combined with a mathematical concept.
Dependent claims 9 and 16, when analyzed as a whole, are not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claims are not directed to an abstract idea. The additional limitation added by these claims covers a mental process of evaluation, judgement, or opinion to generate/detect the input sample based on the received/observed input. This can be performed as a mental process. A person can merely generate an input sample by detecting/identifying the sample based on observing the received input data.
Thus, this limitation does nothing to alter the analysis of claims 1 and 10.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite the additional element: “the input received at the convolutional layer.”
This is an insignificant extra-solution activity that is not integrated into the claims as a whole and does not add a meaningful limitation to the above-noted mental processes and mathematical concept specified in these claims. That is, “the input received” at a generically-recited “convolutional layer” amounts to mere data gathering (i.e., receiving provided/transmitted input data) (See MPEP § 2106.05(g)).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Receiving, transmitting and communicating data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitations of “the input received at the convolutional layer” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Mere instructions to apply the mental process electronically, or to implement an abstract idea or other exception on a computer (i.e., with the “non-transitory computer-readable storage mediums [sic – media] having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations” of base claim 1, and the apparatus comprising “one or more processors” and memory of base claim 10) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, there are no additional elements recited that impose any meaningful limits on practicing the abstract idea. Therefore, the additional element of these dependent claims is not sufficient to amount to significantly more than the abstract idea. These claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 9-13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Luo et al. (“Switchable Normalization for Learning-to-Normalize Deep Representation”, 2019, arXiv:1907.10473v1, hereinafter referred “Luo”) in view of non-patent literature Wang et al. (“Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation”, 2016, arXiv:1606.02280v1, hereinafter “Wang”) and further in view of Nemlekar et al. (U.S. Patent Application Pub. No. 2020/0192631 A1, hereinafter “Nemlekar”).
With respect to claim 1, Luo discloses the invention as claimed including instructions that, when executed by one or more processors, cause the one or more processors to perform operations (see, e.g., Abstract, “We address a learning-to-normalize problem by proposing Switchable Normalization (SN), which learns to select different normalizers for different normalization layers of a deep neural network. … maintaining high performance even when small minibatch is presented (e.g. 2 images/GPU). … The code of SN has been released” and pages 1-2, Sect. 1, “The gradients in training are averaged over all GPUs and the statistics of normalizers are estimated in each GPU.”, “We make the code of SN available” [i.e., executable software code/instructions causing a GPU/processor to perform operations]) comprising:
receiving an input at a convolutional layer of a convolutional neural network (CNN) (see, e.g., pages 2, Sect. 1, right column, “learn arbitrary normalization for different convolutional layers.”, 3, Sect. 3, left column, paragraph 6, “We take CNN as an illustrative example. Let h be the input data of an arbitrary normalization layer”, 13, Sect. 4.7, right column, “a convolutional neural network (CNN) is constructed by stacking multiple convolutional cells.” and 15, Sect. 5, right column, “it is valuable to design the algorithm to lean [sic – learn] arbitrary normalization operations for different convolutional layers in a deep ConvNet.” [i.e., receive input data at a convolutional layer of a CNN]);
receiving an input sample at a … stage of the convolutional layer (see, e.g., pages 3, Sects 2-3, left column, paragraphs 2, 4 and 6, “SN trained with a single stage”, “SN can be generally optimized within a single stage in the same dataset”, “Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N, C, H, W), indicating number of samples, number of channels, height and width of a channel respectively”, page 13, Sect. 4.7, right column, “a convolutional neural network (CNN) is constructed by stacking multiple convolutional cells.” and page 15, Sect. 5, right column, “it is valuable to design the algorithm to lean [sic – learn] arbitrary normalization operations for different convolutional layers in a deep ConvNet.” [i.e., receive input sample at a stage of the convolutional layer]);
generating a plurality of soft weights8 based on the input sample (see, e.g., pages 5, Sect. 4.1.2, right column, last paragraph, “using only 16 and 32 samples, such that their batch sizes are the same as (8; 2) and (8; 4).” [i.e., input sample has batch sizes] and 6, Sect 4.1.3, left column, paragraph 4 “Fig.1 (a) and Fig.4 plot histograms to compare the importance weights of SN with respect to different tasks and batch sizes. … SN adapts to various scenarios by changing its importance weights. For example, SN prefers BN when the minibatch is sufficiently large [i.e., based on the size of the input sample], while it selects LN instead when small minibatch is presented” [i.e., generate changeable/updateable soft weights based on the input sample size]); … and
performing conditional normalization on the aggregated value to generate an output for the convolutional layer (see, e.g., pages 2, left column, paragraph 1 and right column, paragraphs 2 and 4, “We introduce Switchable Normalization (SN), which is applicable in … CNNs … enabling each normalization layer in a deep network to have its own operation”, “Dynamic Normalization (DN) to learn arbitrary normalization for different convolutional layers. … we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN”, 5, Sect. 4.1.1, “For each setting, the gradients are aggregated … and the means and variances of the normalization methods are computed” and 16, Appendix A, “Let h^ be the output of the SN layer” [i.e., conditional normalization on an aggregated value to generate the CNN’s convolutional layer output]).
Although Luo substantially discloses the claimed invention, and Luo discloses input “samples per GPU). For each setting, the gradients are aggregated over all GPUs” and “spatial pyramid pooling (i.e. one type of multi-scale global pooling)” (see, pages 5, Sect. 4.1.1 and 13, Sect 4.3), Luo is not relied on for explicitly disclosing receiving an input sample at a pooling stage of the convolutional layer; and
performing conditional aggregation on the input sample utilizing the plurality of soft weights to generate an aggregated value.
In the same field, analogous art Wang teaches receiving an input sample at a pooling stage of the convolutional layer (see, e.g., Fig. 2, reproduced below, an “illustration of the weighted spatial average pooling strategy” where an input sample is received and pages 5, “VGG-16 net uses 3×3 convolution interleaved with max pooling and 3 fully-connected layers. … firstly warp the image data in each region into a form that is compatible with the CNN (VGG-16 net requires inputs” [i.e., receiving input sample a convolutional layer of the CNN] and 6, paragraph 1 “Spatial Average Pooling After the initial discovery, a large number of region proposals are positively detected … We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals to compute a confidence map” [i.e., receive input sample at a pooling stage of the layer after initial discovery/input stage]);
PNG
media_image1.png
200
400
media_image1.png
Greyscale
Wang, FIG. 2
performing conditional aggregation on the input sample utilizing the plurality of soft weights9 to generate an aggregated value (see, e.g., pages 6, paragraph 1 “We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals to compute a confidence map Ct… The resulted confidence map Ct aggregates not only the region-wise score but also their spatial extent. … the weighted spatial average pooling is shown in Fig. 2” [i.e., perform average pooling and conditional aggregation using the weights to generate an aggregated value/score]).
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. Luo teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling at a pooling stage on CNN data. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Luo and Wang by pooling Luo’s input data before further processing. Doing so would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with Luo’s normalization and Wang’s pooling performing the same together as they did separately. (See, MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Although Luo in view of Wang substantially teaches the claimed invention, Luo in view of Wang is not relied on to teach One or more non-transitory computer-readable storage mediums having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations.
In the same field, analogous art Nemlekar teaches One or more non-transitory computer-readable storage mediums10 having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations (see, e.g., paragraphs 28-29, “A computer readable storage medium may include any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system”, “aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.”).
Nemlekar relates to normalization in convolutional neural networks and is analogous to the claimed invention. Luo in view of Wang teaches techniques, methods and an apparatus for performing normalization in convolutional neural networks. The claimed invention improves upon these methods by storing them in the form of instructions on computer hardware. Nemlekar teaches computer hardware for normalization in CNNs, applicable to Luo. Before the effective filing date of the claimed invention, one of ordinary skill in the art would have recognized that storing Luo in view of Wang’s techniques/methods as computer-executable instructions on Nemlekar’s hardware would lead to the predictable result of the method being executable by a computing system, and would improve the known device by allowing it to be performed with real data (See, MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
With respect to independent claim 10, claim 10 is substantially similar to claim 1 and therefore is rejected on the same grounds as claim 1, discussed above. In particular, claim 10 is an apparatus claim with operations that correspond to the operations of claim 1.
Although Luo in view of Wang substantially teaches the claimed invention and Wang discloses “We implement our method using MATLAB and C/C++, with Caffe … on a commodity desktop with a Quad-Core 4.0 GHz processor, 16 GB of RAM, and GTX 980 GPU” (see, page 10, Sect. 3.3), Luo in view of Wang is not relied on to teach an apparatus comprising: one or more processors; and a memory to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers.
In the same field, analogous art Nemlekar teaches an apparatus comprising: one or more processors; and a memory to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers (see, e.g., paragraphs 11, “the CNN includes layers … operations to implement the CNN can be grouped into 3 categories, or phases: a convolution phase, a batch normalization (BN) phase, and an activation phase, … phases can be repeated for each layer of the CNN”, 26, “fusing a convolution phase of a CNN with a reduction phase of a batch normalization phase of the CNN” [i.e., a CNN with multiple layers and a convolution layer] and 29 “aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.”).
Nemlekar relates to normalization in convolutional neural networks and is analogous to the claimed invention. Luo in view of Wang teaches techniques, methods and an apparatus for performing normalization in convolutional neural networks. The claimed invention improves upon these methods by storing them in the form of instructions on computer hardware. Nemlekar teaches computer hardware for normalization in CNNs, applicable to Luo. Before the effective filing date of the claimed invention, one of ordinary skill in the art would have recognized that storing Luo in view of Wang’s techniques/methods as computer-executable instructions on Nemlekar’s hardware would lead to the predictable result of the method being executable by a computing system, and would improve the known device by allowing it to be performed with real data (See, MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claims 2 and 11, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 1 and the apparatus of claim 10.
Luo further discloses wherein the plurality of soft weights11 are generated by at least one soft agent12 module stored in memory to be executed by the one or more processors (see, e.g., page 3, Sect. 2, left column, “a set of modules with different numbers of parameters and computational complexities … choose the module with large complexity to overfit training data”, page 4, Sect. 3.1, left column, equation (5), paragraph 2, “Each wk or w’k is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. …
PNG
media_image2.png
87
563
media_image2.png
Greyscale
Here each wk [i.e., soft weight] is computed by using a softmax function with λin, λln, and λbn as the control parameters, which can be learned by back-propagation (BP). w’k are defined similarly by using another three control parameters λ’in, λ’ln, and λ’bn.” and 6, Sect 4.1.3, left column, paragraph 4 “Fig.1 (a) and Fig.4 plot histograms to compare the importance weights of SN with respect to different tasks and batch sizes. … SN adapts to various scenarios by changing its importance weights.” and page 8, Sect 4.2, right column, “memory footprint and therefore leading to small minibatch size, such as 2 samples per GPU … we employ a fast implementation … we train all models on 4 GPUs” [i.e., updateable/changeable soft weights wk and w’k are computed/generated by a soft agent/module executed by one or more GPUs/processors]).
Additionally or alternatively, in the same field, analogous art Nemlekar also teaches a module stored in memory to be executed by the one or more processors (see, e.g., paragraphs 14 and 16, “the CUs [compute units] of the GPU 100 are logically divided into different groups, (e.g. CU group 102). … the logical grouping of the CUs, and the designation of the corresponding operations, is performed by a … control module that assigns operations to the individual CUs. … the CU s of the GPU 100 are assigned to more than one group, such that a CU may be assigned to perform different designated operations” and “the matrix B represents the weights for a node of the CNN. Further, the matrices A and B are composed (e.g., by the GPU 100 or other processor module)” [i.e., a module executed by one or more GPUs/processors]).
The motivation to combine Luo, Wang and Nemlekar is the same as discussed above with respect to claims 1 and 10.
Regarding claims 9 and 16, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 1 and the apparatus of claim 10.
Luo further discloses performing convolution and detection to generate the input sample from the input received at the convolutional layer (see, e.g., pages 2, Sect. 2, right column, “learn arbitrary normalization for different convolutional layers”, 3, Sect. 3, left column, “We take CNN as an illustrative example. Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N;C;H;W), indicating number of samples, number of channels, height and width of a channel respectively” and right column, “a mean value and a variance value are computed in (C;H;W) for each one of the N samples”, 8, Sects. 4.2-4.2.1, right column, “SN selects different operations in different components of a detection system”, “we implement object detection … on existing detection softwares of PyTorch and Caffe2-Detectron … Faster R-CNN … R-CNN+FPN … Mask R-CNN” [i.e., performing convolution and detection operations], 13, Sect. 4.7, right column, “a convolutional neural network (CNN) is constructed by stacking multiple convolutional cells.” and 14-15, Sect. 5, “This work has demonstrated SN in multiple tasks of CV [computer vision] such as recognition, detection … it is valuable to design the algorithm to lean [sic – learn] arbitrary normalization operations for different convolutional layers in a deep ConvNet.” [i.e., perform convolution and detection to generate the input sample N from input data received at the convolutional layer]).
With respect to independent claim 17, claim 17 is substantially similar to claim 1 and therefore is rejected on the same grounds as claim 1, discussed above. In particular, claim 17 is an system claim with operations that correspond to the operations of claim 1.
Luo further discloses one or more processors; a data storage to store data including instructions for the one or more processors (see, e.g., Abstract, “We address a learning-to-normalize problem by proposing Switchable Normalization (SN), which learns to select different normalizers for different normalization layers of a deep neural network. … maintaining high performance even when small minibatch is presented (e.g. 2 images/GPU). … The code of SN has been released” and pages 1-2, Sect. 1, “The gradients in training are averaged over all GPUs and the statistics of normalizers are estimated in each GPU.”, “We make the code of SN available” [i.e., software code/instructions for one or more GPUs/processors]),
wherein the plurality of soft weights13 are generated by at least one soft agent14 (see, e.g., pages 4, Sect. 3.1, left column, equation (5), paragraph 2, “Each wk or w’k is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. …
PNG
media_image2.png
87
563
media_image2.png
Greyscale
Here each wk [i.e., soft weight] is computed by using a softmax function with λin, λln, and λbn as the control parameters, which can be learned by back-propagation (BP). w’k are defined similarly by using another three control parameters λ’in, λ’ln, and λ’bn.” and 6, Sect 4.1.3, left column, paragraph 4 “Fig.1 (a) and Fig.4 plot histograms to compare the importance weights of SN with respect to different tasks and batch sizes. … SN adapts to various scenarios by changing its importance weights.” [i.e., updateable/changeable soft weights wk and w’k are computed/generated by a soft agent/function]).
Although Luo in view of Wang substantially teaches the claimed invention, and Wang discloses “We implement our method using MATLAB and C/C++, with Caffe … on a commodity desktop with a Quad-Core 4.0 GHz processor, 16 GB of RAM, and GTX 980 GPU” (see, page 10, Sect. 3.3), Luo in view of Wang is not relied on to teach a computing system comprising: one or more processors; a data storage to store data including instructions for the one or more processors; and a memory including random access memory (RAM) to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers.
In the same field, analogous art Nemlekar teaches a computing system comprising: one or more processors; a data storage to store data including instructions for the one or more processors; and a memory including random access memory (RAM) to store data, including data of a convolutional neural network (CNN), the CNN having a plurality of layers including one or more convolutional layers (see, e.g., paragraphs 11, “the CNN includes layers … operations to implement the CNN can be grouped into 3 categories, or phases: a convolution phase, a batch normalization (BN) phase, and an activation phase, … phases can be repeated for each layer of the CNN”, 26, “fusing a convolution phase of a CNN with a reduction phase of a batch normalization phase of the CNN” [i.e., a CNN with multiple layers and a convolution layer], and 29, “aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.”).
Nemlekar relates to normalization in convolutional neural networks and is analogous to the claimed invention. Luo in view of Wang teaches techniques, methods and an apparatus for performing normalization in convolutional neural networks. The claimed invention improves upon these methods by storing them in the form of instructions on computer hardware. Nemlekar teaches computer hardware for normalization in CNNs, applicable to Luo. Before the effective filing date of the claimed invention, one of ordinary skill in the art would have recognized that storing Luo in view of Wang’s techniques/methods as computer-executable instructions on Nemlekar’s hardware would lead to the predictable result of the method being executable by a computing system, and would improve the known device by allowing it to be performed with real data (See, MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claims 3, 12 and 18, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 2, the apparatus of claim 11, and the system of claim 17.
Luo further discloses wherein the at least one soft agent15 is to perform: … scaling of the mapped input sample to generate the plurality of soft weights (see, e.g., page 3, Table 1, “
γ
,
β
denote the scale and shift parameters … wk is an importance weight of each kind”, and right column, paragraphs 1 and 4-5, “
γ
and
β
are a scale and a shift parameter respectively … each pixel is normalized by using μ and Ϭ, and then re-scale and re-shift by
γ
and
β
.”, “mean value and a variance value are computed in (C;H;W) for each one of the N samples.”, “normalizing the hidden feature maps of CNNs.” [i.e., mapped input samples/images with pixels], and 4, equation (3), Sect. 3.1, left column, paragraphs 1-2, “SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω
is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}
the same as above where
μ
k
and
σ
k
can be calculated by following Eqn. (2).”, “each wk [i.e., soft weight] is computed by using a softmax function with λin, λln, and λbn as the control parameters, which can be learned by back-propagation (BP). w’k are defined similarly by using another three control parameters λ’in, λ’ln, and λ’bn.” [i.e., scaling input sample to generate/compute updateable/changeable soft weights wk and w’k]).
Although Luo substantially discloses the claimed invention, and Luo discloses input “samples per GPU). For each setting, the gradients are aggregated over all GPUs” and “spatial pyramid pooling (i.e. one type of multi-scale global pooling)” (see, pages 5, Sect. 4.1.1 and 13, Sect 4.3), Luo is not relied on for explicitly disclosing global aggregation of the input sample to aggregate the input sample along all but one input dimension; and
mapping of the aggregated input sample to generate a mapped input sample.
In the same field, analogous art Wang teaches global aggregation of the input sample to aggregate the input sample along all but one input dimension (see, e.g., pages 6, paragraph 1 “We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals [i.e., global aggregation and pooling] to compute a confidence map Ct… The resulted confidence map Ct aggregates not only the region-wise score but also their spatial extent. … the weighted spatial average pooling is shown in Fig. 2” [i.e., perform global aggregation/over all the proposals of the input sample/image data/frames to aggregate the input sample along all but one spatial dimension of the input image]); and
mapping of the aggregated input sample to generate a mapped input sample (see, e.g., pages 6, paragraph 1 “aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals to compute a confidence map Ct… The resulted confidence map Ct aggregates not only the region-wise score but also their spatial extent.” [i.e., map the aggregated input image frame/sample to generate/create a mapped input sample]).
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. Luo teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling at a pooling stage on CNN data. Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Luo and Wang by pooling Luo’s input data before further processing. Doing so would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with Luo’s normalization and Wang’s pooling performing the same together as they did separately. (See, MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Regarding claims 4 and 13, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 3 and the apparatus of claim 12.
Luo further discloses wherein the at least one soft agent includes … a second soft agent16 to support the conditional normalization (see, e.g., pages 2, left column, paragraph 1 and right column, paragraphs 2 and 4, “We introduce Switchable Normalization (SN), which is applicable in … CNNs … enabling each normalization layer in a deep network to have its own operation”, “Dynamic Normalization (DN) to learn arbitrary normalization for different convolutional layers. … we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN”, 4, Sect. 3.1, left column, paragraph 2, “each wk is computed by using a softmax function with λin, λln, and λbn as the control parameters” and 5, Sect. 4.1.1, “For each setting, the gradients are aggregated … and the means and variances of the normalization methods are computed” [i.e., conditional normalization is supported/computed by a 2nd soft agent/function computing method for the normalization]).
Although Luo substantially discloses the claimed invention, and Luo discloses input “samples per GPU). For each setting, the gradients are aggregated over all GPUs” and “spatial pyramid pooling (i.e. one type of multi-scale global pooling)” (see, pages 5, Sect. 4.1.1 and 13, Sect 4.3), Luo is not relied on for explicitly disclosing wherein the at least one soft agent includes a first soft agent to support the conditional aggregation.
In the same field, analogous art Wang teaches wherein the at least one soft agent includes a first soft agent17 to support the conditional aggregation (see, e.g., pages 6, paragraph 1 “We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal
r
i
, we rescore it by multiplying its score and classification confidence, which is denoted by
s
~
r
i
=
s
r
i
∙
c
r
i
. We then generate score map
S
r
i
of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score
s
~
r
i
. We perform an average pooling over the score maps of all the proposals to compute a confidence map Ct… The resulted confidence map Ct aggregates not only the region-wise score but also their spatial extent.” and 10, Sect. 3.3, “Implementation We implement our method using MATLAB and C/C++, with Caffe … on a commodity desktop with a Quad-Core 4.0 GHz processor, 16 GB of RAM, and GTX 980 GPU” [i.e., a 1st soft agent/C/C++ software module to calculate/support the conditional aggregation]).
The motivation to combine Luo, Wang and Nemlekar is the same as discussed above with respect to claims 3 and 12.
Regarding claim 5, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 4.
Luo further discloses wherein the first soft agent includes a fully connected layer for mapping and a layer for scaling (see, e.g., pages 2, Sect. 1, left column, “to use SN in modern deep neural networks. (3) By enabling each normalization layer in a deep network to have its own operation, SN helps ease the usage of normalizers” [i.e., each layer of agent has its own operation] and Sect. 2, right column, “batch normalization in fully-connected feedforward neural networks”, 3, Table 1, “
γ
,
β
denote the scale and shift parameters”, and Sect. 3, right column, paragraphs 1 and 5-6, “
γ
and
β
are a scale and a shift parameter respectively … each pixel is normalized by using μ and Ϭ, and then re-scale and re-shift by
γ
and
β
.”, “normalizing the hidden feature maps of CNNs.”, 6, Sect. 4.1.3, right column, “the SN layers in different places of a network may select distinct operations” [i.e., layers of agent have distinct operation] and 14, Table 11 listing “practices that help training CNNs with SN.” Including “Adding 0:5 dropout in the last fully-connected layer helps generalization in ImageNet.” and “Do not put SN after global pooling when feature map size is 1x1” [i.e., 1st agent includes a fully-connected layer for mapping and another layer for scaling]).
Regarding claim 6, as discussed above, Luo in view of Wang and Nemlekar teaches the computer-readable storage medium of claim 4.
Luo further discloses wherein the second soft agent includes a long short-term memory (LSTM) block to provide mapping and scaling (see, e.g., pages 1-2, Sect. 1, “SN is applied to LSTM”, “We introduce Switchable Normalization (SN), which is applicable in both CNNs and RNNs/LSTMs”, 3, Table 1, “
γ
,
β
denote the scale and shift parameters”, and Sect. 3, right column, paragraphs 1 and 5-6, “
γ
and
β
are a scale and a shift parameter respectively … each pixel is normalized by using μ and Ϭ, and then re-scale and re-shift by
γ
and
β
.”, “normalizing the hidden feature maps of CNNs.”, 13-14, Sect. 4.7, “We investigate SN in LSTM for efficient neural architecture … (CNN) is constructed by stacking multiple convolutional cells. … training controllers … A controller is a LSTM whose parameters are trained”, and 14, Table 11 listing “practices that help training CNNs with SN.” including “Do not put SN after global pooling when feature map size is 1x1” [i.e., 2nd agent includes LTSM block for mapping and scaling]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record, listed on form PTO-892, and not relied upon, is considered pertinent to applicant's disclosure.
The references listed on form PTO-892 are all generally related to techniques, methods and systems for normalization of data used in convolutional neural networks (CNNs).
For example, Quader et al. (U.S. Patent Application Pub. No. 2021/0142106 A1, hereinafter “Quader”) discloses that a method for “learning the set of location-based attention multipliers may include: performing average pooling to obtain an averaged weight for each convolution kernel; feeding the averaged weights of the convolution kernels through one or more fully connected layers, to learn the attention multiplier for each convolution kernel.” where “The neural network 200 includes a preprocessing block 202, which may perform various operations (e.g., normalization) on the input data to generate pre-processed data (e.g. an input activation map) for input to the CNN” (see, paragraphs 10 and 48).
Also, for example, non-patent literature Luo, Ping, et al. ("Differentiable dynamic normalization for learning deep representation." International Conference on Machine Learning. PMLR, 2019, hereinafter “Luo 2019 NPL”) discloses a dynamic normalization (DN) method that builds on and incorporates aspects of Luo’s “Switchable Normalization” method (see, Abstract and pages 2-3, Sects. 1.1-2).
The examiner requests, in response to this office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the reference cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111 (c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RANDY K BALDWIN whose telephone number is (571)270-5222. The examiner can normally be reached on Mon - Fri 9:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RANDALL K. BALDWIN/Primary Examiner, Art Unit 2125
1 Paragraph 39 of Applicant’s specification discloses “As used here, a soft weight refers to a weight value that is determined in operation based on certain values or conditions.” Therefore, “soft weights”, under the broadest reasonable interpretation (BRI), in light of the specification are any weight values that can be determined or updated/changed in operations based on values or conditions.
2 As noted above “soft weights”, under the BRI, in light of the specification are any weight values that can be determined or updated/changed in operations based on values or conditions.
3 Paragraph 39 of the instant specification discloses “As used here, a soft weight refers to a weight value that is determined in operation based on certain values or conditions.” Therefore, “soft weights”, under the broadest reasonable interpretation (BRI), in light of the specification are any weight values that can be determined or updated/changed in operations based on values or conditions.
4 Paragraphs 35 and 42 of applicant’s specification mention “a soft agent 330 for generating soft weights conditional on input samples to regulate the aggregation and normalization blocks” and “the soft agent 400 thus provides easily implementable operations, and can be effectively trained using forward or backward propagation … the soft agent 400 can serve as a general bridge between the entire input sample 405 and local operations”. Therefore, a “soft agent” under the BRI, in view of the specification any combination of software functions and/or hardware capable of performing the claimed weight generation.
5 As noted in the rejections of these claims under 112(b) above, the 1st and 2nd “soft agent” have been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software agents or modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules) capable of performing the claimed functions.
6 Paragraphs 35, 39 and 55 of the specification mention “a soft agent 330 for generating soft weights conditional on input samples to regulate the aggregation and normalization blocks”, “a soft agent for dynamic conditional pooling” and “two soft agents are implemented to provide conditional aggregation and conditional normalization blocks separately”, and paragraphs 42 and 55-56 of the specification state “the soft agent 400 thus provides easily implementable operations, and can be effectively trained using forward or backward propagation algorithms in deep learning. Further, the soft agent 400 can serve as a general bridge between the entire input sample 405 and local operations”, “a first soft agent includes global average pooling (GAP) … for global aggregation, a fully-connected (FC) layer 730 with N output units for mapping, and a SoftMax layer 732 for scaling.” and “a second soft agent again includes the global average pooling (GAP) … for global aggregation, and further includes a long short-term memory (LSTM) block 750 (LSTM referring to an RNN architecture) included to provide mapping and scaling”.
Therefore, the 1st and 2nd “soft agent”, under the BRI, in view of the specification, are as any combination of software (i.e., a set of instructions, code, one or more functions or software agents or modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules) capable of performing the claimed functions.
7 As noted in the objections to these claims above, “the standardized feature map” should recite “the standardized representation of the feature map”.
8 Paragraph 39 of the specification discloses “As used here, a soft weight refers to a weight value that is determined in operation based on certain values or conditions.” Therefore, “soft weights”, under the BRI, in light of the specification are any weight values that can be determined or updated/changed in operations based on values or conditions.
9 As indicated above “soft weights”, under the BRI, in light of the specification, are any weight values that can be determined or updated/changed in operations based on values or conditions.
10 As noted above in the objection to this claim, it appears “mediums” should recite “media”.
11 As indicated above “soft weights”, under the BRI, in light of the specification, are any weight values that can be determined or updated/changed in operations based on values or conditions.
12 As indicated above, a “soft agent” under the BRI, in view of the specification, is any combination of software functions and/or hardware capable of performing the claimed weight generation.
13 As indicated above “soft weights”, under the BRI, in light of the specification, are any weight values that can be determined or updated/changed in operations based on values or conditions.
14 As indicated above, a “soft agent” under the BRI, in view of the specification any combination of software functions and/or hardware capable of performing the claimed weight generation.
15 As noted above, the “soft agent”, under the BRI, in view of the specification, is any combination of software (i.e., a set of instructions, code, one or more functions or software agents or modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules) capable of performing the claimed functions.
16 As noted above, under the BRI, in view of the specification, the 2nd “soft agent” is as any combination of software (i.e., a set of instructions, code, one or more functions or software agents or modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules) capable of performing the claimed functions.
17 As noted above, the 1st “soft agent”, under the BRI, in view of the specification, is any combination of software (i.e., a set of instructions, code, one or more functions or software agents or modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules) capable of performing the claimed functions.
