DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-26 are rejected under 35 USC 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, the claim recites the following limitations b) Forming first situation data (Y(t3)) based on the first output values (Output11), determining initial assessments (Output2l) by the evaluation level indicating whether or to what degree the initial situation data meets predetermined initial conditions, influencing the determination of the first output values in the working level based on the first evaluations; whereby steps a) - d) are carried out repeatedly, determining second output values (Outputl2) from the second input values by the working level, according to a second classification, wherein the determination of the second output values is influenced by the first output values; f) forming second situation data (Y(t4)) based on the second output values; g;) inputting the second situation data to the evaluation level and determining second assessments (output22) by the evaluation level indicating whether or to what degree the second situation data satisfy predetermined second conditions, the determination of the second assessments being influenced by the first assessments; h) influencing the determination of the second output values in the working level based on the second evaluations.
The above forming of data, determining and influencing steps pertain to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
The claim does not integrate the judicial exceptions into a practical application. The claim includes additional limitations including: a) inputting first input values (Xi (t1)) to the working level and determining first output values (Output 11) from the first input values by the working level, according to a first classification; c) inputting the initial situation data to the evaluation level. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)). The claim further recites the additional element of whereby steps e) - h) are carried out repeatedly; wherein the first and/or the second output values are used as overall output values (Output) of the overall system, wherein the overall output values are used as control parameters and/or state parameters of the machine. These limitations are recited at a high level of generality and amount to field of use and technological environment (2106.05(h)).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As noted above, the inputting step relates to insignificant extra solution activity (MPEP 2106.05(g)). Further, the steps e) - h) are carried out repeatedly; wherein the first and/or the second output values are used as overall output values (Output) of the overall system, wherein the overall output values are used as control parameters and/or state parameters of the machine amounts to field of use and technological environment (2106.05(h)). Additionally, the inputting step can also be understood as pertaining to well-understood, routine, and conventional activity i.e. data gathering and transmission issuance since the MPEP dictates that the courts have recognized data gathering and data transmission/issuance to be well-understood, routine and conventional activity (MPEP 2106.05(d)(II)). Even when considered in combination, these additional elements represent mere instruction to apply an exception and insignificant extra solution activity which do not provide an inventive concept. Therefore, the claim is not eligible.
Therefore, claim 1 is rejected.
Claim 2 recites wherein steps a) - d) are repeatedly performed until a predetermined first time period has elapsed and/or the first output values no longer change between successive repetitions within predetermined first tolerances and/or the first evaluations indicate that the first conditions are fulfilled at least to some degree; wherein preferably the first output values are used as overall output values when this repeated performance is completed. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 2 is rejected.
Claim 3 recites wherein steps e) h) are repeatedly performed until a predetermined second time period has elapsed and/or the second output values no longer change between successive repetitions within predetermined second tolerances and/or the second evaluations indicate that the second conditions are fulfilled at least to some degree; wherein preferably the second output values are used as overall output values when this repeated performance is completed. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 3 is rejected.
Claim 4 recites storing, in an overall sequence memory (760,860),overall sequences of overall records each comprising mutually corresponding input values and/or first output values and/or first situation data and/or first evaluations and/or second output values and/or second situation data and/or second evaluations; wherein preferably the overall records and/or the values or data comprised in the overall records are provided with respective time information and/or numbering. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 4 is rejected.
Claim 5 recites supplementing the first and/or second conditions so that, for first and second situation data respectively, for which the first and second conditions respectively are not satisfied prior to the supplementation, the supplemented first and second conditions respectively are satisfied or at least to some degree satisfied; wherein preferably only the second conditions are changed and the first conditions remain 10 unchanged. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 5 is rejected.
Claim 6 recites wherein steps e) h) are repeatedly performed until a predetermined second time period has elapsed repetitions within predetermined second tolerances and/or the second evaluations indicate that the second conditions are fulfilled at least to some degree; wherein preferably the second output values are used as overall output values when wherein, when the repetition of steps e)- h) is aborted because the second time period has expired or, preferably, because the second output values no longer change within the second tolerances, the second conditions are supplemented so that the situation data present at abort satisfy the supplemented second conditions. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 6 is rejected.
Claim 7 recites storing, in an overall sequence memory (760, 860), overall sequences of overall records each comprising mutually corresponding input values and/or first output values and/or first situation data and/or first evaluations and/or second output values and/or second situation data and/or second evaluations; wherein preferably the overall records and/or the values or data comprised in the overall records are provided with respective time information and/or numbering wherein the supplementing of the first and/or second conditions takes place based on stored total sequences for which the first or second conditions, respectively, could not be fulfilled. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 7 is rejected.
Claim 8 recites wherein the overall system comprises a projection level and the forming of the first and/or the second situation data is performed by the projection level. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 8 is rejected.
Claim 9 recites wherein said second classification classifies at least one class of said first classification into a plurality of subclasses and/or wherein for at least one of said first conditions said one first condition is implied by a plurality of said second conditions. The above classification pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 9 is rejected.
Claim 10 recites wherein the first conditions are given in the form of rules and the second conditions are given in the form of rule classifications; wherein each rule is assigned a rule classification which represents a subdivision, in particular into several levels, of the respective rule; wherein preferably memories are provided in which the rules and the rule classifications are stored; wherein further preferably the rule classifications are subdivided into levels which are linked by means of a blockchain, wherein the rules and/or rule classifications are implemented in the form of a smart contract and/or wherein, if dependent on claim 5, a further level of the subdivision is added when supplementing the second conditions. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 10 is rejected.
Claim 11 recites wherein the working level is designed in such a way that the determination of the first output values in step a) requires a shorter period of time and the determination of the second output values in step e) requires a longer period of time; and/or wherein the evaluation level is designed in such a way that the determination of the first evaluations in step c) requires a shorter period of time and the determination of the second evaluations in step g) requires a longer period of time; wherein preferably in both cases independently of one another the longer period of time is longer than the shorter period of time by at least a factor of 2, in particular by at least a factor of 5. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 11 is rejected.
Claim 12 recites wherein the first and second input values are given as time-continuous input signals or as time-discrete time series, preferably wherein the first and second input values are wholly or partially identical. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 12 is rejected.
Claim 13 recites wherein said work plane comprises first and second artificially learning work units (810, 820); wherein said first artificial learning work unit (810) is adapted to receive said first input values (X; (t1)) and to determine said first output values; wherein said second artificial learning work unit (820) is adapted to receive said second input values (Xi (t2)) and to determine said second output values; and wherein in the working level one or more first modulation functions (fmodu, fmod2_w) are formed based on the first output values and/or values derived therefrom, the formed one or more first modulation functions being applied to one or more parameters (foutA2 , faktA2 f , -transA2, WiA2 ) of the second artificial learning working unit (820), wherein the one or more parameters influence the processing of input values and the obtaining of output values in the second artificial learning working unit. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 13 is rejected.
Claim 14 recites wherein one or more second modulation functions (fmodzi, foionw) are formed based on the first evaluations and/or values derived therefrom, wherein the formed one or more second modulation functions are applied to one or more parameters (foutAl , - f aktAl , ftransAl, WiAl ) of the first artificially learning working unit (810), wherein the one or more parameters influence the processing of input values and the obtaining of output values in the first artificially learning working unit. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 14 is rejected.
Claim 15 recites wherein the first evaluations and/or values derived therefrom are used as evaluation input values of the first artificially learning work unit (810); and/or wherein the second evaluations and/or values derived therefrom are used as evaluation input values of the second artificially learning work unit (820). The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 15 is rejected.
Claim 16 recites A method according to wherein the evaluation level comprises a first and a second artificially learning evaluation unit (830, 840); wherein the first artificially learning evaluation unit (830) is arranged to receive the first situation data (Y(t3)) and to determine the first evaluations; wherein the second artificially learning evaluation unit (840) is arranged to receive the second situation data (Y(t4)) and to determine the second evaluations; and wherein in the evaluation level one or more third modulation functions (fmodu, fmon w) are formed based on the first evaluations and/or values derived therefrom, wherein the formed one or more second modulation functions are applied to one or more parameters (fouts2, faktB2 , ftransB2, Wil32) of the second artificially learning evaluation unit (840), wherein the one or more parameters influence the processing of input values and the obtaining of output values in the second artificially learning evaluation unit. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 16 is rejected.
Claim 17 recites further comprising storing, in a first sequence memory, a first evaluation sequence of first evaluation sets comprising input values of the first evaluation unit and associated first evaluations, the first evaluation sets being provided in particular with respective time information and/or numbering; and/or storing, in a second sequence memory (832), a second evaluation sequence of second evaluation sets comprising input values of the second evaluation unit and associated second evaluations, the second evaluation sets being provided in particular with respective time information and/or numbering; wherein preferably the determination of the first and/or the second evaluations is carried out taking into account the stored first or second evaluation sequences. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 17 is rejected.
Claim 18 recites wherein the storing is done in cryptographically secured form; wherein preferably respectively one blockchain is used, wherein blocks of the respective blockchain contain at least one of the first evaluation sets, the second evaluation sets and the overall sets, respectively. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 18 is rejected.
Claim 19 recites forming first and/or second situation data from the received output values; determining first and/or second evaluations by the evaluation level based on the first or second situation data formed from the received output values; determining that the other system is compatible if the determined first and/or second evaluations indicate that the first or second conditions are respectively met. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 19 is rejected.
Claim 20 recites wherein the working level is arranged to receive the input values and preferably the evaluation level is not capable of receiving the input values. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 20 is rejected.
Claim 21 recites wherein the working level and the evaluation level are each implemented in at least one computing unit. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 12 is rejected.
Claim 22 recites wherein the at least one computing unit in which the working level is implemented is different, in particular separate, from the at least one computing unit in which the evaluation level is implemented. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 22 is rejected.
Claim 23 recites a projection level and/or an overall sequence memory. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 23 is rejected.
Claim 24 recites wherein the working level comprises first and second artificially learning working units (810, 820) and wherein the evaluation level comprises first and second artificially learning evaluation units (830, 840); wherein the artificially learning working units and/or evaluation units preferably each comprise a neural network having a plurality of nodes, wherein further preferably the one or more parameter(s) are each at least one of: a weighting for a node of the neural network, an activation function of a node, an output function of a node, a propagation function of a node. The above steps pertains to mental processes since these steps can be performed in the mind using pen and paper and thus falls under the mental process grouping of abstract ideas (MPEP 2106.04(a)(2)(III)).
Therefore, claim 24 is rejected.
Claim 25 recites wherein the first and second artificially learning evaluation units are implemented and/or executed as hardware and/or computer program in a first and/or second computing unit, wherein the first and second computing units are interconnected by a first interface; wherein, if dependent on claim 13, the first interface is arranged to form the one or more first modulation functions; and/or wherein the first and second artificially learning evaluation units are implemented and/or executed as hardware and/or computer program in a third and/or fourth computing unit, the third and fourth computing unit being interconnected by a third interface; wherein, if dependent on claim 16, the third interface is arranged to form the one or more third modulation functions; and/or wherein the third computing unit and the first computing unit are interconnected by a second interface; wherein, if dependent on claim 14, the second interface is arranged to form the one or more second modulation functions. This limitation are at a high level of generality and amount to field of use and technological environment (2106.05(h)).
Therefore, claim 25 is rejected.
Claim 26 recites preferably all, computing units is/are assigned a memory which is connected to or included in the respective computing unit; wherein preferably the memory assigned to the first computing unit is arranged to store the first classification, and/or the memory assigned to the second computing unit is arranged to store the second classification, and/or the memory assigned to the third computing unit is arranged to store the first conditions, and/or the memory assigned to the fourth computing unit is arranged to store the second conditions. These limitations are recited at a high level of generality and amount to mere data gathering which is a form of insignificant extra solution activity (MPEP 2106.05(g)).
Therefore, claim 26 is rejected.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 2, 4, 7, 11, and 14-18 recites the limitation "the first evaluations". There is insufficient antecedent basis for this limitation in the claim.
Therefore, any form of evaluation will be considered the first evaluations.
Claims 13-15 recites the limitation "the work plane". There is insufficient antecedent basis for this limitation in the claim.
Therefore, any form of work plane will be considered the work plane.
All instances of antecedent basis must be corrected throughout the claims.
Claims 2-7, 10-12, 17-18, 20, 24, and 26 recite the word preferably. The word preferably is deemed indefinite for failing to particularly point out and distinctly claim the subject matter since it is not clear what follows the word is required by the claim.
For the purpose of examination, any limitation that is preferably claimed will not be required by the broadest reasonable interpretation of the claim.
Claims 10, 11, 17 and 20 recite the phrase in particular. The phrase in particular is deemed indefinite for failing to particularly point out and distinctly claim the subject matter since it is not clear what follows the phrase is required by the claim.
For the purpose of examination, any limitation that is in particularly claimed will not be required by the broadest reasonable interpretation of the claim.
Claims 10, 18, and 25 recite “if dependent on claim…”. This has been deemed indefinite for failing to particularly point out and distinctly claim the subject matter since none of the claims are dependent on the claims mentioned in the aforementioned claims.
For the purpose of examination, any limitation that recites this phrase will be understood as being optional since none of the claims depend from the claims mentioned.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 13-15 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 13 recites "method according to any of the preceding claims" which does not constitute a reference to a claim previously set forth and specifies a further limitation of the claim previously set forth. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 9-11,13-17, and 20-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yan et al (NPL “HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition”, 2015 IEEE International Conference on Computer Vision, hereinafter Yan) in view of Kimura et al (US PUB. 20200372332, herein Kimura).
Regarding claim 1, Yan teaches A method, executed in a controller of a machine, for processing input values (Xi) comprising sensor data detected by one or more sensors in an overall system having a working level (710) and an evaluation level (730) which are artificial learning systems (2742 first paragraph “Shared layers (left of Fig 1 (b)) receive raw image pixels as input and extract low-level features. The configuration of shared layers is set to be the same as the preceding layers in the building block CNN” 2741 third paragraph “CNN-based models hold state-of-the-art performance in various computer vision tasks, including image classification [18], object detection [10, 13], and image parsing”), comprising
a) inputting first input values (Xi (t1)) to the working level and determining first output values (Output 11) from the first input values by the working level, according to a first classification (page 2741 fig. b
PNG
media_image1.png
167
643
media_image1.png
Greyscale
, page 2740 last paragraph right side “An HD-CNN follows the coarse-to fine classification paradigm and probabilistically integrates predictions from fine category classifiers” the image is the first input being fed to the coarse component which corresponds to the working level and the coarse prediction corresponds to the first output value according to a first classification);
b) Forming first situation data (Y(t3)) based on the first output values (Output11) (2742 column 1 paragraph 3 “coarse category probabilities serve two purposes. First, they are used as weights for combining the predictions made by fine category components {Fk}K k=1. Second, when thresholded, they enable conditional execution of fine category components whose corresponding coarse probabilities are sufficiently large”);
c) inputting the initial situation data to the evaluation level and determining initial assessments (Output2l) by the evaluation level indicating whether or to what degree the initial situation data meets predetermined initial conditions (2742 1st column third paragraph “In the bottom right of Fig 1 (b) are independent layers of a set of fine category classifiers {Fk}K k=1, each of which makes fine category predictions. As each fine component only excels in classifying a small set of categories, they produce a fine prediction over a partial set of categories”);
d) influencing the determination of the first output values in the working level based on the first evaluations; whereby steps a) - d) are carried out repeatedly (2743 1st column paragraph 1 “we decompose the HD-CNN training into multiple steps instead of training the complete HD-CNN from scratch” 2746 1st column last paragraph “HD-CNN corrects the mistakes made by the building block net. In Fig 3, we collect four testing cases. In the first case, the building block net fails to predict the label of the tiny hermit crab in the top 5 guesses. In HD-CNN, two coarse categories, #6 and #11, receive most of the coarse probability mass. The fine category component #6 specializes in classifying crab breeds and strongly suggests the ground truth label”);
e) inputting second input values (Xi (t2)) to the working level and determining second output values (Outputl2) from the second input values by the working level, according to a second classification, wherein the determination of the second output values is influenced by the first output values (2742 first column first paragraph “Shared layers (left of Fig 1 (b)) receive raw image pixels as input and extract low-level features. The configuration of shared layers is set to be the same as the preceding layers in the building block CNN. On the top of Fig 1(b) are independent layers of coarse category component B, which reuses the configuration of rear layers from the building block CNN and produces an intermediate fine prediction {Bf ij}C j=1 for an image xi. To produce a coarse category prediction {Bik}K k=1, we append a fine-to-coarse aggregation layer (not shown in Fig 1(b)), which reduces fine predictions into coarse using a mapping P : [1, C] → [1, K]. The coarse category probabilities serve two purposes. First, they are used as weights for combining the predictions made by fine category components {Fk}K k=1. Second, when thresholded, they enable conditional execution of fine category components whose corresponding coarse probabilities are sufficiently large”, 2746 1st column last paragraph “HD-CNN corrects the mistakes made by the building block net. In Fig 3, we collect four testing cases. In the first case, the building block net fails to predict the label of the tiny hermit crab in the top 5 guesses. In HD-CNN, two coarse categories, #6 and #11, receive most of the coarse probability mass. The fine category component #6 specializes in classifying crab breeds and strongly suggests the ground truth label” 2746 second column first paragraph “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”);
f) forming second situation data (Y(t4)) based on the second output values (2742 first column first paragraph “Shared layers (left of Fig 1 (b)) receive raw image pixels as input and extract low-level features. The configuration of shared layers is set to be the same as the preceding layers in the building block CNN. On the top of Fig 1(b) are independent layers of coarse category component B, which reuses the configuration of rear layers from the building block CNN and produces an intermediate fine prediction {Bf ij}C j=1 for an image xi. To produce a coarse category prediction {Bik}K k=1, we append a fine-to-coarse aggregation layer (not shown in Fig 1(b)), which reduces fine predictions into coarse using a mapping P : [1, C] → [1, K]. The coarse category probabilities serve two purposes. First, they are used as weights for combining the predictions made by fine category components {Fk}K k=1. Second, when thresholded, they enable conditional execution of fine category components whose corresponding coarse probabilities are sufficiently large” 2746 1st column last paragraph “HD-CNN corrects the mistakes made by the building block net. In Fig 3, we collect four testing cases. In the first case, the building block net fails to predict the label of the tiny hermit crab in the top 5 guesses. In HD-CNN, two coarse categories, #6 and #11, receive most of the coarse probability mass. The fine category component #6 specializes in classifying crab breeds and strongly suggests the ground truth label” 2746 second column first paragraph “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”);
g;) inputting the second situation data to the evaluation level and determining second assessments (output22) by the evaluation level indicating whether or to what degree the second situation data satisfy predetermined second conditions, the determination of the second assessments being influenced by the first assessments (2742 first column first paragraph “Shared layers (left of Fig 1 (b)) receive raw image pixels as input and extract low-level features. The configuration of shared layers is set to be the same as the preceding layers in the building block CNN. On the top of Fig 1(b) are independent layers of coarse category component B, which reuses the configuration of rear layers from the building block CNN and produces an intermediate fine prediction {Bf ij}C j=1 for an image xi. To produce a coarse category prediction {Bik}K k=1, we append a fine-to-coarse aggregation layer (not shown in Fig 1(b)), which reduces fine predictions into coarse using a mapping P : [1, C] → [1, K]. The coarse category probabilities serve two purposes. First, they are used as weights for combining the predictions made by fine category components {Fk}K k=1. Second, when thresholded, they enable conditional execution of fine category components whose corresponding coarse probabilities are sufficiently large” 2746 second column “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”);
h) influencing the determination of the second output values in the working level based on the second evaluations (2746 second column “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”).
Kimura teaches whereby steps e) - h) are carried out repeatedly (0051 “in the processing of a large-scale CNN, it is impossible to process all hierarchical layers at once, so that processing of the CNN is performed time divisionally. In such cases, the process returns to step S204 via step S207 to perform calculation processing on the remaining hierarchical layers.”);
wherein the first and/or the second output values are used as overall output values (Output) of the overall system, wherein the overall output values are used as control parameters and/or state parameters of the machine (0051 0052 “control unit 105 supplies a parameter corresponding to a feature amount operation to be performed for the control parameter stored in the holding unit 104 to the calculation unit 101 and the management unit 103 to control these operations. Then, the control unit 105 instructs the calculation unit 101 and the management unit 103 to start processing”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to have modified the HD-CNN teachings of Yan with the control parameter teachings using CNNs teachings of Kimura since Kimura teaches a means for time divisionally processing CNN when it is impossible to process all hierarchical layers at once (0052).
Regarding claim 2, the cited prior art teach A method according to claim 1.
Yan teaches wherein steps a) - d) are repeatedly performed until a predetermined first time period has elapsed and/or the first output values no longer change between successive repetitions within predetermined first tolerances and/or the first evaluations indicate that the first conditions are fulfilled at least to some degree; wherein preferably the first output values are used as overall output values when this repeated performance is completed (2746 second column “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”, multiple classifications until conditions are fulfilled);
Regarding claim 3, the cited prior art teach A method according to claim 1.
Yan teaches wherein steps e) h) are repeatedly performed until a predetermined second time period has elapsed and/or the second output values no longer change between successive repetitions within predetermined second tolerances and/or the second evaluations indicate that the second conditions are fulfilled at least to some degree; wherein preferably the second output values are used as overall output values when this repeated performance is completed (2746 second column “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”, multiple classifications until conditions are fulfilled).
Regarding claim 4, the cited prior art teach A method according claim 1.
Kimura teaches comprising storing, in an overall sequence memory (760,860),overall sequences of overall records each comprising mutually corresponding input values and/or first output values and/or first situation data and/or first evaluations and/or second output values and/or second situation data and/or second evaluations; wherein preferably the overall records and/or the values or data comprised in the overall records are provided with respective time information and/or numbering (0096 “according to the present embodiment, the memory configuration according to the first embodiment can also be applied to the feature image of the side-output hierarchical layer. The feature data (block group) required by the second calculation unit 1001 can be supplied all at once in relation to a plurality of access patterns that are variable for each hierarchical layer in accordance with the presence or absence of sampling, sampling rate, and the like, and the utilization efficiency of the second calculation unit 1001 can be improved”).
Regarding claim 5, the cited prior art teach A method according to claim 1.
Yan teaches comprising supplementing the first and/or second conditions so that, for first and second situation data respectively, for which the first and second conditions respectively are not satisfied prior to the supplementation, the supplemented first and second conditions respectively are satisfied or at least to some degree satisfied; wherein preferably only the second conditions are changed and the first conditions remain 10 unchanged (2744 second column first paragraph “Our HD-CNN achieves a testing error of 32.62%, which improves the building block net by 2.65%”).
Regarding claim 6 the cited prior art teach A method according to claim 5.
Yan teaches wherein steps e) h) are repeatedly performed until a predetermined second time period has elapsed repetitions within predetermined second tolerances and/or the second evaluations indicate that the second conditions are fulfilled at least to some degree (2746 second column “For HD-CNN, the coarse category component is also not confident about which coarse category the object belongs to and thus assigns even probability mass to the top coarse categories. For the top 3 fine category classifiers, #74 strongly predicts ground truth label while the other two ,#49 and #40, rank the ground truth label at the 2nd and 4th place, respectively. Overall, the HD-CNN ranks the ground truth label at the 1st place. This demonstrates HD-CNN needs to rely on multiple fine category classifiers to make correct predictions for difficult cases”, multiple classifications until conditions are fulfilled)
wherein preferably the second output values are used as overall output values when wherein, when the repetition of steps e)- h) is aborted because the second time period has expired or, preferably, because the second output values no longer change within the second tolerances, the second conditions are supplemented so that the situation data present at abort satisfy the supplemented second conditions.
Regarding claim 7, the cited prior art teach A method according to claim 5.
Kimura teaches comprising storing, in an overall sequence memory (760, 860), overall sequences of overall records each comprising mutually corresponding input values and/or first output values and/or first situation data and/or first evaluations and/or second output values and/or second situation data and/or second evaluations; wherein preferably the overall records and/or the values or data comprised in the overall records are provided with respective time information and/or numbering wherein the supplementing of the first and/or second conditions takes place based on stored total sequences for which the first or second conditions, respectively, could not be fulfilled (0096 “according to the present embodiment, the memory configuration according to the first embodiment can also be applied to the feature image of the side-output hierarchical layer. The feature data (block group) required by the second calculation unit 1001 can be supplied all at once in relation to a plurality of access patterns that are variable for each hierarchical layer in accordance with the presence or absence of sampling, sampling rate, and the like, and the utilization efficiency of the second calculation unit 1001 can be improved”).
Regarding claim 9, the cited prior art teach A method according claim 1.
Yan teaches wherein said second classification classifies at least one class of said first classification into a plurality of subclasses and/or wherein for at least one of said first conditions said one first condition is implied by a plurality of said second conditions (2742 “Both coarse (B) and fine ({Fk}K k=1) components share common layers. The reason is threefold. First, it is shown in [35] that preceding layers in deep networks response to class-agnostic low-level features such as corners and edges, while rear layers extract more class-specific features such as a dog’s face and bird’s legs. Since low-level features are useful for both coarse and fine classification tasks, we allow the preceding layers to be shared by both coarse and fine components”).
Regarding claim 10, the cited prior art teach a method according to claim 1.
Kimura teaches wherein the first conditions are given in the form of rules and the second conditions are given in the form of rule classifications; wherein each rule is assigned a rule classification which represents a subdivision, in particular into several levels, of the respective rule; wherein preferably memories are provided in which the rules and the rule classifications are stored; wherein further preferably the rule classifications are subdivided into levels which are linked by means of a blockchain, wherein the rules and/or rule classifications are implemented in the form of a smart contract and/or wherein, if dependent on claim 5, a further level of the subdivision is added when supplementing the second conditions (2742 second column “group confusing fine categories into the same coarse category for which a dedicated fine category classifier will be trained…disjoint coarse categories, the overall classification depends heavily on the coarse category classifier. If an image is routed to an incorrect fine category classifier, then the mistake cannot be corrected, as the probability of ground truth label is implicitly set to zero there. Removing the separability constraint between coarse categories can make the HD-CNN less dependent on the coarse category classifier.”).
Regarding claim 11, the cited prior art teach the method according to claim 1.
Kimura teaches wherein the working level is designed in such a way that the determination of the first output values in step a) requires a shorter period of time and the determination of the second output values in step e) requires a longer period of time; and/or wherein the evaluation level is designed in such a way that the determination of the first evaluations in step c) requires a shorter period of time and the determination of the second evaluations in step g) requires a longer period of time; wherein preferably in both cases independently of one another the longer period of time is longer than the shorter period of time by at least a factor of 2, in particular by at least a factor of 5 (0075 “even if the processing target kernel size to be processed, the presence or absence of pooling, and the pooling window size are dynamically changed for each hierarchical layer, the management unit 103 can supply the pixel block sequence required by the calculation unit 101 every clock cycle. Therefore, the waiting time and the stopping period of the calculation unit 101 are shortened, and the utilization efficiency is improved”).
Regarding claim 13, A method according to any of the preceding claims.
Yan teaches wherein said work plane comprises first and second artificially learning work units (810, 820); wherein said first artificial learning work unit (810) is adapted to receive said first input values (X; (t1)) and to determine said first output values; wherein said second artificial learning work unit (820) is adapted to receive said second input values (Xi (t2)) and to determine said second output values; and wherein in the working level one or more first modulation functions (fmodu, fmod2_w) are formed based on the first output values and/or values derived therefrom, the formed one or more first modulation functions being applied to one or more parameters (foutA2 , faktA2 f , -transA2, WiA2 ) of the second artificial learning working unit (820), wherein the one or more parameters influence the processing of input