DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the application filed on 01/23/2023. Claims 1-21 are presented in the case. Claims 8 and 15 are independent claims.
Priority
Applicant's claim for the benefit of PCT Application No. PCT/JP2020/030021, filed on August 5, 2020 is acknowledged.
Information Disclosure Statement
The information disclosure statement submitted on 01/23/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Inference Processing Apparatus inferring a feature of input data through a learned neural network.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a data filter configured to extract only specific input data from the input data that have been received” and “an inference operator configured to use the specific input data extracted by the data filter and the weight as inputs” in claim 8.
“a comparator configured to compares a difference of the input data with a preset threshold” in claim 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may:
(1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or
(2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim limitations “a data filter configured to extract only specific input data from the input data that have been received”, “an inference operator configured to use the specific input data extracted by the data filter and the weight as inputs” and “a comparator configured to compares a difference of the input data with a preset threshold” invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed function. There is no disclosure of any particular structure, either explicitly or inherently for performing these functions by these units. The specification does not provide sufficient details such that one of ordinary skill in the art would understand which structure or structures perform(s) these claimed functions.
Therefore, these claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 8-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 8-14 are directed to a device and claims 15-21 are directed to a method. Therefore, the claims are eligible under Step 1 for being directed to a machine and a process respectively.
Step 2A Prong 1:
Independent claims 8 and 15 recite:
extract only specific input data from the input data that have been received - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data and selecting data based on judgement, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
use the specific input data extracted by the data filter and the weight as inputs, perform inference operation of the learned neural network, and infer the feature of the input data - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data and selecting data based on judgement to infer data, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Dependent claims 9 and 16 recite:
determine similarity between the input data that has been received and input data of a previous inference operation - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data and comparing data, which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
extract the input data that has been received as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are not similar to each other - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
not extract the input data that has been input as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are similar to each other - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Dependent claims 10 and 17 recite:
wherein the data filter includes a comparator configured to compares a difference of the input data with a preset threshold, and wherein the data filter is configured to determine presence or absence of the similarity based on a comparison result of the comparator - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Dependent claims 11 and 18 recite:
the data filter is configured to:
determine similarity between the plurality of pieces of input data;
when determining that there is no similar input data in the plurality of pieces of input data, extract the plurality of pieces of input data as the specific input data to the inference operator - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
when determining that there are pieces of similar data in the plurality of pieces of input data, extract input data that is not similar and any one piece of input data in pieces of input data that are similar, among the plurality of pieces of input data, as the specific input data to the inference operator - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Dependent claims 12 and 19 recite:
the data filter is configured to:
determine both similarity between the plurality of pieces of input data and similarity between the plurality of pieces of input data that have been received and input data of previous inference operation - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data and comparing data which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
when determining that each of the plurality of pieces of input data that have been received is not similar to another piece of input data of the plurality of pieces of input data and the input data of the previous inference operation, extract the plurality of pieces of input data that have been received as the specific input data to the inference operator - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
when determining that there are pieces of input data that are similar in the plurality of pieces of input data, extract any one piece of input data from the pieces of input data that are similar - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
when determining that the input data that has been extracted is not similar to the input data of the previous inference operation, extract the input data that has been extracted as the specific input data to the inference operator - Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating data, comparing data and selecting data based on judgement which is observing, evaluating and judging that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because they recite the additional elements:
Independent claims 8 and 15:
An inference processing device, a first storage circuit configured to store the input data, a second storage circuit configured to store a weight of the learned neural network, a data filter, an inference operator - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).
storing, by the inference processing device, the input data - the steps recited at a high level of generality, and amounts to mere data storing which is well known which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
storing, by the inference processing device, a weight of the learned neural network - the steps recited at a high level of generality, and amounts to mere data storing which is well known which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Dependent claims 11 and 18 recite:
the first storage circuit receives and stores a plurality of pieces of input data from a plurality of different data generation sources - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).
Dependent claims 12 and 19 recite:
the first storage circuit is configured to receive and store a plurality of pieces of input data from a plurality of different data generation sources - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).;
Dependent claims 13 and 20 recite:
wherein the data filter is configured to use output data of the inference operator as input data to the data filter - the step recited at a high level of generality, and amounts to mere data inputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Dependent claims 14 and 21 recite:
wherein the inference operator is configured to use output data of the inference operator as the input data to the inference operator - the step recited at a high level of generality, and amounts to mere data inputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to the abstract idea.
Step 2B: The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements:
Independent claims 8 and 15:
An inference processing device, a first storage circuit configured to store the input data, a second storage circuit configured to store a weight of the learned neural network - These limitations amount to components of a general purpose computer that applies a judicial exception, by use of conventional computer functions (see MPEP § 2106.05(b)).
store the input data; store a weight of the learned neural network; storing, by the inference processing device, the input data; storing, by the inference processing device, a weight of the learned neural network - which is a well-understood, routine, conventional activity similar to Storing and retrieving information in memory described in MPEP 2106.05(d)(II);
Dependent claims 11 and 18:
receives and stores a plurality of pieces of input data from a plurality of different data generation sources - which is a well-understood, routine, conventional activity similar to Storing and retrieving information in memory described in MPEP 2106.05(d)(II).
Dependent claims 12 and 19 recite:
receive and store a plurality of pieces of input data from a plurality of different data generation sources - which is a well-understood, routine, conventional activity similar to Storing and retrieving information in memory described in MPEP 2106.05(d)(II).
Dependent claims 13 and 20 recite:
wherein the data filter is configured to use output data of the inference operator as input data to the data filter - the step recited at a high level of generality, and amounts to mere data inputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Dependent claims 14 and 21 recite:
wherein the inference operator is configured to use output data of the inference operator as the input data to the inference operator - the step recited at a high level of generality, and amounts to mere data inputting, which is a form of insignificant extra-solution activity (see MPEP § 2106.05(g)).
Accordingly, these additional elements do not amount to significantly more than the judicial exception. As such, the claims are ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 8 and 15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Habbecke et al. (hereinafter Habbecke), US 20190286652 A1.
Regarding independent claim 8, Habbecke teaches an inference processing device inferring a feature of input data through a learned neural network ([0013] Described herein are embodiments of a system, apparatus, and method for retrieving relevant surgical videos based, at least in part, on one or more preoperative images and/or user selected metadata; [0021] FIG. 1 illustrates a system 100 to retrieve surgical videos, in accordance with an embodiment of the present disclosure. System 100 includes medical imaging device 101, display 103, processor 105, database 107 (including surgical videos 111, and associated preoperative scans), and network 109; [0035] FIG. 2C illustrates an example graphical user interface (GUI) 220 output to a display (e.g., display 103 illustrated in FIG. 1) for retrieving relevant surgical videos based, at least in part, on one or more preoperative images), the inference processing device comprising:
a first storage circuit configured to store the input data (Fig. 1, 107; [0022] Database 107 includes surgical videos and surgical scans (e.g., pre-operative MR and CT scans corresponding to each surgical video); [0024] a pre-operative scan (e.g., MR/CT scan of a tumor) of a patient may be captured and then a video 111 of the surgery to remove the tumor may be captured. Both the pre-operative scan and the surgical video 111 may be stored in database 107);
a second storage circuit configured to store a weight of the learned neural network (Fig. 2B; [0032] an optimization algorithm (e.g., gradient descent) is used to adjust or otherwise update the weights of the machine learning model to reduce the triplet loss. This is an iterative process in which the machine learning model has weights updated for each of the triplet set of images until training is completed; [0037] Parameter weighting section 226 includes a plurality of parameters 228, which may be ranked, or otherwise weighted by a user of the system to indicate relevancy. Parameters may include any of the previously discussed metadata (e.g., patient age, patient body mass index, type of disease, patient gender, or patient preexisting conditions), organ specific parameters (e.g., size, shape, position), disease specific information (e.g., at least one of a shape of the disease, a location of the disease, or a stage of the disease));
a data filter configured to extract only specific input data from the input data that have been received ([0040] As illustrated in FIG. 2D, block 262 illustrates receiving input information (e.g., metadata constraints, parameter weighting values, and preoperative images from a database or otherwise); [0041] Block 266 shows filtering a plurality of videos from a database including a plurality of surgical videos. In other words, based on the inputs (e.g., metadata constraints, parameter weighting values, or the like) surgical videos deemed irrelevant may be filtered by block 266 such that not every video indexed within the database 264 is passed to block 268); and
an inference operator configured to use the specific input data extracted by the data filter and the weight as inputs, perform inference operation of the learned neural network, and infer the feature of the input data ([0039] FIG. 2D illustrates a flow chart 260 to retrieve surgical videos based, at least in part, on one or more preoperative images, in accordance with an embodiment of the present disclosure; [0042] Block 268 illustrates segmenting and/or classifying surgical videos to generate a feature vector associated with the preoperative images, surgical videos, and the like; [0044] In one embodiment, block 268 includes a 3D deeply supervised network that generates an image similarity score; [0046] Block 270 receives each feature vector associated with each of the preoperative images; [0047] In some embodiments, block 270 proceeds through route A to block 274 in which similarity scoring occurs; [0050] In the same or other embodiments, block 270 may proceed to block 272, in which individual features (i.e., elements of the feature vectors) are ranked (e.g., according to the input parameter weighting and/or as determined via a machine learning model); [0051] Once the similarity score for each of the feature vectors included in the feature list/array is known, block 274 proceeded to block 276, which determined which surgical videos in the database are indexed to which feature vector in order to associate respective surgical videos with the corresponding similarity score; [0052] Block 276 proceeds to block 278, which retrieved relevant surgical videos. In particular, the relevancy of the surgical videos within the database may be determined and retrieved based on their similarity score in relation to a threshold value or range).
Regarding independent claim 15, it is a method claim that corresponding to the device of claim 8. Therefore, it is rejected for the same reason as claim 8 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9-12 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Habbecke as applied in claims 8 and 15, in view of Stengel et al. (hereinafter Stengelet), US 12405660 B2.
Regarding dependent claim 9, Habbecke teaches all the limitations as set forth in the rejection of claim 8 that is incorporated.
Habbecke does not explicitly disclose wherein the data filter is configured to:
determine similarity between the input data that has been received and input data of a previous inference operation;
extract the input data that has been received as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are not similar to each other; and
not extract the input data that has been input as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are similar to each other.
However, in the same field of endeavor, Stengel teaches
wherein the data filter is configured to:
determine similarity between the input data that has been received and input data of a previous inference operation (Fig. 4; Col 8, lines 41-43 In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position);
extract the input data that has been received as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are not similar to each other (Col 8, lines 45-50 In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold, then that image can be analyzed 412 using a coarse CNN to obtain a coarse pupil position estimate); and
not extract the input data that has been input as the specific input data when determining that the input data that has been received and the input data of the previous inference operation are similar to each other (Col 8, lines 52-56 In at least one embodiment, if it is determined that there was not a significant or actionable change since a prior image or frame, then a prior coarse pupil estimate can be reused 414 without performing another coarse inference).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of an architecture which reduces latency as needed by skipping a fine inference on at least some passes and utilizing position from a coarse inference, which can be reused for frames within a fixation period as suggested in Stengel into Habbecke’s system because both of these systems are addressing inferring operations. This modification would have been motivated by the desire for an architecture that provide for high overall accuracy with a lower inferencing time, as well as a flexible trade-off between inference time and pupil localization estimation (Stengel, Col 7, lines 24-27).
Regarding dependent claim 10, the combination of Habbecke and Stengel teaches all the limitations as set forth in the rejection of claim 9 that is incorporated.
Stengel further teaches
wherein the data filter includes a comparator configured to compares a difference of the input data with a preset threshold, and wherein the data filter is configured to determine presence or absence of the similarity based on a comparison result of the comparator (Fig. 4, 408; Col 8, lines 41-48 In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position. In at least one embodiment, this can be based on intensities or other information in those images. In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold).
Regarding dependent claim 11, Habbecke teaches all the limitations as set forth in the rejection of claim 8 that is incorporated.
Habbecke teaches wherein:
the first storage circuit receives and stores a plurality of pieces of input data from a plurality of different data generation sources (Fig. 1, 107; [0022] Database 107 includes surgical videos and surgical scans (e.g., pre-operative MR and CT scans corresponding to each surgical video); [0024] a pre-operative scan (e.g., MR/CT scan of a tumor) of a patient may be captured and then a video 111 of the surgery to remove the tumor may be captured. Both the pre-operative scan and the surgical video 111 may be stored in database 107).
Habbecke does not explicitly disclose
the data filter is configured to:
determine similarity between the plurality of pieces of input data;
when determining that there is no similar input data in the plurality of pieces of input data, extract the plurality of pieces of input data as the specific input data to the inference operator; and
when determining that there are pieces of similar data in the plurality of pieces of input data, extract input data that is not similar and any one piece of input data in pieces of input data that are similar, among the plurality of pieces of input data, as the specific input data to the inference operator.
However, in the same field of endeavor, Stengel teaches
the data filter is configured to:
determine similarity between the plurality of pieces of input data (Fig. 4; Col 8, lines 41-43 In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position);
when determining that there is no similar input data in the plurality of pieces of input data, extract the plurality of pieces of input data as the specific input data to the inference operator (Col 8, lines 45-50 In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold, then that image can be analyzed 412 using a coarse CNN to obtain a coarse pupil position estimate); and
when determining that there are pieces of similar data in the plurality of pieces of input data, extract input data that is not similar and any one piece of input data in pieces of input data that are similar, among the plurality of pieces of input data, as the specific input data to the inference operator (Col 8, lines 52-56 In at least one embodiment, if it is determined that there was not a significant or actionable change since a prior image or frame, then a prior coarse pupil estimate can be reused 414 without performing another coarse inference).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of an architecture which reduces latency as needed by skipping a fine inference on at least some passes and utilizing position from a coarse inference, which can be reused for frames within a fixation period as suggested in Stengel into Habbecke’s system because both of these systems are addressing inferring operations. This modification would have been motivated by the desire for an architecture that provide for high overall accuracy with a lower inferencing time, as well as a flexible trade-off between inference time and pupil localization estimation (Stengel, Col 7, lines 24-27).
Regarding dependent claim 12, Habbecke teaches all the limitations as set forth in the rejection of claim 8 that is incorporated.
Habbecke teaches wherein:
the first storage circuit is configured to receive and store a plurality of pieces of input data from a plurality of different data generation sources (Fig. 1, 107; [0022] Database 107 includes surgical videos and surgical scans (e.g., pre-operative MR and CT scans corresponding to each surgical video); [0024] a pre-operative scan (e.g., MR/CT scan of a tumor) of a patient may be captured and then a video 111 of the surgery to remove the tumor may be captured. Both the pre-operative scan and the surgical video 111 may be stored in database 107).
Habbecke does not explicitly disclose the data filter is configured to:
determine both similarity between the plurality of pieces of input data and similarity between the plurality of pieces of input data that have been received and input data of previous inference operation;
when determining that each of the plurality of pieces of input data that have been received is not similar to another piece of input data of the plurality of pieces of input data and the input data of the previous inference operation, extract the plurality of pieces of input data that have been received as the specific input data to the inference operator;
when determining that there are pieces of input data that are similar in the plurality of pieces of input data, extract any one piece of input data from the pieces of input data that are similar; and
when determining that the input data that has been extracted is not similar to the input data of the previous inference operation, extract the input data that has been extracted as the specific input data to the inference operator.
However, in the same field of endeavor, Stengel teaches
the data filter is configured to:
determine both similarity between the plurality of pieces of input data and similarity between the plurality of pieces of input data that have been received and input data of previous inference operation (Fig. 4; Col 8, lines 41-43 In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position);
when determining that each of the plurality of pieces of input data that have been received is not similar to another piece of input data of the plurality of pieces of input data and the input data of the previous inference operation, extract the plurality of pieces of input data that have been received as the specific input data to the inference operator (Col 8, lines 45-50 In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold, then that image can be analyzed 412 using a coarse CNN to obtain a coarse pupil position estimate);
when determining that there are pieces of input data that are similar in the plurality of pieces of input data, extract any one piece of input data from the pieces of input data that are similar (Col 8, lines 52-56 In at least one embodiment, if it is determined that there was not a significant or actionable change since a prior image or frame, then a prior coarse pupil estimate can be reused 414 without performing another coarse inference); and
when determining that the input data that has been extracted is not similar to the input data of the previous inference operation, extract the input data that has been extracted as the specific input data to the inference operator (Col 8, lines 52-56 In at least one embodiment, if it is determined that there was not a significant or actionable change since a prior image or frame, then a prior coarse pupil estimate can be reused 414 without performing another coarse inference).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of an architecture which reduces latency as needed by skipping a fine inference on at least some passes and utilizing position from a coarse inference, which can be reused for frames within a fixation period as suggested in Stengel into Habbecke’s system because both of these systems are addressing inferring operations. This modification would have been motivated by the desire for an architecture that provide for high overall accuracy with a lower inferencing time, as well as a flexible trade-off between inference time and pupil localization estimation (Stengel, Col 7, lines 24-27).
Regarding dependent claim 16, it is a method claim that corresponding to the device of claim 9. Therefore, it is rejected for the same reason as claim 9 above.
Regarding dependent claim 17, it is a method claim that corresponding to the device of claim 10. Therefore, it is rejected for the same reason as claim 10 above.
Regarding dependent claim 18, it is a method claim that corresponding to the device of claim 11. Therefore, it is rejected for the same reason as claim 11 above.
Regarding dependent claim 19, it is a method claim that corresponding to the device of claim 12. Therefore, it is rejected for the same reason as claim 12 above.
Claims 13-14 and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Habbecke as applied in claims 8 and 15, in view of Das et al. (hereinafter Das), US 10108850 B1.
Regarding dependent claim 13, Habbecke teaches all the limitations as set forth in the rejection of claim 8 that is incorporated.
Habbecke does not explicitly disclose wherein the data filter is configured to use output data of the inference operator as input data to the data filter.
However, in the same field of endeavor, Das teaches wherein the data filter is configured to use output data of the inference operator as input data to the data filter (Col 38, lines 8-16 Recurrent neural networks (RNNs) are a family of feedforward neural networks that include feedback connections between layers. RNNs enable modeling of sequential data by sharing parameter data across different parts of the neural network. The architecture for a RNN includes cycles. The cycles represent the influence of a present value of a variable on its own value at a future time, as at least a portion of the output data from the RNN is used as feedback for processing subsequent input in a sequence; Fig. 14; Col 40, lines 23-41 FIG. 14 illustrates an exemplary recurrent neural network 1400. In a recurrent neural network (RNN), the previous state of the network influences the output of the current state of the network. RNNs can be built in a variety of ways using a variety of functions. The use of RNNs generally revolves around using mathematical models to predict the future based on a prior sequence of inputs. For example, an RNN may be used to perform statistical language modeling to predict an upcoming word given a previous sequence of words. The illustrated RNN 1400 can be described has having an input layer 1402 that receives an input vector, hidden layers 1404 to implement a recurrent function, a feedback mechanism 1405 to enable a ‘memory’ of previous states, and an output layer 1406 to output a result. The RNN 1400 operates based on time-steps. The state of the RNN at a given time step is influenced based on the previous time step via the feedback mechanism 1405. For a given time step, the state of the hidden layers 1404 is defined by the previous state and the input at the current time step).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a portion of the output data from the RNN is used as feedback for processing subsequent input in a sequence as suggested in Das into Habbecke’s system because both of these systems are addressing performing inferencing using a trained model. This modification would have been motivated by the desire for a mechanism for facilitating recognition, reidentification, and security in machine learning at autonomous machines (Das, Col 1, lines 7-10).
Regarding dependent claim 14, Habbecke teaches all the limitations as set forth in the rejection of claim 8 that is incorporated.
Habbecke does not explicitly disclose wherein the data filter is configured to use output data of the inference operator as input data to the data filter.
However, in the same field of endeavor, Das teaches wherein the inference operator is configured to use output data of the inference operator as the input data to the inference operator (Col 38, lines 8-16 Recurrent neural networks (RNNs) are a family of feedforward neural networks that include feedback connections between layers. RNNs enable modeling of sequential data by sharing parameter data across different parts of the neural network. The architecture for a RNN includes cycles. The cycles represent the influence of a present value of a variable on its own value at a future time, as at least a portion of the output data from the RNN is used as feedback for processing subsequent input in a sequence; Fig. 14; Col 40, lines 23-41 FIG. 14 illustrates an exemplary recurrent neural network 1400. In a recurrent neural network (RNN), the previous state of the network influences the output of the current state of the network. RNNs can be built in a variety of ways using a variety of functions. The use of RNNs generally revolves around using mathematical models to predict the future based on a prior sequence of inputs. For example, an RNN may be used to perform statistical language modeling to predict an upcoming word given a previous sequence of words. The illustrated RNN 1400 can be described has having an input layer 1402 that receives an input vector, hidden layers 1404 to implement a recurrent function, a feedback mechanism 1405 to enable a ‘memory’ of previous states, and an output layer 1406 to output a result. The RNN 1400 operates based on time-steps. The state of the RNN at a given time step is influenced based on the previous time step via the feedback mechanism 1405. For a given time step, the state of the hidden layers 1404 is defined by the previous state and the input at the current time step).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a portion of the output data from the RNN is used as feedback for processing subsequent input in a sequence as suggested in Das into Habbecke’s system because both of these systems are addressing performing inferencing using a trained model. This modification would have been motivated by the desire for a mechanism for facilitating recognition, reidentification, and security in machine learning at autonomous machines (Das, Col 1, lines 7-10).
Regarding dependent claim 20, it is a method claim that corresponding to the device of claim 13. Therefore, it is rejected for the same reason as claim 13 above.
Regarding dependent claim 21, it is a method claim that corresponding to the device of claim 14. Therefore, it is rejected for the same reason as claim 14 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
OKUDA et al. (US 20230035526 A1) discloses an inference device, a driving assistance device, an inference method, and a server that perform inference using a learned model in machine learning.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY P HOANG whose telephone number is (469)295-9134. The examiner can normally be reached M-TH 8:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER WELCH can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMY P HOANG/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143