Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
CLAIM REJECTIONS - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-26 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention.
As to claims 1 & 24, for enablement, the critical inquiry is: Does the specification provide enough information so that one of ordinary skill in the art can make and/or use the full scope of the claim invention without "undue experimentation"? Factors to be weighed when evaluating whether a disclosure satisfies the enablement requirement and whether any necessary experimentation is “undue” (i.e., “Wands” factors): − Breadth of the claims; − Nature of the invention; − State of the prior art; − Level of one of ordinary skill; − Level of predictability in the art; − Amount of direction provided by the inventor; − Existence of working examples; and − Quantity of experimentation needed to make or use the invention based on the content of the disclosure. In this case, the breadth of the claims are not enabled due to lack of metes and bounds for the limitation "training data based, preferably self-learning module, by comparing a determined image structure with image structures of a reference database and then outputting control instructions to activate image optimization”. In this case, specification is silent to providing enough information so that one of ordinary skill in the art can make and/or use the full scope of the claim invention without "undue experimentation" to determine metes and bounds. The claims are broad enough to cover any machine learning model, any training data structure, any image structure representation, any reference database comparison, and any resulting control instruction for image optimization.
As to claims 1 & 24, for enablement, the critical inquiry is: Does the specification provide enough information so that one of ordinary skill in the art can make and/or use the full scope of the claim invention without "undue experimentation"? Factors to be weighed when evaluating whether a disclosure satisfies the enablement requirement and whether any necessary experimentation is “undue” (i.e., “Wands” factors): − Breadth of the claims; − Nature of the invention; − State of the prior art; − Level of one of ordinary skill; − Level of predictability in the art; − Amount of direction provided by the inventor; − Existence of working examples; and − Quantity of experimentation needed to make or use the invention based on the content of the disclosure. In this case, the breadth of the claims are not enabled due to lack of metes and bounds for the limitation
(1) "to determine at least one image structure and determining a quality value by comparing the determined image structure with image structures of a reference database”.
(2) “image optimization control”
(3) “trained model” designed to carry out” the method
In this case, specification is silent to providing enough information so that one of ordinary skill in the art can make and/or use the full scope of the claim invention without "undue experimentation" to determine metes and bounds.
As to claims 2-23 & 25-26, these claims are rejected due to their dependence on claims 1 & 24 and are rejected for the same reasons.
CLAIM REJECTIONS - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
As to claims 1, 24 & 26 claim elements “unit for activating image optimization, external unit, adjacent trained model language” is a limitation that invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for the claimed function “unit for activating image optimization, external unit, adjacent trained model language”. In this case, the above phrase is unclear to the sufficient structure.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph; or
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the claimed function, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
As to claims 2-23 & 25-26, these claims are rejected due to their dependence on claims 1 & 24 and are rejected for the same reasons.
CLAIM INTERPRETATIONS - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Sufficient structure, material or acts for performing the claimed function are not present in the claim.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure`, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “control unit” in claim 24.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
CLAIM REJECTIONS - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to as ineligible under subject eligibility test. In the Subject Matter Eligibility Test for Products and Processes (Federal Register, Vol. 79, No. 241, dated Tuesday, December 16, 2014, page 74621), The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional device elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea.
Claims 1 & 24
Step 1
This step inquires “is the claim to a process, article of machine, manufacture or composition of matter?” Yes,
Claim 1 – “Method” is a process.
Claim 24 - “Systems” are machines.
Step 2A - Prong 1
This step inquires “does the claim recite an abstract idea, law or natural phenomenon”. This claim appears to directed to an abstract idea.
The limitation of “acquiring image data by means of an image acquisition device of an endoscope; receiving the image data by a control unit, pre-analyzing at least a subset of the image data of one or more consecutive images by a control unit to determine at least one image structure; determining a quality value by means of a training data-based, preferably self-learning, module by comparing the at least one determined image structure with image structures of a reference database stored in a memory unit, and on the basis of the determined quality value, manually or automatically outputting control instructions from the control unit to a unit for activating image optimization.”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity) but for the recitation of generic computer components. That is, other than reciting “control unit, memory unit, a unit, external unit” nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “control unit, memory unit, a unit, external unit” language, “receiving, determine, comparing, output, activating” in the context of this claim encompasses covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity).
STEP 2A – PRONG 1 - CONCLUSION
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A - Prong 2
This step inquires “does the claim recite additional elements that integrate the judicial exception into a practical application”. This judicial exception is not integrated into a practical application. In particular, the claim recites two additional element – using a “control unit, memory unit, a unit, external unit” to perform “receiving, determine, comparing, output, activating” steps. The “control unit, memory unit, a unit, external unit” are recited at a high-level of generality (i.e., as a generic processor) “acquiring image data by means of an image acquisition device of an endoscope; receiving the image data by a control unit, pre-analyzing at least a subset of the image data of one or more consecutive images by a control unit to determine at least one image structure; determining a quality value by means of a training data-based, preferably self-learning, module by comparing the at least one determined image structure with image structures of a reference database stored in a memory unit, and on the basis of the determined quality value, manually or automatically outputting control instructions from the control unit to a unit for activating image optimization.”, such that it amounts no more than mere instructions to apply the exception using a generic computer component.
STEP 2A – PRONG 2 - CONCLUSION
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B
The critical inquiry here is does the claim recite additional elements that amount to “significantly more” than the judicial exception? The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a “control unit, memory unit, a unit, external unit” to perform “receiving, determine, comparing, output, activating “steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Dependent Claims
As to claim 2, this claim is directed to generic computer components (“the unit”), mental process (“recognizing that cleaning is needed ”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 3, this claim is directed to generic computer components (“”), mental process (“a human could visually assess brightness/contamination and assign a contamination degree”) and insignificant extra-solution activity (“classification”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 4, this claim is directed to generic computer components (“self learning module, a model, image acquisition device”), mental process (“classification/probability is image quality judgment computation”) and insignificant extra-solution activity (“generic computer implementation”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 5, this claim is directed to generic computer components (“memory unit, trained model, self learning module”), mental process (“categorizing images into contamination classes can be performed via human review”) and insignificant extra-solution activity (“training classification data organization”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 6, this claim is directed to generic computer components (“self-learning module, neural network, processor”), mental process (“pre-labeling/checking is mental observation/judgement”) and insignificant extra-solution activity (“data labeling and algorithm correction are preparatory training activity”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 7, this claim is directed to generic computer components (“self learning module, neural network model”) and insignificant extra-solution activity (“choosing a generic neural network architecture is generic computational implementation of the abstract classification”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 8, this claim is directed to mental process (“humans can compare images and notice smoke deterioration, contamination or object structure changes.”) and insignificant extra-solution activity (“detection only.”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 9, this claim is directed to mental process (“segmenting an image and judging regions can be done mentally/visually”) and insignificant extra-solution activity (“image segmentation data analysis unless it drives targeted physical cleaning”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 10, this claim is directed to generic computer components (“cleaning module”), mental process (“choosing a contaminated region or weight is mental”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 11, this claim is directed to generic computer components (“cleaning module”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 12, this claim is directed to generic computer components (“the unit”), mental process (“selecting an optimization mode can be mental”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 13, this claim is directed to generic computer components (“control unit”), mental process (“comparing smoke/condensation characteristics and judging blindness can be visual/mental”) and insignificant extra-solution activity (“classification”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 15, this claim is directed to generic computer components (“control unit”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 16, this claim is directed to generic computer components (“cleaning module”), mental process (“selecting parameters based on class can be mental.”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 17, this claim is directed to generic computer components (“self-learning module”), mental process (“yes for algorithmic selection probability optimization”) and insignificant extra-solution activity (“algorithm is abstract/generic”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 18, this claim is directed to mental process (“model/decision logic”) and insignificant extra-solution activity (“computational parameter selection”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 19, this claim is directed to generic computer components (“self-cleaning module”), mental process (“collecting labeling reference examples are mental”) and insignificant extra-solution activity (“Building a training reference database is preparatory data gathering/training activity.”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 20, this claim is directed to generic computer components (“cleaning and image acquisition devices”), mental process (“comparing device identity to stored data can be mental”) and insignificant extra-solution activity (“database lookup is IESA/generic”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 21, this claim is directed to mental process (“recognizing whether structures are present and deciding continue/stop can be mental”) and insignificant extra-solution activity (“gating analysis”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 22, this claim is directed to generic computer components (“cleaning module, control unit”), mental process (“evaluating cleaning success can be mental”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 23, this claim is directed to mental process (“severity classification is mental”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 25, this claim is directed to generic computer components (“computer”) , mental process (“see claim 1 rationale”) and insignificant extra-solution activity (“see claim 1 rationale”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 26, this claim is directed to generic computer components (“memory unit, self-learning module”), mental process (“classification of images into contamination dependent classes is a mental observational judgment and/or mathematical classification”) and insignificant extra-solution activity (“Training data, model weights and classification output are data/model preparation and generic computational classification”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8-10, 13, 16, 19, 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over
Shelton et al. (U.S. Publication 2020/0405401) in view of Venkataraman et al. (U.S. Publication 2019/0362834)
As to claims 1 & 24, Shelton discloses a control system for image optimization (Fig. 27 & [0377] discloses the surgical visualization system 13500 includes a control circuit 13524 configured to perform the processes 13520, 13540. Fig. 26a & [0376] discloses detecting 13542 an excessive deterioration of lens transparency based on the monitored parameter, and automatically activating 13544, or triggering activation, of a lens cleaning system 13510 to remedy the excessive deterioration of the lens transparency.) of at least one image acquired at the distal end of an endoscope, wherein the system comprises an image acquisition device (Fig. 24-25 & [0372] discloses the surgical visualization assembly 13502 includes an imaging device 13503 and an outer housing 13504. [0372] discloses a distal end 13505 of the imaging device 13503, which includes a visualization lens 13506 and a light source 13508 is exposed. [0386] discloses frames captured by the imaging device 13503); a control unit (Fig. 27 & [0377] discloses the surgical visualization system 13500 includes a control circuit 13524 configured to perform the processes 13520, 13540.) with a self-learning module in communication with the image acquisition device for receiving the image data and a memory unit ([0377] discloses the processes 13520, 13540 can be embodied as a set of computer executable instructions stored in a memory 13534. [0381] discloses the memory 13534 of the control circuit 13524 may store an algorithm, an equation, or a look up table for determining correlations between measurements of one or more of the capacitive sensors 13530 and the lens occlusion or transparency levels of the visualization lens 13506. [0382] discloses a look up table or database can be accessed by the control circuit to determine the lens occlusion or transparency levels based on the measured values of the optical sensors 13532.)([0386] discloses the imaging module 138 can be utilized to analyze and/or compare frames captured by the imaging device 13503. See wherein one or more frames captured by the imaging device 13503 through visualization lens 13506);
Shelton discloses wherein the control unit is configured to pre-analyze at least a subset of the image data to determine at least one image structure. ([0386] discloses the imaging module 138 can be utilized to analyze and/or compare frames captured by the imaging device 13503 looking for either known markers on instruments or distinguishable objects within the field of view of the visualization lens 13506 to identify irregular distortions or blurriness accepted predetermined threshold.)
wherein the control unit is configured to determine a quality value by comparing the at least one determined image structure with image structures of a reference database stored in the memory unit.
([0380] discloses a lookup table or database can be accessed by the control circuit to determine the lens occlusion or transparency levels based on the measured values of the capacitive sensors 13530. [0381] discloses the memory 13534 of the control circuit 13524 may store an algorithm, an equation, or a look-up table for determining correlations between measurements of one or more of the capacitive sensors 135 and the lens occlusion or transparency level of the visualization lens 13506. [0382] discloses a look up table or database can be accessed by the control circuit to determine the lens occlusion or transparency levels based on the measured values of the optical sensors 13532. [0386] discloses to analyze and/or compare frames captured by the imaging device 13503 looking for either known markers or distinguishable objects within the field of view of the visualization lens 13506.)
Shelton is silent to the reference database or comparison structure is built or improved by a self learning module trained on annotated surgical image structures.
However, Venkataraman discloses a self-learning module ([0198] discloses these annotated video data include accurately labeled image objects such as tools, anatomies, tasks, and complications, which themselves become training data for supervised learning problems. Also See wherein these annotated videos can be used to train machine learning classifiers to automatically detect and identify different MLTs. [0201] discloses spatial tagging subsystem 508 receive sets of machine learning descriptors 534, such as tools, anatomies, events, and complications for identifying and tagging or labeling various tools, anatomies, events, and complications within different video segments 518 and generate tagged video segments 520. [0202] discloses model training subsystem 510 is configured to receive tagged video segments 520 as input and train various machine learning classifiers based on tagged video segments 520. These automatically tagged objects can be used as additional training data for model training subsystem 510 to iteratively improve the accuracy of trained machine learning classifiers 522. );
Wherein the control unit is configured to automatically output control instructions for activating at least one unit for image optimization. (Fig. 26a & [0376] discloses detecting 13542 an excessive deterioration of lens transparency based on the monitored parameter, and automatically activating 13544, or triggering activation, of a lens cleaning system 13510 to remedy the excessive deterioration of the lens transparency. Fig. 29 & [0385] discloses the lens cleaning system 13510 is activated by the control circuit 13524 when the lens occlusion level 13548 as derived from the measurements of the parameter detector 13529, passes 13545 the visibility threshold 13546. [0386] discloses the control circuit can trigger the activation of the cleaning system 13510 based on input from the imaging module 138 indicative of identification of irregular distortions from one or more frames captured by the imaging device 13503 through the visualizations lens 13506. )
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton’s disclosure to include the above limitations in order to improve automated recognition of surgical image structures and image quality defects before activating image optimization.
As to claim 2, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein the unit that can be activated for image optimization is a unit of the endoscope; and preferably comprises a cleaning module for cleaning at least one distal window by means of at least one fluid and the image optimization takes place via the activation of the cleaning module. (Fig. 24-25 & [0374]])
As to claim 3, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein the quality value depends on the detected brightness of the image and/or contamination of the at least one distal window ([0378 & 0382]), wherein, in the step of determining the quality value by means of the self-learning module, a classification into contamination probabilities and/or degree of contamination takes place. Shelton’s ([0380-0383]) & Venkataraman ([0202])
As to claim 8, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein the at least one image structure represents at least one object to be examined that is depicted in the acquired image; and wherein contamination detection, deterioration or smoke detection is based on a change in the image structure. ([0386])
As to claim 9, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein, in the step of pre-analyzing, a division of one of the acquired images into image sections is carried out and/or a division into individual regions based on pixels is carried out on the basis of analysis values, wherein the quality value can be determined for each of the image sections and/or regions. ([0381, 0383])
As to claim 10, Shelton in view of Venkataraman discloses everything as disclosed in claim 2. In addition, Shelton discloses wherein the cleaning module is designed to activate a regionally targeted cleaning depending on a definable weighting of regions and/or depending on a permissible percentage of contamination, wherein preferably the region of the distal window that can be assigned to the image center can be cleaned in a targeted manner. ([0381, 0385, 0393])
As to claim 13, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein the determination of a quality value comprises a blindness value due to smoke or condensation, wherein the control unit compares acquired characteristics for smoke or condensation with characteristics of a collection of characteristics stored in the reference database. ([0378, 0380-0381, 0387, 0389])
As to claim 16, Shelton in view of Venkataraman discloses everything as disclosed in claim 1 but is silent to wherein, the cleaning module is designed to adapt the type of cleaning by changing the cleaning parameters depending on the database classes and/or a degree of contamination, wherein one or more cleaning parameters are selected from a group comprising: type of fluid, fluid volume, fluid volumes, fluid velocity, pressure, pulse duration, number of pulses, pulse-pause ratio and/or total cleaning duration.
However, Shelton discloses wherein, the cleaning module is designed to adapt the type of cleaning by changing the cleaning parameters depending on the database classes and/or a degree of contamination ([0378, 0380, 0382, 0385, 0393]), wherein one or more cleaning parameters are selected from a group comprising: type of fluid, fluid volume, fluid volumes, fluid velocity, pressure, pulse duration, number of pulses, pulse-pause ratio and/or total cleaning duration. ([0374, 0392-0393])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to tailor the cleaning action to the detected contamination condition.
As to claim 19, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Venkataraman discloses wherein the self-learning module for building up the reference database receives training data of image structures of characteristic images of objects to be examined, preferably organs or tissue structures, and/or of smoke via an input interface automatically or by an input via a user. ([0006, 0193-0198, 0202])
As to claim 25, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to claim 1. ([0377])
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Shelton et al. (U.S. Publication 2020/0405401) in view of Venkataraman et al. (U.S. Publication 2019/0362834) as applied in claim 1 above further in view of Seo (U.S. Publication 2020/0090322)
As to claim 4, Shelton in view of Venkataraman discloses everything as disclosed in claim 1 but is silent to wherein the self-learning module comprises a model with a neural network, wherein the input data comprises the image data acquired by the image acquisition device, which data can be extracted as individual images, pixels and/or image sections; and wherein the output data comprises the probability of contamination or blindness of the image acquisition device.
However, Seo discloses wherein the self-learning module comprises a model with a neural network ([0022-0023]), wherein the input data comprises the image data acquired by the image acquisition device, which data can be extracted as individual images, pixels and/or image sections ([0029-0030, 0034]); and wherein the output data comprises the probability of contamination or blindness of the image acquisition device. ([0035, 0126])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to improve confidence based determination of whether the acquired image data is visually impaired before activating image optimization.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Shelton et al. (U.S. Publication 2020/0405401) in view of Venkataraman et al. (U.S. Publication 2019/0362834) as applied in claim 1 above further in view of Murthy et al. (U.S. Publication 2018/0247107)
As to claim 7, Shelton in view of Venkataraman discloses everything as disclosed in claim 1 but is silent to wherein the self-learning module has a model with a neural network based on machine learning or deep learning, wherein the neural network is selected from the group comprising: an open neural network; a closed neural network; a single-layer neural network; a multi-layer feedforward network with hidden layers; a feedback neural network; and combinations thereof.
However, Murthy discloses wherein the self-learning module has a model with a neural network based on machine learning or deep learning, wherein the neural network is selected from the group comprising: an open neural network; a closed neural network; a single-layer neural network; a multi-layer feedforward network with hidden layers; a feedback neural network; and combinations thereof. ([0022, 0024, 0028])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to improve automated classification of endoscopic image states from acquired image data.
Claims 12, 14-15 & 22 are rejected under 35 U.S.C. 103 as being unpatentable over
Shelton et al. (U.S. Publication 2020/0405401) in view of Venkataraman et al. (U.S. Publication 2019/0362834) as applied in claim 1 above further in view of Banik et al. (U.S. Publication 2006/0069306)
As to claim 12, Shelton in view of Venkataraman discloses everything as disclosed in claim 1. In addition, Shelton discloses wherein the unit that can be activated for image optimization is selected from the group comprising: one or more light sources of the endoscope for changing the illuminance ([0372] discloses distal end of the imaging device including a visualization lens and a light source); at least one filter for optimizing a contrast; autofocus system for adjusting the sharpness of the image; color spectrum change unit; end lens heating; monitor; user-dependent and/or endoscope- or light guide-dependent customizable software; room illumination of the operating room; and combinations thereof.
Shelton in view of Venkataraman is silent to changing luminance.
However, Banik [0021] discloses adjusting illumination intensity by controlling current supplied to LEDs.
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to improve image visibility when acquired image quality requires optimization.
As to claim 14, Shelton in view of Venkataraman discloses everything as disclosed in claim 2. In addition, Shelton discloses a control instruction to a smoke evacuator and/or an insufflator or a flushing pump ([0136, 0144] discloses a surgical hub including smoke evacuation and suction/irrigation modules.) in order to optimize the image by means of smoke evacuation; and/or to optimize the fluid management in the body cavity to be examined with the endoscope by means of the insufflator or the flushing pump as a function of fluid flows during smoke evacuation or cleaning.
Shelton in view of Venkataraman is silent to express image analysis based control of insufflation or flushing pump fluid management.
However, Banik discloses express image analysis based control of insufflation or flushing pump fluid management. [0028] discloses using images from the distal image sensor to determine whether irrigation or insufflation is required. [0033] discloses activating valves to deliver irrigation liquid and vacuum aspiration. To optimize the fluid management in the body cavity [0036] discloses using environmental measurements to adjust delivery/aspiration volume, rate and washing liquid composition.
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to provide concrete fluid management actuation when image analysis indicates that the field of view requires optimization.
As to claim 15, Shelton in view of Venkataraman discloses everything as disclosed in claim 2 but is silent to wherein the intracorporeal pressure in the body cavity is measured with a pressure sensor, and the control unit controls the intracorporeal pressure in an event and/or time-controlled manner at least during the duration of a cleaning by means of a control of the at least one pump or a control of a pressure regulator such that the intracorporeal pressure does not exceed a predetermined maximum limit value.
However, Banik discloses wherein the intracorporeal pressure in the body cavity is measured with a pressure sensor ([0029, 0036]), and the control unit controls the intracorporeal pressure in an event and/or time-controlled manner at least during the duration of a cleaning by means of a control of the at least one pump or a control of a pressure regulator such that the intracorporeal pressure does not exceed a predetermined maximum limit value ([0039]).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to maintain visualization while preventing excessive body cavity pressure during cleaning.
As to claim 22, Shelton in view of Venkataraman discloses everything as disclosed in claim 2. In addition, Shelton discloses a monitoring routine with the following steps: after cleaning by means of the cleaning module, analyzing the image data by a control unit ([0386] discloses analyzing whether irregular distortions remain after cleaning is completed.); comparing the image data with stored parameters for a positive cleaning result ([0385]); and depending on the cleaning result, interrupting the cleaning or repeating the cleaning ([0386] discloses delaying retriggering if distortions remain after cleaning. [0394] discloses a waiting period between consecutive cleaning activations).
Shelton in view of Venkataraman is silent to manual interruption of a wash routine or aspiration.
However, Banik discloses manual interruption of a wash routine or aspiration. ([0038] discloses manual interruption of a wash routine or aspiration.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to stop or repeat cleaning based on whether image visibility has sufficiently improved.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Shelton et al. (U.S. Publication 2020/0405401) in view of Venkataraman et al. (U.S. Publication 2019/0362834) as applied in claim 1 above further in view of Pang et al. (U.S. Publication 2007/0132839)
As to claim 20, Shelton in view of Venkataraman discloses everything as disclosed in claim 1 but is silent to wherein the following initialization steps take place before or during the acquisition of image data: providing at least one further database with technical data of usable image acquisition devices and/or cleaning modules; comparing the provided image acquisition device with the technical data and detecting the provided image acquisition device; and, depending on the detected image acquisition device, transmitting stored cleaning parameters to the cleaning module for cleaning activation.
However, Pang discloses wherein the following initialization steps take place before or during the acquisition of image data: providing at least one further database with technical data of usable image acquisition devices and/or cleaning modules; comparing the provided image acquisition device with the technical data and detecting the provided image acquisition device; and, depending on the detected image acquisition device, transmitting stored cleaning parameters to the cleaning module for cleaning activation. ([0017-0019, 0020-0023, 0034, 0385, 0393])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Shelton in view of Venkataraman’s disclosure to include the above limitations in order to apply device specific cleaning parameter to the cleaning module.
CONCLUSION
No prior art has been found for claims 5-6, 11, 17-18, 21, 23 & 26 in their current form.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Stephen P. Coleman
Primary Examiner
Art Unit 2675
/STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675
Prosecution Insights