Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Method and Apparatus for Training an Encoder-Decoder Model on a Formation of Drones to Inspect Chemical Fiber Products in a Spinning Workshop.
The abstract of the disclosure is objected to because it does not “enable the Office and the public generally to determine quickly from a cursory inspection the nature and gist of the technical disclosure.” 37 CFR 1.72(b). Specifically, the abstract appears to describe a generic, known, encoder-decoder model.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
While the legal analysis below focuses on dependency, the underlying issue is whether Applicant needs to redraft claims such that they are correctly charged for additional independent claims.
Claim 18 references claim 7, but does not properly depend from claim 7 because the instructions can exist without performance of any of the method steps. Here, claim 7 is a method but claim 18 is an apparatus, and the apparatus claim can be met without necessarily practicing the method. MPEP 608.01(n)(III) addresses the “test for proper dependency.”
MPEP 607(III) states:
Any claim which is in dependent form but which is so worded that it, in fact, is not a proper dependent claim, as for example it does not include every limitation of the claim on which it depends, will be required to be canceled as not being a proper dependent claim; and cancellation of any further claim depending on such a dependent claim will be similarly required. The applicant may thereupon amend the claims to place them in proper dependent form, or may redraft them as independent claims, upon payment of any necessary additional fee.
Claim 18 is such a claim because it is directed to a storage medium rather than a method as in referenced claim 7. MPEP 608.01(n)(III). While, in the interest of compact prosecution, claim 18 has been examined, claim 18 is required to be cancelled.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-18 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claims are replete with errors. The examiner has provided examples below.
Note that many of these indefiniteness rejections appear to be due to the use of translations that are not high enough quality for US patent practice. US patent law requires precise language. See, for example, MPEP 2111 discussing how claims are interpreted. Here, for example, it appears likely that issues such as “new terminology” may be overcome with a better translation.
Claims 1, 9, and 14 recite “fault detection model,” but this is new terminology. MPEP 2173.05(a). One way to overcome this issue is to specify what type of fault is being detected, such that the claimed type of fault serves as a description.
Claims 1, 9, and 14 recite “designated device,” but this is subjective terminology. MPEP 2173.05(b)(IV). Here, different people can have different opinions as to which device is designated because the claim does not specify how the designation occurs, or alternatively, how to know which device was designated.
Claims 1, 9, and 14 recite “preset drone formation,” but “preset” is subjective terminology. MPEP 2173.05(b)(IV). The analysis for “preset” matches the above analysis for “designated.”
Claims 1, 9, and 14 recite “sample sequence,” but this is new terminology. MPEP 2173.05(a). What is the sequence of? Does the word “sample” mean that it was created by sampling or that it is a sample of the sequence?
Claims 1, 9, and 14 recite “sampling a designated device based on a preset drone formation to obtain a sample sequence,” but this does not make sense because the claim lacks sufficient context. Is the designated device a drone? What else might it be? Does the sampling obtain the entire sample sequence?
Claims 1, 9, and 14 recite “position encoding.” While position encoding has a technical meaning, that meaning is generally in the context of servo motors, and is not what is described in the specification. Here, Applicant is using a different meaning without a sufficient redefinition in the specification. MPEP 2173.05(a)(III).
Claims 1, 9, and 14 recite “drone formation encoding result,” but this is new terminology. MPEP 2173.05(a).
Claims 1, 9, and 14 recite “each sample image in the sample sequence,” but this lacks sufficient antecedent basis because there is not a prior recitation that the sample sequence has sample images.
Claims 1, 9, and 14 recite “a multi-scale feature of each sample image,” but it is unclear if the literal meaning is intended (that the feature needs to be of the entire image) or if this is a translation issue and “of” should be “from.”
Claims 1, 9, and 14 twice refer to “scale” in the limitation that begins “perform feature fusion,” but these references are to a singular scale, whereas the only previous recitation of “scale” was “multi-scale.” Thus, the antecedent basis is unclear.
Claims 1, 9, and 14 recite “perform feature fusion on fused features in all scales to obtain a target feature,” but the plain meaning of this phrase doesn’t make sense in English.
Claims 1, 9, and 14 recite “fault sample map,” but this is new terminology. MPEP 2173.05(a).
Claim 7 recites “applied to the fault detection model of claim 1,” but it is unclear whether or not this means that claim 7 requires all of claim 1 to occur. If the intent is that claim 1 occurs, Applicant may wish to write claim 7 as a more standard dependent claim. If the intent is that not all of claim 1 is required, Applicant may wish to write claim 7 as an independent claim that does not reference claim 1.
Claims 14, 15, and 18 recite “computer instruction is used” but this is improperly including a method step in an apparatus claim. MPEP 2173.05(p)(II).
Claims 14, 15, and 18 recite “computer instruction is used” but this is confusing, because in context this appears to be a poor translation of storing instructions (plural) to perform the recited steps, but the literal meaning is a single instruction that could be used for various things (e.g., a load or add instruction could be used to accomplish a wide range of programs).
Dependent claims are likewise rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 (all claims) are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more.
Step 1: Claim 1 (and its dependents) recite a method, and processes satisfy Step 1 of the eligibility test.
Claim 8 (and its dependents) recite a device, and machines satisfy Step 1 of the eligibility test.
Claim 14 (and its dependents) recite a non-transitory computer-readable storage medium, and manufactures satisfy Step 1 of the eligibility test. Claim 18 also recites a non-transitory computer-readable storage medium.
Step 2A, prong one: All of the elements of the claims are a mental process because the claims are directed to generically using artificial intelligence. Further, the various models are also mental processes, see example 47, claim 2, element (d) (from the July 2024 AI subject matter eligibility examples). MPEP 2106.04(a)(2)(III)(C) explains that use of a generic computer or in a computer environment is still a mental process. In particular, this section begins by citing Gottschalk v. Benson, 409 US 63 (1972). “The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea.” In Benson the Supreme Court did not separately analyze the computer hardware at issue; the specifics of what hardware was claimed is only included in an appendix to the decision.
Because there are no additional elements, no further analysis is required for Step 2A, prong two or Step 2B.
Additionally, specification [0003] explains that the present invention is automating what had been a manual process. Thus, the claim elements are merely placing the abstract idea of visually inspecting spinning processes in the technological environment of drones that use computer vision.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 7-11, 13-16, and 18 (the remaining claims are rejected under 103, below) are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nogueira JC, Hadano FS, Deschamps F, Marques A, Teodoro A, Valle PD. Pattern Recognition and Oxidation Classification in Metal Structures of Industrial Roofs Using Artificial Intelligence. In Transdisciplinarity and the Future of Engineering 2022 (pp. 433-442). IOS Press. (“Nogueira”)
1. A training method for a fault detection model, comprising:
sampling a designated device based on a preset drone formation to obtain a sample sequence; (Nogueira, section 1.2, “202 images of Plant A obtained by UAV A”)
performing position encoding on the sample sequence according to the drone formation to obtain a drone formation encoding result; (Nogueira, section 1.2, “202 images of Plant A obtained by UAV A and 89 images of Plant A captured by UAV B” UAV A, as opposed to B teaches the claimed position encoding (i.e., A is before B))
inputting the sample sequence and the drone formation encoding result into a model to be trained to obtain a fault detection result output by the model to be trained; (Nogueira, section 2.2, “four different training arrangements were done on two distinct test sprints.” The data from section 1.2, cited above, is the present training data.)
determining a loss value based on the fault detection result and a true value of a fault detection result of the sample sequence; and (Nogueira, section 2.2, table 2. Nogueira’s validation teaches the claimed loss value and true value.)
adjusting a model parameter of the model to be trained based on the loss value to obtain the fault detection model; (Nogueira, section 2.2, “A small reduction on Q1 and Q3 values was observed as data from the Cap dataset was introduced to the training process. Analyzing Table 2, the greatest change was observed when the training phase was done with data from Cap’s rooftops and oxidations, where the median value suffered a reduction from 0.51 to 0.55.”)
wherein the model to be trained comprises an encoder and a decoder; (See the below mappings)
the encoder is configured to perform feature extraction on each sample image in the sample sequence to obtain a multi-scale feature of each sample image; (Nogueira, section 1.4, Fig. 1, “Feature extractor for rooftops”)
perform feature fusion on features in a same scale of the sample sequence based on the drone formation encoding result to obtain a fused feature corresponding to each scale; and (Nogueira, section 1.4, Fig. 1, “Feature extractor for oxidations”)
perform feature fusion on fused features in all scales to obtain a target feature; and (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations”)
the decoder is configured to determine a fault detection result of the designated device based on the target feature, wherein the fault detection result comprises a fault prediction type and a fault prediction box of a same fault position in a fault sample map; and (Nogueira, abstract, “In the third stage, the network performed the criticality classification of the detected failures.”)
splice sample images in the sample sequence with reference to the drone formation to obtain the fault sample map. (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations.” Fig. 1 shows that the final image (i.e., the claimed map) is a result of splicing together the subdivisions.)
2. The method of claim 1, further comprising: determining the true value of the fault detection result of the sample sequence by:
splicing the sample sequence into the fault sample map according to the drone formation, wherein the fault sample map describes a state of the designated device from a plurality of drone perspectives; (Nogueira, section 2.1.1, Fig. 2)
constructing first prompt information based on the fault sample map, wherein the first prompt information comprises a fault point of at least one fault in the fault sample map, and position information of detection boxes of a same fault in different drone perspectives in the fault sample map is used as sub-position parameters; (Nogueira, section 1.4, Fig. 1, “Original image”)
performing position encoding on the sub-position parameters of the same fault to obtain a fault position code of the same fault; and (Nogueira, section 1.4, Fig. 1, “Feature extractor for rooftops” The rooftop oxidations (i.e., rust) teach the claimed faults.)
for each fault, performing following operations:
inputting a fault point of the fault and a fault position code of the fault as second prompt information into an everything segmentation model, so that the everything segmentation model segments out a fault mask map of the fault from the fault sample map; and (Nogueira, section 1.4, Fig. 1, “Image with extracted rooftop”)
obtaining a true class label of the fault mask map of the fault, and constructing a detection box label of the fault based on position information of the fault mask map in the fault sample map, to obtain the true value of the fault detection result. (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations”)
3. The method of claim 1, wherein a loss function of the model to be trained comprises following loss items:
position loss between the fault prediction box and detection box label; and (Nogueira, section 2.2. Nogueira’s rooftops teach the claimed positions of the faults because the oxidation (i.e., faults) occur on rooftops))
classification loss between the fault prediction type and true class label. (Nogueira, section 2.2)
7. A device fault detection method, applied to the fault detection model of claim 1, comprising:
obtaining an initial image set of a target device based on a drone queue, wherein the drone queue is used to collect images of the target device from a plurality of perspectives to obtain the initial image set; (Nogueira, section 1.2, “202 images of Plant A obtained by UAV A and 89 images of Plant A captured by UAV B” UAV A, as opposed to B teaches the claimed position encoding (i.e., A is before B))
denoising each initial image in the initial image set to obtain a set of images to be detected; (Nogueira, section 2.1.1, Fig. 2. The use of the less blurry image teaches the claimed denoising.)
performing position encoding on the drone queue to obtain a drone position encoding result; and (Nogueira, section 1.2, “202 images of Plant A obtained by UAV A and 89 images of Plant A captured by UAV B” UAV A, as opposed to B teaches the claimed position encoding (i.e., A is before B))
inputting the set of images to be detected and the drone position coding result into the fault detection model to obtain a fault detection result of the fault detection model for the target device; (Nogueira, section 1.4, Fig. 1, “Feature extractor for oxidations”)
wherein the fault detection result comprises a fault prediction type and a fault prediction box of a same fault position in a target map; and (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations”)
the target map is obtained by splicing the images to be detected in the set of images to be detected with reference to the drone queue. (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations”)
8. The method of claim 7, further comprising:
screening out at least one key prediction box from a plurality of fault prediction boxes of the same fault position; (Nogueira, section 1.4, Fig. 1, “640x640 subdivisions”)
separating out an image to be detected of each key prediction box from the target map based on the at least one key prediction box; and (Nogueira, section 1.4, Fig. 1, “Feature extractor for oxidations”)
constructing and outputting a three-dimensional effect graph of the same fault position based on the image to be detected of each key prediction box. (Nogueira, section 1.4, Fig. 1, “Final image with extracted oxidations”)
Claims 9-11, 13-16, and 18 are rejected as per their counterpart claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 5, 12, and 17 (the remaining claims are rejected either above or below) are rejected under 35 U.S.C. 103 as being unpatentable over Nogueira JC, Hadano FS, Deschamps F, Marques A, Teodoro A, Valle PD. Pattern Recognition and Oxidation Classification in Metal Structures of Industrial Roofs Using Artificial Intelligence. In Transdisciplinarity and the Future of Engineering 2022 (pp. 433-442). IOS Press. (“Nogueira”) in view of Frenkel L, Goldberger J. Network calibration by temperature scaling based on the predicted confidence. In 2022 30th European Signal Processing Conference (EUSIPCO) 2022 Aug 29 (pp. 1586-1590). IEEE. (“Frenkel”)
The equation is not reproduced correctly due to word processing limitations, see the claims for the original.
4. Nogueira teaches the method of claim 3, but is not relied on for the below claim language.
However, Frenkel teaches wherein the classification loss is determined based on a following formula: lcls=-1N∑iNlogef(ai,bi)/ρτ∑b'ef(ai,b')/ρτ wherein N is a quantity of fault positions in the fault sample map, and a same fault position is counted once in N when there are a plurality of sample images describing the same fault position in the fault sample map; ai is an i-th fault position; f(ai, bi) is a true class label of the i-th fault position; f(ai, b') is statistic of a prediction score of each sample image for the i-th fault position when the i-th fault position appears in a plurality of sample images of the fault sample map; and ρτ is a temperature scalar. (Frenkel, equation (4))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Frenkel to the teachings of Nogueira such that Frenkel’s network calibration is used on Nogueira’s neural network(s) for the purpose of improving calibration. Frenkel, abstract.
Based on the above, this is an example of “combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
5. The method of claim 4, wherein the statistic is a mean value, a mode or a maximum value. (Frenkel, equation (4). See also the text right above equation (4), “average confidence”)
Claims 12 and 17 are rejected as per their counterpart claims.
Claim 6 (the other claims are rejected above) is rejected under 35 U.S.C. 103 as being unpatentable over Nogueira JC, Hadano FS, Deschamps F, Marques A, Teodoro A, Valle PD. Pattern Recognition and Oxidation Classification in Metal Structures of Industrial Roofs Using Artificial Intelligence. In Transdisciplinarity and the Future of Engineering 2022 (pp. 433-442). IOS Press. (“Nogueira”) in view of Žbontar J, LeCun Y. Stereo matching by training a convolutional neural network to compare image patches. Journal of Machine Learning Research. 2016;17(65):1-32. (“Žbontar”)
6. Nogueira teaches the method of claim 3, but is not relied on for the remainder of claim 6. The remainder of claim 6 is taught by Žbontar, section 4.1, including Fig. 4. As explained in the abstract, Žbontar teaches aligning stereo images with supervised training of a convolutional neural network. As referenced in the abstract and detailed in section 4.1, Žbontar uses cross-based cost aggregation, which includes the claimed position loss sub-items.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Žbontar to the teachings of Nogueira such that Žbontar’s cross-based cost aggregation is used to identify objects in Nogueira’s images for the purpose of improving the error rate of object detection. Žbontar, section 4.1.
Based on the above, this is an example of “combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
20240070850 – title, “Apparatus and method for image stitching based on artificial intelligence for inspecting wind turbines”
20220036537 – title, “Systems and methods for detecting blight and code violations in images”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/Primary Examiner, Art Unit 2663
Prosecution Insights