DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “proper spatial frequency range” in claims 1, 8, and 9 is a relative term which renders the claim indefinite. The term “proper” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Thereby it is unclear what is meant by “features in a proper spatial frequency range”.
Regarding claims 1, 8, and 9, the phrase "such as" as used in the limitation “morphological characteristics, such as a size or aspect ratio”, renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claims 7 and 16 recites the limitation "the plurality of feature modules" in lines 2 and 1 respectively. There is insufficient antecedent basis for these limitations in the claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-6 and 9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that Claims 1, 3-6 and 9 are directed to an abstract idea as shown below:
STEP 1: Do the claims fall within one of the statutory categories (i.e. process, a computer readable medium, i.e. a system)? YES. Claims 1-8 and 11-20 are directed to a system, Claims 9-10 are directed to a method.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? YES, the claims are directed towards an abstract idea – mental process.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
- Mathematical concepts — mathematical relationships, mathematical formulas or equations, mathematical calculations;
- Certain methods of organizing human activity — fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations
- Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgement, opinion).
Regarding Claim 1, representative of Claim 9, the claim recites a 3D-object identification and quality assessment system, comprising:
a correspondence indication facility configured to provide for an inspected 3D-object respective sets of correspondence indications, wherein each set of correspondence indications indicates respective correspondences between respective characteristics of the inspected 3D-object and the respective characteristics of a reference object class (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can evaluate characteristics between a 3D object and a reference object), respective characteristics comprising one or more of image features in a proper spatial frequency range, image features in a specific part of the visual spectrum, morphological characteristics, such as a size or aspect ratio, and grades of reflectivity (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can evaluate characteristics between a 3D object and a reference object, particularly size);
an evaluation module configured to receive the respective sets of correspondence indications and to perform the following procedures in a parallel manner:
generating a class indication signal indicative for a most probable object class identified for the inspected 3D object (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can evaluate an object type); and
generating a quality assessment signal that indicates respective values for an extent to which the inspected 3D object meets quality requirements for respective object classes in that the respective values indicate the extent to which the inspected 3D object matches each characteristic of the respective object classes including a value for an extent to which the inspected 3D object meets quality requirements for the most probable one of the object classes (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can judge whether or not an object matches other objects and to what degree depending on a number of matching qualities).
Regarding Claim 3, the claim recites the system according to claim 1, wherein the class indication signal further indicates respective probabilities that the inspected 3D object belongs to respective object classes (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can judge a likelihood of whether or not an object matches other objects).
Regarding Claim 4, the claim recites the system according to claim 1, wherein generating a class indication signal comprises indicating an inspected 3D object as being a member of an object class in accordance with at least a subset of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can judge whether or not an object matches other objects based on matching characteristics).
Regarding Claim 5, the claim recites the system according to claim 1 wherein generating a quality assessment signal comprises indicating that the inspected 3D object meets the quality requirements for an object class in accordance with each of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class (Mental process – concepts easily performed in the human mind including observation, evaluation, and judgement. Examiner notes a human can judge whether or not an object matches other objects and to what degree depending on a number of matching qualities)
Regarding Claim 6, the claim recites the system according to claim 1 wherein the one or more sensing devices comprise one or more of a Time Of Flight based laser scanner, an Infrared grid projection based laser scanner, an RGB, an RGBD camera, a sonar sensors, a radar scanners, an X-ray scanner and a CT-scanner, digital holography imaging device, weighting device (see step 2A prong 2 – insignificant extra solution activity putting limitations on a data gathering step).
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same). As such a human could mentally observe correspondences between characteristics of a 3D object and characteristics of a reference object class, generate a classification for the 3D object, and generate a quality assessment for the 3D object indicating an extent of matching characteristics. The mere nominal recitation that the various steps are being executed by a device/in a device (e.g. processing unit) does not take the limitations out of the mental process grouping. Thus, the claims recite a mental process. Examiner notes that Claims 2, 7, and 8 contain feature vector processing recited as processing that cannot practically be performed in the human mind. Further, claim 11 recites a method of training involving generating loss and backpropagating which are processes that cannot practically be performed in the human mind.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claims 1, 3-6 and 9 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? NO, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claims 1, 3-6 and 9 do not recite any additional elements that are not well-understood, routine or conventional.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-7 and 9-10 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable by Dal (US 20190096135 A1).
Regarding Claim 1, representative of Claim 9, Dal teaches a 3D-object identification and quality assessment system, comprising:
a correspondence indication facility configured to provide for an inspected 3D-object respective sets of correspondence indications, wherein each set of correspondence indications indicates respective correspondences between respective characteristics of the inspected 3D-object and the respective characteristics of a reference object class ([0130]: 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. In operation 1316, the feature vectors for each view are combined (e.g., using max pooling, as described in more detail below) to generate a descriptor for the 3-D model and to classify the object based on the descriptor. This feature vector may contain salient and characteristic aspects of the object's shape, and is used for subsequent classification or retrieval steps, [0131]: shape retrieval will be considered as a special case of classification, in which each shape in the database represents a class in itself, and a shape s is classified with the label of the most similar shape in the database), respective characteristics comprising one or more of image features in a proper spatial frequency range, image features in a specific part of the visual spectrum, morphological characteristics, such as a size or aspect ratio, and grades of reflectivity ([0130]: 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. In operation 1316, the feature vectors for each view are combined (e.g., using max pooling, as described in more detail below) to generate a descriptor for the 3-D model and to classify the object based on the descriptor. This feature vector may contain salient and characteristic aspects of the object's shape, and is used for subsequent classification or retrieval steps);
an evaluation module configured to receive the respective sets of correspondence indications and to perform the following procedures in a parallel manner ([abstract]: A system for visual inspection includes:…an inspection system configured to: compute a descriptor of the object based on the 3-D model of the object; retrieve metadata corresponding to the object based on the descriptor; and compute a plurality of inspection results based on the retrieved metadata and the 3-D model of the object. Examiner interpreting the object classification and defect classification as in a parallel manner since both are derived from the same feature/descriptor data):
generating a class indication signal indicative for a most probable object class identified for the inspected 3D object ([0133]: a convolutional neural network (CNN) is used to process the synthesized 2-D views to generate the classification of the object); and
generating a quality assessment signal that indicates respective values for an extent to which the inspected 3D object meets quality requirements for respective object classes in that the respective values indicate the extent to which the inspected 3D object matches each characteristic of the respective object classes including a value for an extent to which the inspected 3D object meets quality requirements for the most probable one of the object classes ([0022]: The inspection system may be configured to detect defects by: retrieving, from the metadata, a convolutional stage of a convolutional neural network and a defect detector; rendering one or more views of the 3-D model of the object; computing a descriptor by supplying the one or more views of the 3-D model of the object to the convolutional stage of the convolutional neural network; supplying the descriptor to the defect detector to compute one or more defect classifications of the object; and outputting the one or more defect classifications of the object).
Regarding Claim 2, representative of Claim 10, Dal teaches the system according to claim 1. In addition, Dal teaches wherein the correspondence indication facility comprises:
an object information extraction unit comprising one or more sensing devices configured to provide sensing data indicative for sensed physical aspects of a 3D object and to provide a plurality of object information signals, each indicative for a respective characteristic in the sensing data ([0010]: one or more sensing components may include a depth camera, [0091]: image sensors 102a and 104a of the cameras 102 and 104 are RGB-IR image sensors, [0130]: 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. In operation 1316, the feature vectors for each view are combined (e.g., using max pooling, as described in more detail below) to generate a descriptor…feature vector may contain salient and characteristic aspects of the object's shape, and is used for subsequent classification or retrieval steps);
a database configured for storing respective reference feature vector sets for respective characteristics wherein each respective reference feature vector set comprises respective reference feature vectors for respective object classes of 3D-objects, wherein a respective reference feature vector in a respective reference feature vector set is the feature vector expected to be extracted for the respective characteristic in accordance with the 3D-object being a specimen of the respective one of the plurality of object classes ([0145]: the inspection agent 300 performs object identification using a multi-view CNN that has been pre-retrained on generic object classification …search within a database of features vectors of possible identities to be retrieved, [0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects);
a respective comparator for each respective characteristic configured to compare the feature vector for the respective characteristic extracted by its associated feature module with each of the reference feature vectors in the respective feature set for the respective characteristic and to output the respective correspondence indication for the correspondence of the extracted feature vector for the respective characteristic with each of the reference feature vectors for the respective characteristic ([0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects, and a known object corresponding to the current object (e.g., the scanned object or “query object”) can be identified by searching for the closest (e.g. most similar) descriptor in the multi-dimensional space of descriptors, with respect to the descriptor of the current object).
Regarding Claim 3, Dal teaches the system according to claim 1. In addition, Dal teaches wherein the class indication signal further indicates respective probabilities that the inspected 3D object belongs to respective object classes ([0133]: each of the M layers of the second stage CNN.sub.2 is a fully connected layer. The output p of the second stage is a class-assignment probability distribution. For example, if the entire CNN is trained to assign input images to one of k different classes, then the output of the second stage CNN.sub.2 is a vector p that includes k different values, each value representing the probability (or “confidence”) that the input image should be assigned the corresponding class).
Regarding Claim 4, Dal teaches the system according to claim 1. In addition, Dal teaches wherein generating a class indication signal comprises indicating an inspected 3D object as being a member of an object class in accordance with at least a subset of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class ([0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects, and a known object corresponding to the current object (e.g., the scanned object or “query object”) can be identified by searching for the closest (e.g. most similar) descriptor, [0144]: similarity metric is defined to measure the distance between any two given descriptors (vectors)).
Regarding Claim 5, Dal teaches the system according to claim 1. In addition, Dal teaches wherein generating a quality assessment signal comprises indicating that the inspected 3D object meets the quality requirements for an object class in accordance with each of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class ([0019] The inspection system may be configured to detect defects by: retrieving, from the metadata, a reference 3-D model of a canonical instance of a class corresponding to the object; aligning the 3-D model of the object with the reference 3-D model; comparing the 3-D model of the object to the reference 3-D model to compute a plurality of differences between corresponding regions of the 3-D model of the object and the reference 3-D model; and detecting one or more defects in the object when one or more of the plurality of differences exceeds a threshold).
Regarding Claim 6, Dal teaches the system according to claim 1. In addition, Dal teaches wherein the one or more sensing devices comprise one or more of a Time Of Flight based laser scanner ([0031] The one or more sensing components may include a depth camera, [0106]: also be used with other depth camera systems such as structured light time of flight cameras and LIDAR cameras), an Infrared grid projection based laser scanner, an RGB camera ([0091] In some embodiments, the image sensors 102a and 104a of the cameras 102 and 104 are RGB-IR image sensors), an RGBD camera, a sonar sensors, a radar scanners, an X-ray scanner and a CT-scanner, digital holography imaging device, weighting device.
Regarding Claim 7, Dal teaches the system according to claim 1. In addition, Dal teaches wherein the plurality of feature modules comprises at least two feature modules that are configured to extract a respective feature vector for a respective one of a plurality of characteristics from output data obtained from a common sensing device ([0130] In particular, in the embodiment shown in FIG. 7, the descriptor is computed from 2-D views 16 of the 3-D model 240, as rendered by the view generation module 312 in operation 1312. In operation 1314, the synthesized 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. Examiner interpreting feature/descriptor extraction for multiple views as feature modules).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8 and 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Dal (US 20190096135 A1) in view of Lin (US 20060008151 A1).
Regarding Claim 8, Dal teaches a 3-dimensional (3D) object manufacturing system, comprising:
a specification receiving facility configured to receive product specification data ([0077]: As noted above, defect detection is a component of quality control in contexts such as manufacturing, where individual objects may be inspected and analyzed for compliance with particular quality standards);
one or more manufacturing system units configured for manufacturing 3D-objects in accordance with the product specification data ([0171] If the differences between the scanned model and the reference model exceed a threshold value, then the quality control system may flag the scanned object as falling outside of the quality control standards in operation);
an object information extraction unit comprising one or more sensing devices configured to provide sensing data indicative for sensed physical aspects of a 3D object and to provide a plurality of object information signals, each indicative for a respective characteristic in the sensing data, respective characteristics comprising one or more of image features in a proper spatial frequency range, image features in a specific part of the visual spectrum, morphological characteristics, such as a size or aspect ratio, and grades of reflectivity ([0010]: one or more sensing components may include a depth camera, [0091]: image sensors 102a and 104a of the cameras 102 and 104 are RGB-IR image sensors, [0130]: 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. In operation 1316, the feature vectors for each view are combined (e.g., using max pooling, as described in more detail below) to generate a descriptor…feature vector may contain salient and characteristic aspects of the object's shape, and is used for subsequent classification or retrieval steps);
a database configured for storing respective reference feature vector sets for respective characteristics wherein each respective reference feature vector set comprises respective reference feature vectors for respective object classes of 3D-objects, wherein a respective reference feature vector in a respective reference feature vector set is the feature vector expected to be extracted for the respective characteristic in accordance with the 3D-object being a specimen of the respective one of the plurality of object classes ([0145]: the inspection agent 300 performs object identification using a multi-view CNN that has been pre-retrained on generic object classification …search within a database of features vectors of possible identities to be retrieved, [0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects);
a respective comparator for each respective characteristic configured to compare the feature vector for the respective characteristic extracted by its associated feature module with each of the reference feature vectors in the respective feature set for the respective characteristic and to output a respective correspondence indication for the correspondence of the extracted feature vector for the respective characteristic with each of the reference feature vectors for the respective characteristic ([0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects, and a known object corresponding to the current object (e.g., the scanned object or “query object”) can be identified by searching for the closest (e.g. most similar) descriptor in the multi-dimensional space of descriptors, with respect to the descriptor of the current object);
an evaluation module configured to receive the respective sets of correspondence indications and to perform the following procedures in a parallel manner ([abstract]: A system for visual inspection includes:…an inspection system configured to: compute a descriptor of the object based on the 3-D model of the object; retrieve metadata corresponding to the object based on the descriptor; and compute a plurality of inspection results based on the retrieved metadata and the 3-D model of the object. Examiner interpreting the object classification and defect classification as in a parallel manner since both are derived from the same feature/descriptor data):
generating a class indication signal indicative for a most probable object class identified for the inspected 3D object ([0133]: a convolutional neural network (CNN) is used to process the synthesized 2-D views to generate the classification of the object); and
generating a quality assessment signal that indicates respective values for an extent to which the inspected 3D object meets quality requirements for respective object classes in that the respective values indicate the extent to which the inspected 3D object matches each characteristic of the respective object classes including a value for an extent to which the inspected 3D object meets quality requirements for the most probable one of the object classes ([0022]: The inspection system may be configured to detect defects by: retrieving, from the metadata, a convolutional stage of a convolutional neural network and a defect detector; rendering one or more views of the 3-D model of the object; computing a descriptor by supplying the one or more views of the 3-D model of the object to the convolutional stage of the convolutional neural network; supplying the descriptor to the defect detector to compute one or more defect classifications of the object; and outputting the one or more defect classifications of the object);
at least an actuator system unit configured to receive an inspected 3D-object and to act on the 3D-object in accordance with the identification signal and the quality assessment signal ([0148] Defect Detection, [0149] Once an object has been identified, the 3-D model analysis module 350 of the inspection agent 300 analyzes the input 3-D model (and, in some instances, its frames), in the context of the retrieved data about the identified class, in order to provide some insights and analysis about the object itself, [0057] FIG. 5A is a schematic diagram of a scanning system configured to scan objects on a conveyor belt according to one embodiment of the present invention), wherein:
Although Dal teaches an actuator system unit in at least [0057], Dal does not explicitly teach the remaining limitations of Claim 8.
Lin explicitly teaches the actuator system unit redirects the 3D-object to a further manufacturing/processing stage so as to achieve that a further manufacturing/processing step is applied according to the class of the 3D-object in case the 3D-object is a semi-finished product meeting all quality requirements; and/or
the actuator system unit redirects the 3D-object to a packaging line where the 3D- object is to be packaged in accordance with its identified 3D-object class in case the 3D-object is a finished product meeting all quality requirements ([0108]: if the system determines that an item being inspected matches a reference object or symbol that corresponds to a malformed or "bad" item, a motion control operation may be initiated to remove the offending item from the conveyor belt, [0190]: For example, in an automated manufacturing application where manufactured parts are automatically inspected for quality, a part successfully characterized as a "pass" may be sent on for packaging or further assembly); and/or
the actuator system unit having identified an 3D object of a particular object-class that does not meet all quality requirements returns the 3D-object to a previous manufacturing/processing stage for that object-class to achieve that a preceding manufacturing/processing step is repeated in order to achieve that the 3D-object meets all quality requirements; and/or
the actuator system unit having identified a 3D object of a particular object-class that does not meet all quality requirements forwards the 3D-object to a manual inspection station; and/or
the actuator system unit having encountered an unidentified 3D object forwards the 3D-object to a manual inspection station; and/or
the actuator system unit having identified a 3D object of a particular object-class that does not meet all quality requirements forwards the 3D-object to recycling station; and/or
the actuator system unit having encountered an unidentified 3D object forwards the 3D-object to a recycling station.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Dal to include the teachings of Lin by including an actuator system that automatically moves the object to a packaging line when quality assessments are met. Doing so would provide an obvious application of 3D object inspection and defect detection, improving the accuracy of 3D object manufacturing by packaging the objects that meet quality requirements.
Regarding Claim 12, the Dal and Lin combination teaches the system according to claim 8. In addition, Dal teaches wherein the class indication signal further indicates respective probabilities that the inspected 3D object belongs to respective object classes ([0133]: each of the M layers of the second stage CNN.sub.2 is a fully connected layer. The output p of the second stage is a class-assignment probability distribution. For example, if the entire CNN is trained to assign input images to one of k different classes, then the output of the second stage CNN.sub.2 is a vector p that includes k different values, each value representing the probability (or “confidence”) that the input image should be assigned the corresponding class).
Regarding Claim 13, representative of Claim 17, the Dal and Lin combination teaches the system according to claim 8. In addition, Dal teaches wherein generating a class indication signal comprises indicating an inspected 3D object as being a member of an object class in accordance with at least a subset of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class ([0143]: the descriptor vector is used to query a database of objects for which are associated with descriptors that were previously computed using the same technique. This database of objects constitutes a set of known objects, and a known object corresponding to the current object (e.g., the scanned object or “query object”) can be identified by searching for the closest (e.g. most similar) descriptor, [0144]: similarity metric is defined to measure the distance between any two given descriptors (vectors)).
Regarding Claim 14, representative of Claim 18, the Dal and Lin combination teaches the system according to claim 8. In addition, Dal teaches wherein generating a quality assessment signal comprises indicating that the inspected 3D object meets the quality requirements for an object class in accordance with each of the correspondence indications being indicative for a correspondence of the inspected 3D object with the object class ([0019] The inspection system may be configured to detect defects by: retrieving, from the metadata, a reference 3-D model of a canonical instance of a class corresponding to the object; aligning the 3-D model of the object with the reference 3-D model; comparing the 3-D model of the object to the reference 3-D model to compute a plurality of differences between corresponding regions of the 3-D model of the object and the reference 3-D model; and detecting one or more defects in the object when one or more of the plurality of differences exceeds a threshold).
Regarding Claim 15, representative of Claim 19, the Dal and Lin combination teaches the system according to claim 8. In addition, Dal teaches wherein the one or more sensing devices comprise one or more of a Time Of Flight based laser scanner ([0031]: the one or more sensing components may include a depth camera, [0106]: also be used with other depth camera systems such as structured light time of flight cameras and LIDAR cameras), an Infrared grid projection based laser scanner, an RGB camera ([0091] In some embodiments, the image sensors 102a and 104a of the cameras 102 and 104 are RGB-IR image sensors), an RGBD camera, a sonar sensors, a radar scanners, an X-ray scanner and a CT-scanner, digital holography imaging device, weighting device.
Regarding Claim 16, representative of Claim 20, the Dal and Lin combination teaches the system according to claim 8. In addition, Dal teaches wherein the plurality of feature modules comprises at least two feature modules that are configured to extract a respective feature vector for a respective one of a plurality of characteristics from output data obtained from a common sensing device ([0130] In particular, in the embodiment shown in FIG. 7, the descriptor is computed from 2-D views 16 of the 3-D model 240, as rendered by the view generation module 312 in operation 1312. In operation 1314, the synthesized 2-D views are supplied to a descriptor generator 314 to extract a descriptor or feature vector for each view. Examiner interpreting feature/descriptor extraction for multiple views as feature modules).
Claim(s) 11 is rejected under 35 U.S.C. 103 as being unpatentable over Dal (US 20190096135 A1) in view of Memo (US 20180322623 A1).
Regarding Claim 11, Dal teaches a method of training the 3D-object identification and quality assessment system of claim 1. However, Dal does not explicitly teach the remaining limitations of Claim 11.
Memo teaches wherein the system comprises a plurality of trainable feature modules to be trained for generating a respective feature vector from an object information input signal ([0003] Quality control in manufacturing typically involves inspecting manufactured products to detect defects. [0006]: computing, by the processor, a descriptor by supplying the one or more views of the 3-D model to a convolutional stage of a convolutional neural network; supplying, by the processor, the descriptor to a defect detector to compute one or more defect classifications of the target object; and outputting the one or more defect classifications of the target object), and wherein the method of training comprises:
acquiring a respective object information input signal for each of a plurality of object classes ([0010] The convolutional neural network may be trained based on an inventory including: a plurality of 3-D models of a plurality of defective objects, each 3-D model of the defective objects having a corresponding defect classification; and a plurality of 3-D models of a plurality of non-defective objects);
for each object information input signal generating mutually different types of augmented sets of object information input signals, wherein each type of augmented set is obtained by varying N-1 of a number of N signal properties, wherein N is a natural number that is at least equal to the number of augmented sets and wherein a respective one of the N properties is not varied in each of the mutually different augmented sets ([0168] In some embodiments of the present invention, the size of the training set is increased by synthetically generating samples of defective surfaces, [0169]: Furthermore, the size and shape of the wrinkles may be modified (in accordance with the expected distribution of shapes and sizes of wrinkles.) The model thus obtained may represent an additional synthetic defective sample that can be used for training the classifier);
providing a plurality of supersets, wherein each superset comprises all augmented sets of a respective type ([0168] In some embodiments of the present invention, the size of the training set is increased by synthetically generating samples of defective surfaces from a probability distribution that is assumed to represent the variability of surfaces affected by that defect, [0054]: a system may be trained using labeled training data, which may include captured images of defective objects 14d and captured images of good (or “clean”) objects 14c);
using each of the supersets to train a respective feature module, wherein training a respective feature module with a respective superset comprises repeating the following steps:
selecting a pair of a first object information input signal and a second object information input signal from the respective superset ([0106]: In some embodiments, the descriptor of a target object is compared against a descriptor corresponding to one or more non-defective or clean objects),
with the trainable feature module computing a respective feature vector for the first and the second object information input signal ([0103]: representation of the object 10 which is summarized in a feature vector or “descriptor” F, ([0106]: In some embodiments, the descriptor of a target object is compared against a descriptor corresponding to one or more non-defective or clean objects),
computing a distance value indicative for a distance between the feature vectors ([0106]: In some embodiments, the descriptor of a target object is compared against a descriptor corresponding to one or more non-defective or clean objects, and any discrepancy or distance between the descriptor of the target object and the one or more descriptors of the non-defective objects is used as an indication of the possible presence of a defect),
generating a loss value by comparing the distance value with a predetermined distance value depending on whether or not the first and the second object information input signals are augmented object information input signals derived from an object of a same object class ([0181]: The quality of the resulting classification depends on the ability of the descriptors (computed as described above) to convey discriminative information about the surfaces. In some embodiments, the network used to compute the descriptors is tuned based on the available samples. This can be achieved, for example, using a “Siamese network” trained with a contrastive loss…contrastive loss encourages descriptors of objects within the same class (defective or non-defective) to have small Euclidean distance, and penalizes descriptors of objects from different classes with similar Euclidean distance),
training the trainable feature module by backpropagation of the loss value ([0153] The parameters of the neural network (e.g., the weights of the connections between the layers) can be learned from the training data using standard processes for training neural network such as backpropagation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Dal to include the teachings of Memo by including a training method involving augmented datasets. Doing so would improve the accuracy of classification by reducing class imbalance.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights