DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-15, as amended, are currently pending and have been considered below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “learning data acquisition part which acquires,” “learning part which uses,” “multi-filter processing part for processing,” “feature extraction image generation part for generating,” “feature extraction part for processing,” “feature matching part for comparing” and “control part for controlling” in claims 1, 6-8, 10 and 12-15.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 14 and 15 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kobayashi et al., U.S. Publication No. 2019/0370974, hereinafter, “Kobayashi”.
As per claim 14, Kobayashi discloses a feature extraction device for extracting a feature of a workpiece from an image in which the workpiece is captured (Kobayashi, ¶0024, FIG. 2B is a diagram illustrating the relationship among the imaging unit 101, the targets 301, the environment 302, and the conveying unit 303. The imaging unit 101 disposed over the environment 302 captures an image of the plurality of targets 301 disposed in the environment 302; Kobayashi, ¶0043, extracting regions in the vicinity of a specific edge detected), the device comprising:
a multi-filter processing part for processing the image in which the workpiece is captured using a plurality of different filters to generate a plurality of filtered images (Kobayashi, ¶0037, The edge detection unit 204 performs the edge detection by applying the Canny filter to the conversion images 311a to 311d. The edge detection unit 204 generates a luminance gradient value map storing luminance gradient values of individual pixels and a luminance gradient direction map storing luminance gradient directions of the individual pixels for each of the conversion images 311a to 311d while performing the edge detection using the Canny filter ... “Fx” and “Fy” indicate kernels in horizontal and vertical directions of the Prewitt filter; Kobayashi, ¶0075-0078) and
a feature extraction image generation part for generating and outputting a feature extraction image of the workpiece by compositing the plurality of filtered images based on a composite ratio for each corresponding section of the plurality of filtered images (Kobayashi, ¶0018, generates conversion images by reducing a size of a captured image in different magnifications and performs edge detection on the individual conversion images. By reducing a size of an image in a plurality of magnifications, optimum edges are detected relative to the image deterioration in one of the conversion images. Since degrees of image deterioration are uneven in an image, optimum edges are detected in different conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of a luminance gradient value ... The reliability is defined by a difference between luminance gradients of adjacent pixels. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region; Kobayashi, ¶0026, An image conversion unit 202 converts the input captured image 310 by image processing to generate a plurality of conversion images 311a to 311d. The image conversion unit 202 transmits the generated conversion images 311a to 311d to an edge detection unit 204; Kobayashi, ¶0027, The edge detection unit 204 performs an edge detection process on the conversion images 311a to 311d. The edge detection unit 204 generates a detection edge list 320 and a luminance gradient value map to be transmitted to a reliability computation unit 205; Kobayashi, ¶0029, The edge selection unit 206 obtains the detection edge reliability 334 from the detection edge list 320 and selects one of the edges in the conversion images 311a to 311d which has a highest reliability to generate a selection edge list 340. The edge selection unit 206 transmits the generated selection edge list 340 to the comparison unit 103; Kobayashi, ¶0037, The edge detection unit 204 performs the edge detection by applying the Canny filter to the conversion images 311a to 311d. The edge detection unit 204 generates a luminance gradient value map storing luminance gradient values of individual pixels and a luminance gradient direction map storing luminance gradient directions of the individual pixels for each of the conversion images 311a to 311d while performing the edge detection using the Canny filter ... “Fx” and “Fy” indicate kernels in horizontal and vertical directions of the Prewitt filter; Kobayashi, ¶0041, The edge detection unit 204 generates the detection edge lists 320 for the individual conversion images 311a to 311d; Kobayashi, ¶0043, FIG. 5 is a concept diagram illustrating content of the process performed by the reliability computation unit 205 in step S1040. Partial images 312a to 312d are obtained by extracting regions in the vicinity of a specific edge detected in the conversion images 311a to 311d; Kobayashi, ¶0046, The edge selection unit 206 obtains the detection edge lists 320 and selects edges of high reliability to generate the selection edge list 340; Kobayashi, ¶0069, Different regions may be obtained for different portions of the targets 301; Kobayashi, ¶0073, the information processing apparatus 200 generates conversion images by reducing a size of a captured image in a plurality of magnifications and performs edge detection on the individual conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of luminance gradient values. The reliability indicates stability of the edge detection. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region. The comparison unit 103 computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, even in a case where uneven image deterioration is generated for each region in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed; Kobayashi, ¶0077-0078).
As per claim 15, Kobayashi discloses a controller for controlling operations of a machine based on at least one of a position and posture of a workpiece detected from an image in which the workpiece is captured (Kobayashi, ¶0024, FIG. 2B is a diagram illustrating the relationship among the imaging unit 101, the targets 301, the environment 302, and the conveying unit 303. The imaging unit 101 disposed over the environment 302 captures an image of the plurality of targets 301 disposed in the environment 302; Kobayashi, ¶0043, extracting regions in the vicinity of a specific edge detected), the controller comprising:
a feature extraction part for processing the image in which the workpiece is captured with a plurality of different filters to generate a plurality of filtered images (Kobayashi, ¶0037, The edge detection unit 204 performs the edge detection by applying the Canny filter to the conversion images 311a to 311d. The edge detection unit 204 generates a luminance gradient value map storing luminance gradient values of individual pixels and a luminance gradient direction map storing luminance gradient directions of the individual pixels for each of the conversion images 311a to 311d while performing the edge detection using the Canny filter ... “Fx” and “Fy” indicate kernels in horizontal and vertical directions of the Prewitt filter),
compositing the plurality of filtered images based on a composite ratio for each corresponding section of the plurality of filtered images and extracting a feature of the workpiece (Kobayashi, ¶0018, generates conversion images by reducing a size of a captured image in different magnifications and performs edge detection on the individual conversion images. By reducing a size of an image in a plurality of magnifications, optimum edges are detected relative to the image deterioration in one of the conversion images. Since degrees of image deterioration are uneven in an image, optimum edges are detected in different conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of a luminance gradient value ... The reliability is defined by a difference between luminance gradients of adjacent pixels. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region; Kobayashi, ¶0026, An image conversion unit 202 converts the input captured image 310 by image processing to generate a plurality of conversion images 311a to 311d. The image conversion unit 202 transmits the generated conversion images 311a to 311d to an edge detection unit 204; Kobayashi, ¶0027, The edge detection unit 204 performs an edge detection process on the conversion images 311a to 311d. The edge detection unit 204 generates a detection edge list 320 and a luminance gradient value map to be transmitted to a reliability computation unit 205; Kobayashi, ¶0029, The edge selection unit 206 obtains the detection edge reliability 334 from the detection edge list 320 and selects one of the edges in the conversion images 311a to 311d which has a highest reliability to generate a selection edge list 340. The edge selection unit 206 transmits the generated selection edge list 340 to the comparison unit 103; Kobayashi, ¶0037, The edge detection unit 204 performs the edge detection by applying the Canny filter to the conversion images 311a to 311d. The edge detection unit 204 generates a luminance gradient value map storing luminance gradient values of individual pixels and a luminance gradient direction map storing luminance gradient directions of the individual pixels for each of the conversion images 311a to 311d while performing the edge detection using the Canny filter ... “Fx” and “Fy” indicate kernels in horizontal and vertical directions of the Prewitt filter; Kobayashi, ¶0041, The edge detection unit 204 generates the detection edge lists 320 for the individual conversion images 311a to 311d; Kobayashi, ¶0043, FIG. 5 is a concept diagram illustrating content of the process performed by the reliability computation unit 205 in step S1040. Partial images 312a to 312d are obtained by extracting regions in the vicinity of a specific edge detected in the conversion images 311a to 311d; Kobayashi, ¶0046, The edge selection unit 206 obtains the detection edge lists 320 and selects edges of high reliability to generate the selection edge list 340; Kobayashi, ¶0069, Different regions may be obtained for different portions of the targets 301; Kobayashi, ¶0073, the information processing apparatus 200 generates conversion images by reducing a size of a captured image in a plurality of magnifications and performs edge detection on the individual conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of luminance gradient values. The reliability indicates stability of the edge detection. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region. The comparison unit 103 computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, even in a case where uneven image deterioration is generated for each region in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed),
a feature matching part for comparing the extracted feature of the workpiece with a model feature extracted from a model image in which the workpiece, for which at least one of position and posture is known, is captured (Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model; Kobayashi, ¶0030, The comparison unit 103 computes position and orientations of the targets 301 which have been captured in the captured image 310 based on the selection edge list 340 generated by the edge selection unit 206 and a model edge list stored in an edge model storage unit 104. The comparison unit 103 transmits the computed position and orientations of the targets 301 to the holding unit 102; Kobayashi, ¶0050, The comparison unit 103 obtains the selection edge list 340 generated by the edge selection unit 206 and the model edge list stored in the edge model storage unit 104 and computes position and orientations of the targets 301 which have been captured as the captured image 310 by means of a repetitive process), and
detecting at least one of the position and posture of the workpiece, for which at least one of position and posture is unknown (Kobayashi, ¶0073, the information processing apparatus 200 generates conversion images by reducing a size of a captured image in a plurality of magnifications and performs edge detection on the individual conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of luminance gradient values. The reliability indicates stability of the edge detection. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region. The comparison unit 103 computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, even in a case where uneven image deterioration is generated for each region in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed), and
a control part for controlling the operations of the machine based on at least one of the detected position and posture of the workpiece (Kobayashi, ¶0023, the holding unit 102 is realized by a combination of a robot arm included in the conveying unit 303 and an end effector, which performs a holding operation on the targets 301).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wilbee, A. J. A Framework For Learning Scene Independent Edge Detection. Rochester Institute of Technology, 2015, hereinafter, “Wilbee”, and further in view of Kobayashi et al., U.S. Publication No. 2019/0370974, hereinafter, “Kobayashi”.
As per claim 1, Wilbee discloses a machine learning device, comprising:
a learning data acquisition part which acquires, as a learning data set, data regarding a plurality of different filters applied to images in which a workpiece is captured (Wilbee, Abstract, a framework for a system which will intelligently assign an edge detection filter to an image based on features taken from the image is introduced. The framework has four parts: the learning stage, image feature extraction, training filter creation, and filter selection training; Wilbee, page 3, Introduction, In this thesis a framework is developed for creating a system which will match an input image with its corresponding edge filter ... the fields of: manufacturing, fault detection ... For applications in mobile robotics and photography), and
data indicating a state of each predetermined section of a plurality of filtered images processed by the plurality of filters (Wilbee, page 61-63, 4.1.1 First Filter Set for Non-Fuzzy Prototype System, Five filter sets were trained each against the same ground truth index of 400 training images ... To illustrate the outcome of the prototype system a representative test image was selected with its accompanying Canny and Sobel edge images, Figure 4.1. The ground truths which accompany this image are given by, Figure 4.2 ... Figure 4.3 gives the edge images produced by the five systems ... For each system, the filter selected was different but the edge image was very similar), and
a learning part which uses the learning data set to generate a learning model that outputs a composition parameter for the plurality of filtered images for each corresponding section (Wilbee, page 49, 3.3 Edge Detection Filter Creation, The proposed output of a system built from the proposed framework are edge filters ... The training set for the input is created by taking the training images and extracting the features from them. The training set for the output is created by taking the images and determining the best filters or methodologies in the chosen search space for that image ... The set of all filters in the training set are the training filters; Wilbee, page 56, 3.4.1 Linear CA Neural Network, For a stand alone Linear CA filter, the only information needed to distinguish between filters is which pixels in its neighborhood are to be used to determine its next state. In this system, the next state determines if the pixel of interest is an edge pixel ... The input to this network will be the features gathered ... The end result of this training will be a neural network which takes in a new image and then outputs a new filter based on the features of the image, Figure 3.5; Wilbee, page 58, 3.4.2 Fuzzy CA Neural Network, this method will select from a filter bank).
Wilbee does not explicitly disclose the following limitations as further recited however Kobayashi discloses
for compositing the plurality of filtered images for each corresponding section (Kobayashi, (Kobayashi, ¶0018, generates conversion images by reducing a size of a captured image in different magnifications and performs edge detection on the individual conversion images. By reducing a size of an image in a plurality of magnifications, optimum edges are detected relative to the image deterioration in one of the conversion images. Since degrees of image deterioration are uneven in an image, optimum edges are detected in different conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of a luminance gradient value ... The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region; Kobayashi, ¶0026, An image conversion unit 202 converts the input captured image 310 by image processing to generate a plurality of conversion images 311a to 311d. The image conversion unit 202 transmits the generated conversion images 311a to 311d to an edge detection unit 204; Kobayashi, ¶0027, The edge detection unit 204 performs an edge detection process on the conversion images 311a to 311d. The edge detection unit 204 generates a detection edge list 320 and a luminance gradient value map to be transmitted to a reliability computation unit 205; Kobayashi, ¶0029, The edge selection unit 206 obtains the detection edge reliability 334 from the detection edge list 320 and selects one of the edges in the conversion images 311a to 311d which has a highest reliability to generate a selection edge list 340; Kobayashi, ¶0037, The edge detection unit 204 performs the edge detection by applying the Canny filter to the conversion images 311a to 311d. The edge detection unit 204 generates a luminance gradient value map storing luminance gradient values of individual pixels and a luminance gradient direction map storing luminance gradient directions of the individual pixels for each of the conversion images 311a to 311d while performing the edge detection using the Canny filter ... “Fx” and “Fy” indicate kernels in horizontal and vertical directions of the Prewitt filter; Kobayashi, ¶0041, The edge detection unit 204 generates the detection edge lists 320 for the individual conversion images 311a to 311d; Kobayashi, ¶0043, FIG. 5 is a concept diagram illustrating content of the process performed by the reliability computation unit 205 in step S1040. Partial images 312a to 312d are obtained by extracting regions in the vicinity of a specific edge detected in the conversion images 311a to 311d; Kobayashi, ¶0046, The edge selection unit 206 obtains the detection edge lists 320 and selects edges of high reliability to generate the selection edge list 340; Kobayashi, ¶0069, Different regions may be obtained for different portions of the targets 301; Kobayashi, ¶0073; Kobayashi, ¶0077-0078)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Kobayashi and Wilbee because they are in the same field of endeavor. One skilled in the art would have been motivated to substitute the compositing of edge sections from multiple images for the compositing of multiple filters as taught by Wilbee in order to provide an alternate means to detect and determine the edges of an image with the highest reliability (Kobayashi, ¶0018; ¶0073)
As per claim 2, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning model includes at least one of a first learning model that outputs a composite ratio for each corresponding section of the plurality of filtered images (Wilbee, page 49, 3.3 Edge Detection Filter Creation, The proposed output of a system built from the proposed framework are edge filters ... The training set for the input is created by taking the training images and extracting the features from them. The training set for the output is created by taking the images and determining the best filters or methodologies in the chosen search space for that image; Kobayashi, ¶0018; Kobayashi, ¶0073), and a second learning model that outputs a set of a specified number of filters (Wilbee, page 58, 3.4.2 Fuzzy CA Neural Network, this method will select from a filter bank; Wilbbe, page 73, Table 4.7, Fuzzy-Filter, list of filters used from filter bank; Kobayashi, ¶0077, The filter selection unit 203 selects a plurality of filters to be used in edge detection in a captured image 310. FIG. 7 is a diagram illustrating examples of the filters to be selected by the filter selection unit 203. Filters 401a to 401c are kernels of the Prewitt filters which detect edges in a vertical direction and filters 402a to 402c are kernels of the Prewitt filters, which detect edges in a horizontal direction).
As per claim 3, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the data regarding the plurality of filters includes data regarding at least one of types and sizes of the plurality of filters (Wilbee, pages 50-51, 3.3.1 Linear Cellular Automata, filter search to the set of Linear CA. This set is built out of a 3x3 pixel neighborhood ... The different automata are created by including or excluding different pixels from the neighborhood. This is indicated by the pixel in the neighborhood mask being a 0 for exclusion or 1 for inclusion. The automata are indexed by the binary representation of their included and excluded cells ... determining the best filter for the Non-fuzzy linear CA approach. Because the filter space was reduced to a small 512 possibilities it is simple to apply each CA filter to each image and then use the BDM to gauge how well the filter performed. The filter with the lowest BDM is then determined to be the best filter).
As per claim 4, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the data indicating a state of each predetermined section of the plurality of filtered images includes data indicating variations in values of peripheral sections of the predetermined section, or data indicating a reaction for each of the predetermined sections after threshold-processing of the plurality of filtered images (Kobayashi, ¶0018, generates conversion images by reducing a size of a captured image in different magnifications and performs edge detection on the individual conversion images. By reducing a size of an image in a plurality of magnifications, optimum edges are detected relative to the image deterioration in one of the conversion images. Since degrees of image deterioration are uneven in an image, optimum edges are detected in different conversion images. The information processing apparatus 200 computes reliability of results of the edge detection based on sharpness of a luminance gradient value ... The reliability is defined by a difference between luminance gradients of adjacent pixels. The information processing apparatus 200 selects an edge having a highest reliability in the edges detected in the plurality of conversion images as an optimum edge for each region; Wilbee, Abstract, a framework for a system which will intelligently assign an edge detection filter to an image based on features taken from the image ... Feature extraction is performed using a GIST methodology which extracts color, intensity, and orientation information. The set of image features are used as the input to a single hidden layer feed forward neural network trained using back propagation. The system trains against a set of linear cellular automata filters which are determined to best solve the edge image).
As per claim 5, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the data indicating a state of each predetermined section of the plurality of filtered images includes label data indicating a degree from a normal state to an abnormal state for each predetermined section (Kobayashi, ¶0042, The reliability computation unit 205 computes reliability of the edges detected by the edge detection unit 204 based on the detection edge list 320 and the luminance gradient value map; Kobayashi, ¶0045, In S1040, the reliability computation unit 205 computes reliability of all the edges included in the detection edge list 320 in accordance with Equation 2 and writes values of the reliability in the detection edge reliability 334; Kobayashi, ¶0068, an edge may not be selected when the detection edge reliability 334 is low. In step S1530, in a case where the largest detection edge reliability 334m is smaller than a predetermined threshold value, the edge selection unit 206 omits the process; Kobayashi, ¶0074, The comparison unit 103 selects an edge in which a difference between an image coordinate of the model edge and the detection edge coordinate 332a is smaller than a predetermined threshold value and in which a direction of the model edge is similar to a selection edge direction 353a).
As per claim 6, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning part converts a state of the learning model so that a feature of the workpiece extracted from a composite image composed of the plurality of filtered images based on the composite ratio of each corresponding section approaches a model feature of the workpiece extracted from a model image in which the workpiece, for which at least one of position and posture is known, is captured (Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, in this embodiment, even in a case where uneven image deterioration is generated for individual regions in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed; Kobayashi, ¶0030, The comparison unit 103 computes position and orientations of the targets 301 which have been captured in the captured image 310 based on the selection edge list 340 generated by the edge selection unit 206 and a model edge list stored in an edge model storage unit 104. The comparison unit 103 transmits the computed position and orientations of the targets 301 to the holding unit 102; Wilbee, page 43, 3.1 Learning Step, The learning step is the method by which the system learns what is and is not considered a good edge image. This system will have input image features training to an output of edge filters ... The methodology used in this research is a reference based measure. Such a measure use a training set with ground truth images and a method for rating the output edge images).
As per claim 7, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning data acquisition part calculates the difference between the filtered images and a model feature extraction image extracted from a model image in which the workpiece, for which at least one of position and posture is known, is captured, and acquires label data indicating the degree from a normal state to an abnormal state for each of the predetermined sections of the plurality of filtered images (Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, in this embodiment, even in a case where uneven image deterioration is generated for individual regions in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed; Kobayashi, ¶0030, The comparison unit 103 computes position and orientations of the targets 301 which have been captured in the captured image 310 based on the selection edge list 340 generated by the edge selection unit 206 and a model edge list stored in an edge model storage unit 104. The comparison unit 103 transmits the computed position and orientations of the targets 301 to the holding unit 102; Wilbee, page 43, 3.1 Learning Step, The learning step is the method by which the system learns what is and is not considered a good edge image. This system will have input image features training to an output of edge filters ... The methodology used in this research is a reference based measure. Such a measure use a training set with ground truth images and a method for rating the output edge images).
As per claim 8, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning data acquisition part acquires label data indicating a degree from a normal state to an abnormal state for each predetermined section of the plurality of filtered images using one or more model feature extraction images extracted from the model image when one or more changes are made to the model image in which the workpiece, for which at least one of position and posture is known, is captured (Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, in this embodiment, even in a case where uneven image deterioration is generated for individual regions in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed; Kobayashi, ¶0030, The comparison unit 103 computes position and orientations of the targets 301 which have been captured in the captured image 310 based on the selection edge list 340 generated by the edge selection unit 206 and a model edge list stored in an edge model storage unit 104. The comparison unit 103 transmits the computed position and orientations of the targets 301 to the holding unit 102; Kobayashi, ¶0050, The comparison unit 103 computes translation and rotation components of the targets 301 of a smallest coordinate difference in the captured image 310 between the model edge and the selection edge information 35; Wilbee, page 43, 3.1 Learning Step, The learning step is the method by which the system learns what is and is not considered a good edge image. This system will have input image features training to an output of edge filters ... The methodology used in this research is a reference based measure. Such a measure use a training set with ground truth images and a method for rating the output edge images).
As per claim 9, Wilbee and Kobayashi disclose the machine learning device according to claim 8, wherein the one or more changes made to the model image include one or more changes that are used when comparing features of the workpiece extracted from an image of the workpiece and model features of the workpiece extracted from the model image (Kobayashi, ¶0050, he comparison unit 103 obtains the selection edge list 340 generated by the edge selection unit 206 and the model edge list stored in the edge model storage unit 104 and computes position and orientations of the targets 301 which have been captured as the captured image 310 by means of a repetitive process. Specifically, the comparison unit 103 compares a coordinate and a direction obtained by projecting a 3D coordinate and a direction of a model edge to the captured image 310 in accordance with position and orientations of the current targets 301 with the selection edge list 340 and performs association. The comparison unit 103 computes translation and rotation components of the targets 301 of a smallest coordinate difference in the captured image 310 between the model edge and the selection edge information 350).
As per claim 10, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning part generates the learning model using a result of detecting at least one of the position and posture of the workpiece by comparing the feature of the workpiece extracted from the image in which the workpiece is captured with a model feature extracted from a model image in which the workpiece, for which at least one of position and posture is known, is captured (Wilbee, Abstract, a framework for a system which will intelligently assign an edge detection filter to an image based on features taken from the image ... Feature extraction is performed using a GIST methodology which extracts color, intensity, and orientation information; Wilbee, pages 45-47, 3.2 Image Feature Extraction, a GIST feature set .. The feature set is constructed from orientation, color and intensity information ... Four orientation channels were created using Garbor filters ... Each feature image which was created, 34 in total, was then subdivided into 16 equal regions; Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, in this embodiment, even in a case where uneven image deterioration is generated for individual regions in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed).
As per claim 11, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the data indicating a state of each predetermined section of the plurality of filtered images includes data indicating a reaction for each predetermined section after threshold-processing of the plurality of filtered images processed by the plurality of filters exceeding a specified number (Wilbee, page 52, 3.3.2 Fuzzy Cellular Automata, The fuzzy transition rule ... is a near neighborhood CA [Equations 3.8, 3.9] where t is the edge pixel threshold; Kobayashi, ¶0068, an edge may not be selected when the detection edge reliability 334 is low. In step S1530, in a case where the largest detection edge reliability 334m is smaller than a predetermined threshold value, the edge selection unit 206 omits the process; Kobayashi, ¶0077, The filter selection unit 203 selects a plurality of filters to be used in edge detection in a captured image 310. FIG. 7 is a diagram illustrating examples of the filters to be selected by the filter selection unit 203. Filters 401a to 401c are kernels of the Prewitt filters which detect edges in a vertical direction and filters 402a to 402c are kernels of the Prewitt filters, which detect edges in a horizontal direction).
As per claim 12, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning part generates the learning model that outputs a set of a specified number of filters using a model image in which the workpiece, for which at least one of position and posture is known, is captured (Kobayashi, ¶0017, an apparatus which accurately performs the edge comparison of a target so that a position and orientation is computed even in a case where a motion blur occurs in a captured image due to a shift of an imaging unit or the target); Kobayashi, ¶0018, A comparison unit 103 described below computes a position and orientation of a target by comparing a selected edge with an edge of a defined 3D geometric model. By this, in this embodiment, even in a case where uneven image deterioration is generated for individual regions in a captured image due to a motion blur, for example, comparison with an edge of the target is accurately performed so that a position and orientation is computed; Kobayashi, ¶0030, The comparison unit 103 computes position and orientations of the targets 301 which have been captured in the captured image 310 based on the selection edge list 340 generated by the edge selection unit 206 and a model edge list stored in an edge model storage unit 104. The comparison unit 103 transmits the computed position and orientations of the targets 301 to the holding unit 102; Kobayashi, ¶0077, The filter selection unit 203 selects a plurality of filters to be used in edge detection in a captured image 310. FIG. 7 is a diagram illustrating examples of the filters to be selected by the filter selection unit 203. Filters 401a to 401c are kernels of the Prewitt filters which detect edges in a vertical direction and filters 402a to 402c are kernels of the Prewitt filters, which detect edges in a horizontal direction).
As per claim 13, Wilbee and Kobayashi disclose the machine learning device according to claim 1, wherein the learning part generates the learning model for outputting a set of a specified number of filters so that after threshold-processing of the plurality of filtered images processed by the plurality of filters exceeding the specified number, the reaction for each predetermined section becomes a maximum for each predetermined section (Wilbee, page 58, 3.4.2 Fuzzy CA Neural Network, this method will select from a filter bank; Wilbee, page 59, Results, The number and type of filter which is selected; Kobayashi, ¶0077, The filter selection unit 203 selects a plurality of filters to be used in edge detection in a captured image 310. FIG. 7 is a diagram illustrating examples of the filters to be selected by the filter selection unit 203. Filters 401a to 401c are kernels of the Prewitt filters which detect edges in a vertical direction and filters 402a to 402c are kernels of the Prewitt filters, which detect edges in a horizontal direction).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Primary Examiner, Art Unit 2668
Prosecution Insights