DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/15/2024 in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 8 and 15 recite limitations – “ in a case where a machine learning model classifies a first image based on a value less than a threshold, generating training data in which a classification result of a second region of a second image that corresponds to a position of a first region of the first image is labeled to the first region, based on a classification result obtained by classifying the second image based on a value equal to or more than the threshold by the machine learning model”, appears to be directed to matching position of first image region with second image region, and compares first image region with threshold and second image region with threshold where second image region position is similar to first image region position.
However, it is not clear where it recites that if the threshold of first image region is less than a threshold then training is generated where the first image is labelled to the first region. It is not clear as to the implying of the feature of the first image being labelled to the first region. It is also not clear as to it is based on second image region higher than a value equal or higher than the threshold value if the classification is generated based on this comparison or first region being less than threshold. It appears to be a run on statement without clear scope.
Dependent claims do not remedy the deficiencies introduced by the independent claims and therefore similarly rejected.
Therefore, Examiner suggests amending the claims to clearly define the features above as disclosed in the embodiments of the original specification.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Takahashi et al. (US Pub No. 20210365731 A1) in view of Murata et al. (JP 2020144755 A, as provided).
Regarding Claim 1,
Takahashi discloses A non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute processing comprising: (Takahashi, [0011], [0058], [0194], discloses these comprehensive or specific aspects may be implemented as systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or may be implemented as any combination of systems, methods, integrated circuits, computer programs, and recording media; these comprehensive or specific aspects may be implemented as systems, devices, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or may be implemented as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media; Moreover, the technique disclosed in the present disclosure may be the program described above, or may be a non-transitory computer-readable recording medium that records the program described above. It goes without saying that the program described above may be distributed via a transmission medium such as the Internet. For example, the program described above and digital signals formed by the program described above may be transmitted via an electric communication line, a wireless or wired communication line, a network typified by the Internet, or data broadcastings. The program described above and digital signals formed by the program described above may be implemented as an independent different computer system by being recorded and transferred on a recording medium or by being transferred via a network or other systems)
in a case where a machine learning model classifies a first image based on a value less than a threshold, generating training data in which a classification result of a second region of a second image that corresponds to a position of a first region of the first image is labeled to the first region, based on a classification result obtained by classifying the second image based on a value equal to or more than the threshold by the machine learning model; training the machine learning model based on the training data. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) and
Takahashi does not explicitly disclose in a case where a machine learning model classifies a first image based on a value less than a threshold, generating training data in which a classification result of a second region of a second image that corresponds to a position of a first region of the first image is labeled to the first region, based on a classification result obtained by classifying the second image based on a value equal to or more than the threshold by the machine learning model;
Murata discloses in a case where a machine learning model classifies a first image based on a value less than a threshold, generating training data in which a classification result of a second region of a second image that corresponds to a position of a first region of the first image is labeled to the first region, based on a classification result obtained by classifying the second image based on a value equal to or more than the threshold by the machine learning model; (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Takahashi in view of Murata having a method of detecting image region in one image and matching the position and pixel values in another image, with the teachings of Murata having, having a method of object classification compared with threshold to determine classification of object using machine learning classifier in order to improve classification accuracy of image regions of similar regions in multiple images.
Regarding Claim 2,
The combination of Takahashi and Murata further discloses wherein a case where the machine learning model classifies the first image based on the value less than the threshold is a case where an average of output values when each of a plurality of regions that includes the first region in the first image is classified is less than the threshold. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Regarding Claim 3,
The combination of Takahashi and Murata further discloses wherein a case where the machine learning model classifies the first image based on the value less than the threshold is a case where an output value when the first region in the first image is classified is less than the threshold, and the classification result obtained by classifying the second image based on the value equal to or more than the threshold by the machine learning model is a classification result of the second region obtained by classifying the second region of the second image obtained by inputting the second image into the machine learning model based on the value equal to or more than the threshold. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Regarding Claim 4,
The combination of Takahashi and Murata further discloses wherein the output value is a value that indicates confidence of the classification result by the machine learning model. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Regarding Claim 5,
The combination of Takahashi and Murata further discloses wherein the processing of generating the training data includes processing of generating the training data in which a classification result of a third region is labeled to the third region of the first image, in a case where the third region of the first image is classified based on the value equal to or more than the threshold. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Regarding Claim 6,
The combination of Takahashi and Murata further discloses wherein the processing of generating the training data includes processing of generating training data in which a classification result of the second region of the second image that corresponds to a position of a fourth region of a third image generated by using at least one of the first image or the second image or a combination of the first image and the second image is labeled to the fourth region. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Regarding Claim 7,
The combination of Takahashi and Murata further discloses wherein the classification result of the second region is a probability that the second region is classified into each of a plurality of classes, and the processing of labeling includes assigning a label that corresponds to a class with the highest probability that the second region is classified to the first region, based on a classification result of the second region of a plurality of the second images. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Claims 8-14 apparatus with elements corresponding to the computer readable medium instructions recited in Claims 1-7 respectively. Therefore, the recited elements of the apparatus Claims 8-14 are mapped to the proposed combination in the same manner as the corresponding instructions of Claims 1-7 respectively. Additionally, the rationale and motivation to combine the Takahashi and Murata references presented in rejection of Claim 1, apply to these claims.
Furthermore, the combination of Takahashi and Murata further discloses A machine learning apparatus comprising: a memory; and a processor coupled to the memory and configured to: in a case where a machine learning model classifies (Takahashi, [0011], [0068], [0191], discloses processor memory apparatus interface).
Regarding Claim 19,
The combination of Takahashi and Murata further discloses wherein a processing of generating the training data includes processing of generating the training data in which a classification result of a third region is labeled to the third region of the first image, in a case where the third region of the first image is classified based on the value equal to or more than the threshold. (Takahashi, [0014], [0075], [0078-0079], [0108], discloses aforementioned training label image correction method according to the first aspect, the comparing the labels may include acquiring a matching portion and a non-matching portion with the training label image in the determination label image for a label of interest by comparing the determination label image with the training label image, and the correcting the label areas may include correcting a range of a label area with the label of interest based on the matching portion and the non-matching portion. Accordingly, it is possible to understand, from the matching portions and the non-matching portions between the determination label image and the training label image, how the label area of the training label image deviates from the determination label image that serves as a reference for variations in the boundaries of many training label images. Therefore, the label area can be easily corrected based on the matching portion and the non-matching portion such that a variation in the boundary is reduced; server device 30 is an information processing device (training data set generator) that generates a training data set used in relearning of a trained model for object detection (object detection model). For example, server device 30 is operated by a manufacturer that has manufactured the object detection model installed in object detector 11 of mobile unit 10, or by other operators; determination detector 32 is a processing unit that performs object detection processing on an image included in the log information. Determination detector 32 performs computation on a larger scale than object detector 11 and thus can more accurately detect objects. In the present embodiment, determination detector 32 includes an object detection model that has been trained so as to be capable of executing image segmentation (semantic segmentation), and uses the object detection model to execute image segmentation on an image. Executing image segmentation refers to executing processing for labeling each of a plurality of pixels in the image with a meaning indicated by the pixel. This corresponds to labelling each pixel with an object class, i.e., with a category; determination detector 32 may include object classes that can be detected by object detector 11 (in the present embodiment, “passenger car” and “person”) and an object detection model that has been trained so as to be capable of detecting a larger number of object classes than the above object classes, and may use the trained model to execute object detection processing; in the case where there is an object that hides part of target object 110 (e.g., an object located between mobile unit and target object 110 and overlapping with target object 110 when viewed from mobile unit 10) in image 101, generator 34 may cut out this object together with target object 110 as an integral unit. For example, in the case where there is another object that hides target object 110 between target object 110 to be cut out and the vehicle (mobile unit 10) and this other object can also be cut out using a display-area threshold value (e.g., the size of cutout image 120) set in advance in accordance with the segmentation result, these objects may be cut out in a cluster. Target object 110 and the other object may, for example, be of the same object class (e.g., passenger car). By reflecting the cutout image cut out in a cluster in other images, a natural image with less discomfort can be generated as training data; training data is generated based on classifying region in first and second image to classify the object in specific class by comparing image regions) (Murata, Claims, discloses when the reliability of the recognition result exceeds a predetermined threshold value, the classification unit classifies the sensor data into the related object, and when the reliability is equal to or less than the predetermined threshold value, the sensor data is classified. Classify as unrelated objects; object classification is compared with threshold to determine of its classification accuracy with respect to region in image). Additionally, the rational and motivation to combine the references Takahashi and Murata as applied in rejection of claim 1 apply to this claim.
Claims 15-18 recite method with steps corresponding to the computer readable medium instructions recited in Claims 1-3 and 5 respectively. Therefore, the recited steps of the method Claims 15-18 are mapped to the proposed combination in the same manner as the corresponding instructions of Claims 1-3 and 5 respectively. Additionally, the rationale and motivation to combine the Takahashi and Murata references presented in rejection of Claim 1, apply to these claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20200327360 A1 (A segmentation neural network is extended to provide classification at the segment level. An input image of a document is received and processed, utilizing a segmentation neural network, to detect pixels having a signature feature type. A signature heatmap of the input image can be generated based on the pixels in the input image having the signature feature type. The segmentation neural network is extended from here to further process the signature heatmap by morphing it to include noise surrounding an object of interest. This creates a signature region that can have no defined shape or size. The morphed heatmap acts as a mask so that each signature region or object in the input image can be detected as a segment. Based on this segment-level detection, the input image is classified. The classification result can be provided as feedback to a machine learning framework to refine training)
US 20200293786 A1 (The present disclosure provides a video identification method, a video identification device and a storage medium. The video identification device extracts an image and an optical flow from a video, classifies the image by using a first machine learning model to obtain a first classification result, classifies the optical flow by using a second machine learning model to obtain a second classification result, wherein a depth of the first machine learning model is larger than a depth of the second machine learning model, and fuses the first classification result and the second classification result to obtain the identification result of the video)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673