DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Remarks pages 7-8, filed 07/15/2026, with respect to the interpretation of claim 12 under Superguide have been fully considered and are persuasive. The interpretation of claim 12 has been withdrawn.
Applicant’s arguments, see Remarks pages 8-9, filed 07/15/2026, with respect to the rejections of claim(s) 1 and 13-14 under 35 U.S.C. 103 have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below) necessitated by Applicant's amendment to the claim(s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 10-14, and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba et al. (US-20190012790-A1) hereinafter referenced as Aoba, in view of Shrivastava (Handling Class Imbalance by Introducing Sample Weighting in the Loss Function).
Regarding claim 1, Aoba discloses: A learning apparatus comprising: at least one processor; and at least one memory having stored thereon instructions which, when executed by the at least one processor (Aoba: 0010), cause the learning apparatus at least to:
train an estimator configured to execute a region detection task that calculates a mixing ratio of class labels (Aoba: 0006: “a training apparatus comprises: an extraction unit configured to extract a feature amount of an identified image for training of an estimator; an obtaining unit configured to obtain, as supervisory information, distribution-related information which relates to distribution of regions having attributes different from each other in the identified image; and a training unit configured to perform the training of the estimator using a combination of the feature amount of the identified image and the supervisory information, wherein the estimator is trained to estimate the distribution from the feature amount.”;
0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525. For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”);
obtain a plurality of training data items including input data and supervisory data corresponding to the input data, wherein the input data is image data (Aoba: 0036: “A plurality of training images and supervisory information of distributions are stored in advance in the training data storage unit 5100. The training image indicates an image used for training of the estimator. The training image can be, for example, image data captured by a digital camera or the like. The format of the image data is not particularly limited and can be, for example, JPEG, PNG, BMP, or the like. The number of training images prepared is represented by N, and the nth training image is represented by In (n=1, . . . N) below.”), and the supervisory data is label data indicating a class label of each region in the image data (Aoba: 0036: “The supervisory information of a distribution indicates a distribution in a predetermined region of the training image. The supervisory information is prepared in advance and, for example, a human can create it while viewing the training image. In this embodiment, a plurality of regions each serving as an identification unit are set in the training image, and supervisory information is prepared for each region. The image in the predetermined region of the training image, which is one identification unit, will be referred to as an identified image hereinafter.”);
calculate, for each of the plurality of training data items, the mixing ratio of the class labels in the training data item (Aoba: 0055: “When performing training of the estimator, the training unit 2200 compares the value of the output signal obtained by the output layer 640 with the supervisory information when the identified image obtained from the predetermined region i of the training image In is input to the CNN.”); and
control training of the estimator based on the mixing ratio of the class labels (Aoba: 0049-0050: “As described above, the distribution can be expressed using various methods. The distribution may be expressed using any one of these expressions. For example, if the distribution is represented only by the area ratio, it is defined as C=r…
Let L be the number of dimensions of the vector representing the distribution C. A distribution vector in a predetermined region i on an image In is expressed as Cni, and the Ith (l=l, ... , L) element of the distribution vector Cni is expressed as c(n, i, I).”;
0055: “When performing training of the estimator, the training unit 2200 compares the value of the output signal obtained by the output layer 640 with the supervisory information when the identified image obtained from the predetermined region i of the training image In is input to the CNN. Here, let Xni be the feature amount obtained by inputting the predetermined region i of the training image In to the feature extraction unit 610 and yl(Xni) be the output signal of the Ith output element of the output layer 640 obtained as the result of inputting the feature amount to the estimator 620. In addition, the supervisory signal in the Ith output element of the output layer 640 is represented by the Ith element c(n, i, I) of the distribution Cni. In this case, the error between the output signal and the supervisory information is calculated by
PNG
media_image1.png
47
179
media_image1.png
Greyscale
”).
Aoba does not disclose expressly: control training of the estimator based on frequencies of occurrence of the mixing ratio of the class labels to increase a correction coefficient for a training data item whose mixing ratio has a frequency of occurrence lower than a frequency of occurrence of a mixing ratio of another training data item.
Shrivastava discloses: controlling the training of an estimator based on frequencies of occurrence of class labels (Shrivastava: Section: Sample Weighting in Loss Function: “Let’s consider a Loss Function for our Multi Label Classification running example. I used PyTorch’s implementation of Binary Cross Entropy: torch.nn.BCEWithLogitLoss which combines a Sigmoid Layer and the Binary Cross Entropy loss for numerical stability and can be expressed mathematically as:
PNG
media_image2.png
191
683
media_image2.png
Greyscale
…It’s Wn_c which is the Sample Weight that we wish to compute for every sample in a batch which enables us to weigh the contribution of a particular sample towards the overall loss. It can be assigned using the argument ‘weight’ and has to be a Tensor of size N*C (C is Total Number of Classes).”) to increase a correction coefficient for a training data item whose class label has a frequency of occurrence lower than a frequency of occurrence of a class label of another training data item (Shrivastava: Section: Inverse of Number of Samples (INS): “As the name suggests, we weight the samples as the inverse of the class frequency for the class they belong to.
PNG
media_image3.png
107
646
media_image3.png
Greyscale
PNG
media_image4.png
72
693
media_image4.png
Greyscale
The function above shows a simple implementation that computes the weights and normalizes them over different classes.“).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the sample weight term determined based on the Inverse of Number of Samples weighting scheme taught by Shrivastava into the error function used for training the estimator based on the predicted class ratio compared to the supervisory information disclosed by Aoba. The suggestion/motivation for doing so would have been “Introducing Sample Weights in the Loss Function is a pretty simple and neat technique for handling Class Imbalance in your training dataset. The idea is to weigh the loss computed for different samples differently based on whether they belong to the majority or the minority classes. We essentially want to assign a higher weight to the loss encountered by the samples associated with minor classes.” (Shrivastava: Section: Sample Weighting in Loss Function). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba with Shrivastava to obtain the invention as specified in claim 1.
Regarding claim 10, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1, wherein subcategories are set for the class labels of a target object in the supervisory data, and the mixing ratio is calculated for each of the subcategories (Aoba: 0040: “Classes and class labels are defined variously, and the class classification method is not particularly limited. In the example shown in FIGS. 3A and 3B, class classification is performed in accordance with the types of the objects. Other examples of class labels are a skin region, a hair region, animals such as a dog and a cat, and artifacts such as an automobile and a building. A class label representing a specific object such as a component A or a component B used in a factory can also be used. On the other hand, the pixels may be classified into main object regions and a background region.”; Wherein the classifications of larger classes, such as animals and artifacts, based on subcategories, such as dogs and cats, constitutes the calculation of mixing ratios for each of the subcategories).
Regarding claim 11, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1, wherein camera parameters used when the input data was shot are obtained (Aoba: 0080: “the size of the identified region or how to cut it can be changed based on various kinds of capturing condition information. For example, in a strongly blurred region, a fine texture is lost as information. For this reason, the distribution estimation accuracy may be improved by performing estimation for a wider identified region.”; Wherein the capturing condition, which constitute the camera parameters, are obtained for the estimation of distribution accuracy based on the size of the identified region), and wherein frequencies of occurrence of the mixing ratio of the class labels are calculated based on the camera parameters (Aoba: 0086: “Depending on the difference in the region setting pattern, the supervisory information of the distribution can change even at the same position on the image. FIG. 3C shows rectangular regions 551, 552, and 553 of various sizes at the same position of a training image. In the smallest rectangular region 551, the area ratio of sky: non-sky is r=1. On the other hand, each of the rectangular regions 552 and 553 includes a non-sky region, and the area ratios are r=0.9 and r=0.8, respectively.”;
0087: “The training unit 2200 performs training of the estimator corresponding to each region setting pattern. That is, the training unit 2200 performs training of the estimator corresponding to a region setting pattern of interest based on the identified region set in accordance with the region setting pattern of interest and the supervisory information given for the identified region. As a result, the training unit 2200 generates an estimator corresponding to each of the plurality of region setting patterns. For example, letting q be the index of a region setting pattern, and Q be the total number of region setting patterns, Q types of estimators yq can be obtained by training. Training of the estimator can be done as in the first embodiment.”;
0088: “In step S2400, the evaluation unit 2400 evaluates the identification accuracy of the estimator obtained in step S2200 together with the capturing condition information and generates a region setter.”;
Wherein the generation of a region setter based on capturing conditions and estimators generated based on region setting patterns constitutes the calculation of mixing ratios frequencies of occurrence based on camera parameters since the class ratio frequencies are modified based on the size of identifier regions)
Regarding claim 12, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 11, wherein the camera parameters include at least one of a Bv value, an exposure time, an F value, an ISO sensitivity value, a shooting date and time, or GPS information (Aoba: 0081: “The capturing condition information includes information unique to an image capturing apparatus and information unique to a captured image. As the information unique to the image capturing apparatus, the size or the allowable diameter of a circle of confusion of a sensor, the brightness or the focal length of an optical system, and the like are usable. As the information unique to the captured image, an aperture value, a focus distance, a By value, a RAW image, an exposure time, a gain (ISO sensitivity), a white balance coefficient, distance information, position information by a GPS, time information such as a date/time, and the like are usable”).
As per claim(s) 13, arguments made in rejecting claim(s) 1 are analogous.
As per claim(s) 14, arguments made in rejecting claim(s) 1 are analogous. In addition, paragraphs 0009-0010 of Aoba disclose the use of “A non-transitory computer-readable recording medium storing a program”.
Regarding claim 16, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1, wherein a degree of importance of each training data item included in the plurality of training data items is determined based on the frequencies of occurrence of the mixing ratio of the class labels
(Shrivastava: Section: Sample Weighting in Loss Function: “It’s Wn_c which is the Sample Weight that we wish to compute for every sample in a batch which enables us to weigh the contribution of a particular sample towards the overall loss. It can be assigned using the argument ‘weight’ and has to be a Tensor of size N*C (C is Total Number of Classes).”;
Section: Inverse of Number of Samples (INS): “As the name suggests, we weight the samples as the inverse of the class frequency for the class they belong to.
PNG
media_image3.png
107
646
media_image3.png
Greyscale
”).
Regarding claim 17, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1, wherein the mixing ratio of class labels for a region is a value obtained by dividing the number of pixels having one of the class labels in the region by the total number of pixels included in the region (Aoba: 0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Claim(s) 6 & 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Shrivastava, and further in view of Fukuda (US-20120327127-A1).
Regarding claim 6, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1, wherein, in calculating the mixing ratio of the class labels in the plurality of training data items, each training data item is divided into a plurality of partial regions, and for each partial region, a first ratio is calculated, the first ratio being a mixing ratio of class labels within the partial region, and wherein the training of the estimator is controlled based on the first ratio (Aoba: 0006: “a training apparatus comprises: an extraction unit configured to extract a feature amount of an identified image for training of an estimator; an obtaining unit configured to obtain, as supervisory information, distribution-related information which relates to distribution of regions having attributes different from each other in the identified image; and a training unit configured to perform the training of the estimator using a combination of the feature amount of the identified image and the supervisory information, wherein the estimator is trained to estimate the distribution from the feature amount.”;
0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510. The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525. For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Aoba in view of Shrivastava does not disclose expressly: wherein, for each partial region, a first ratio and a second ratio are calculated, the first ratio being a mixing ratio of class labels within the partial region, and the second ratio being a mixing ratio of class labels including a peripheral partial region, and wherein the training of the estimator is controlled based on the first ratio and the second ratio.
Thus, Aoba in view of Shrivastava does not disclose expressly: the incorporation of a calculated a second ratio weight, based on a mixing ratio of class labels including a peripheral partial region, into the training of the estimator.
Fukuda discloses: a method for calculating an average size of regions within an image classified as facial regions, based on the determined weight values of the facial regions. The average size of the facial regions is determined based on a weighted average of the face regions, wherein the weight of each face region is determined based on the facial image’s proximity to the center of the picture (Fukuda: 0041: “The CPU 104 calculates an average value SAVE of the areas of face rectangles as the typical size of the face region. Of course, the CPU 104 may calculate a weighted average. As an example of weighting, weight values according to the positions in a picture are set in advance, and a weight value is determined based on the coordinates of a face rectangle. For example, the weight value of a central part of a picture is set to be larger than those of peripheral parts of the picture, and a larger weight is attached to a face image located at the center of the picture. Alternatively, the reliability of each face rectangle detected by the face-detection process may be used as a weight value.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the method for calculating a weighted average based on distance of each facial region to the center taught by Fukuda into the training of the estimator disclosed by Aoba in view of Shrivastava by weighing pixel values based on their distance from an identified image’s center. The suggestion/motivation for doing so would have been “Alternatively, not all face rectangles need to be used in determination of the typical size of a face region. A picture captured in urban area or the like often includes images of persons (to be referred to as images of non-subject persons hereinafter) other than an image of a person to be captured (to be referred to as an image of a subject person hereinafter). The images of the non-subject persons are normally sufficiently smaller than the image of the subject person. Hence, face rectangles, the area of each of which is less than, for example, ¼ that of a maximum face rectangle, may be excluded from determination of the typical size of a face region. A standard deviation d of the areas of face rectangles may be calculated, face rectangles each having an area less than SAVE —d may be excluded, and the average value SAVE of the areas of the remaining face rectangles may be calculated. Alternatively, when the weight values of peripheral parts of a picture are set to be zero, face rectangles can be excluded depending on the positions of the face rectangles (the positions where face images appear).” (Fukuda: 0042; Wherein the weighted average allows for the weights to be adjusted based on their perceived relevancy.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Shrivastava with Fukuda to obtain the invention as specified in claim 6.
Regarding claim 18, Aoba in view of Shrivastava and Fukuda discloses: The learning apparatus according to claim 6, wherein the first ratio is calculated by counting a number of pixels belonging to one of the class labels inside the partial region and dividing the counted number by a total number of pixels constituting the partial region (Aoba: 0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510. The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Regarding claim 19, Aoba in view of Shrivastava and Fukuda discloses The learning apparatus according to claim 6, wherein the peripheral partial region includes the partial region and the second ratio is obtained by calculating a weighted average value of the class labels, weights being set to
decrease as a distance from a center position of the partial region increases (Fukuda: 0041: “The CPU 104 calculates an average value SAVE of the areas of face rectangles as the typical size of the face region. Of course, the CPU 104 may calculate a weighted average. As an example of weighting, weight values according to the positions in a picture are set in advance, and a weight value is determined based on the coordinates of a face rectangle. For example, the weight value of a central part of a picture is set to be larger than those of peripheral parts of the picture, and a larger weight is attached to a face image located at the center of the picture. Alternatively, the reliability of each face rectangle detected by the face-detection process may be used as a weight value.”).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Shrivastava and Fukuda, and further in view of Kondo et al. (US-6965693-B1) hereinafter referenced as Kondo.
Regarding claim 7, Aoba in view of Shrivastava and Fukuda discloses: The learning apparatus according to claim 6.
Aoba in view of Shrivastava and Fukuda does not disclose expressly: wherein the second ratio includes statistic information pertaining to a direction to a center of gravity position of each of the class labels from a center position of a partial region of interest.
Kondo discloses: statistic information pertaining to a direction to a center of gravity position of each of the class labels from a center position of an image (Kondo: Col 17: lines 12-27: “As mentioned above, the value X that the gravity center has in the X-axis direction is applied, in order to detect the orientation of the face. Rather, the value Y that the gravity center has in the Y-axis direction may be applied, to detect the orientation of the face. As described already, the orientation of the face is detected from the positional relation between the gravity center of the face image and the gravity center of the region to which the class number is allocated. The orientation of the face can be therefore detected correctly, regardless of the position of the face. In addition, the angle of the face can be accurately detected. This is because the correlation value is calculated by applying a large weight coefficient W to any region prominently related with the orientation of the face and a small weight coefficient W to any region weakly related with the orientation of the face.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the orientation calculation method taught by Kondo into the training of the estimator disclosed by Aoba in view of Shrivastava and Fukuda by weighing pixel labels based on their relation to an object’s center of gravity. The suggestion/motivation for doing so would have been “In this case, the value ( difference Dxfrequency C) and the angle V are correlated firmly, and the class that is allocated to the region has a large weight coefficient. On the other hand, if the distribution of points, which is determined by ( difference Dxfrequency C) and angle V, diverges is illustrated in FIG. 27B, the value (difference Dxfrequency C) and the angle V are correlated but a little. In this case, the class that is allocated to the region has a small weight coefficient” (Kondo: Col 16: Lines 45-53; Wherein the pixels may be labeled based upon their correlation to an image class’s center of gravity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Shrivastava and Fukuda with Kondo to obtain the invention as specified in claim 7.
Claim(s) 9 & 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Shrivastava, and further in view of Lin et al. (US-20200334501-A1) hereinafter referenced as Lin.
Regarding claim 9, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1.
Aoba in view of Shrivastava does not disclose expressly: wherein a padding amount of each training data item used to train the estimator is controlled such that a total number of training data items having a high degree of importance low frequency of occurrence is larger than a total number of other training data.
Lin discloses: wherein a padding amount of each training data item used to train an estimator is controlled such that a total number of training data items having a low frequency of occurrence is larger than a total number of other training data (Lin: 0055-0056: “In some datasets, the ratio of occurrence of the most common to the least common object class may be one or more orders of magnitude larger than that of simple dataset. Thus, certain embodiments of the present disclosure include performing offline oversampling of the images containing the rare classes to balance the class distribution to create a balanced training set. For example, if an object class is relatively rare (e.g., class 1), class balancing may be performed by selecting a proportionately higher number of images from an initial dataset that include objects from class 1, so that in the initial training set, the same number of objects are represented from each object class.”; Wherein the oversampling of images, by selecting a proportionately higher number of images, based on frequency of occurrence constitutes a total number of training data items having a low frequency of occurrence being larger than a total number of other training data.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image oversampling with data augmentation taught by Lin into the training of the estimator disclosed by Aoba in view of Shrivastava by oversampling identified region images based on their mixing ratios. The suggestion/motivation for doing so would have been “class balancing (as described in FIGS. 3 and 4) results in reduced sampling diversity. Low data diversity may result from oversampling or repeating training images of rare object classes. Therefore, data augmentation techniques such as horizontal flipping of images may be used to increase the level of diversity.” (Lin: 0062; Wherein the oversampling with augmentation allows for the classes to be equally represented while also maintaining data diversity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Shrivastava with Lin to obtain the invention as specified in claim 9.
Regarding claim 15, Aoba in view of Shrivastava discloses: The learning apparatus according to claim 1.
Aoba in view of Shrivastava does not disclose expressly: wherein the training of the estimator is controlled such that unevenness of the frequencies of occurrence of the mixing ratio of the class labels among the plurality of training data items is reduced.
Lin discloses: wherein the training of an estimator is controlled such that unevenness of the frequencies of occurrence of the class labels among a plurality of training data items is reduced (Lin: 0055-0056: “In some datasets, the ratio of occurrence of the most common to the least common object class may be one or more orders of magnitude larger than that of simple dataset. Thus, certain embodiments of the present disclosure include performing offline oversampling of the images containing the rare classes to balance the class distribution to create a balanced training set. For example, if an object class is relatively rare (e.g., class 1), class balancing may be performed by selecting a proportionately higher number of images from an initial dataset that include objects from class 1, so that in the initial training set, the same number of objects are represented from each object class.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image oversampling with data augmentation taught by Lin into the training of the estimator disclosed by Aoba in view of Shrivastava by oversampling identified region images based on their mixing ratios. The suggestion/motivation for doing so would have been “class balancing (as described in FIGS. 3 and 4) results in reduced sampling diversity. Low data diversity may result from oversampling or repeating training images of rare object classes. Therefore, data augmentation techniques such as horizontal flipping of images may be used to increase the level of diversity.” (Lin: 0062; Wherein the oversampling with augmentation allows for the classes to be equally represented while also maintaining data diversity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Shrivastava with Lin to obtain the invention as specified in claim 15.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights