DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/26/2026 has been entered.
Response to Arguments
Applicant’s arguments, see Remarks page 9, filed 01/26/2026, with respect to the rejection of claim(s) 1 and 13-14 under 35 U.S.C. 103 have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below) necessitated by Applicant’s amendment to the claim(s).
Claim Interpretation
Note that according to the Federal Circuit’s 2004 Superguide v. DirecTV decision, “at least one of … and …” requires at least one instance of each and every item listed.
Claim(s) 12 recite(s): “wherein the camera parameters include at least one of a Bv value, an exposure time, an F value, an ISO sensitivity value, a shooting date and time, and GPS information”.
If Applicant intends for an interpretation of only one of these items being required for claim interpretation, Applicant can amend the claim language to, instead recite, “at least one of … or …”. In SuperGuide, the Federal Circuit held that the plain meaning of “at least one of A, B, and C” means: at least one of A, at least one of B, and at least one of C. The Court held that if the applicant intended “at least one of A, B, and C” to mean A, B, or C, they should have used “OR.” For the purposes of examination, claim(s) 12 is/are interpreted as disjunctive, “or,”as disclosed by Paragraphs 0158 & 0164-0165 of the Specification.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8, 10-14, and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba et al. (US-20190012790-A1) hereinafter referenced as Aoba, in view of Fernando et al. (Dynamically Weighted Balanced Loss: Class Imbalanced Learning Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks) hereinafter referenced as Fernando.
Regarding claim 1, Aoba discloses: A learning apparatus comprising: at least one processor; and at least one memory having stored thereon instructions which, when executed by the at least one processor (Aoba: 0010), cause the learning apparatus at least to:
train an estimator that executes a region detection task that calculates a mixing ratio of class labels (Aoba: 0006: “a training apparatus comprises: an extraction unit configured to extract a feature amount of an identified image for training of an estimator; an obtaining unit configured to obtain, as supervisory information, distribution-related information which relates to distribution of regions having attributes different from each other in the identified image; and a training unit configured to perform the training of the estimator using a combination of the feature amount of the identified image and the supervisory information, wherein the estimator is trained to estimate the distribution from the feature amount.”;
0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”);
obtain a plurality of training data items including input data and supervisory data corresponding to the input data, wherein the input data is image data (Aoba: 0036: “A plurality of training images and supervisory information of distributions are stored in advance in the training data storage unit 5100 . The training image indicates an image used for training of the estimator. The training image can be, for example, image data captured by a digital camera or the like. The format of the image data is not particularly limited and can be, for example, JPEG, PNG, BMP, or the like. The number of training images prepared is represented by N, and the nth training image is represented by In (n=1, . . . N) below.”), and the supervisory data is label data indicating a class label of each region in the image data (Aoba: 0036: “The supervisory information of a distribution indicates a distribution in a predetermined region of the training image. The supervisory information is prepared in advance and, for example, a human can create it while viewing the training image. In this embodiment, a plurality of regions each serving as an identification unit are set in the training image, and supervisory information is prepared for each region. The image in the predetermined region of the training image, which is one identification unit, will be referred to as an identified image hereinafter.”);
calculate, for each of the plurality of training data items, the mixing ratio of the class labels in the plurality of training data items (Aoba: 0055: “When performing training of the estimator, the training unit 2200 compares the value of the output signal obtained by the output layer 640 with the supervisory information when the identified image obtained from the predetermined region i of the training image In is input to the CNN.”).
Aoba does not disclose expressly: control training of the estimator based on frequencies of occurrence of the mixing ratio of the class labels.
Fernando discloses: controlling the training of an estimator based on frequencies of occurrence of class labels (Fernando: Section: IV.A. Loss Function Formulation: “We define Dynamically Weighted Balanced (DWB) Loss as:
PNG
media_image1.png
106
614
media_image1.png
Greyscale
where wj is the class weight of class j, yij is the jth element of one-hot encoded label of instance xi and pij is the predicted probability of the class j of instance xi… The class weights wj can be handled as a hyper-parameter that is learned from data by cross validation or set proportional to inverse class frequency. We set wj equal to the log ratio of the class frequency of the majority class and the class frequency nj (computed over the training dataset) as follows:
PNG
media_image2.png
87
478
media_image2.png
Greyscale
”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the error function used for training the CNN based on the predicted class ratio compared to the supervisory information disclosed by Aoba by incorporating the class weight calculation based on class frequency taught by Fernando. The suggestion/motivation for doing so would have been “Training samples from classes with fewer observations producing lower class probabilities are expected to be the harder instances…In this context, we introduce a dynamic weighting based classifier objective function based on the prediction probability of ground truth class to assign higher weights to hard to train instances” (Fernando: Section: IV.A. Loss Function Formulation). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba with Fernando to obtain the invention as specified in claim 1.
Regarding claim 8, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein the training of the estimator is controlled by increasing a correction coefficient for a training data item whose frequency of occurrence is lower than a frequency of occurrence of other training data (Fernando: Section: IV.A. Loss Function Formulation: “We set wj equal to the log ratio of the class frequency of the majority class and the class frequency nj (computed over the training dataset) as follows:
PNG
media_image2.png
87
478
media_image2.png
Greyscale
As such, misclassification errors for a class j with class-wise cost of wj will have wj-times more penalty than misclassification errors for the majority class with weight equals to 1.”; Wherein the class weights increase when the class’s frequency of occurrence is lower than a majority class’s.).
Regarding claim 10, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein subcategories are set for the class labels of a target object in the supervisory data, and the mixing ratio is calculated for each of the subcategories (Aoba: 0040: “Classes and class labels are defined variously, and the class classification method is not particularly limited. In the example shown in FIGS. 3A and 3B, class classification is performed in accordance with the types of the objects. Other examples of class labels are a skin region, a hair region, animals such as a dog and a cat, and artifacts such as an automobile and a building. A class label representing a specific object such as a component A or a component B used in a factory can also be used. On the other hand, the pixels may be classified into main object regions and a background region.”; Wherein the classifications of larger classes, such as animals and artifacts, based on subcategories, such as dogs and cats, constitutes the calculation of mixing ratios for each of the subcategories).
Regarding claim 11, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein camera parameters used when the input data was shot are obtained (Aoba: 0080: “the size of the identified region or how to cut it can be changed based on various kinds of capturing condition information. For example, in a strongly blurred region, a fine texture is lost as information. For this reason, the distribution estimation accuracy may be improved by performing estimation for a wider identified region.”; Wherein the capturing condition, which constitute the camera parameters, are obtained for the estimation of distribution accuracy based on the size of the identified region), and wherein frequencies of occurrence of the mixing ratio of the class labels are calculated based on the camera parameters (Aoba: 0086: “Depending on the difference in the region setting pattern, the supervisory information of the distribution can change even at the same position on the image. FIG. 3C shows rectangular regions 551, 552, and 553 of various sizes at the same position of a training image. In the smallest rectangular region 551, the area ratio of sky:non-sky is r=1. On the other hand, each of the rectangular regions 552 and 553 includes a non-sky region, and the area ratios are r=0.9 and r=0.8, respectively.”;
0087: “The training unit 2200 performs training of the estimator corresponding to each region setting pattern. That is, the training unit 2200 performs training of the estimator corresponding to a region setting pattern of interest based on the identified region set in accordance with the region setting pattern of interest and the supervisory information given for the identified region. As a result, the training unit 2200 generates an estimator corresponding to each of the plurality of region setting patterns. For example, letting q be the index of a region setting pattern, and Q be the total number of region setting patterns, Q types of estimators yq can be obtained by training. Training of the estimator can be done as in the first embodiment.”;
0088: “In step S2400, the evaluation unit 2400 evaluates the identification accuracy of the estimator obtained in step S2200 together with the capturing condition information and generates a region setter.”;
Wherein the generation of a region setter based on capturing conditions and estimators generated based on region setting patterns constitutes the calculation of mixing ratios frequencies of occurrence based on camera parameters since the class ratio frequencies are modified based on the size of identifier regions)
Regarding claim 12, Aoba in view of Fernando discloses: The learning apparatus according to claim 11, wherein the camera parameters include at least one of a Bv value, an exposure time, an F value, an ISO sensitivity value, a shooting date and time, and GPS information (Claim limitation is interpreted according to the SuperGuide interpretation of claim 12 disclosed above) (Aoba: 0081: “The capturing condition information includes information unique to an image capturing apparatus and information unique to a captured image. As the information unique to the image capturing apparatus, the size or the allowable diameter of a circle of confusion of a sensor, the brightness or the focal length of an optical system, and the like are usable. As the information unique to the captured image, an aperture value, a focus distance, a By value, a RAW image, an exposure time, a gain (ISO sensitivity), a white balance coefficient, distance information, position information by a GPS, time information such as a date/time, and the like are usable”).
As per claim(s) 13, arguments made in rejecting claim(s) 1 are analogous.
As per claim(s) 14, arguments made in rejecting claim(s) 1 are analogous. In addition, paragraphs 0009-0010 of Aoba disclose the use of “A non-transitory computer-readable recording medium storing a program”.
Regarding claim 16, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein a degree of importance of each training data item included in the plurality of training data items is determined based on the frequencies of occurrence of the mixing ratio of the class labels (Fernando: Section: IV.A. Loss Function Formulation: “The class weights wj can be handled as a hyper-parameter that is learned from data by cross validation or set proportional to inverse class frequency. We set wj equal to the log ratio of the class frequency of the majority class and the class frequency nj (computed over the training dataset) as follows:
PNG
media_image2.png
87
478
media_image2.png
Greyscale
…The loss function defined in equation (7) optimizes a dynamically weighted training loss which reflects labels’ importance level based on class frequency”).
Regarding claim 17, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein the mixing ratio of class labels for a region is a value obtained by dividing the number of pixels having one of the class labels in the region by the total number of pixels included in the region (Aoba: 0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Claim(s) 6 & 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Fernando, and further in view of Fukuda (US-20120327127-A1).
Regarding claim 6, Aoba in view of Fernando discloses: The learning apparatus according to claim 1, wherein, in calculating the mixing ratio of the class labels in the plurality of training data items, each training data item is divided into a plurality of partial regions, and for each partial region, a first ratio is calculated, the first ratio being a mixing ratio of class labels within the partial region, and wherein the training of the estimator is controlled based on the first ratio (Aoba: 0006: “a training apparatus comprises: an extraction unit configured to extract a feature amount of an identified image for training of an estimator; an obtaining unit configured to obtain, as supervisory information, distribution-related information which relates to distribution of regions having attributes different from each other in the identified image; and a training unit configured to perform the training of the estimator using a combination of the feature amount of the identified image and the supervisory information, wherein the estimator is trained to estimate the distribution from the feature amount.”;
0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Aoba in view of Fernando does not disclose expressly: wherein, for each partial region, a first ratio and a second ratio are calculated, the first ratio being a mixing ratio of class labels within the partial region, and the second ratio being a mixing ratio of class labels including a peripheral partial region, and wherein the training of the estimator is controlled based on the first ratio and the second ratio.
Thus, Aoba in view of Fernando does not disclose expressly: the incorporation of a calculated a second ratio weight, based on a mixing ratio of class labels including a peripheral partial region, into the training of the estimator.
Fukuda discloses: a method for calculating an average size of regions within an image classified as facial regions, based on the determined weight values of the facial regions. The average size of the facial regions is determined based on a weighted average of the face regions, wherein the weight of each face region is determined based on the facial image’s proximity to the center of the picture (Fukuda: 0041: “The CPU 104 calculates an average value SAVE of the areas of face rectangles as the typical size of the face region. Of course, the CPU 104 may calculate a weighted average. As an example of weighting, weight values according to the positions in a picture are set in advance, and a weight value is determined based on the coordinates of a face rectangle. For example, the weight value of a central part of a picture is set to be larger than those of peripheral parts of the picture, and a larger weight is attached to a face image located at the center of the picture. Alternatively, the reliability of each face rectangle detected by the face-detection process may be used as a weight value.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the method for calculating a weighted average based on distance of each facial region to the center taught by Fukuda into the training of the estimator disclosed by Aoba in view of Fernando by weighing pixel values based on their distance from an identified image’s center. The suggestion/motivation for doing so would have been “Alternatively, not all face rectangles need to be used in determination of the typical size of a face region. A picture captured in urban area or the like often includes images of persons (to be referred to as images of non-subject persons hereinafter) other than an image of a person to be captured (to be referred to as an image of a subject person hereinafter). The images of the non-subject persons are normally sufficiently smaller than the image of the subject person. Hence, face rectangles, the area of each of which is less than, for example, ¼ that of a maximum face rectangle, may be excluded from determination of the typical size of a face region. A standard deviation d of the areas of face rectangles may be calculated, face rectangles each having an area less than SAVE —d may be excluded, and the average value SAVE of the areas of the remaining face rectangles may be calculated. Alternatively, when the weight values of peripheral parts of a picture are set to be zero, face rectangles can be excluded depending on the positions of the face rectangles (the positions where face images appear).” (Fukuda: 0042; Wherein the weighted average allows for the weights to be adjusted based on their perceived relevancy.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Fernando with Fukuda to obtain the invention as specified in claim 6.
Regarding claim 18, Aoba in view of Fernando and Fukuda discloses: The learning apparatus according to claim 6, wherein the first ratio is calculated by counting a number of pixels belonging to one of the class labels inside the partial region and dividing the counted number by a total number of pixels constituting the partial region (Aoba: 0042: “ distribution-related information is information representing the ratio of each of regions of attributes in a target image. As a detailed example, distribution-related information can be the area ratio of regions belonging to the respective classes in a target image. A case in which there are two classes “sky” and “non-sky” will be described with reference to an example shown in FIGS. 4A to 4C. FIGS. 4A and 4B show a training image 510 and a class label 520 thereof, respectively. The class label 520 represents a pixel whose class is “sky” by white and a pixel whose class is “non-sky” by black. FIG. 4B shows an enlarged view 526 that enlarges a region 525 on the class label 520 corresponding to an identified image 515 in the training image 510 . The enlarged view 526 shows a non-sky region 521 and a sky region 522 . At this time, the distribution of the identified image 515 can be represented by an area ratio r of the sky region and the non-sky region in the corresponding region 525 . For example, if the number of sky region pixels is 192, and the number of non-sky region pixels is 64 in the rectangular region of 16×16 pixels, r=192/256=0.75.”).
Regarding claim 19, Aoba in view of Fernando and Fukuda discloses The learning apparatus according to claim 6, wherein the peripheral partial region includes the partial region and the second ratio is obtained by calculating a weighted average value of the class labels, weights being set to
decrease as a distance from a center position of the partial region increases (Fukuda: 0041: “The CPU 104 calculates an average value SAVE of the areas of face rectangles as the typical size of the face region. Of course, the CPU 104 may calculate a weighted average. As an example of weighting, weight values according to the positions in a picture are set in advance, and a weight value is determined based on the coordinates of a face rectangle. For example, the weight value of a central part of a picture is set to be larger than those of peripheral parts of the picture, and a larger weight is attached to a face image located at the center of the picture. Alternatively, the reliability of each face rectangle detected by the face-detection process may be used as a weight value.”).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Fernando and Fukuda, and further in view of Kondo et al. (US-6965693-B1) hereinafter referenced as Kondo.
Regarding claim 7, Aoba in view of Fernando and Fukuda discloses: The learning apparatus according to claim 6.
Aoba in view of Fernando and Fukuda does not disclose expressly: wherein the second ratio includes statistic information pertaining to a direction to a center of gravity position of each of the class labels from a center position of a partial region of interest.
Kondo discloses: statistic information pertaining to a direction to a center of gravity position of each of the class labels from a center position of an image (Kondo: Col 17: lines 12-27: “As mentioned above, the value X that the gravity center has in the X-axis direction is applied, in order to detect the orientation of the face. Rather, the value Y that the gravity center has in the Y-axis direction may be applied, to detect the orientation of the face. As described already, the orientation of the face is detected from the positional relation between the gravity center of the face image and the gravity center of the region to which the class number is allocated. The orientation of the face can be therefore detected correctly, regardless of the position of the face. In addition, the angle of the face can be accurately detected. This is because the correlation value is calculated by applying a large weight coefficient W to any region prominently related with the orientation of the face and a small weight coefficient W to any region weakly related with the orientation of the face.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the orientation calculation method taught by Kondo into the training of the estimator disclosed by Aoba in view of Fernando and Fukuda by weighing pixel labels based on their relation to an object’s center of gravity. The suggestion/motivation for doing so would have been “In this case, the value ( difference Dxfrequency C) and the angle V are correlated firmly, and the class that is allocated to the region has a large weight coefficient. On the other hand, if the distribution of points, which is determined by ( difference Dxfrequency C) and angle V, diverges is illustrated in FIG. 27B, the value (difference Dxfrequency C) and the angle V are correlated but a little. In this case, the class that is allocated to the region has a small weight coefficient” (Kondo: Col 16: Lines 45-53; Wherein the pixels may be labeled based upon their correlation to an image class’s center of gravity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Fernando and Fukuda with Kondo to obtain the invention as specified in claim 7.
Claim(s) 9 & 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoba in view of Fernando, and further in view of Lin et al. (US-20200334501-A1) hereinafter referenced as Lin.
Regarding claim 9, Aoba in view of Fernando discloses: The learning apparatus according to claim 1.
Aoba in view of Fernando does not disclose expressly: wherein a padding amount of each training data item used to train the estimator is controlled such that a total number of training data items having a high degree of importance low frequency of occurrence is larger than a total number of other training data.
Lin discloses: wherein a padding amount of each training data item used to train an estimator is controlled such that a total number of training data items having a low frequency of occurrence is larger than a total number of other training data (Lin: 0055-0056: “In some datasets, the ratio of occurrence of the most common to the least common object class may be one or more orders of magnitude larger than that of simple dataset. Thus, certain embodiments of the present disclosure include performing offline oversampling of the images containing the rare classes to balance the class distribution to create a balanced training set. For example, if an object class is relatively rare (e.g., class 1), class balancing may be performed by selecting a proportionately higher number of images from an initial dataset that include objects from class 1, so that in the initial training set, the same number of objects are represented from each object class.”; Wherein the oversampling of images, by selecting a proportionately higher number of images, based on frequency of occurrence constitutes a total number of training data items having a low frequency of occurrence being larger than a total number of other training data.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image oversampling with data augmentation taught by Lin into the training of the estimator disclosed by Aoba in view of Fernando by oversampling identified region images based on their mixing ratios. The suggestion/motivation for doing so would have been “class balancing (as described in FIGS. 3 and 4) results in reduced sampling diversity. Low data diversity may result from oversampling or repeating training images of rare object classes. Therefore, data augmentation techniques such as horizontal flipping of images may be used to increase the level of diversity.” (Lin: 0062; Wherein the oversampling with augmentation allows for the classes to be equally represented while also maintaining data diversity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Fernando with Lin to obtain the invention as specified in claim 9.
Regarding claim 15, Aoba in view of Fernando discloses: The learning apparatus according to claim 1.
Aoba in view of Fernando does not disclose expressly: wherein the training of the estimator is controlled such that unevenness of the frequencies of occurrence of the mixing ratio of the class labels among the plurality of training data items is reduced.
Lin discloses: wherein the training of an estimator is controlled such that unevenness of the frequencies of occurrence of the class labels among a plurality of training data items is reduced (Lin: 0055-0056: “In some datasets, the ratio of occurrence of the most common to the least common object class may be one or more orders of magnitude larger than that of simple dataset. Thus, certain embodiments of the present disclosure include performing offline oversampling of the images containing the rare classes to balance the class distribution to create a balanced training set. For example, if an object class is relatively rare (e.g., class 1), class balancing may be performed by selecting a proportionately higher number of images from an initial dataset that include objects from class 1, so that in the initial training set, the same number of objects are represented from each object class.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image oversampling with data augmentation taught by Lin into the training of the estimator disclosed by Aoba in view of Fernando by oversampling identified region images based on their mixing ratios. The suggestion/motivation for doing so would have been “class balancing (as described in FIGS. 3 and 4) results in reduced sampling diversity. Low data diversity may result from oversampling or repeating training images of rare object classes. Therefore, data augmentation techniques such as horizontal flipping of images may be used to increase the level of diversity.” (Lin: 0062; Wherein the oversampling with augmentation allows for the classes to be equally represented while also maintaining data diversity.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Aoba in view of Fernando with Lin to obtain the invention as specified in claim 15.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights