DETAILED ACTION
This communication is in response to the pre-brief appeal conference decision mailed 06/17/25.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/02/2025 was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed 01/08/2026 have been fully considered but they are not persuasive. In regard to Rejection under 35 U.S.C. 101, (see applicant’s remarks, pgs. 5-7), applicant argues “generate the mini-batch data by selecting divided images divided from a learning input image and the annotation image such that a second area ratio of the rare class with respect to a total area of the divided images included in the mini-batch data is equal to or higher than a second setting value that is higher than the first area ratio of the rare class;”, helps assist in achieving a higher and improved accuracy. Examiner would like to point out that generating mini batch data is insignificant solution activity as is it is gathering data, (see MPEP 2106.05(g)). This insignificant extra solution activity is well understood routine and conventional, see see MPEP 2106.05(d)(II)(i)). Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362. Applicant also points out that the newly amended limitation “input the generated mini-batch data, including the selected divided images that were selected based on the ratio of the rare class, for learning to the machine learning model.”, is to further explain inputting the mini-batch data being put into the machine learning model for performing semantic segmentation to determine a class. Examiner would like to point out, that with how the new limitation is states, that it is mere instructions to apply the judicial exception using generic computer components, not showing an improvement to the invention. Therefore the 35 USC 101 rejection is maintained.
In regard to the Rejections under 35 U.S.C. 103, see Applicant’s remarks pgs. 7-9, applicant argues that amended claim 1 overcomes the prior art. Examiner would like to point out that prior art Yamada et al (US Published Patent Application No. 20180276500, "Yamada") teaches the newly added limitation. Yamada teaches having ratio that is smaller than threshold and picking that data and inputting the mini-batch data into a model.
Specifically:
(Yamada, paragraph 0187, “Accordingly, it is determined whether or not the recognition target 3D model is sufficiently taken by the camera, and the model having the masking ratio that is equal to or smaller than the threshold is used as teacher data [divided images that were selected based on the ratio of the rare class, examiner would like to point out that in the applications specification, the rare class is chosen due to a low ratio therefore the mask ration being smaller or equal to a threshold is being interpreted as this].” and paragraph 0251, ““A learning control unit 31 controls generation of teacher data, and inputs mini-batches to the deep learning execution unit [inputting the generated mini-batch data…for learning to the machine learning model] 204.”
Therefore the 35 USC 103 rejection is maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Claim 1 is directed to a machine.
Step 1: yes.
Step 2A, prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
calculate, from an annotation image which is a source of the mini-batch data, a first area ratio of each of the plurality of classes with respect to an entire area of the annotation image, the plurality of classes being types of objects that appear in the annotation image; (limitation is directed to a mathematical concept).
specify from the plurality of classes that appear in the annotation image, a rare class of which the first area ratio is lower than a first setting value; and (limitation is directed to a mental process. One can mentally decide a rare class by use of pen and paper with respect to the plurality of classes and the objects in the image.)
Step 2A, prong 1: yes.
Step 2A, prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
generate the mini-batch data by selecting divided images divided from a learning input image and the annotation image such that a second area ratio of the rare class with respect to a total area of the divided images included in the mini-batch data is equal to or higher than a second setting value that is higher than the first area ratio of the rare class. (e.g., insignificant extra solution activity of mere data gathering or data output), see MPEP2106.05(g).
input the generated mini-batch data, including the selected divided images that were selected based on the ratio of the rare class, for learning to the machine learning model. (e.g., mere instructions to apply the judicial exception using generic computer components, see MPEP 2106.05(f)).
Step 2A, prong 2: Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
generate the mini-batch data by selecting divided images divided from a learning input image and the annotation image such that a second area ratio of the rare class with respect to a total area of the divided images included in the mini-batch data is equal to or higher than a second setting value that is higher than the first area ratio of the rare class. (e.g., insignificant extra solution activity of mere data gathering or data output), see MPEP 2106.05(g), using components and functions claimed at a high level of generality have been determined by the courts as being well- understood, routine and conventional activities in the field of computer functions (see MPEP 2106.05(d)(II)(i)). Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
input the generated mini-batch data, including the selected divided images that were selected based on the ratio of the rare class, for learning to the machine learning model. (e.g., mere instructions to apply the judicial exception using generic computer components, see MPEP 2106.05(f)).
Step 2B: Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 2,
Claim 2 incorporates the analysis of the machine of claim 1.
Step 2A, prong 2/Step 2B:
receive a selection instruction as to whether or not to perform processing of generating the mini-batch data. (e.g., insignificant extra solution activity of mere data gathering or data output), see MPEP 2106.05(g), using components and functions claimed at a high level of generality have been determined by the courts as being well- understood, routine and conventional activities in the field of computer functions (see MPEP 2106.05(d)(II)(i)). Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
Step 2A, prong 2: Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 3,
Claim 3 incorporates the analysis of the machine of claim 1.
Step 2A, prong 2/Step 2B:
generate a plurality of pieces of the mini-batch data according to a certain rule, and selects, among the plurality of pieces of the mini-batch data generated according to the certain rule, the mini-batch data in which the second area ratio is equal to or higher than the second setting value, for use in the learning. (e.g., insignificant extra solution activity of mere data gathering or data output), see MPEP 2106.05(g), using components and functions claimed at a high level of generality have been determined by the courts as being well- understood, routine and conventional activities in the field of computer functions (see MPEP 2106.05(d)(II)(i)). Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
Step 2A, prong 2: Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
Step 2B: Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 4,
Claim 4 incorporates the analysis of the machine of claim 1.
detect a bias region and a non-bias region of the rare class in the annotation image, and set the number of cut-outs of an image which is a source of the mini-batch data in the bias region to be larger than the number of cut-outs of the image in the non-bias region. (limitation is directed to a mental process. One can mentally set the number of cut-outs of an image by use of pen and paper with respect to detecting the bias and non-bias regions.)
Regarding claim 5,
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Claim 5 is directed to a manufacture.
Step 1: yes
The rest of the analysis for claim 5 is analogous to claim 1.
Regarding claim 6,
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Claim 6 is directed to a process.
Step 1: yes
The rest of the analysis for claim 6 is analogous to claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1-3 and 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Yamada et al (US Published Patent Application No. 20180276500, "Yamada"), in view of Shimizu et al (Balanced Mini-batch Training for Imbalanced Image Data Classification with Neural Network, "Shimizu").
In regard to claim 1 and analogous claims 5 and 6, Yamada teaches A mini-batch learning apparatus that inputs mini-batch data for learning to a machine learning model for performing semantic segmentation, which determines a plurality of classes in an image in units of pixels, the apparatus comprising: a memory; and a processor coupled to the memory and configured to: (Yamada, paragraph 0049, “The teacher data refers to a pair of "input data" and "correct label" that is used in supervised deep learning. Deep learning is performed by inputting the "input data" to a neural network having a lot of parameters to update a difference between an inference label and the correct label (weight during learning) and find a learned weight. Thus, the mode of the teacher data depends on an issue to be learned (thereinafter the issue may be referred to as "task"). Some examples of the teacher data are illustrated in a following table 1.” And Table 1,
PNG
media_image1.png
177
477
media_image1.png
Greyscale
[determines a plurality of classes in an image in units of pixels], paragraph 0085, “In practice, another embodiment using teacher data (Object Detection, Semantic Segmentation [performing semantic segmentation], and so forth) in a task other than the image classification task is possible.” and paragraph 0251, “A learning control unit 31 controls generation of teacher data, and inputs mini-batches to the deep learning execution unit [inputting mini-batch data to the machine learning model,] 204.” and paragraph 0061, “The image processing apparatus 100 in FIG. 1 includes the CPU 1, a Random access memory (RAM) 2, the GPU 3, and a video random access memory (VRAM) 4. [a memory; and a processor]”))
calculate, from an annotation image which is a source of the mini-batch data, a first area ratio of each of the plurality of classes with respect to an entire area of the annotation image, the plurality of classes being types of objects that appear in the annotation image; (Yamada, paragraph 0182, “The intermediate data 196 is data inputted to the masking ratio calculation unit [a first area ratio of each of the plurality of classes with respect to an entire area of the annotation image;] 198. The intermediate data 196 is image data having the same number of pixels as the teacher data image [a learning input image and an annotation image which are sources of the mini-batch data,]. Two images are rendered under the same rendering conditions as the teacher data image.” And paragraph 0088, “The "generation target label" refers to the type of the generation target, and includes, for example, automobiles (for example, family car, truck, bus, bicycle), animals (for example, bird, dog, cat, cow, horse, and monkey), plants (for example, strawberry, tomato, rose), and other objects viewable to human.” [types of objects that appear in the annotation image])
input the generated mini-batch data, including the selected divided images that were selected based on the ratio of the rare class, for learning to the machine learning model. (Yamada, paragraph 0187, “Accordingly, it is determined whether or not the recognition target 3D model is sufficiently taken by the camera, and the model having the masking ratio that is equal to or smaller than the threshold is used as teacher data [divided images that were selected based on the ratio of the rare class, examiner would like to point out that in the applications specification, the rare class is chosen due to a low ratio therefore the mask ration being smaller or equal to a threshold is being interpreted as this].” and paragraph 0251, “A learning control unit 31 controls generation of teacher data, and inputs mini-batches to the deep learning execution unit [inputting the generated mini-batch data…for learning to the machine learning model] 204.”
However, Yamada does not explicitly teach specify from the plurality of classes that appear in the annotation image, a rare class of which the first area ratio is lower than a first setting value; and
generate the mini-batch data by selecting divided images divided from a learning input image and the annotation image such that a second area ratio of the rare class with respect to a total area of the divided images included in the mini-batch data is equal to or higher than a second setting value that is higher than the first area ratio of the rare class.
Shimizu teaches specify from the plurality of classes that appear in the annotation image, a rare class of which the first area ratio is lower than a first setting value; and (Shimizu, pg. 29, Col. 2, D., paragraph 1, “Tables 3 and 4 are contingency tables for the test data provided using conventional and our balanced mini-batch with imbalanced MNIST (0.1%). F-measures for each class [plurality of classes that appear in the annotation image] calculated from the contingency tables are shown in Fig. 3. The classifier trained with minibatch failed to classify class 6 samples; therefore, F-measure of class 6 was quite low.” And paragraph 2, “Only our balanced mini-batch succeeded in improving F-measures for all classes [first area ratio is lower], and the average F-measure was 10% or more higher [lower than a first setting value] than that of over-sampling and under-sampling.”)
generate the mini-batch data by selecting divided images divided from a learning input image and the annotation image (Shimizu, pg. 28, Col. 2, paragraph 6, “We used two imbalanced image datasets; imbalanced MNIST and egg image [the annotation image]. Their sample images are shown in Fig. 2.” And paragraph 7, “The original MNIST dataset has images [learning input image] of handwritten digits from “0” to “9” and consists of 60,000 samples used for training and 10,000 samples for test, but we intentionally extracted some images [selecting divided images] of “5” and “6” from the training data to create an imbalanced two-class training dataset. We changed the ratio of class 6 to class 5 in training samples for 100%, 10%, 1%, and 0.1%”)
such that a second area ratio of the rare class with respect to a total area of the divided images included in the mini-batch data is equal to or higher than a second setting value that is higher than the first area ratio of the rare class. (Shimizu, pg. 30, Col. 1, paragraph 1, “method with imbalanced data made the classifier have weak recall for minority class 2, then F-measure for the class was low. With over-sampling or down-sizing, F-measure for class 2 [second area ratio of the rare class] improved over 90% [equal to or higher than a second setting value] but that for class 0 worsened. Only our balanced mini-batch [a total area of the divided images included in the mini-batch data] succeeded in improving F-measures for all classes, and the average F-measure was 10% or more higher than that of over-sampling and under-sampling.”)
Yamada, Li and Shimizu are related to the same field of endeavor (i.e. image processing). In view of the teachings of Shimizu, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Shhimizu to Yamada and Li before the effective filing date of the claimed invention in order to achieve higher classification ability. (Shimizu, pg. 27, Abstract, “In an experiment, the neural network trained with the proposed method achieved higher classification ability than that trained with over-sampled or undersampled data for two types of imbalanced image datasets.”)
In regard to claim 2, Yamada and Shimizu teach the apparatus of claim 1.
Yamada further teaches receive a selection instruction as to whether or not to perform processing of generating the mini-batch data. (Yamada, paragraph 0194, “In step S602, the teacher image generation control unit 192 outputs the inputted masking ratio threshold 191, recognition target 3D model variation 159, background 3D model 193, illumination model 194, and camera model 162 to the 3D space renderer 195, the processing then proceeds to step S603.”)
In regard to claim 3, Yamada and Shimizu teach the apparatus of claim 1.
Yamada further teaches wherein the processor is further configured to generate a plurality of pieces of the mini-batch data according to a certain rule, and selects, among the plurality of pieces of the mini-batch data generated according to the certain rule, the mini-batch data in which the second area ratio is equal to or higher than the second setting value, for use in the learning. (Yamada, paragraph 0197, “In step S604, the masking ratio calculation unit 198 calculates the masking ratio using the intermediate data 196, the processing then proceeds to step S605.” and paragraph 0198, “In step S605, the teacher image generation control unit 192 determines whether the masking ratio is higher than a masking ratio threshold, or is equal to or smaller than the masking ratio threshold.”)
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Yamada, in view of Shimizu and in further view Shi et al (Normalized Cuts and Image Segmentation, "Shi").
In regard to claim 4, Yamada and Shimizu teach the apparatus of claim 1. However, Yamada and Li do not explicitly teach wherein the generation unit detects a bias region and a non-bias region of the rare class in the annotation image, and sets the number of cut-outs of an image which is a source of the mini-batch data in the bias region to be larger than the number of cut-outs of the image in the non-bias region.
Shi teaches processor is further configured to detect a bias region and a non-bias region of the rare class in the annotation image, and set the number of cut-outs of an image which is a source of the mini-batch data in the bias region to be larger than the number of cut-outs of the image in the non-bias region. (Shi, pg. 732, Col. 2, paragraph 2, “To avoid this unnatural bias for partitioning out small sets of points, we propose a new measure of disassociation between two groups [detects a bias region and a non-bias region of the rare class in the annotation image,]. Instead of looking at the value of total edge weight connecting the two partitions, our measure computes the cut cost as a fraction of the total edge connections to all the nodes in the graph… With this definition of the disassociation between the groups, the cut that partitions out small isolated points will no longer have small Neut value, since the cut value will almost certainly be a large percentage of the total connection from that small set to all other nodes. [a source of the mini-batch data in the bias region to be larger than the number of cut-outs of the image in the non-bias region.]”)
Yamada and Shi are related to the same field of endeavor (i.e. image processing). In view of the teachings of Shi, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Shi to Yamada before the effective filing date of the claimed invention in order to make segmentation more efficient. (Shi, Abstract, “The normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups. We show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion. We have applied this approach to segmenting static images and found results very encouraging.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKYLAR K VANWORMER whose telephone number is (703)756-1571. The examiner can normally be reached M-F 6:00am to 3:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.K.V./ Examiner, Art Unit 2129
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights