DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 5-8, and 11-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lilaonitul (U.S. PG-PUB NO. 2024/0395023) in view of Chen (U.S. PG-PUB NO. 2021/0319266).
-Regarding claim 1, Lilaonitul discloses an information processing apparatus that executes active learning by repeating image selection and retraining of a learning model with the selected images (FIG. 1-2, 23), the information processing apparatus comprising: at least one memory storing a program (memory 994, FIG. 23); and at least one processor (processor 993, FIG. 23) that, upon execution of the program is configured to operate as: an acquisition unit configured to acquire a trained learning model (S3, methods checkpoint (acquire the weights of) the model using validation and apply the (trained) model on a labelled test set, paragraph 75-76); and a second selection unit configured to select an image used to retrain the learning model and the acquired learning model (selected images are effectively added to the previous training examples, to form a new (or newly designed) training set for future iterations of the training process, paragraph 64).
Lilaonitul is silent to teaching that a first selection unit configured to select an image transformation method executed on an image by using the acquired learning model ; by using the selected image transformation method. However, the claimed limitation is well known in the art as evidenced by Chen.
In the same field of endeavor, Chen teaches a first selection unit configured to select an image transformation method executed on an image by using the acquired learning model (the performance of the proposed framework was evaluated when applying augmentations individually or in pairs, paragraph 66); by using the selected image transformation method (stochastic data augmentation module, paragraph 44, 50).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Lilaonitul with the teaching of Chen in order to provide improved visual representations.
-Regarding claim 2, the combination further discloses execution of the stored program further configures the first selection unit to calculate a score for an individual image transformation method candidate by using an annotated image including an annotation representing ground truth information and the acquired learning model, and selects an image transformation method based on the score (Chen, linear evaluation results under individual and composition of transformations, paragraph 66-69).
-Regarding claim 3, the combination further discloses execution of the stored program further configures the first selection unit to operate as a first calculation unit that provides, as input to the acquired learning model, a transformed image of the annotated image, the transformed image having been obtained by executing image transformation based on an image transformation method candidate and calculates an uncertainty of an output result obtained by the acquired learning model, and a second calculation unit that calculates the score for an individual image transformation method candidate based on an uncertainty calculated by the first calculation unit (Lilaonitul, For each prediction output, one may get a corresponding uncertainty from which one can derive a score for image selection, paragraph 125; optionally apply image perturbations, paragraph 67; Chen, the performance of the proposed framework was evaluated when applying augmentations individually or in pairs, paragraph 66).
-Regarding claim 5, the combination further discloses the first selection unit selects an image transformation method whose score is high or is equal to or more than a threshold (Chen, One composition of augmentations stands out: random cropping and random color distortion, paragraph 69; Lilaonitul, uncertainty, paragraph 75).
-Regarding claim 6, the combination further discloses execution of the stored program further configures the second selection unit to select an image from unannotated images having no ground truth information added (Lilaonitul, select images from the unlabelled training set, paragraph 64), and use the selected unannotated image an annotation addition target (Lilaonitul, present the new samples to the expert annotator, paragraph 76).
-Regarding claim 7, the combination further discloses execution of the stored program further configures the second selection unit to include a third calculation unit that provides, as an input to the acquired learning model, a transformed image of the unannotated image, the transformed image having been obtained by executing image transformation based on a selected image transformation method and calculates an uncertainty of an output result obtained by the learning model, and wherein the second selection unit includes a fourth calculation unit that calculates a priority of the unannotated image based on the calculated uncertainty (Lilaonitul, validate the trained computer vision model on unlabelled, previously unseen (by the training model) images, paragraph 64; result is a single scalar value, which is used for ranking the images according to high to low uncertainty, paragraph 90; Chen, draw two augmentation functions t~J, t′~ J, paragraph 50).
-Regarding claim 8, the combination further discloses the learning model is used to execute a task including at least one of image classification, object detection, and segmentation (Lilaonitul, active learning for segmentation in digital images, paragraph 58-60).
-Regarding claim 11, the combination further discloses in a case where the output result includes both a classification result and location or area information, the second calculation unit calculates the score based on a combination of the uncertainty calculated based on the classification result and the uncertainty calculated based on the location or area or based on one of the uncertainties (Lilaonitul, uncertainty metric is then the sum of the bounding box uncertainty and the bounding box class uncertainty, paragraph 125).
-Regarding claim 12, the combination further discloses execution of the stored program further configures the first selection unit to select an image transformation method from candidates including at least one of geometrical transformation, color tone transformation, noise addition, blurring, and mosaic (Chen, paragraph 65).
-Regarding claim 13, the combination further discloses execution of the stored program further configures the at least one processor to operate as an update that updates the acquired learning model by retraining the acquired learning model using the selected image, wherein the acquisition unit acquires the updated learning model (Lilaonitul, new model may be again used to calculate the uncertainty and start a new active learning iteration, paragraph 75).
-Regarding claim 14, Lilaonitul discloses an information processing apparatus that executes active learning by repeating image selection and retraining of a learning model with the selected images (FIG. 1-2, 23), the information processing apparatus comprising: at least one memory storing a program (memory 994, FIG. 23); and at least one processor (processor 993, FIG. 23) that, upon execution of the program is configured to operate as: an acquisition unit configured to acquire a trained learning model (S3, methods checkpoint (acquire the weights of) the model using validation and apply the (trained) model on a labelled test set, paragraph 75-76); and a selection unit configured to select an image used to retrain the learning mode and the acquired learning model (selected images are effectively added to the previous training examples, to form a new (or newly designed) training set for future iterations of the training process, paragraph 64).
Lilaonitul is silent to teaching a setting unit configured to set an image transformation method for an image by using the set image transformation method; wherein the setting unit changes a currently set image transformation method, depending on progress of training of the learning model acquired by the acquisition unit. However, the claimed limitation is well known in the art as evidenced by Chen.
In the same field of endeavor, Chen teaches a setting unit configured to set an image transformation method for an image by using the set image transformation method (the performance of the proposed framework was evaluated when applying augmentations individually or in pairs, paragraph 66); wherein the setting unit changes a currently set image transformation method, depending on progress of training of the learning model acquired by the acquisition unit (One composition of augmentations stands out: random cropping and random color distortion, paragraph 69).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Lilaonitul with the teaching of Chen in order to provide improved visual representations.
-Regarding claim 15, Lilaonitul discloses an information processing method that executes active learning by repeating image selection and retraining of a learning model with the selected images (FIG. 1-2, 23), the information processing method comprising: acquiring a trained learning model (S3, methods checkpoint (acquire the weights of) the model using validation and apply the (trained) model on a labelled test set, paragraph 75-76); and executing second selection for selecting an image used to retrain the learning model and the acquired learning model (selected images are effectively added to the previous training examples, to form a new (or newly designed) training set for future iterations of the training process, paragraph 64).
Lilaonitul is silent to teaching executing first selection for selecting an image transformation method executed on an image by using the acquired learning model; by using the selected image transformation method. However, the claimed limitation is well known in the art as evidenced by Chen.
In the same field of endeavor, Chen teaches executing first selection for selecting an image transformation method executed on an image by using the acquired learning model (the performance of the proposed framework was evaluated when applying augmentations individually or in pairs, paragraph 66); by using the selected image transformation method (One composition of augmentations stands out: random cropping and random color distortion, paragraph 69).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Lilaonitul with the teaching of Chen in order to provide improved visual representations.
-Regarding claim 16, Lilaonitul discloses a non-transitory computer readable storage medium that stores a program (memory 994, FIG. 23) causing a computer of an information processing apparatus that executes active learning by repeating image selection and retraining of a learning model with the selected images (processor 993, FIG. 23) to function as: an acquisition unit configured to acquire a trained learning model (S3, methods checkpoint (acquire the weights of) the model using validation and apply the (trained) model on a labelled test set, paragraph 75-76); and a second selection unit configured to select an image used to retrain the learning model and the acquired learning model (selected images are effectively added to the previous training examples, to form a new (or newly designed) training set for future iterations of the training process, paragraph 64).
Lilaonitul is silent to teaching that a first selection unit configured to select an image transformation method executed on an image by using the acquired learning model; by using the selected image transformation method. However, the claimed limitation is well known in the art as evidenced by Chen.
In the same field of endeavor, Chen teaches a first selection unit configured to select an image transformation method executed on an image by using the acquired learning model (the performance of the proposed framework was evaluated when applying augmentations individually or in pairs, paragraph 66); by using the selected image transformation method (stochastic data augmentation module, paragraph 44, 50).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Lilaonitul with the teaching of Chen in order to provide improved visual representations.
Claim(s) 4 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lilaonitul (U.S. PG-PUB NO. 2024/0395023) in view of Chen (U.S. PG-PUB NO. 2021/0319266) and further in view of Mahendran (U.S. PG-PUB NO. 2023/0290132).
-Regarding claim 4, the combination is silent to teaching that the first calculation unit calculates the uncertainty by comparing the output result and the ground truth information.. However, the claimed limitation is well known in the art as evidenced by Mahendran.
In the same field of endeavor, Mahendran teaches the first calculation unit calculates the uncertainty by comparing the output result and the ground truth information (compare the object recognition output to the ground truth annotations of the one or more objects in the set of training images, paragraph 76).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of Mahendran in order to correctly correlate locations of virtual objects in relation to real objects.
-Regarding claim 10, the combination further discloses in a case where the output result includes location information or area information, the first calculation unit calculates the uncertainty based on a degree of overlapping between a location or an area obtained from the ground truth information and a location or an area included in the output result (Mahendran, localization score is a prediction of an intersection-over-union overlap between a predicted object bounding box and a ground-truth object bounding box, paragraph 60).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lilaonitul (U.S. PG-PUB NO. 2024/0395023) in view of Chen (U.S. PG-PUB NO. 2021/0319266) and further in view of Omi (U.S. PG-PUB NO. 2025/0086943).
-Regarding claim 9, the combination is silent to teaching that . However, the claimed limitation is well known in the art as evidenced by Omi.
In the same field of endeavor, Omi teaches in a case where the output result includes a classification result, the first calculation unit calculates the uncertainty based on a probability distribution distance between a probability distribution obtained from the ground truth information and the classification result transformed into a probability distribution (by using a cross entropy loss as a loss function, the second training section 12 carries out training so as to minimize an error between an output from the machine learning model and correct data, paragraph 35).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of Omi in order to improve images for training.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PING Y HSIEH whose telephone number is (571)270-3011. The examiner can normally be reached Monday-Friday, 9am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PING Y HSIEH/ Primary Examiner, Art Unit 2664
Prosecution Insights