Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
The status of claims 1-17 is:
Claims 1-17 were pending as of the Non-Final Rejection mailed 03/25/2026.
Claims 1, 3, 6-7, 10, and 16-17 are amended as of the arguments and remarks received 06/22/2026.
Claims 4-5, 8-9, and 11-15 remain as originally presented as of the arguments and remarks received 06/22/2026.
Claim 2 is cancelled as of the arguments and remarks received 06/22/2026.
Claim Rejections - 35 USC § 112
The 112(b) rejections of the claims are withdrawn in light of the amendments and remarks received 06/22/2026.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 12, and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Makhzani et al. (Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015). Adversarial autoencoders. arXiv preprint arXiv:1511.05644., hereinafter “Makhzani”) in view of Noguchi (U.S. Patent Publication No 2021/0090312 listed in the IDS received 03/05/2026, hereinafter “Noguchi”) and Hamedi et al. (U.S. Patent Publication No 2023/0260250, hereinafter “Hamedi”).
Regarding claim 1, Makhzani discloses a similar image set creating apparatus configured to:
acquire a plurality of images (Makhzani Fig. 8: x);
extract a plurality of first features from the images by using a first model that executes an image classification task (Makhzani Page 10: “The architecture that we use is similar to Figure 8, with the difference that we remove the semi supervised classification stage and thus no longer train the network on any labeled mini-batch. Another difference is that the inference network q(y|x) predicts a one-hot vector whose dimension is the number of categories that we wish the data to be clustered into. Figure 9 illustrates the unsupervised clustering performance of the AAE on MNIST when the number of clusters is 16. Each row corresponds to one cluster. The first image in each row shows the cluster heads, which are digits generated by fixing the style variable to zero and setting the label variable to one of the 16 one-hot vectors”; Makhzani Fig. 8: top adversarial network);
extract a plurality of second features from the images by using a second model that executes an image classification task (Makhzani Page 10: “The architecture that we use is similar to Figure 8, with the difference that we remove the semi supervised classification stage and thus no longer train the network on any labeled mini-batch. Another difference is that the inference network q(y|x) predicts a one-hot vector whose dimension is the number of categories that we wish the data to be clustered into. Figure 9 illustrates the unsupervised clustering performance of the AAE on MNIST when the number of clusters is 16. Each row corresponds to one cluster. The first image in each row shows the cluster heads, which are digits generated by fixing the style variable to zero and setting the label variable to one of the 16 one-hot vectors”; Makhzani Fig. 8: bottom adversarial network), the second model being trained in such a manner that mutually similar images in a latent space are continuously distributed compared to the first model (Makhzani Fig. 8 description: “the top adversarial network imposes a Categorical distribution on the label representation and the bottom adversarial network imposes a Gaussian distribution on the style representation”); and
create an image set based on the images (Makhzani Page 11: “We used the following evaluation protocol: Once the training is done, for each cluster i, we found the validation example xn that maximizes q(yi|xn), and assigned the label of xn to all the points in the cluster i. We then computed the test error based on the assigned class labels to each cluster. As shown in Table 3, the AAE achieves the classification error rate of 9.55% and 4.10% with 16 and 30 total labels respectively. We observed that as the number of clusters grows, the classification rate improves”).
Makhzani does not explicitly disclose the apparatus comprising processing circuitry, the processing circuitry being configured to:
select, from the images, an image of interest serving as a reference of a similar image set, and an auxiliary image similar to the image of interest, based on the first features and the second features.
However, Noguchi teaches the apparatus comprising processing circuitry (Noguchi [0008]: “The image processing apparatus may comprise a processor”), the processing circuitry configured to:
select, from the images, an image of interest serving as a reference of a similar image set (Noguchi [0089]: “In a case where the centroid is determined, the image closest to the determined centroid (image distributed at the closest position) is determined by the CPU 2 as a representative image of the group having the centroid (step S37 in FIG. 5). In a case where the representative image is determined, the image classification result is displayed in the classification result window by using the determined representative image (step S38 in FIG. 5). This classification result window is displayed on the display screen of the display apparatus 3”; Noguchi [0091]: “In the classification result display window 80, representative images IR1 to IR10 of each group classified into 10 groups are displayed under the control of the CPU 2 (a representative image display control device). For example, each of the representative images IR1 to IR10 is an image representing each of the groups G1 to G10 shown in FIG. 12. Each of the representative images IR1 to IR10 is the image closest to the centroid C1 to the centroid C10 of each of the groups G1 to G10 shown in FIG. 12”; Noguchi [0093]: “In a case where any one of the representative image IR1 to the representative image IR10 displayed in the classification result display window 80 is double-clicked (any device other than the mouse 9 may be used as long as it can select an image, such as touching the touch panel display) by the mouse 9 (a representative image selection device) (YES in step S39 of FIG. 5), the double-clicked representative image is selected. Images included in the group represented by the selected representative image (images similar to each other) are displayed in a list in the second classification window 70A (step S40 in FIG. 5)”), and an auxiliary image similar to the image of interest, based on the first features and the second features (Noguchi [0089]: “In a case where the centroid is determined, the image closest to the determined centroid (image distributed at the closest position) is determined by the CPU 2 as a representative image of the group having the centroid (step S37 in FIG. 5). In a case where the representative image is determined, the image classification result is displayed in the classification result window by using the determined representative image (step S38 in FIG. 5). This classification result window is displayed on the display screen of the display apparatus 3”; Noguchi [0091]: “In the classification result display window 80, representative images IR1 to IR10 of each group classified into 10 groups are displayed under the control of the CPU 2 (a representative image display control device). For example, each of the representative images IR1 to IR10 is an image representing each of the groups G1 to G10 shown in FIG. 12. Each of the representative images IR1 to IR10 is the image closest to the centroid C1 to the centroid C10 of each of the groups G1 to G10 shown in FIG. 12”; Noguchi [0093]: “In a case where any one of the representative image IR1 to the representative image IR10 displayed in the classification result display window 80 is double-clicked (any device other than the mouse 9 may be used as long as it can select an image, such as touching the touch panel display) by the mouse 9 (a representative image selection device) (YES in step S39 of FIG. 5), the double-clicked representative image is selected. Images included in the group represented by the selected representative image (images similar to each other) are displayed in a list in the second classification window 70A (step S40 in FIG. 5)”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the image selection as taught by Noguchi with the apparatus of Makhzani because it would allow a user to look at and make decisions based on the objects in the image sets (Noguchi [0089], [0091], [0093]). For example, a doctor examining medical images or an engineer looking at manufactured parts. This motivation for the combination of Makhzani and Noguchi is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention and rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
The Makhzani and Noguchi combination does not explicitly disclose the apparatus, configured to:
select, from the images, two or more images of interest serving as a reference of a similar image set, based on the first features; and
select, from the images, auxiliary images similar to the selected two or more of the images of interest, based on the second features.
However, Hamedi teaches the apparatus, configured to:
select two or more images of interest serving as a reference of a similar image set (Hamedi [0102]: “For example, the image analysis module 125 may be configured to identify the set of candidate images including the first candidate image. The image analysis module 125 may be configured to determine, for each candidate image of the set of candidate images, whether the candidate image is similar to the set of training images based on the first candidate image and each of the first generative model and the second generative model. The image analysis module 125 may also be configured to identify a subset of the set of candidate images that are similar to the set of training images. For example, the image analysis module 125 may be configured to apply a regression algorithm to the first candidate image and each of the first generative model and the second generative model to determine whether the first candidate image is similar to the set of training images”); and
select, from the images, auxiliary images similar to the selected two or more of the images of interest, based on the second features (Hamedi [0102]: “For example, the image analysis module 125 may be configured to identify the set of candidate images including the first candidate image. The image analysis module 125 may be configured to determine, for each candidate image of the set of candidate images, whether the candidate image is similar to the set of training images based on the first candidate image and each of the first generative model and the second generative model. The image analysis module 125 may also be configured to identify a subset of the set of candidate images that are similar to the set of training images. For example, the image analysis module 125 may be configured to apply a regression algorithm to the first candidate image and each of the first generative model and the second generative model to determine whether the first candidate image is similar to the set of training images”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate comparing multiple images at a time as taught by Hamedi with the apparatus of Makhzani and Noguchi because it would allow for faster comparisons of large sets of images. This motivation for the combination of Makhzani, Noguchi, and Hamedi is supported by KSR exemplary rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding claim 16, it is rejected under the same analysis as claim 1 above.
Regarding claim 17, it is rejected under the same analysis as claim 1 above along with Noguchi’s teaching of a non-transitory computer readable medium (Noguchi Claim 17: “A non-transitory recording medium storing a computer-readable program”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the non-transitory computer readable medium as taught by Noguchi with the method of Makhzani because it would allow the method to be run on any computer. This motivation for the combination of Makhzani and Noguchi is supported by KSR exemplary rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding claim 3, Makhzani discloses the apparatus, wherein the processing circuitry is configured to classify the images into a plurality of clusters (Makhzani Page 10: “Figure 9 illustrates the unsupervised clustering performance of the AAE on MNIST when the number of clusters is 16. Each row corresponds to one cluster. The first image in each row shows the cluster heads, which are digits generated by fixing the style variable to zero and setting the label variable to one of the 16 one-hot vectors”), based on the first features, to select a plurality of representative images that represent the clusters (Makhzani Page 10: “Figure 9 illustrates the unsupervised clustering performance of the AAE on MNIST when the number of clusters is 16. Each row corresponds to one cluster. The first image in each row shows the cluster heads, which are digits generated by fixing the style variable to zero and setting the label variable to one of the 16 one-hot vectors”).
The Makhzani and Noguchi combination does not explicitly disclose the apparatus, configured to select, as the images of interest, representative images that are selected via an input device from among the representative images.
However, Hamedi teaches the apparatus, configured to select, as the images of interest, representative images that are selected via an input device from among the representative images (Hamedi [0101]: “In some implementations, the candidate image can be provided by a user. For example, the GUI module 130 may be configured to provide a graphical user interface to be displayed on a computing device, such as the client computing platform(s) 110. The graphical user interface may display a plurality of indications, such as thumbnails or titles, corresponding to a set of candidate images. For example, the set of candidate images can be any or all of the images stored in a photo gallery application on the computing device. In some implementations, the GUI module 130 may be configured to receive a user selection of a first indication or the plurality of indications corresponding to the selected candidate image whose degree of similarity to the training images is of interest”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate having the user select the images of interest as taught by Hamedi with the apparatus of Makhzani and Noguchi because it allowing the user to select which images they are interested in would allow them to find relevant similar images more quickly in contrast to the system deciding images of interest without user input and the user having to sort through the auxiliary image groups. This motivation for the combination of Makhzani, Noguchi, and Hamedi is supported by KSR exemplary rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding claim 4, the Makhzani and Noguchi combination does not explicitly disclose the apparatus, wherein the processing circuitry is configured to cause a display device to display the representative images in such a manner that the representative images are selectable via the input device.
However, Hamedi teaches the apparatus, wherein the processing circuitry is configured to cause a display device to display the representative images in such a manner that the representative images are selectable via the input device (Hamedi [0101]: “In some implementations, the candidate image can be provided by a user. For example, the GUI module 130 may be configured to provide a graphical user interface to be displayed on a computing device, such as the client computing platform(s) 110. The graphical user interface may display a plurality of indications, such as thumbnails or titles, corresponding to a set of candidate images. For example, the set of candidate images can be any or all of the images stored in a photo gallery application on the computing device. In some implementations, the GUI module 130 may be configured to receive a user selection of a first indication or the plurality of indications corresponding to the selected candidate image whose degree of similarity to the training images is of interest”).
It would have been obvious to combine Hamedi with the apparatus of Makhzani and Noguchi for the same reasons as for claim 3 above.
Regarding claim 12, Makhzani discloses the apparatus, wherein the first model is trained in such a manner that a distance between mutually similar images in a latent space becomes greater than in the second model (Makhzani Fig. 8 description: “the top adversarial network imposes a Categorical distribution on the label representation and the bottom adversarial network imposes a Gaussian distribution on the style representation”).
Regarding claim 14, Makhzani does not explicitly disclose the apparatus, wherein the processing circuitry is configured to cause a display device to display the image of interest and the auxiliary image as the similar image set.
However, Noguchi teaches the apparatus, wherein the processing circuitry is configured to cause a display device to display the image of interest and the auxiliary image as the similar image set (Noguchi [0089]: “In a case where the centroid is determined, the image closest to the determined centroid (image distributed at the closest position) is determined by the CPU 2 as a representative image of the group having the centroid (step S37 in FIG. 5). In a case where the representative image is determined, the image classification result is displayed in the classification result window by using the determined representative image (step S38 in FIG. 5). This classification result window is displayed on the display screen of the display apparatus 3”; Noguchi [0091]: “In the classification result display window 80, representative images IR1 to IR10 of each group classified into 10 groups are displayed under the control of the CPU 2 (a representative image display control device). For example, each of the representative images IR1 to IR10 is an image representing each of the groups G1 to G10 shown in FIG. 12. Each of the representative images IR1 to IR10 is the image closest to the centroid C1 to the centroid C10 of each of the groups G1 to G10 shown in FIG. 12”; Noguchi [0093]: “In a case where any one of the representative image IR1 to the representative image IR10 displayed in the classification result display window 80 is double-clicked (any device other than the mouse 9 may be used as long as it can select an image, such as touching the touch panel display) by the mouse 9 (a representative image selection device) (YES in step S39 of FIG. 5), the double-clicked representative image is selected. Images included in the group represented by the selected representative image (images similar to each other) are displayed in a list in the second classification window 70A (step S40 in FIG. 5)”).
It would have been obvious to combine Makhzani and Noguchi for the same reasons as stated for claim 1 above.
Regarding claim 15, Makhzani does not explicitly disclose the apparatus, wherein the processing circuitry is configured to create a limit sample, based on the image of interest and the auxiliary image.
However, Noguchi teaches the apparatus, wherein the processing circuitry is configured to create a limit sample, based on the image of interest and the auxiliary image (Noguchi [0097]: “In a case where the second designation classification button B11 is pressed (YES in step S41 of FIG. 5), a classification command is generated and input to the CPU 2. The image displayed in the second classification window 70A becomes an image to be newly classified, and the CPU 2 (an image classification device) performs the first stage classification that classifies the images into a plurality of groups for each similar image (step S26 in FIG. 3). The similar images displayed in the second classification window 70A are further classified into groups having the number of groups input in the group number input window 73 for each more similar image”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the limit sample as taught by Noguchi with the method of Makhzani because it would create an even more precise sample for a user to look at and act on. This motivation for the combination of Makhzani and Noguchi is supported by KSR exemplary rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over the Makhzani, Noguchi, and Hamedi combination in view of Tao et al. (U.S. Patent Publication No 2021/0248246 listed in the IDS received 02/28/2024, hereinafter “Tao”).
Regarding claim 13, the Makhzani, Noguchi, and Hamedi combination does not explicitly disclose the apparatus, wherein the first model and the second model are generated by unsupervised representation learning and are trained based on mutually different loss functions.
However, Tao teaches the apparatus, wherein the first model and the second model are generated by unsupervised representation learning and are trained based on mutually different loss functions (Tao [0076]: “The update unit 20G receives the first loss for the plurality of first target data XA from the first loss calculation unit 20E. The update unit 20G receives the second loss for the element classes G from the second loss calculation unit 20F. The update unit 20G updates the parameters such that both of the received first loss and the received second loss become lower. Specifically, the update unit 20G updates the parameters such that the received first loss and the received second loss are lower than the first loss and the second loss calculated by the above-mentioned processing by using the learning model 30 the parameters of which are currently stored in the storage unit 20A”; Tao [0077]: “Specifically, the update unit 20G calculates the parameters such that a loss function expressed by Equation (5) below becomes lower”; Tao [0078]: “In Equation (5), L represents the loss function. L_1 represents the first loss. L_2 represents the second loss. a is a weighting of the second loss, and is a positive real number. a only needs to be determined in advance depending on the type of target data X. In the present embodiment, the case where a=1 is described as an example”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the loss functions as taught by Tao with the apparatus of Makhzani, Noguchi, and Hamedi because it would improve the accuracy of the method by minimizing loss and it would allow for the refining of each model for its specific purpose. This motivation for the combination of Makhzani, Noguchi, Hamedi, and Tao is supported by KSR exemplary rationale (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Allowable Subject Matter
Claims 5-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Amendment
The amendment filed 6/22/2026 incorporated the subject matter of original claim 2 into independent claim 1 and removed original subject matter from claim 1. Therefore, Applicant has changed the scope the claim objected to in the Non-Final Rejection mailed 3/25/2026. As noted in the 112(b) rejection of the claims in the Non-Final, it was contradictory for the invention to select both “an image” and “two or more images” at the same time. As such, Examiner was unable to apply art which selected “an image” and then “two or more images”. Now that the claims have been amended to only include the selection of multiple images, the scope is different and Examiner is able to apply prior art for a Final Rejection. Therefore, this Final Rejection is proper as Examiner is issuing a new grounds of rejection necessitated by the amendment. See MPEP 1207.03.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AIDAN KEUP whose telephone number is (703)756-4578. The examiner can normally be reached Monday - Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN KEUP/ Examiner, Art Unit 2666
/Molly Wilburn/Primary Examiner, Art Unit 2666
Prosecution Insights