DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In the Reply filed 07/16/2026, Applicant has amended claims 1, 11, and 12 and argues that this/those new limitation(s) was/were not taught with the reference(s) cited in the previous action dated 01/16/2026. However, the Examiner respectfully disagrees for the reasons laid out below.
Response to Arguments
Applicant's arguments filed 07/16/2026 regarding the previous 112(b) rejections have been fully considered but they are not persuasive.
Starting on page 6 of the Remarks, Applicant argues that the previous 112(b) rejections against the independent claims regarding “determining the object of the second class…depending on a label for the first class and/or depending on a least one pixel representing an object of the first class”. Applicant states that for unknown or novel objects i.e. objects of a second class, labels of the first class (i.e. known objects) would be used as a “use-what-is-available” approach and therefore that portion of the claims is clear. However, this is not persuasive because nowhere in the originally filed application does it state that known labels/labels of the first class are used in such a way. In fact, paragraph [0091] of the published application appears to contradict this by reciting “The method may comprise adding a new label identifying the class for novel or unknown objects to the bounding box comprising the novel object”. Furthermore, it is unclear how a system using an image with a novel object but labeling it incorrectly with a label of a known object would be beneficial in computer vision systems. For example, if a new, unknown type of road sign were augmented into an image, what would be the benefit of labeling it as a previously known object such as a car or traffic light? It appears this would actually create false positives for a system being trained with such images. Therefore, the 112(b) rejections regarding “determining the object of the second class” are maintained.
Applicant's arguments filed 07/16/2026 regarding the previous 112(b) rejections have been fully considered but they are not persuasive.
On page 7 of the Remarks, Applicant argues that the amended portions of the claims overcome the previous rejection under Sabatini because the reference allegedly does not use classes of objects in connection with “rare objects” and “common objects”. This is not persuasive because paragraph [0004] of Sabatini specifically recites “Deep learning vision-based perception systems used for object recognition are trained by a huge amount of training data and prove to recognize reliably objects belonging to classes of objects including a large number of the objects that are to be recognized. However, deep learning vision-based perception systems of the art tend to fail recognizing objects for which only a few examples are present in the training data sets (rare objects) and cannot recognize unknown (untrained) objects” (emphasis added) showing that their classifier is acting on classes when analyzing known vs rare or unknown objects. Paragraph [0022] also contemplates “unknown objects” being on the road.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding independent claims 1, 11, and 12, each in effect recite
“determining the object of the second class with a generative model depending on a label for the first class”. (emphasis added)
It is unclear how generating the object of the second class would depend on a label for the first class. The plain wording does not seem to make sense as it would be contradictory to use a label from a different class when generating an object. When looking to the Specification, both paragraphs [0040]-[0041] talk about a label for a pasted object in an image with a class of novel or unknown, and paragraph [0045] makes it clear that “known objects” are the first class and “novel or unknown objects” are the second class. Therefore, these paragraphs do not mention a label of the first class. Paragraph [0043] then gives a situation where no label is present, which does not fit the claim. Paragraph [0052] talks broadly about a label of the first class and paragraph [0059] merely repeats the claim language. The rest of the paragraphs describing labels continue to talk about labels for “novel or unknown objects” which would be a label for the second class.
Therefore, one of ordinary skill in the art would find the meets and bounds of “determining the object of the second class with a generative model depending on a label for the first class” to be unclear.
Dependent claims 2-10 do not cure independent claim 1 of these issues, and are similarly rejected.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 2, 5, 11, and 12 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Sabatini et al, U.S. Publication No. 2025/0157174.
Regarding claim 11, Sabatini teaches a device configured to determine a class for at least a part of a digital image (see Sabatini Abstract), the device comprising:
at least one processor; and at least one storage; wherein the storage is adapted to store instructions for determining the class for the at least a part of a digital image, the instructions, when executed by the at lease one processor, causing the at least one processor to perform the following steps (see paragraph [0032]):
providing a classifier (see Figure 1, first neural network 132 and paragraph [0042], “The first neural network 132 may comprise a neural network operating as a classifier”) for a first class and a second class (see Figure 2, step 230, wherein the first class would be recognized objects and the second class would be initially unrecognized objects, which is in line with the classes used in present application as explained above),
generating a digital image including an object of the second class in at least the part of the digital image (see Figure 7, synthetic image with “rare object” and paragraph [0051], “In a rare object retrieval procedure 320, a rare object (for example, a rare vehicle) that cannot be recognized is automatically determined (detected) and a 3D bounding box corresponding to/containing the object is automatically extracted from the Lidar point clouds” indicating that rare and unrecognized are synonymous), and
determining the class for at least the part of the digital image with the classifier (see Figure 2, step 260), wherein the generating of the digital image includes determining the object of the second class (see paragraph [0064]) with a generative model (see Figure 2, second neural network 139 and paragraph [0044]) depending on a label for the first class and/or depending on at least one pixel representing an object of the first class (see Figure 7, wherein “common object” in the real image is replaced with “rare object” in the synthesized image and paragraph [0063]),
wherein the classifier is configured to use the first class for known objects and the second class for novel or unknown objects (see Figure 2, step 230, wherein the first class would be recognized objects and the second class would be initially unrecognized objects), wherein the classifier is configured for known objects including vehicles, pedestrians, traffic signs, or pavement, and wherein the classifier is configured to classify other road features in the second class (see paragraph [0004] which talks about object classes which can be included as “rare” and “unknown” as noted above and paragraph [0022] which refers to unknown objects lying in the road).
Independent claims 1 and 12 recite similar limitations as claim 11, and are rejected under similar rationale.
Regarding claim 2, Sabatini teaches all the limitations of claim 1, and further teaches training the classifier to determine the second class for the object of the second class in the digital image (see Sabatini Figure 2, step 260).
Regarding claim 5, Sabatini teaches all the limitations of claim 1, and further teaches wherein the generating of the digital image includes replacing or modifying a part of the digital image to form at least the part of the digital image (see Sabatini Figure 7, wherein “common object” in the real image is replaced with “rare object” in the synthesized image and paragraph [0063]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 4, 8-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sabatini et al, U.S. Publication No. 2025/0157174 in view of Erignac, U.S. Patent No. 9,547,805.
Regarding claim 3, Sabatini teaches all the limitations of claim 1, but does not expressively teach training the classifier to determine the second class for a pixel in the digital image forming at least a part of the object of the second class.
However, Erignac in a similar invention in the same field of endeavor teaches a method involving a classifier (see Erignac Figure 2, classifier 240) configured to determine a second class of an object in a digital image (see Figure 3, step 310 and column 6, “Classifier 240 may score each pixel with a roadness score that represents a likelihood that the pixel is a part of a road”) as taught in Sabatini comprising
training the classifier to determine the second class for a pixel in the digital image forming at least a part of the object of the second class (see Figure 3, step 350 and column 6, “After classifier 240 has been trained, classifier 240 is configured to process one or more images and identify roads in the one or more images. Classifier 240 may score each pixel with a roadness score that represents a likelihood that the pixel is a part of a road. Alternatively, or additionally, classifier 240 may label each pixel as a road or some other object”).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of training a classifier to label pixels as taught in Erignac with the method taught in Sabatini, the motivation being to utilize the granularity of individual pixel labeling for more accurate further analysis of a labeled image.
Regarding claim 4, Sabatini teaches all the limitations of claim 1, but does not expressively teach training the classifier to determine the second class for a bounding box in the digital image including the object of the second class.
However, Erignac in a similar invention in the same field of endeavor teaches a method involving a classifier (see Erignac Figure 2, classifier 240) configured to determine a second class of an object in a digital image (see Figure 3, step 310 and column 6, “Classifier 240 may score each pixel with a roadness score that represents a likelihood that the pixel is a part of a road”) as taught in Sabatini comprising
training the classifier to determine the second class for a bounding box in the digital image including the object of the second class (see column 6, “After classifier 240 has been trained, classifier 240 is configured to process one or more images and identify roads in the one or more images… Alternatively, or additionally, classifier 240 may output pixel coordinates, bounding boxes, or any other reference to the location of pixels determined to be a road”).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of training a classifier to output bounding boxes as taught in Erignac with the method taught in Sabatini, the motivation being to allow humans to easily visualize the determined object via such boxes.
Regarding claim 8¸ Sabatini in view of Erignac teaches all the limitations of claim 4, and further teaches
(i) generating at least the part of the digital image with the generative model depending on a mask, or
(ii) generating at least the part of the digital image with the generative model depending on the bounding box (see Sabatini paragraph [0059]).
Regarding claim 10¸ Sabatini in view of Erignac teaches all the limitations of claim 8, and further teaches training the generative model to synthesize at least the part of the digital image depending on the mask or the bounding box (see Sabatini paragraphs [0059]-[0060]).
Regarding claim 8¸ Sabatini in view of Erignac teaches all the limitations of claim 4, and but in the above embodiment does not expressively teach
generating at least the part of the digital image with the generative model depending on a d mask.
However, as noted above, Sabatini does teach (ii) generating at least the part of the digital image with the generative model depending on a bounding box (see Sabatini paragraph [0059]).
Furthermore, Erignac goes on teach using a mask for outlining an object as an alternative to a bounding box (see Erignac column 6, “Classifier 240 may output a mask showing pixels having a roadness score above a pre-determined threshold and/or pixels labeled as a road. Alternatively, or additionally, classifier 240 may output pixel coordinates, bounding boxes, or any other reference to the location of pixels determined to be a road”).
Therefore, one of ordinary skill in the art before the effective filing date of the invention would have found it obvious as a matter of simple substitution to replace the bounding box of Sabatini with a mask as taught in Erignac to yield the predictable results of successfully switching a known object with a rare object in an image.
Regarding claim 9¸ Sabatini in view of Erignac teaches all the limitations of claim 8, and further teaches determining the mask identifying the pixels of at least the part of the digital image with the generative model (see Sabatini paragraphs [0059]-[0060] as combined with Erignac column 6, “Classifier 240 may output a mask showing pixels having a roadness score above a pre-determined threshold and/or pixels labeled as a road. Alternatively, or additionally, classifier 240 may output pixel coordinates, bounding boxes, or any other reference to the location of pixels determined to be a road”).
Claim(s) 6 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sabatini et al, U.S. Publication No. 2025/0157174 in view of Sainburg et al, “Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourages convex latent distributions” (published at https://arxiv.org/abs/1807.06650, April 2019).
Regarding claim 6¸Sabatini teaches all the limitations of claim 1, but does not expressively teach generating the part of the digital image with the generative model, from random noise or with latent space interpolation.
However, Sainburg in a similar invention in the same field of endeavor teaches a method involving a generative model (see Sainburg Figure 1, “Generator”) configured to generate part of a digital image (see Figure 1, middle image after “Generator” which has replaced the woman’s face with the man’s face) as taught in Sabatini comprising
generating the part of the digital image with the generative model, from random noise or with latent space interpolation (see section 1, final paragraph, “We propose a novel AE that hybridizes features of an AE and a GAN. Our network is trained explicitly to control for the structure of latent representations and promotes convexity in latent space by adversarially constraining interpolations between data samples in latent space to produce realistic samples”).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious as a matter of simple substitution to replace the method of determining the part of the digital image taught in Sabatini with that of Sainburg to yield the predictable results of successfully generating a synthesized image.
Regarding claim 7¸ Sabatini in view of Sainburg teaches all the limitations of claim 6, and further teaches training the generative model to synthesize at least the part of the digital image, from random noise or with latent space interpolation (see Sainburg section 2, final paragraph, “We also train the network on interpolations in the generator, to explicitly train the generator to produce interpolations (Gd(zint)) which deceive the discriminator and are closer to the distribution in X than interpolations from an unconstrained AE”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CASEY L KRETZER whose telephone number is (571)272-5639. The examiner can normally be reached M-F 10:00-7:00 PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Payne can be reached at (571)272-3024. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CASEY L KRETZER/Primary Examiner, Art Unit 2635
Prosecution Insights