Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a non-Final office action in merits. Claims 1-20, as originally filed, are presently pending and have been elected and considered below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/6/2023 and 4/11/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 USC § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. The claims 1-20 are directed to abstract idea.
Although claim 1 is directed to a statutory machine, it recites abstract mathematical concepts (neural network-based modification of bounding regions and label updates) without integrating them into a practical application, and without additional elements that amount to significantly more than the abstract idea. “update the one or more labels based … on the modified bounding regions,” which is a logical rule for revising annotations and, absent specific technical constraints, characterizes an outcome that could be decided mentally. There is no transformation and the operations concern image/label data only (modifying bounding regions, updating labels), which is insignificant data manipulation/extra-solution activity.
Similarly, claim 12 recites “update a model …; modify bounding regions … obtain scores …; minimize loss …” which is rejected with the same reason as set forth in claim 1.
Claim 20 recites a computer-implemented method performing similar features as claim 1 and/or claim 12 with abstract mathematical concepts without significantly more and without integrating them into a practical application. It is rejected with the same reason.
Dependent claims 2-11 do not add more meaningful limitations to claim 1, thus are rejected with the same reason.
Dependent claims 13-19 do not add more meaningful limitations to claim 12, thus are rejected with the same reason.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 9-11 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US 2020/0005468 A1, Paul et al. (hereinafter Paul).
As to claim 1, Paul discloses a processor comprising: one or more circuits to:
modify, using a neural network and based at least on images from a first dataset (pars 0031, 0034-0036, 0047), one or more bounding regions of one or more labels corresponding to one or more images from the first dataset to obtain one or more modified bounding regions (Figs 2, 4; pars 0031, 0059, 0065, 0095, 0108, 0123, bounding boxes/regions being generated, refined, and/or modified by the object bounds generator or Region of Interest Generation Unit); and update the one or more labels based at least on the modified bounding regions (Fig 4; pars 0059-0060, 0108, 0123, Label updating logic in the RoI generation unit updates the label of the region accordingly).
As to claim 2, Paul discloses the processor of claim 1, wherein at least one label of the one or more labels comprises a bounding region surrounding one or more objects depicted in at least one image of the first dataset (Figs 2-4; pars 0031, 0035-0037, 0047).
As to claim 3, Paul discloses the processor of claim 2, wherein the one or more circuits are to modify the bounding region based on changes in pixels proximate to the bounding region between at least two different images of the first dataset (Fig 3, 6A; pars 0042, 0044-0045, 0055, 0069).
As to claim 9, Paul discloses the processor of claim 1, wherein at least one label of the one or more labels comprises a semantic or panoptic characterization of one or more objects in the bounding region (pars 0051, 0111).
As to claim 10, Paul discloses the processor of claim 1, wherein at least one bounding region of the one or more bounding regions includes an identifiable object in a corresponding first image of the first dataset (Figs 2-4; pars 0031, 0035-0036, 0047-0050).
As to claim 11, Paul discloses the processor of claim 1, wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine (pars 0030, 0165, automatic operation or system); a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations (Fig 2; pars 0031-0032, a deep learning system/operations); a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 4-8, 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Paul in view of US 2022/0084204 A1, Li et al. (hereinafter Li).
As to claim 4, Paul discloses the processor of claim 1, wherein the neural network is updated by using a generator to modify the one or more bounding regions in the one or more images of the first dataset (see rejection in claim 1) but does not expressly disclose using a discriminator to obtain a first score for the one or more images of the first data set and the modified bounding regions; using a discriminator to obtain a second score for images of a second dataset and one or more bounding regions of the images of the second dataset; and minimizing, using at least one of the first score or the second score, a first loss of the generator and a second loss of the discriminator.
Li, in the same or similar field of endeavor, further teaches a GAN with a generator network and a discriminator network in which the generator may be used to modify the one or more bounding regions in the one or more images of the first dataset (pars 0072, 0083, 0089, 0092, 0098, generating and make regression for the region of synthetic images by a generator); using a discriminator to obtain a first score for the one or more images of the first data set and the modified bounding regions (Figs 6-7; pars 0078, 0104-0106, 0108-0110, determining a first and a second scores using a discriminator); using a discriminator to obtain a second score for images of a second dataset and one or more bounding regions of the images of the second dataset (Figs 6-7; pars 0078, 0104-0106, 0108-0110); and minimizing, using at least one of the first score or the second score, a first loss of the generator and a second loss of the discriminator (pars 0068, 0084-0086, 0093-0095, loss functions with gradient decent algorithm and the like aiming at minimizing the loss function).
Therefore, consider Paul and Li’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Li’s teachings in Paul’s processor to utilize a GAN with a generator and a discriminator for adjusting or modifying a box region as well as associated label corresponding object(s) in an image.
As to claim 5, Paul as modified discloses the processor of claim 4, wherein the first dataset comprises images depicting synthetically generated data (Li: Figs 3A-3B, 4, 6), and the second dataset comprises images of real objects or scenes (Li: Fig 7; pars 0075-0076, 0078-0079, 0103-0105).
As to claim 6, Paul as modified discloses the processor of claim 4, wherein the second loss comprises a function to minimize differences between the images of the first dataset and the images of the second dataset (see rejection in claim 4).
As to claim 7, Paul as modified discloses the processor of claim 1, wherein the one or more circuits are to: identify pixels proximate to the bounding regions of the one or more images of the first dataset (Paul: pars 0045, 0055); and modify, using the identified pixels, the bounding regions of the one or more images of the first dataset (Paul: pars 0104-0108, 0123-0124; Li: pars 0081, 0089).
As to claim 8, Paul as modified discloses the processor of claim 1, wherein the neural network is a generative adversarial network (GAN) (Li: (Li: Figs 3-4, 6-7).
As to claim 12, Paul discloses a processor comprising: one or more circuits to: update a model to modify labels of first images in a source dataset (Figs 6C, 9; pars 0036, 0048-0051, 0059, 0065, generate and update labels of images by training of a neural network), which includes: using a generator to modify bounding regions of the first images (Figs 2, 4; pars 0031, 0059, 0065, 0095, 0108, 0123, bounding boxes/regions being generated, refined, and/or modified by the object bounds generator or Region of Interest Generation Unit). Paul does not expressly disclose using a discriminator to obtain a first score for the first images and the modified bounding regions, and a second score for second images in a reference dataset and bounding regions of the second images; and minimizing a first loss of the generator and a second loss of the discriminator, using the first score and the second score.
Li, in the same or similar field of endeavor, further teaches using a discriminator to obtain a first score for the first images and the modified bounding regions, and a second score for second images in a reference dataset and bounding regions of the second images (Figs 6-7; pars 0075-0076, 0078-0079, 0104-0106, 0108-0110, 0115-0117, determining a first and a second scores using a discriminator; the reference dataset being the one with real images); and minimizing a first loss of the generator and a second loss of the discriminator, using the first score and the second score (pars 0068, 0084-0086, 0093-0095, loss functions with gradient decent algorithm and the like aiming at minimizing the loss function).
Therefore, consider Paul and Li’s teachings as a whole, it would have been obvious to one of skill in the art before the fling date of invention to incorporate Li’s teachings in Paul’s processor to utilize a GAN with a generator and a discriminator for adjusting or modifying a box region as well as associated label corresponding object(s) in an image.
As to claim 13, Paul as modified discloses the processor of claim 12, wherein the source dataset comprises synthetic data, and the reference dataset comprises real data (see citation/rejection in claim 12).
As to claim 14, Paul as modified discloses the processor of claim 12, wherein the second loss comprises a function to minimize differences between the first images and the second images (Li: pars 0068, 0081, 0083, 0085-0086, 0091, 0093, 0101, 0106, loss function representing the difference between synthetic image and input image and gradient decent algorithm being used to minimize/optimize the loss function).
As to claim 15, Paul as modified discloses the processor of claim 12, wherein the one or more circuits are to: identify pixels proximate to the bounding regions of the first images (Paul: Figs 6C, 8; pars 0045-0046, neighboring pixels surrounding the regions and associated labels being set and identified); and modify the bounding regions of the first images, using the identified pixels (Paul: Figs 6C, 8; pars 0045-0046, 0050, 0085, 0089, 0091, 0099).
As to claim 16, Paul as modified discloses the processor of claim 12, wherein the model is a generative adversarial network (GAN) model (Li: Figs 3-4, 7).
As to claim 17, Paul as modified discloses the processor of claim 12, wherein the one or more circuits are to use the updated model to modify the labels in the first images according to the modified bounding regions (Paul: Fig 4; pars 0059-0060, 0108, 0123, Label updating logic in the RoI generation unit updates the label of the region accordingly).
As to claim 18, Paul as modified discloses the processor of claim 17, wherein each label in the first images comprises a bounding region surrounding one or more objects (Paul: Figs 2-4, 6C; pars 0031, 0036, 0052, 0065), and wherein modifying the label in an image in the source dataset comprises modifying the bounding region (Paul” pars 0050, 0059, 0065, 0101, 0106, 0108, 0124).
As to claim 19, Paul as modified discloses the processor of claim 17, wherein modifying the bounding region comprises changing a shape or dimension of the bounding region (Paul: pars 0062, 0075, 0087, change of dimension).
As to claim 20, it recites similar limitations with some variation and/or broader scope of claim 12. Similarly, Paul as modified discloses a computer-implemented method comprising: using a generative adversarial network (GAN) model to modify bounding regions in synthetic images (Li: Figs 3A-3B, 4, 7; pars 0067, 0072-0077), the bounding regions corresponding to objects in the synthetic images, the GAN model comprising a generator and a discriminator, the generator is to generate modified bounding regions (Paul: Figs 2, 4; pars 0031, 0059, 0065, 0095, 0108, 0123; Li: Figs 6-7; pars 0075-0076, 0078-0079, 0104-0106, 0108-0110, 0115-0117), and the discriminator is to evaluate the modified bounding regions (Li: Figs 6-7; pars 0075-0076, 0078-0079, 0104-0106, 0108-0110, 0115-0117). Also see rejection and motivation statement in claim 12.
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUN SHEN whose telephone number is (571)270-7927. The examiner can normally be reached on Mon-Fri 8:30-5:50 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUN SHEN/
Primary Examiner, Art Unit 2662