DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot based on new grounds of rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Atsmon, publication number: US 2024/0403717 in view of Bu, publication number: US 2024/0290027.
As per claim 1, Atsmon teaches a method comprising:
generating a simulated object based at least on extracting at least a portion of the simulated object from a simulated textured representation (simulated objects, [0020][0087]);
combining the simulated object with an existing image to generate a training image (Inserting, [0045], objects, [0065]); and
updating one or more parameters of a machine learning model based at least on the training image and ground truth data corresponding to the training image (training model with generated data and ground truth, [0063-0064][0093], Data describing new scenes, [0065]).
Atsmon does not teach generating, as a simulated textured representation, a synthetically generated image having a randomly generated texture corresponding thereto,
and a shape of a first portion being determined based at least on a number of randomly distributed points or a convex polygon in which a particular shape of the convex polygon is generated at random.
In an analogous art, Bu teaches teach generating, as a simulated textured representation, a synthetically generated image having a randomly generated texture corresponding thereto (object with randomly generated texture, [0021][0045-0046]),
and a shape of a first portion being determined based at least on a number of randomly distributed points or a convex polygon in which a particular shape of the convex polygon is generated at random (object with randomly generated dimensions [0021][0045][0047][0069]).
Therefore, it would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention to modify Atsmon’s training system to include randomly generated training data as described in Bu’s machine learning system for the advantage of having a broader training base for the system to improve object detection.
As per claim 2, the combination teaches wherein the simulated object comprises at least the first portion combined with a second portion (Atsmon: Inserting, [0045], objects [0065]).
As per claims 3 and 13, the combination teaches wherein the portion of the simulated object is retrieved from a portion of the simulated textured representation based at least on one of a random number of randomly distributed points or a convex polygon (Atsmon: randomly selected simulation objects with shapes and sizes, [0020][0087]).
As per claim 4, the combination teaches wherein:
the simulated object comprises the first portion that is randomly rotated and randomly translated and a second portion that is randomly rotated and randomly translated; and
the first portion is joined with the second portion based at least on joining a first randomly selected pixel from the first portion with a second randomly selected pixel from the second portion (Atsmon: Placing rotated objects in different locations, [0045][0067]).
As per claim 5, the combination teaches wherein the existing image includes an object mask associated therewith that directs one or more object locations within the existing image that the simulated object may be located (Atsmon: Placing objects based on outline,[0076]).
As per claim 6, the combination teaches wherein the existing image includes a freespace representation associated therewith that directs one or more object locations within the existing image that the simulated object may be located (Atsmon: Placing objects based on outline,[0076] and ground truth, [0064]).
As per claim 7, the combination teaches further comprising, in response to the training image being generated, generating a bounding shape around the simulated object to be included in the ground truth data (Atsmon: Identified size and shape, [0020][0075]).
As per claims 8 and 14, the combination teaches wherein:
a size of the simulated object comprises an upper threshold and a lower threshold; and
the size, the upper threshold, and the lower threshold are determined based at least on one or more characteristics of an environment as depicted in the existing image (Atsmon: Identified size and shape, [0020][0075]).
As per claims 9 and 15, the combination teaches wherein the size of the simulated object is determined using heuristics based on at least one of a focal distance and a relative size of at least one existing object in the environment (Atsmon: Size and shape relative to base distance, [0075]).
As per claims 10 and 18, the combination teaches further comprising: segmenting the training image; and
labelling a portion of the segmented training image that includes the simulated object as the anomaly object (not adhering to contraints, [0063]),
wherein a resulting label from the labelling is included in the ground truth data (Atsmon: Ground truths and data describing new scenes, [0064-0065]).
As per claim 11, the combination teaches wherein the simulated object is automatically combined with the existing image to generate the training image (Atsmon: Generating synthetic training data, [0020][0057]).
As per claim 12, Atsmon teaches a system comprising:
one or more processing units to:
generate one or more simulated objects from one or more textured images based at least on one or more randomly generated shapes (simulated objects, [0020][0087], placing objects based on outline [0076]); and
updating one or more parameters of a machine learning model using the one or more simulated objects and ground truth data corresponding to the one or more simulated objects (training model with generated data and ground truth, [0063-0064][0093], Data describing new scenes, [0065]).
Atsmon does not teach the generated shapes extracted from the one or more textured images such that the one or more simulated objects are formed as abstracts objects lacking correspondence to any specific real-world objects.
In an analogous art, Bu teaches the generated shapes extracted from the one or more textured images such that the one or more simulated objects are formed as abstracts objects lacking correspondence to any specific real-world objects (generating objects with random textures and shapes, [0021][0045-0047]).
Therefore, it would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention to modify Atsmon’s training system to include randomly generated training data as described in Bu’s machine learning system for the advantage of having a broader training base for the system to improve object detection.
As per claims 16 and 20, the combination teaches wherein the computing system comprises one or more of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting at least one virtual reality content, augmented reality content, or mixed reality content;
a system implemented using a robot;
a system for performing conversational AI operations; a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs) (Atsmon: Autonomous system, [0098]);
As per claim 17, Atsmon teaches a processor One or more processors comprising processing circuitry to perform operations comprising
generating a simulated anomaly object lacking correspondence to any specific real- world object based at least on one or more randomly generated shapes (simulated objects, [0020][0087]);
combining the simulated anomaly object with an existing image of an operating environment to form a training image (Inserting, [0045], objects, [0065]); and
providing the training image to a machine learning system to train the machine learning system to detect an anomaly object in an operating environment based at least on the simulated anomaly object (training model with generated data and ground truth, [0063-0064][0093], Data describing new scenes, [0065]).
Atsmon does not teach generating anomaly objects lacking correspondence to any specific real world object.
In an analogous art, Bu teaches generating anomaly objects lacking correspondence to any specific real world object. (generating objects with random textures and shapes, [0021][0045-0047][0069]).
Therefore, it would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention to modify Atsmon’s training system to include randomly generated training data as described in Bu’s machine learning system for the advantage of having a broader training base for the system to improve object detection.
As per claim 19, the combination teaches further comprising, in response to the forming of the training image, generating a bounding object around the simulated anomaly object, and associating the bounding shape with the training image (Atsmon: Generating synthetic training data, [0020][0057]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUGBENGA O IDOWU whose telephone number is (571)270-1450. The examiner can normally be reached Monday-Friday 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jung Kim can be reached at 5712723804. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUGBENGA O IDOWU/Primary Examiner, Art Unit 2494
Prosecution Insights