DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
This office actions is in response to application filed on 11/20/2024.
Limitations appearing inside of {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Claims 1-12 are pending in the application.
Information Disclosure Statement
Information Disclosure Statement (IDS) filed on 11/20/2024 has been considered.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 5, 7-8, 11 and 12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Hinterstoisser” (Hinterstoisser et al., US Patent Application Publication No. US 20210327127 A1), as provided in the IDS.
1. Hinterstoisser teaches a non-transitory computer-readable storage medium storing a set of program instructions for a computer, the set of program instructions, when executed by the computer, causing the computer to:
acquire object image data (each foreground layer can include rendering(s) of foreground 3D object model(s); Hinterstoisser, ¶[0038]) and a plurality of source image data (The background layers can be generated from a corpus/dataset of textured background 3D object models; Hinterstoisser, ¶[0033]), the object image data indicating an object image including an object,
each of the plurality of source image data indicating a source image not including the object, the plurality of source image data indicating respective ones of a plurality of source images (the corpus of background 3D object models are disjoint from the corpus of foreground 3D object models. Put another way, none of the background 3D object models can be included amongst the foreground 3D object models, Hinterstoisser, ¶[0033]);
perform a first combining process by using the plurality of source image data to generate background image data indicating a background image, the first combining process including combining at least some of the plurality of source images (the background engine 112 generates a background layer by successively selecting regions in the background where no other background 3D object model has been rendered (a “bare region”), and rendering, onto each selected region, a random background 3D object model; Hinterstoisser, ¶[0048]);
perform a second combining process by using the object image data and the background image data to generate input image data indicating an input image, the second combining process including combining the background image and the object image where the background image is background and the object image is foreground (generating synthetic images based on fusing a corresponding foreground layer background layer, and optionally occlusion layer; and generating training instances that include the synthetic images; Hinterstoisser, ¶[0066]); and
perform a particular process by using the input image data and a machine learning model, the particular process including inputting the input image data into the machine learning model and generating output data (training instances for initially training a machine learning model; Hinterstoisser, ¶[0068]).
2. Hinterstoisser teaches the non-transitory computer-readable storage medium according to claim 1. Hinterstoisser further teaches wherein the generating the background image data includes: selecting N use image data from among M source image data, where N is an integer satisfying 2≤N≤M and M is an integer of 3 or more (the corpus of background 3D objects includes (e.g., is restricted to) objects that are specific to an environment for which synthetic images are being generated. For example, if the synthetic images are being generated for a home environment, typical household objects can be included in the background 3D objects of the corpus; Hinterstoisser, ¶[0060]. The number of objects specific to an environment (i.e., N is more than 2) is less than the number of images in the corpus( i.e., M), meeting the claim limitation.);
performing the first combining process by using the selected N use image data to generate the background image data (block 206, the system renders, in the background layer at a corresponding location, the background 3D object; Hinterstoisser, [0061]; block 204, where it randomly selects an additional background 3D object. The system then proceeds to block 206 and renders the additional background 3D object, in the background layer at the next location of block 210; Hinterstoisser, ¶[0063]); and
performing a repetition process of repeating the first combining process while changing a combination of the N use image data to generate a plurality of background image data (block 214, the system determines whether to generate an additional background layer. If so, the system proceeds back to block 202 … and performs multiple iterations of blocks 204, 206, 208, and 210 in generating an additional background layer. Hinterstoisser, ¶[0065]); and
wherein the generating the input image data includes generating a plurality of input image data by using the object image data and the plurality of background image data (a foreground 3D object model in a rendering …utilized in generating the corresponding background layer; for a foreground 3D object model it will be rendered in multiple different foreground layers (once in each), and each rendering will include the corresponding object; Hinterstoisser, ¶[0038]).
5. Hinterstoisser teaches the non-transitory computer-readable storage medium according to claim 1. Hinterstoisser further teaches wherein the first combining process includes generating the background image data indicating the background image in which a plurality of source images are arranged (See Hinterstoisser, Fig 5B exhibits source images 526 and 514 arranged in the background layer); and
wherein the second combining process includes combining the object image with the background image such that the object image is located on a boundary of the plurality of source images arranged in the background image (he background engine 112 generates a background layer by successively selecting regions in the background where no other background 3D object model has been rendered (a “bare region”), and rendering, onto each selected region, a random background 3D object model; Hinterstoisser, ¶[0048]).
7. Hinterstoisser teaches the non-transitory computer-readable storage medium according to claim 1. Hinterstoisser further teaches wherein the particular process is a training process of training the machine learning model by using a plurality of input image data (create purely synthetic training data for training a machine learning model, such as an object detection machine learning model; Hinterstoisser, ¶[0045]).
8. Hinterstoisser teaches the non-transitory computer-readable storage medium according to claim 7. Hinterstoisser further teaches wherein the machine learning model is an object detection model configured to detect a region at which an object is located in an image (object detection machine learning model; Hinterstoisser, ¶[0045]; the training engine compares predictions to labels; Hinterstoisser, ¶[0054], where labels can be locations of foreground objects; Hinterstoisser, ¶[0055]);
wherein the set of program instructions, when executed by the computer, causes the computer to further perform (Storage subsystem 724 stores programming and data constructs that provide the functionality of some or all of the modules described herein; Hinterstoisser, ¶[0107]):
generating region information indicating a region at which the object is located in the input image, based on a position at which the object image is arranged in the background image in the second combining process (The label engine 120 can determine the labels from, for example, the foreground engine 114 as the foreground engine 114 determines 3D objects and their rotations and locations in generating the foreground layer. Hinterstoisser, [0054]); and
wherein the training process is performed by using the plurality of input image data and a plurality of region information corresponding to the plurality of input image data (The label engine 120 provides each pair of a synthetic image and corresponding label(s) as a training instance for storage in training instances database 156. Hinterstoisser, [0054]; the training engine 130 can process the synthetic images of the training instances to generate predictions, compare those predictions to labels of the corresponding training instances to determine errors, and update weights of the machine learning model 165 based on the errors; Hinterstoisser, ¶[0055]).
Claim 11 and claim 12 are similarly analyzed as analogous claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Hinterstoisser in view of “Krueger” (Krueger US Patent Application Publication No. US 20140281915 A1).
6. Hinterstoisser teaches the non-transitory computer-readable storage medium according to claim 1. Hinterstoisser further teaches wherein the first combining process includes generating the background image data indicating the background image in which a plurality of source images are arranged (the background engine 112 generates a background layer by successively selecting regions in the background where no other background 3D object model has been rendered (a “bare region”), and rendering, onto each selected region, a random background 3D object model; Hinterstoisser, ¶[0048]);
wherein the generating the background image data includes generating a plurality of background image data including first background image data indicating a first background image and second background image data indicating a second background image, the first background image including a first size-adjusted source image, the second background image including a second size-adjusted source image (portion 500A1 illustrates some of the background objects 511, 512, and 513. As can be ascertained by viewing background objects 511, 512, and 513, they are of a similar size/scale relative to one another, and relative to object 501; Hinterstoisser, ¶[0089]); and
wherein the first size-adjusted source image and the second size-adjusted source image are generated based on a same source image, {the first size-adjusted source image and the second size-adjusted source image having different aspect ratios} (As described herein, the foreground layers with rendered objects at corresponding size(s) can be fused with background layers, having background object sizes that are based on the size(s), in generating synthetic images. After processing all foreground objects at all rotations for a given size/scale level, the process can be repeated for the next (smaller) scale level; Hinterstoisser, ¶[0041]. Hinterstoisser teaches first and second sizes, where in the next synthetic image, the background images may be similar to the second size: “At block 406, the system selects, upon the completion of block 404, additional training instances with synthetic images having foreground object(s) at smaller size(s),…, foreground objects that are of a second size or scale, or within a threshold percentage (e.g., 10%) of the second size or scale, …, background objects of the synthetic images of the training instances can be of a similar size as the foreground objects”; Hinterstoisser ¶[0084])
Hinterstoisser does not explicitly disclose the first size-adjusted source image and the second size-adjusted source image having different aspect ratios.
However, Krueger, a similar field of endeavor of generation of backgrounds for images, teaches the first size-adjusted source image and the second size-adjusted source image having different aspect ratios (the aspect ratio of the corner images are not fixed and can vary based on the adjustment of the aspect ratio of the background image; Krueger, ¶[0015]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include varying the aspect ratios as taught by Krueger to the invention of Hinterstoisser. The motivation to do so would be to generate backgrounds dynamically that are and vertically and horizontally tileable.
Allowable Subject Matter
Claims 3, 4, 9, and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ye et al., US Patent Application Publication No. US 20150206169 A1, teaches automatically generating images, and would have been relied upon for teaching a flexible layout, see at least Fig. 11B.
O’Donnavan, US Patent Application Publication No. US 20180276182 A1, teaches responsive grid layouts for graphic design, and would have been relied upon for teaching vertical and horizontal divisions, see at least Fig. 5.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANDHANA PEDAPATI whose telephone number is 571-272-5325. The examiner can normally be reached M-F 8:30am-6pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANDHANA PEDAPATI/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights