DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/30/2026 has been entered.
Response to Arguments
Applicant arguments regarding claim rejections under 103 are considered, but are moot in view of new ground of rejections. However, Examiner would answer the following arguments.
Applicant argues:
Reference Li does not teach the amended feature, which is generating a layout overlaid on the input image based on the estimated position; wherein the layout indicates potential positions for a synthetic object to be added;
Examiner disagrees: Rong teaches these features. Rong teaches generating a potential position for an object to be added in the map. In order to do that, the potential position is decided based on existing objects in the map to avoid collision. Once the position is decided, it is overlaid on the existing map. (Rong [0178]-[0179], “FIG. 9 depicts one possible implementation of the system's insertion location determination and object data selection steps. In this implementation, the insertion location determination step includes sampling locations 902 in the environment to determine where the insertion location 906 is going to be. The system can sample the locations and determine whether the locations are viable locations for an object to be placed. The location needs to meet the dynamics of the environment without leading to a collision. In this implementation, the system is aware of the movement of objects in the scene 902, and once a sampling location 906 is determined, the system determines whether the placement leads to a collision 904. When an insertion location is finally determined to be a viable location for placement that does not lead to a collision, an object data set can be selected.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 11-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 2021/0261148 A1) in view of Rong et al. (US 2021/0383616 A1).
Regarding claim 1, Li teaches:
A method for generating a synthetic image, the method comprising:
receiving an input image; ([0053], “In an exemplary embodiment, the data reception module 118 of the action prediction application 106 may be configured to receive image data that may be associated with images captured of the surrounding environment of the ego vehicle 102 that may be provided by the vehicle camera system 110 of the ego vehicle 102.”)
removing at least one pre-existing object from the input image; ([0057], “In an exemplary embodiment, the neural network 108 may complete image inpainting to electronically remove and replace each of pixels associated with each of the dynamic objects independently,”)
inpainting a region where the at least one pre-existing object was removed; ([0057], “In an exemplary embodiment, the neural network 108 may complete image inpainting to electronically remove and replace each of pixels associated with each of the dynamic objects independently,”)
estimating a position of another pre-existing object from the input image; (
PNG
media_image1.png
460
472
media_image1.png
Greyscale
)
However, Li does not explicitly, but Rong teaches:
generating a layout overlaid on the input image based on the estimated position; ([0178] , ”FIG. 9 depicts one possible implementation of the system's insertion location determination and object data selection steps. In this implementation, the insertion location determination step includes sampling locations 902 in the environment to determine where the insertion location 906 is going to be. The system can sample the locations and determine whether the locations are viable locations for an object to be placed. The location needs to meet the dynamics of the environment without leading to a collision. In this implementation, the system is aware of the movement of objects in the scene 902, and once a sampling location 906 is determined, the system determines whether the placement leads to a collision 904.” Layout is shown in FIG. 9 first graph.)
wherein the layout indicates potential positions for a synthetic object to be added;([0178]-[0179], “FIG. 9 depicts one possible implementation of the system's insertion location determination and object data selection steps. In this implementation, the insertion location determination step includes sampling locations 902 in the environment to determine where the insertion location 906 is going to be. The system can sample the locations and determine whether the locations are viable locations for an object to be placed. The location needs to meet the dynamics of the environment without leading to a collision. In this implementation, the system is aware of the movement of objects in the scene 902, and once a sampling location 906 is determined, the system determines whether the placement leads to a collision 904. When an insertion location is finally determined to be a viable location for placement that does not lead to a collision, an object data set can be selected. The system may take data sets from an object bank 908 to process for selection.”)
generating a synthetic object based on the layout. ([0181], “FIG. 10 depicts an example input and output of one implementation of the system. In this implementation, the input is an input video 1002, captured while a car is driving down the street. The output is an output simulated video 1006 that includes a new car 1008 in the input video 1002. In this embodiment, the output is photorealistic 1012, physically plausible 1014, and geometrically consistent 1016. The photorealistic, physically plausible, and geometrically consistent output may have been generated through the use of the method of FIG. 3 or another method or system disclosed herein.”)
Li teaches that a self-driving car system captures an environment, predicts driving actions, but does not teach adding a synthetic vehicle to the environment. Rong also teaches that a self-driving car system captures an environment and analyzes the environment and tests driving safety features by augmented to add another vehicle into the environment.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to have combined the teachings of Li with the specific teachings of Rong to effectively test the driving safety features for a self-driving system.
Regarding claim 2, Li in view of Rong teaches:
The method according to claim 1, wherein the at least one pre-existing object is a foreground object, (Li, FIG. 4A, the object are cars.) wherein the another pre-existing object is a road structure object;(Li, FIG. 4C. ) and wherein the generated synthetic object is a vehicle.
Regarding claim 3, Li in view of Rong teaches:
The method according to claim 2, wherein estimating a position of the another pre-existing object comprises: estimating a depth of the road structure object.(Li, [0064], “he necessity of defining spatial relation arises from that the interactions of two distant objects are usually scarce. To calculate this relation, the neural network 108 may unproject objects from the 2D image plane to the 3D space in the world frame:[x y z 1].sup.T=δ.sub.u, .sub.vP.sup.−1[u v 1].sup.T (3)
where [u v 1].sup.T and [x y z 1].sup.T are homogeneous representations in 2D and 3D coordinate systems, P is the camera intrinsic matrix, and δ.sub.u,v is the relative depth at (u, v) obtained by depth estimation.”)
Regarding claim 4, Li in view of Rong teaches:
The method according to claim 3, wherein the another pre-existing object is a road lane, (Li, FIG. 4C, 404) and wherein estimating a position of the another pre-existing object further comprises: estimating a location of the road lane. (Li, [0064]-[0065] teaches estimating location and distance between different object. FIG. 4C shows one object on the road is road lane.
PNG
media_image1.png
460
472
media_image1.png
Greyscale
)
Regarding claim 5, Li in view of Rong teaches:
The method according to claim 4, wherein removing the at least one object is performed by instance segmentation. (Li [0054], “The I3D 204 is configured to apply instance segmentation and semantic segmentation 210 to detect dynamic objects located within the driving scene and the driving scene characteristics of the driving scene.” After the objects are detected, the following paragraphs [0057] will remove the detected objects.)
Regarding claim 11, Li in view of Rong teaches:
An apparatus for generating a synthetic image, the apparatus comprising: at least one memory storing computer-executable instructions; and at least one processor configured to execute the computer-executable instructions to: (Li, [0005], “According to yet another aspect, non-transitory computer readable storage medium storing instructions that when executed by a computer, which includes a processor perform a method”) The rest of claim 11 recites similar limitations of claim 1, thus is rejected accordingly.
claim 12-15 recites similar limitations of claim 2-5 respectively, thus are rejected accordingly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YANNA WU whose telephone number is (571)270-0725. The examiner can normally be reached Monday-Thursday 8:00-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 5712722330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YANNA WU/Primary Examiner, Art Unit 2615
Prosecution Insights