DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/29/2026 has been entered. Claims 5-7, 13-15, and 17 were amended. Claims 1-20 are pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 8-10, 16-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (US 2017/0109940) in view of Hoang et al. (US 7711044), Cricri et al. (US 2021/0168395), and Lyman et al. (US 11488077).
Regarding claim 1, Guo teaches/suggests: A processor comprising:
one or more processing units (Guo Fig. 3: processor 330) to:
at least one of: (i) a dynamic seam placement that is based at least on one or more locations of one or more detected objects in an environment around an ego-object (Guo [0111] “the electronic device 322 (e.g., image visualization adjuster 336) may adjust 1010 the image visualization based on one or more foreground object distances” [0071] “adjusting the image visualization may be performed based on dynamic data” [0042] “the combined image obtainer 332 may perform image alignment (e.g., registration), seam finding and/or merging”) or (ii) a three-dimensional (3D) bowl that adaptively models the environment with a shape based at least on one or more distances to the one or more detected objects (Guo [0056] “an image (e.g., a combined image) may be rendered in the shape of a bowl”);
Guo does not teach/suggest:
one or more visualizations representing the environment based at least on relaxing or disabling one or more constraints on changes to a previous dynamic seam placement or a previous 3D bowl.
Hoang, however, teaches/suggests relaxing or disabling one or more constraints on changes (Hoang col. 13 ll. 15-20 “it may be beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the combined image of Guo not to be temporally filtered (the one or more constraints) as taught/suggests by Hoang to be beneficial. As such, Guo as modified by Hoang teaches/suggests:
one or more visualizations representing the environment based at least on relaxing or disabling one or more constraints on changes to a previous dynamic seam placement or a previous 3D bowl (Guo [0042] “the combined image obtainer 332 may perform image alignment (e.g., registration), seam finding and/or merging” [0056] “an image (e.g., a combined image) may be rendered in the shape of a bowl” [0071] “adjusting the image visualization may be performed based on dynamic data” Hoang col. 13 ll. 15-20 “it may be beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later”).
Guo as modified by Hoang does not teach/suggest:
predict, for at least one future time interval, that a
based at least on the predicting that the
Cricri, however, teaches/suggests:
predict, for at least one future time interval, that a
Before the effective filing date of the claimed invention, the substitutions of one known element (the predicted scene change of Cricri) for another (the scene change of Hoang) would have been obvious to one of ordinary skill in the art because such substitutions would have yielded predictable results, namely to be beneficial. As such, Guo as modified by Hoang and Cricri teaches/suggests:
based at least on the predicting that the beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later” Cricri [0075] “if a scene transition is detected in the predicted future frames, video processing unit 32 allows the encoding unit 34 to use encoding methods suitable for scene transition when encoding the future frames”).
Hoang and Cricri are silent regarding a threshold change. Lyman, however, teaches/suggests a threshold change (Lyman Claim 1 “in response to predicting the future change in condition satisfies the threshold, performing a function”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the scene change of Guo as modified by Hoang and Cricri to satisfy a threshold as taught/suggests by Lyman as a condition for such a change.
Regarding claim 2, Guo as modified by Hoang, Cricri, and Lyman teaches/suggests: The processor of claim 1, the one or more processing units further to temporally mask the threshold change based at least on temporarily disabling temporal filtering that implements the one or more constraints (Hoang col. 13 ll. 15-20 “it may be beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later” Lyman Claim 1 “in response to predicting the future change in condition satisfies the threshold, performing a function”). The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 8, Guo as modified by Hoang, Cricri, and Lyman teaches/suggests: The processor of claim 1, wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content (Guo [0033] “virtual reality devices (e.g., headsets), augmented reality devices (e.g., headsets), mixed reality devices (e.g., headsets)”);
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding claim 9, Guo as modified by Hoang, Cricri, and Lyman teaches/suggests: A system comprising:
one or more processing units (Guo Fig. 3: processor 330) to:
predict, for at least one future time interval, that a threshold change will occur (Cricri [0075] “if a scene transition is detected in the predicted future frames, video processing unit 32 allows the encoding unit 34 to use encoding methods suitable for scene transition when encoding the future frames” Lyman Claim 1 “in response to predicting the future change in condition satisfies the threshold, performing a function”) to at least one of: (i) a dynamic seam placement that is based at least on one or more locations of one or more detected objects in an environment around an ego-object (Guo [0111] “the electronic device 322 (e.g., image visualization adjuster 336) may adjust 1010 the image visualization based on one or more foreground object distances” [0071] “adjusting the image visualization may be performed based on dynamic data” [0042] “the combined image obtainer 332 may perform image alignment (e.g., registration), seam finding and/or merging”) or (ii) a three-dimensional (3D) bowl that adaptively models the environment with a shape based at least on one or more distances to the one or more detected objects (Guo [0056] “an image (e.g., a combined image) may be rendered in the shape of a bowl”); and
based at least on predicting that the threshold change will occur, generating one or more visualizations that at least partially conceal the threshold change based at least on relaxing or disabling temporal filtering that limits changes to a previous dynamic seam placement or a previous 3D bowl (Guo [0042] “the combined image obtainer 332 may perform image alignment (e.g., registration), seam finding and/or merging” [0056] “an image (e.g., a combined image) may be rendered in the shape of a bowl” [0071] “adjusting the image visualization may be performed based on dynamic data” Hoang col. 13 ll. 15-20 “it may be beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later” Cricri [0075] “if a scene transition is detected in the predicted future frames, video processing unit 32 allows the encoding unit 34 to use encoding methods suitable for scene transition when encoding the future frames” Lyman Claim 1 “in response to predicting the future change in condition satisfies the threshold, performing a function”).
The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Claims 10 and 16 recite limitation(s) similar in scope to those of claims 2 and 8, respectively, and are rejected for the same reason(s).
Claims 17-18 and 20 recite limitation(s) similar in scope to those of claims 1-2 and 8, respectively, and are rejected for the same reason(s).
Claim(s) 3 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (US 2017/0109940) in view of Hoang et al. (US 7711044), Cricri et al. (US 2021/0168395), and Lyman et al. (US 11488077) as applied to claims 1 and 9 above, and further in view of Gardner et al. (US 2014/0247365).
Regarding claim 3, Guo as modified by Hoang, Cricri, and Lyman teaches/suggests: The processor of claim 1, wherein the temporal filtering that smooths changes to the previous dynamic seam placement or the previous 3D bowl (Guo [0042] “the combined image obtainer 332 may perform image alignment (e.g., registration), seam finding and/or merging” [0056] “an image (e.g., a combined image) may be rendered in the shape of a bowl” Hoang col. 1 line 56 – col. 2 line 11 “Digital video noise can be removed by a variety of known digital processing techniques such as … temporal filters”). The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Guo, Hoang, Cricri, and Lyman are silent regarding the relaxing or disabling of the one or more constraints reduces latency introduced by temporal filtering. Gardner, however, teaches/suggests the relaxing or disabling of the one or more constraints reduces latency introduced by temporal filtering (Gardner [0064] “a user may select appropriate user settings 219 to partially or completely disable temporal filter 218 to prevent image lag from being exhibited by filtered signals 222 that might be attributable to temporal filtering”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the temporal filtering of Guo as modified by Hoang, Cricri, and Lyman to be disabled as taught/suggests by Gardner to prevent image lag.
Claim 11 recites limitation(s) similar in scope to those of claim 3, and is rejected for the same reason(s).
Claim(s) 4, 12, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (US 2017/0109940) in view of Hoang et al. (US 7711044), Cricri et al. (US 2021/0168395), and Lyman et al. (US 11488077) as applied to claims 1, 9, and 17 above, and further in view of Vanam et al. (US 11778224).
Regarding claim 4, Guo as modified by Hoang, Cricri, and Lyman does not teach/suggest: The processor of claim 1, the one or more processing units further to temporally mask the threshold change based at least on shortening a temporal window over which temporal filtering is applied. Vanam, however, teaches/suggests temporally mask the threshold change based at least on shortening a temporal window over which the temporal filtering is applied (Vanam col. 5 ll. 35-52 “the temporal window may be set to zero, effectively disabling the temporal filter”). Before the effective filing date of the claimed invention, the substitutions of one known element (shortening the temporal window in Vanam) for another (setting all M values to 0 in Hoang) would have been obvious to one of ordinary skill in the art because such substitutions would have yielded predictable results, namely to disable the temporal filtering.
Claims 12 and 19 recite limitation(s) similar in scope to those of claim 4, and are rejected for the same reason(s).
Claim(s) 5 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (US 2017/0109940) in view of Hoang et al. (US 7711044), Cricri et al. (US 2021/0168395), and Lyman et al. (US 11488077) as applied to claims 1 and 9 above, and further in view of Reif (US 2021/0065461).
Regarding claim 5, Guo as modified by Hoang, Cricri, and Lyman does not teach/suggest: The processor of claim 1, the one or more processing units further to temporally mask the threshold change based at least on initiating the threshold change before the at least one future time interval to compensate for latency introduced by temporal filtering. Reif, however, teaches/suggests initiating before the at least one future time interval to compensate for latency (Reif [0024] “the 3D model of the facility can be generated in the background, and can take advantage of extra computing and resources from the cloud system to reduce the time needed to render the high resolution 3D model of the facility”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the scene change of Guo as modified by Hoang, Cricri, and Lyman to be initiated in the background as taught/suggests by Reif to reduce the time needed for the temporal filtering.
As such, Guo as modified by Hoang, Cricri, Lyman, and Reif teaches/suggests temporally mask the threshold change based at least on initiating the threshold change before the at least one future time interval to compensate for latency introduced by temporal filtering (Hoang col. 13 ll. 15-20 “it may be beneficial to completely disable temporal filtering (i.e. set all M values to 0) immediately after a change of scene is detected and then to resume normal operation one frame later” Cricri [0075] “if a scene transition is detected in the predicted future frames, video processing unit 32 allows the encoding unit 34 to use encoding methods suitable for scene transition when encoding the future frames” Lyman Claim 1 “in response to predicting the future change in condition satisfies the threshold, performing a function” Reif [0024] “the 3D model of the facility can be generated in the background, and can take advantage of extra computing and resources from the cloud system to reduce the time needed to render the high resolution 3D model of the facility”).
Claim 13 recites limitation(s) similar in scope to those of claim 5, and is rejected for the same reason(s).
Allowable Subject Matter
Claims 6-7 and 14-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the limitations “predicting one or more future states of the environment and determining the at least one future time interval in which the dynamic seam placement or the 3D bowl for the one or more future states is predicted to differ from the previous dynamic seam placement or the previous 3D bowl by more than a threshold amount” in claims 6 and 14 and “tracking one or more positions of the one or more detected objects, using a detected trajectory of the ego-object to predict a future location of the ego-object in the at least one future time interval, and determining the dynamic seam placement or the 3D bowl based at least on the one or more positions of the one or more detected objects and the future location of the ego-object” in claims 7 and 15, taken as a whole, render the respective claims patentably distinct over the prior art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2022/0058859 – predict seam changes
US 2024/0161318 – predict scene transitions
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANH-TUAN V NGUYEN whose telephone number is 571-270-7513. The examiner can normally be reached on M-F 9AM-5PM ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached on 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANH-TUAN V NGUYEN/
Primary Examiner, Art Unit 2619
Prosecution Insights