DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/09/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-7, 9-12, 21-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Piek (US 20120033130 A1) in view of Guizilini (US 20220392089 A1), Pan (US 2018/0197294 A1).
Regarding claim 1, Piek teaches a system, comprising: one or more processing units to: determine a first optical flow vector corresponding to a forward motion from a pixel location in a first image to a second image of a sequence of images (Piek, Fig. 4, 403a, determine a first forward optical flow motion vector from a block [pixel location] a first image to a second image in a sequence of images (n, n-1, n-2), [0024, 0058]); determine a second optical flow vector corresponding to a backward motion from the pixel location in the second image to the other image of the sequence of images (Piek, Fig. 4, 403b, determine a second backward optical flow motion vector from the corresponding block [pixel location] in the second image to another image in a sequence of images (n, n-1, n-2), [0025, 0058]); and determine, based at least in part on respective validities of the first optical flow vector and the second optical flow vector, an occlusion status for the pixel location in the first image and the pixel location in the second image, at least one of the respective validities being based at least in part on a corresponding chroma of the pixel location. (Piek, Fig. 4, determine occlusion status for pixel location based on respective match errors [corresponding to validities] of the forward/first vector and backward/second vector for the corresponding pixel location between images, [0058-0060]).
However, Piek fails to teach to determine a second optical flow vector corresponding to a backward motion from a pixel location in the second image to the first image of the sequence of images, at least one of the respective validities being based at least in part on a corresponding chroma of the pixel location. (Guizilini, determine second backward optical flow from a corresponding pixel location in the second image (t+1) to the first image (t) in the sequence of images, [0025]). However, Piek in view of Guizilini fails to teach at least one of the respective validities being based at least in part on a corresponding chroma of the pixel location. (Pan, [0100], includes the pixel comparison engine can compare a pixel value (e.g., a pixel chrominance value) at the selected pixel location to the pixel values of pixels at corresponding pixel locations of one or more history planes within history planes database.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Piek using the teachings of Guizilini to include Guizilini’s determination of the backward and forward optical flow vectors using the same pair of images to Piek's determination of the backward and forward optical flow vectors. Further, it would have been obvious to one of ordinary skill in the art to further modify Piek in view of Guizilini with the teachings of Pan in order to complete the comparison based upon chrominance values. Doing so would improve determination of backward and forward optical flow vectors by providing the same pair of images, and further ensure consistency and accuracy for occlusion status.
Regarding claim 4, the combination of Piek with Guizilini and Pan teaches the system of claim 1, wherein the first optical flow vector is a forward flow vector (Piek, Fig. 4, 403a, first optical flow motion vector is a forward flow vector, [0024, 0058]) and the second optical flow vector is a backward flow vector (Piek, Fig. 4, 403b, second optical flow motion vector is a backward flow vector, [0025, 0058]).
Regarding claim 5, the combination of Piek with Guizilini and Pan teaches the system of claim 4, wherein the one or more processing units are further to determine that the pixel location in the first frame is occluded in the second frame when the forward flow vector is invalid for the pixel location and the backward flow vector is valid for the pixel location (Piek, Fig. 4, 405-406, when forward match error is greater than the backward match error [making the forward vector invalid for the location] and the backward match error is thereby less than the forward match error [making the backward vector valid for the location], then the pixel location in the first frame is determined to be occluded in the second frame (it is determined to be a covering occlusion), [0058-0059]).
Regarding claim 6, the combination of Piek with Guizilini and Pan teaches the system of claim 4, wherein the one or more processing units are further to determine that the pixel location in the second frame is occluded in the first frame when the forward flow vector is valid for the pixel location and the backward flow vector is invalid for the pixel location (Piek, Fig. 4, 405-406, when backward match error is greater than the forward match error [making the backward vector invalid for the location] and the forward match error is thereby less than the backward match error [making the forward vector valid for the location], then the pixel location in the second frame is determined to be occluded in the first frame (it is determined to be an uncovering occlusion), [0058-0059]).
Regarding claim 7, the combination of Piek with Guizilini and Pan teaches the system of claim 1, wherein the one or more processing units are further to determine respective validities based at least on a forward-backward check (Piek, Fig. 4, 405-406, determine match errors [corresponding to validities] based on forward-backward check between frames, [0026-0029]).
Regarding claim 9, Piek with Guizilini and Pan teaches a method, comprising: generating optical flow data for a pixel location in a pair of images (Piek, Fig. 4, generate optical flow data 403a-b for a pixel location/block in a pair of images (any pair of n, n-1, n-2), [0024-0025, 0058]); determining a first validity of a forward flow vector for the pixel location in a first image of the pair of images, wherein the forward flow vector corresponds to a forward motion of the pixel location from the first image to a second image of the pair of images (Piek, Fig. 4, determine first match error 404a [corresponding to validity] of a forward flow vector, which is from the pixel location/block in a first image to a second image of the pair/sequence (n, n-1, n-2), [0026, 0058]); determining a second validity of a backward flow vector for the corresponding pixel location in the second image, the second image being later in time than the first image, wherein the backward flow vector corresponds to a backward motion of the corresponding pixel location from the second image to the other image (Piek, Fig. 4, determine second match error 404b [corresponding to validity] of a backward flow vector, which is from the corresponding pixel location/block in a second image later in time than the other image (n, n-1, n-2), [0027, 0058]); and determining the pixel location is occluded in one of the first image or the second image based, at least in part, on the first validity and the second validity (Piek, Fig. 4, determine if block/pixel location is occluded based on the first and second match error/validities 405-407, [0058-0060]).
Guizilini teaches wherein the backward flow vector corresponds to a backward motion of the corresponding pixel location from the second image to the first image (Guizilini, determine second backward optical flow from a corresponding pixel location in the second image (t+1) to the first image (t) in the sequence of images, [0025]).
Regarding claim 10, the combination of Piek with Guizilini and Pan teaches the method of claim 9, further comprising performing a forward-backward check on the forward flow vector and the backward flow vector (Piek, Fig. 4, 405-406, perform forward-backward check on the forward flow vector and the backward flow vector between frames, [0026-0029]).
Regarding claim 21, it is rejected for similar reasons as to those described in claim 1. (Piek, processor, [0063]) corresponds to the system of claims 4-7, and performs the steps disclosed herein.
Claim(s) 2-3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Piek with Guizilini and Pan, as applied to claim 1 above, in view of Staranowicz (US 20180315174 A1).
Regarding claim 2, the combination of Piek with Guizilini and Pan teaches the system of claim 1.
However, the combination of Piek with Guizilini and Pan fails to teach where Staranowicz teaches wherein the one or more processing units are further to filter at least one of a first image occlusion status or a second image occlusion status (Staranowicz, the occlusion map/status is processed and filtered with a median filter, [0066]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Piek with Guizilini and Pan, using the teachings of Staranowicz to include Staranowicz’s filtering of the occlusion status to Piek with Guizilini and Pan, occlusion status and chroma checking. Doing so would improve the occlusion status by providing filtering, which would be used to smooth inconsistencies.
Regarding claim 3, the combination of Piek with Guizilini with Pan, and Staranowicz teaches the system of claim 2, wherein the filter is a median filter (Staranowicz, the occlusion map/status is processed and filtered with a median filter, [0066]).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Piek with Guizilini and Pan, as applied to claim 1 above, in view of Jiang (US 20190138889 A1).
Regarding claim 8, the combination of Piek with Guizilini and Pan teaches the system of claim 1.
However, the combination of Piek with Guizilini and Pan fails to teach where Jiang teaches wherein the system is comprised in at least one of: a human-machine interface system of an autonomous or semi-autonomous machine (Jiang, “The PPU 300 may be configured to accelerate numerous deep learning systems and applications including autonomous vehicle platforms,” [0074]); a human-machine interface system of a gaming machine (Jiang, “The architecture and/or functionality of the various previous figures may be implemented in the context of…a game console system dedicated for entertainment purposes,” [0127]); a system for performing conversational Al operations; a system for performing simulation operations (Jiang, “The PPU 300 may be configured to accelerate numerous deep learning systems and applications including…molecular simulations…molecular dynamics simulation,” [0074]); a system for performing digital twin operations; a system for performing deep learning operations (Jiang, “The PPU 300 may be configured to accelerate numerous deep learning systems and applications including…deep learning,” [0074]); a system for generating or presenting virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; a system for generating or presenting mixed reality (MR) content; a system for performing video playback; a system for performing rendering operations (Jiang, “In an embodiment, the PPU 300 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline,” [0073]); a system for performing three-dimensional rendering operations (Jiang, “In an embodiment, the PPU 300 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline or processing three - dimensional (3D) graphics data,” [0073]); a system implemented using an edge device; a system implemented using a robot (Jiang, “The PPU 300 may be configured to accelerate numerous deep learning systems and applications including…robotics,” [0074]); a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center (Jiang, “One or more PPUS 300 may be configured to accelerate thousands of High Performance Computing (HPC), data center, and machine learning applications,” [0074]); or a system implemented at least partially using cloud computing resources.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Piek with Guizilini and Pan, using the teachings of Jiang to include Jiang’s variety of applications for occlusion determination to Piek with Guizilini and Pan, occlusion determination. Doing so would improve the use of the occlusion determination by providing a wide breadth of possible applications, which would be used to fully utilize the system.
Claim(s) 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Piek with Guizilini and Pan, as applied to claim 9 above, in view of Liu (US 20240214565 A1).
Regarding claim 13, the combination of Piek with Guizilini and Pan teaches the method of claim 9 further comprising determining at least one of the forward flow vector or the backward flow vector is valid (Piek, Fig. 4, 404-405, determine validities [match errors] of forward flow vector and backward flow vector).
However, the combination of Piek with Guizilini and Pan fails to teach where Liu teaches the method further comprising: determining a difference between a first luma value for the pixel location in the first image and a second luma value for the pixel location in the second image exceeds a luma threshold (Liu, determine if difference between luma component in reference picture [first image] and target picture [second image] is within a threshold, [0195, 0305-0306]); determining a difference between a first chroma value for the pixel location in the first image and a second chroma for the pixel location in the second image is within a chroma threshold (Liu, determine if difference between chroma component in reference picture [first image] and target picture [second image] is within a threshold, [0195, 0305-0306]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Piek with Guizilini and Pan, using the teachings of Liu to include Liu’s comparison of the differences of chroma and luma values to thresholds in occlusion determination to Piek with Guizilini and Pan, occlusion determination. Doing so would improve occlusion determination by providing a way to compare chroma and luma changes between images, which would be used to help determine occlusion if the differences indicate a covered/uncovered object.
Regarding claim 14, the combination of Piek with Guizilini and Pan, and Liu teaches the method of claim 13, further comprising generating an occlusion output identifying an occlusion status of the pixel location in at least one of the first image or the second image (Piek, Fig. 4, 406-407 identify an occlusion status of the block/pixel location in the images, [0059]).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Piek with Guizilini and Pan and Liu, as applied to claim 14 above, further in view of Staranowicz (US 20180315174 A1).
Regarding claim 15, the combination of Piek with Guizilini with Pan and Liu teaches the method of claim 14.
However, the combination of Piek with Guizilini with Pan and Liu fails to teach where Staranowicz teaches wherein the optical flow data is noisy optical flow data (Staranowicz, the noisy optical flow data results from noisy input image frames, [0046]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Piek with Guizilini with Pan and Liu, using the teachings of Staranowicz to include Staranowicz’s noisy optical flow data to Piek with Guizilini with Pan and Liu, optical flow data. Doing so would improve the method’s versatility by providing noisy optical flow data, which would be used to ensure the method was capable of handling noisy data.
Conclusion
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Amandeep Saini whose telephone number is (571)272-3382. The examiner can normally be reached M-F (8AM-4PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights