CTNF 18/732,908 CTNF 77342 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1, 2, 4-7, 9-11, 13-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20190236794 A1 to Nash et al (hereinafter ‘Nash’) . Regarding claim 1, Nash discloses an apparatus (Fig. 1, device 102) comprising: a processing system including one or more memories and one or more processors coupled to the one or more memories (Para [0048], wherein the memory 126 may store instructions and/or data. The processor 112 may access (e.g., read from and/or write to) the memory 126.), the processing system configured to: obtain an image frame (Para [0072, and Fig. 7 wherein a wide-angle image 778, as reference frame , and a telephoto image 780, as a frame , may be provided to an alignment determiner 782); obtain a first delta frame corresponding to a difference between the image frame and a reference frame (Para [0093], wherein the alignment (e.g., transform) may be provided to a transformer 784. The transformer 784 may apply a transform (e.g., scaling, translation, and/or rotation, etc.), as the differences between the frame and the reference frame , to the telephoto image 780 in order to approximately align the telephoto image 780 to the wide-angle image 778. For example, the transformer 784 may produce an aligned telephoto image 786, inherently as the delta frame); obtain an output image frame corresponding to a sum of the first delta frame and the reference frame (Para [0095], wherein the wide-angle image 778 and the aligned telephoto image 786 may be provided to the image fuser 718, as summing . For example, the aligned images may be provided to the image fuser 718. The image fuser 718 may fuse (e.g., combine and/or composite) the aligned images to produce a fused image 788 (e.g., a fused output, a combined image, etc.)); and perform one or more operations using the output image frame (Para [0077], wherein the electronic device 102 may transmit, as the operation , one or more fused images to another device.). Nash does not specifically disclose performing a denoising operation associated with the first delta frame to generate a second delta frame, and that sum the dealt frame with the reference frame. However, Nash discloses in alternative embodiment that the aligned image can be denoised after aligning, before fusing (Para [0035], wherein in some configurations of the systems and methods disclosed herein, spatial and photometric alignment may allow diffusion-based de-noising. Reference image (e.g., wide-angle image or telephoto image) structure may be used as an input to a guided averaging filter . The combination of alignment and smart averaging may result in enhanced image quality. For example, combining images may reduce noise in the resulting image. More specifically, averaging images, inherently as producing averaged aligned images, i.e. averaging aligned first delta frame to produce the second delta frame which inherently include less noise , in accordance with some of the configurations of the systems and methods disclosed herein may suppress noise by combining information from multiple cameras). Therefore, it would have been obvious to one ordinary skill in the art to denoise the first delta frame to produce a second aligned averaged delta frame in order to reduce noise in the produced fused frame generated by combining the aligned averaged images (Para [0035]). Regarding claim 2, Nash discloses wherein the reference frame corresponds to one of a preceding frame that precedes the image frame in a sequence of image frames captured by a camera or a blended frame that is based on multiple preceding frames that precede the image frame in the sequence of image frames (para [0097], wherein the reference frame 890 may be captured at the same time as or in a time period that overlaps with the capture of frame n 892. For instance, both the reference frame 890 and frame n 892 may be captured within a time period n. In other approaches, the reference frame 890 may be captured closely in time relative to frame n 892, as reference frame 890 is capable of being any one of the preceding frames of the camera ). Regarding claim 4, Nash discloses wherein the processing system is further configured to perform a pixel-by-pixel subtraction of reference pixels of the reference frame from pixels of the image frame (Para [0062], wherein using (e.g., combining) the images (e.g., aligned images) may include combining the aligned images in accordance with a weighting based on a similarity measure. The similarity measure may indicate a degree of similarity between images. For example, a photometric similarity measure (e.g., D) may be computed in accordance with Equation (1). D=F(|S.B−S.A|). . .In Equation (1), D is the photometric similarity measure, F is a function, S.B is a second image (e.g., telephoto image, non-reference image , etc.) or a component thereof (e.g., one or more pixels), and S.A is a first image (e.g., wide-angle image, a reference image, S.Aref, etc.) or a component thereof (e.g., one or more pixels).). Regarding claim 5, Nash discloses wherein the processing system is further configured to perform a pixel-by-pixel addition operation of reference pixels of the reference frame to pixels of the second delta frame (para [0057], wherein fusing two images may include producing an image that is based on and/or includes data (e.g., pixel data, a sum of pixel data, etc .) from both images.). Regarding claim 6, Nash discloses wherein the processing system is further configured to perform the denoising operation irrespective of a magnitude of the difference between the image frame and the reference frame (para [0035], wherein in some configurations of the systems and methods disclosed herein, spatial and photometric alignment may allow diffusion-based de-noising. Reference image (e.g., wide-angle image or telephoto image) structure may be used as an input to a guided averaging filter, inherently as just relying on only the reference frame for averaging of the images ). Regarding claim 7, Nash discloses wherein a magnitude of the difference between the image frame and the reference frame exceeds a threshold associated with directly denoising image frames (para [0075] and [0100], wherein the electronic device 102 may fuse 208 the aligned images based on a diffusion kernel. The diffusion kernel may indicate a threshold level over a gray level range, as not exceeding the threshold . Additionally, or alternatively, fusing 208 the aligned images may be based on an averaging filter guided by reference image structure. For example, the averaging filter may be adapted based on information in the reference image, and wherein the adaptive thresholder 896 may determine a similarity mask 898 and/or a difference mask 801. For example, an adaptive threshold (e.g., the diffusion kernel) may be applied to generate a similarity mask and a difference mask, as the limit of magnitude of difference ). Regarding claim 9, Nash discloses wherein the image frame includes a first amount of temporal noise, and wherein the output image frame includes a second amount of temporal noise that is less than the first amount of temporal noise (Para [0035], wherein the combination of alignment and smart averaging may result in enhanced image quality. For example, combining images may reduce noise in the resulting image, inherently as generated less noise than before ). Regarding method claims 10, 11, 13-16 and 18, please refer to the corresponding apparatus claims above for further teachings. Regarding computer-readable medium claim 19, Nash discloses a non-transitory computer-readable medium storing instructions executable by one or more processors to initiate, perform, or control operations (Para [0172], wherein the functions may be stored as one or more instructions on a computer-readable medium), the operations comprising: please refer to the corresponding apparatus claim 1 above for further teachings. Regarding claim 20, please refer to the corresponding apparatus claim 2 above for further teachings . 07-21-aia AIA Claim s 3, 8, 12 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Nash in view of US 2024/0119609 A1 to Goesele et al (hereinafter ‘Goesele’) . Regarding claims 3 and 12, Nash does not specifically disclose wherein the camera is included in a wearable device, and wherein the sequence of image frames is captured in accordance with execution of an extended reality (XR) application by the wearable device. Goesele discloses the camera is included in a wearable device (Para [0046], wherein one or more user- wearable devices such as AR headsets (and/or glasses, wherein the terms may be interchangeably used herein) may be equipped with multiple types of cameras.), and wherein the sequence of image frames is captured in accordance with execution of an extended reality (XR) application by the wearable device (Para [0075], wherein output 295 may be additionally or alternatively comprise a video capture, which may be enhanced in terms of resolution, noise characteristics, dynamic range, a relative absence of artifacts, and/or other visual or imaging aspects. In particular embodiments may be suitable for consumer photography and/or videography in AR applications .). Nash and Goesele are combinable because they both disclose camera image processing. Therefore, before the effective filing data of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the extended reality (XR) application of Goesele’s apparatus/method with Nash’s in order to a selectively sampled output from one or more cameras of camera array 245 that may be suitable for use in particular Augmented Reality use cases, such as object tracking (Para [0075]). Regarding claims 8 and 17, Nash does not specifically disclose wherein the processing system is further configured to: execute a video see-through application that specifies that the output image frame is to be augmented with extended reality (XR) content; and initiate display of video content at a display, wherein the video content includes the output image frame and a virtual overlay of the output image frame. Goesele discloses execute a video see-through application that specifies that the output image frame is to be augmented with extended reality (XR) content (Para [0075], wherein output 295 may be suitable for consumer photography and/or videography in AR applications . In particular embodiments, a selectively sampled output from one or more cameras of camera array 245 may be suitable for use in particular Augmented Reality use cases, as inherently executing a video see-through application base thereon ); and initiate display of video content at a display, wherein the video content includes the output image frame and a virtual overlay of the output image frame (Para [0065], wherein a grid of detail cameras 260 may be overlaid, as the overlay of the output image , over the FoV of guide camera 250, as also the output image , so that for a given minimum depth, the respective FoVs of the detail cameras 260 may jointly cover an FoV of detail camera 250). Nash and Goesele are combinable because they both disclose camera image processing. Therefore, before the effective filing data of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the execute a video see-through application that specifies that the output image frame is to be augmented with extended reality (XR) content of Goesele’s apparatus/method with Nash’s in order to a selectively sampled output from one or more cameras of camera array 245 that may be suitable for use in particular Augmented Reality use cases, such as object tracking (Para [0075]). Contact Information Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERVIN K NAKHJAVAN whose telephone number is (571)272-5731. The examiner can normally be reached Monday-Friday 9:00-12:00 PST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sue Lefkowitz can be reached at (571)272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SHERVIN K NAKHJAVAN/Primary Examiner, Art Unit 2672 Application/Control Number: 18/732,908 Page 2 Art Unit: 2672 Application/Control Number: 18/732,908 Page 3 Art Unit: 2672 Application/Control Number: 18/732,908 Page 4 Art Unit: 2672 Application/Control Number: 18/732,908 Page 5 Art Unit: 2672 Application/Control Number: 18/732,908 Page 6 Art Unit: 2672 Application/Control Number: 18/732,908 Page 7 Art Unit: 2672 Application/Control Number: 18/732,908 Page 8 Art Unit: 2672 Application/Control Number: 18/732,908 Page 9 Art Unit: 2672 Application/Control Number: 18/732,908 Page 10 Art Unit: 2672
Prosecution Insights