CTNF 18/729,081 CTNF 101491 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claims 1-30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claims 1-30, the term “strength” in claims 1, 8, 10, 11, 18, 21, 28, and 30 is a relative term which renders the claim indefinite. The term “strength” is not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Specifically, it is unclear to the examiner what constitutes a first, second, third, or fourth “strength” (e.g. is it a statistical measurement, how is it determined and/or measured, is it related to focal strength, edge strength, shape strength, line strength, etc.). Therefore, it is unclear how one is to transform a portion of an image data with a strength based on an alignment difference, determine a set of values specifying a strength corresponding to a portion of an image data, transform a foreground portion of an image data with strength, determine if a strength is less than another strength, or perform any other task related to strength and respective first, second, third, or fourth strengths. Additionally, the use of “strength” in dependent claims 8, 10, 18, 28, and 30 fails to resolve the indefiniteness of “strength” in independent claims 1, 11, and 21. Thus, independent claims 1, 11, 21, and their corresponding dependent claims 2-10, 12-20, and 22-30 are rejected under 35 U.S.C. 112(b) for being indefinite. Regarding claims 4, 14, and 24, the terms “filling” and “holes” and the phrase “filling holes” in claims 4, 14, and 24 are relative which render the claims indefinite. The terms and/or phrase “filling holes” is not defined by the claim, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Specifically, it is unclear to the examiner what constitutes “filling holes” due to the vague and ambiguous nature of the statement. For example, it is unclear whether “filling holes” refers to some kind of image morphology (e.g. erosion/dilation processing), or whether “filling” comprises using a model, copying neighbor values, inserting random values or another sequence of events to alter the corresponding motion vector map. Further, it is unclear whether the term “holes” refers to missing values, zero values, gaps in sequences, or another structure corresponding to the motion vector map. Although paragraph [0061] of the specification suggests erosion and dilation may be used to “fill holes” of the motion vector map, the claim itself does not positively recite this relationship, or provide further clarity, and therefore the metes and bounds of the claim are unclear. Thus, claims 4, 14, and 24 are rejected under 35 U.S.C. 112(b) for being indefinite. For examination purposes, “filling holes” will be read as using an erosion or dilation techniques. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim (s) 1-2, 9-12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1) . Regarding claims 1 and 11, Watts teaches: A method (claim 1 of present application; see Watts Abstract) and an apparatus ([0012] “a device including a first camera and a second camera”) , comprising: a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations (FIG 8; [0066] “computer system 800 includes a processor … and a main memory 805 and static memory 814, which communicate with each other via a bus … disk drive unit 820 includes a computer-readable medium 822 on which is stored one or more sets of instructions and data structures (e.g., instructions 810) embodying or utilizing any one or more of the methodologies or functions described herein. The instructions 810 may also reside, completely or at least partially, within the main memory 805 and/or within the processors 802 during execution thereof by the computer system 800. The main memory 805 and the processors 802 may also constitute machine-readable media.”) including: receiving first image data from a first camera of an image capture device, wherein the first camera is different from a second camera of the image capture device ([0012] “first camera and the second camera are configured to capture two initial images of the same object from different imaging angles”; FIG. 3, [0031] “ a device 310 equipped with two cameras 312a and 312b”; FIG. [0062] “a schematic process flow 710 utilizing a device with two cameras 712 and 714”; FIG. 7C [0063] “another schematic process flow 720 utilizing a device with two cameras 722 and 724…”) ; determining a first portion of the first image data ([0062]-[0063]; FIG. 7B: foreground detection 716; FIG. 7C: foreground detection 726) ; transforming the first portion of the first image data with a first strength based on an alignment difference between the first camera and the second camera ([0009] “…adaptive aspect may depend on differences in positions of object center lines on the foreground and background portions.”; [0062] “Specifically, operation 718 involves aligning and crossfading the images captured by cameras 712 and 714. This operation yields a combined image, which may be further processed by separating the foreground and background portions and processing the background portion separately from the foreground portion, e.g., detecting and suppressing the background portion and/or enhancing the detected foreground portion (block 719).”; [0063]; FIG 7B: Align and Crossfade images 718, Detect and Suppress Background and/or Enhance Foreground 719; FIG. 7C: 728, 729) ; transforming a second portion of the first image data with a second strength based on the alignment difference between the first camera and the second camera ([0062]-[0063]; FIG. 7B: 718, 719; FIG 7C: 728, 729) ; and determining a first output image frame based on the first image data after transforming the first portion of the first image data and transforming the second portion of the first image data (Examiner interprets an image frame to be equivalent to an image (i.e. image frame of a video is also an image), as explained by Watts “In some embodiments, the image is a frame of a video…” ([0058]). Watts teaches a combined image is yielded from the image data acquired from the two cameras (See [0062]-[0063] and the model’s workflow illustrated in FIG. 7B and FIG. 7C to yield the final image, i.e. determine a first output image frame.) and that images are displayed ([0057])). Watts fails to explicitly disclose a model using a strength for transforming a portion of a first and second portion of an image. In a related art, Ouyang teaches : generating blurring strength based on depth information from a first foreground region and a second background region ([0046]) . Ouyang’s second model teaches using the difference in depth information from two different regions to obtain a corresponding blurring strength depth and blurring a region with corresponding depth information according to the blurring strength ([0057]) . The blurring strength taught by Ouyang is “looked up” based on a difference between depth information of two images and when the difference between the two regions’ depth information is larger, the corresponding blurring strength is larger ([0057]) . In summary, Ouyang teaches using image information and a strength value together to transform a region. It would have been obvious to a person of ordinary skill in the art, prior to the effective filling date of the claimed invention, to have applied a metric of strength corresponding to image data for transforming a region, taught by Ouyang, to Watts’ alignment model, specifically applying a strength metric to the spatial and alignment information taught by Watts, for transforming a first and a second portion of image data based on alignment difference between the first camera and the second camera in order to provide a first and second strength that accounts for the severity of alignment modification required when transforming the first portion and second portions of first image data. Doing so would provide an extra level of continuity across operational iterations when assessing alignment differences between two cameras, resulting in a more consistent overall model and improve the accuracy of outputted images. Both inventions lie in the field of endeavor of image analysis with a particular aim at obtaining images from two different cameras to generate a synthesized target image. Regarding claims 2 and 12, Watts and Ouyang teach the method of claim 1 and apparatus of claim 11. Watts further teaches: the first portion comprises a foreground portion; and the second portion comprises a background portion ([0062]-[0063]; FIG. 7B: Foreground Detection 716, Detect and Suppress Background and/or Enhance Foreground 719; FIG. 7C: Foreground Detection 726, Detect and Suppress Background and/or Enhance Foreground 729). Regarding claims 9 and 19, Watts and Ouyang teach the method of claim 1 and the apparatus of claim 11. Watts further teaches: wherein transforming the first portion reduces an image shift between the first output image frame and a previous output image frame from the second camera (Watts teaches two cameras take pictures from two different angles (e.g. left side of image comes from left camera and right side of image comes from right camera) and the outputted synthesized image (i.e. first output image frame containing first portion) is based on corrected alignment of the foreground portion of the two images from the two cameras using a center of the foreground object ([0032]). Examiner equates Watts’ teaching of a right-side image from right camera to be equivalent to previous output image frame from the second camera because the right-side image was previously output from the right camera (i.e. second camera) to the system for processing. Under the broadest reasonable interpretation of the claim, Examiner interprets the first output image frame taught by Watts (i.e. outputted synthesized image) having a corrected alignment from a second right camera’s previous right side image, based on the center of the foreground object, to be equivalent to “wherein transforming the first portion reduces an image shift between the first output image frame and a previous output image frame from the second camera” because the object in the foreground of the first output image is already centered (i.e. requires no shift of foreground pixel coordinates), while the alignment of the previous image from the second right camera taught by Watts requires a shift based on the center of the image’s foreground object initially be aligned to the right side of the image (i.e. requires a shift of foreground pixel coordinates greater than first output image).). Regarding claim 10, Watts and Ouyang teach the method of claim 1, including receiving first image data from a first camera of an image capture device, determining a first portion of the first image data, transforming the first portion of the first image data with a first strength based on an alignment difference between the first camera and the second camera; transforming a second portion of the first image data with a second strength based on the alignment difference between the first camera and the second camera. Watts further discloses: receiving second image data from the first camera of the image capture device (Watts [0012] “first camera and the second camera are configured to capture two initial images of the same object from different imaging angles”; FIG. 3, [0031] “ a device 310 equipped with two cameras 312a and 312b”; FIG. 7B [0062] “a schematic process flow 710 utilizing a device with two cameras 712 and 714”; FIG. 7C [0063] “another schematic process flow 720 utilizing a device with two cameras 722 and 724…”) ; determining a foreground portion of the second image data (Watts [0062]-[0063]; Watts FIG. 7B: foreground detection 716; Watts FIG. 7C: foreground detection 726; ) ; determining a second foreground portion of the second image data (Watts teaches determining a foreground portion of the second image data (refer back to [0062]-[0063] and FIG. 7B: 716 and FIG 7C: 726). Watts also teaches different objects may be detected in the foreground (i.e. a first foreground portion and a second foreground portion) based on depth, motion, and/or distance cues and dynamically processed ([0042]; [0047]). transforming the second foreground portion of the second image data with a third strength based on the alignment difference between the first camera and the second camera ([0009] “…adaptive aspect may depend on differences in positions of object center lines on the foreground and background portions.”; [0062] “Specifically, operation 718 involves aligning and crossfading the images captured by cameras 712 and 714. This operation yields a combined image, which may be further processed by separating the foreground and background portions and processing the background portion separately from the foreground portion, e.g., detecting and suppressing the background portion and/or enhancing the detected foreground portion (block 719).”; [0063]; FIG 7B: Align and Crossfade images 718, Detect and Suppress Background and/or Enhance Foreground 719; FIG. 7C: 728, 729) ; and transforming a second background portion of the second image data with a fourth strength based on the alignment difference between the first camera and the second camera ([0062]-[0063]; FIG. 7B: 718, 719; FIG 7C: 728, 729) , wherein: the third strength is less than the first strength ; and the fourth strength is less than the second strength. It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have applied the spatial and alignment strength metrics taught by Watts and Ouyang in claim 1 to the further teachings of Watts in order to provide third and fourth strengths for the purpose of transforming the second foreground and background portions of the second image data using a consistent strength metric based on the resulting differences between camera information. Doing so would provide an extra level of continuity across operational iterations when assessing alignment differences between two cameras, resulting in a more consistent overall model and improve the accuracy of outputted images. Watts and Ouyang fail to explicitly disclose: wherein: the third strength is less than the first strength; and the fourth strength is less than the second strength. However, Ouyang further teaches: strength metrics are based on a set of values and information, e.g. coordinates and depth information (Ouyang [0046]; [0051]-[0057]). Ouyang goes on to teach the strength may be correlated to the relevance of a region and/or regions (i.e. portions) have varying levels of strength depending on the values and information ([0057]) ; thus, one region may have a higher or lower strength than another region’s strength depending on values and information, and a relative relevancy of a region may be deduced from comparing strengths . It would obvious to one of ordinary skill in the art, prior to effective filing date of the claimed invention, to have modified Watts and Ouyang’s previously modified teachings, including alignment techniques and strengths based on alignment information, to incorporate Ouyang’s further teachings of applying varying strengths (e.g. a third strength less than a first strength, and a fourth strength less than a second strength) in order to more accurately transform portions of the foreground and/or background portions of images. Doing so would allow for the alignment procedures to align more relevant portions of the image (i.e. foreground portion(s)) when there may be multiple portions of images from a first and second camera, thereby presenting more relevant portions of the images to the user . 07-21-aia AIA Claim (s) 3-6 and 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), in further view of Kim (US 20170104920 A1), and in further view of Mao (US 20210365707 A1) . Regarding claims 3 and 13, Watts and Ouyang teach the method of claim 2 and apparatus of claim 12. Watts and Ouyang fail to explicitly disclose: wherein determining the foreground portion of the first image data comprises determining a motion vector map corresponding to the first image data, wherein the foreground portion is based on the motion vector map. In a related art, Kim teaches: determining the foreground portion of the first image data comprises determining [motion vectors and masks] a motion vector map corresponding to the first image data, wherein the foreground portion is based on the [motion vectors and masks] motion vector map (Kim teaches a process for calculating motion vectors using a plurality of images, including “..continuous imaging within a predetermined time using the image capturer 230, calculates motion vectors using the plurality of images, separates a foreground and a background of a first image among the plurality of images based on the motion vectors and color information of the first image, and performs out focusing based on the separated foreground and background. In this case, the motion vector is information about x-axis motion and y-axis motion of a pixel, and thus may be two-dimensional (2D) information. Further, the color information is red green blue (RGB) information of the video data, and thus may be three-dimensional (3D) information.” ([0070]); See motion vector 360 in FIG. 3 and FIG. 4 that represents depth map information ([0073]-[0080]). Also refer to Kim’s teachings of calculating depth information (i.e. motion vectors) using the plurality of images and separation of foreground of first images to perform further tasks like focusing, found in [0045]-[0048].) . In summary, Kim teaches determining foreground regions based on motion information but fails to explicitly disclose determining a motion vector map (i.e. per-pixel displacement vectors). It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to modify the teachings of Watts and Ouyang to incorporate Kim’s teachings of determining a motion vector map corresponding to the first image data when determining the foreground portion of the first image data in order to more accurately separate a first portion of the image when dealing with various environments and external variables when capturing an image (e.g. motion) that result in blurred or out of focus images. In a related art, Mao teaches: determining motion vector maps representing pixel motion fields between frames and motion is associated with image pixels/locations (i.e. per-pixel motion representation) (see “optical flow maps (also referred to as motion vector maps) can be generated based on the computation of the optical flow vectors between frames. Each optical flow map can include a 2D vector field, with each vector being a displacement vector showing the movement of points from a first frame to a second frame (e.g., indicating horizontal and vertical displacements, such as x- and y-displacements). The optical flow maps can include an optical flow vector for each pixel in a frame, where each vector indicates a movement of a pixel between the frames.”). Therefore, it would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to implement the motion-based foreground determination of Kim using motion vector maps of Mao in order to predictably improve the accuracy and robustness of foreground detection, thereby modifying the teachings of Watts, Ouyang, and Kim to incorporate the teachings of Mao. Both inventions (Mao and Kim) are used to gain insight on motion in image sequences to identify meaning regions (e.g. moving foreground objects). Regarding claims 4 and 14, as best understood based on the 35 U.S.C. 112(b) issue identified above, Watts, Ouyang, Kim and Mao teach the method of claim 3 and the apparatus of claim 13, including determining the foreground portion of the first image data comprises determining a motion vector map corresponding to the first image data, wherein the foreground portion is based on the motion vector map. Kim further teaches: Kim teaches carrying out erosion techniques to separate the foreground region of the first image data using motion vector information and carrying out a dilation operation to separate the background region ([0020]-[0022]; [0077]-[0080]) , i.e., interpreted by examiner to be equivalent to “filling holes”, as explained in the 35 U.S.C. 112(b) section above. Examiner notes, the techniques taught by Kim are used to generate a mask corresponding to the foreground and background regions, rather than a motion vector map . While Watts, Ouyang, Kim and Mao fail to explicitly disclose determining the foreground portion of the first image data further comprises filling holes in the motion vector map to determine a processed motion vector map, wherein the foreground portion is based on the processed motion vector map, it would have been obvious to a person of ordinary skill in the art to implement the “filling holes” (i.e. erosion or dilation techniques interpreted by examiner in 35 U.S.C. 112(b) section above) techniques taught by Kim when determining a processed motion vector map, as previously taught by Watts, Ouyang, Kim and Mao in order to more accurately analyze, modify, and enhance shapes found in the foreground according to pixel position. Regarding claims 5 and 15, Watts, Ouyang, Kim and Mao teach the method of claim 3 and the apparatus of claim 13, including determining the foreground portion of the first image data. Watts, Ouyang, Kim and Mao fail to explicitly disclose: determining the foreground portion of the first image data is further based on region of interest (ROI) information. However, Mao teaches: determining a region of interest from a first frame (i.e. from a first image data) for subsequent operations on second frames, i.e. cropping and/or scaling objects in a second frame (Abstract; [0005] “…determining a region of interest in a first frame of a sequence of frames, the region of interest in the first frame including an object having a size in the first frame…”) . Mao further teaches a system can generate and detect foreground blobs to perform various operations across video frames (i.e. including a singular frame), such as object tracking using bounding regions that include a suitably-shaped region representing a region of interest ( see [0104]). It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to apply well known region of interest concepts, exemplified by the teachings of Mao, to the teachings of Watts, previously modified by Ouyang, Kim and Mao, when determining the foreground portion of the first image data to more accurately track an object in the foreground of the image, and/or to maintain a consistent size for a target object (i.e. foreground region) in one or more frames (see Mao [0002]). Regarding claims 6 and 16, Watts, Ouyang, Kim, and Mao teach the method of claim 5 and the apparatus of claim 15, including the region of interest (ROI) information corresponding to the first image data. Kim further teaches: “motion vectors 360 may be depth map type information” (FIG. 4, [0073]). While Watts, Ouyang, Kim, and Mao fail to explicitly disclose wherein the region of interest (ROI) information comprises a depth map corresponding to the first image data, it would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the teachings of Watts, Ouyang, Kim, and Mao to further incorporate well known concepts of depth map information in relation to motion vectors, as taught by Kim, in order to provide a more accurate and robust model that accounts for and/or simulates depth-of-field views, 3D modeling, and/or provides greater insight to special structure of a first image and/or an area of interest with respect to the first image contents . 07-21-aia AIA Claim (s) 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), in further view of Kim (US 20170104920 A1), in further view of Mao (US 20210365707 A1), and in further view of Li (US 20180293735 A1) . Regarding claims 7 and 17, Watts, Ouyang, Kim and Mao teach the method of claim 5 and the apparatus of claim 15, including determining the foreground portion of the first image data and: Mao further teaches: Generating a motion vector map ([0218] “optical flow maps (also referred to as motion vector maps) can be generated based on the computation of the optical flow vectors between frames.”; see 35 U.S.C. 103 rejections for claims 13 and 14 above for further corresponding motion vector map teachings) . Mao further teaches identifying and segmenting moving objects (i.e., regions of interest (ROI)) using blob detection and segmentation masks (Blob detection is used to segment moving objects for subsequent processing tasks such as mask generation and filtering ([0115]). Further, bounding boxes correspond to detected objects/regions of interest ([0077]; [0104]) . Mao also teaches assigning confidence scores to detected objects (Generating a confidence score indicating that a predicted bounding box actually encloses an object (i.e. a region of interest information) and selecting and/or generating objects based on a threshold ([0337])) . Mao further teaches generating segmentation masks and weighting pixel classification (A background subtraction engine 412 determines segmentation masks to detect foreground pixels ([0115]). Further, Mao teaches weighting using statistical models including weights representing the probability in a Gaussian mixture model (GMM) that adapts to local changes ([0117]-[0118]). Mao also teaches learned weights may be parameters derived from training of a neural network and updated during propagation using filters ([0313]-[0319])) . Mao further explicitly teaches flexibility in the order of operation, stating “The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes” [0308] . However, Watts, Ouyang, Kim and Mao fail to explicitly disclose determining a weight mask by fusing the motion vector map with the regions of interest (ROI) information based on the confidence level. It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the invention, to combine Mao’s further teachings of motion vector maps representing motion, region based segmentation identifying objects and regions of interest, and confidence and weighting techniques to determine a weight mask by fusing the motion vector map with the region of interest (ROI) information based on confidence level because Mao already teaches motion estimation, region-based masking and confidence, weighting techniques, and the order of operations may be combined in any order. Further, combining these teachings is a predictable use of known imaging processing techniques. Doing so would emphasize reliable motion information, decrease noisy or low-confidence regions, and improve segmentation and motion based process’ accuracy. Watts, Ouyang, Kim and Mao further fail to explicitly disclose: determining a confidence level associated with determining the foreground portion based on the motion vector map and wherein the foreground portion is determined based on the weight mask. In a related art, Li teaches: determining a confidence level associated with determining the foreground portion based on the motion vector map (Li teaches determining confidence values based on motion vectors in a foreground region of an entire image, i.e. optical flow map or motion vector map (examiner interprets motion vector of an entire image and optical flow map to be equivalent to an motion vector map because it represents the motion vectors of all pixels in an image) (See Li’s teachings of generating optical flow map based on a difference of pixel values of the plurality of pixels in the current image frame and the previous image frame found in paragraph [0023]; [0025] “The image-processing apparatus 102 may be further configured to determine a confidence score for the computed plurality of first motion vector values based on a set of defined parameters. For example, the set of defined parameters may include, but is not limited to, an area covered by a foreground object(s) in an image frame with respect to total area of the image frame and/or a contrast level of the image frame.”) and “The confidence level may be represented numerically by a confidence score.” ([0025]). Li goes on to teach a confidence map is generated based on the confidence score ([0026] “…the image-processing apparatus 102 may be further configured to generate a confidence map based on the determined confidence score and the determined similarity parameter for each of the plurality of pixels.”). Li further teaches: wherein the foreground portion is determined based on the weight mask [confidence map and corresponding confidence score] (Li teaches a foreground region of a current image frame is detected based on the confidence map and having a higher confidence score than background regions ([0067]).) . It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to modify the teachings of Watts, Ouyang, Kim and Mao to incorporate the teachings of Li to determine a confidence level associated with determining the foreground portion based on the motion vector map and determine a foreground portion. Doing so would increase the accuracy of the model by providing a metric that ensures confidence, i.e., reliability, in the foreground portion being correctly determined from the motion vector map information. Additionally, it would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to substitute the use of a weight mask taught by Mao with the confidence map and corresponding confidence score taught by Li when determining the foreground portion because both techniques already use a confidence score and/or confidence level (e.g. the weight mask taught by Mao’s modified teaching is fused from the motion vector map with the region of interest (ROI) information based on confidence level) and doing so would not only process the foreground determination based on confidence in a map’s motion, but more specifically, it would process the foreground determination based on the confidence or reliability the model has in a particular region of interest’s motion, thereby making the model more robust and accurate . 07-21-aia AIA Claim (s) 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), and in further view of Pyojae Kim (US 20200051265 A1; referred to as “Pyojae Kim” below to distinguish from Kim reference used in earlier claims) . Regarding claims 8 and 18, Watts and Ouyang teach the method of claim 1 and apparatus of claim 11, including at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations. Watts and Ouyang’s method of claim 1 and apparatus of claim 11, specifically the “transforming of a first portion of the first image data with a first strength…” and the “transforming of a second portion of with a second strength...” also established a first strength corresponds to a first portion of the first image data and a second strength corresponds to a second portion of the first image data. Ouyang further teaches: Ouyang further teaches: determining a weight map comprising: determining a first set of values specifying the first strength corresponding to the first portion of the first image data; and determining a second set of values specifying the second strength corresponding to the second [a] portion of the first image data (Ouyang teaches depth strength is based on first and second depth information ([0046]). Ouyang’s second mode (embodiment used in claim 11 rejection found above) determines a depth map of unfocused regions (i.e. a second portion) and determines depth information of the focus region (i.e. a first portion) ([0051]). With regard to the first focus region, Ouyang teaches determining a first depth information of the foreground region in the front of the focus region and a second depth information of the background region in the rear of the focus region by using the difference in pixel-point coordinates (values) of corresponding first image data obtained from two cameras to determine a “phase difference”, then looking up the blur strength that corresponds to the “phase difference” value, the first depth information, and the second depth information ([0051]-[0057]). Under the broadest reasonable interpretation of the claim, Examiner interprets Ouyang’s teaching of determining a focus region’s “phase difference” of pixel points, and the focus region’s first and second depth information, and their corresponding relationship’s structure-like set of values, used for looking up a corresponding depth strength to be equivalent to teaching a first set of values specifying the first strength corresponding to the first portion of the first image data.). wherein determining the first output image frame is based on the weight map. Additionally, Ouyang teaches processed image data is displayed using output devices ( [0084]). It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have applied Ouyang’s further teachings of determining a set of values specifying the strength corresponding to a portion of the first image data to the teachings of Watts, previously modified by Oyang, by applying Ouyang’s further teachings to a first strength corresponding to a first portion of the first image data (previously taught by Watts in view of Ouyang) and applying Ouyang’s further teachings to a second strength corresponding to a second portion of the first image data (previously taught by Watts in view of Ouyang) in order to provide consistent metrics for specifying strengths of two different portions of a first image data. Doing so would increase the robustness and accuracy of the model by providing a consistent and measurable relationship for determining the strength used in processing image data. Watts and Ouyang fail to disclose: determining a weight map and determining the first output image frame is based on the weight map. In a related art, Pyojae Kim teaches: determining a weight map (Pyojae Kim teaches generating a weight map based on focus region and depth map information, similar to information derived from Ouyang’s teachings ([0100] “…generate a blur weight map 502 based on the focus region 511 and the depth map 403.”).) and determining the first output image frame based on the weight map (Pyojae Kim teaches generating a “second corrected image”, i.e. a first image, by applying effects based on the weight map ([0101]) and displaying the “second corrected image” ([0109]). Examiner interprets Pyojae Kim’s teachings to be equivalent to determining the first output image frame based on the weight map because the “second corrected image” is the first frame displayed (i.e. outputted) after applying the weight map information.). It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the teachings of Watts and Ouyang to incorporate the teachings of Pyojae Kim in order to provide pixel information and its quality in a mapped format that represents an image for output purposes (i.e. to display, store, send to another device, etc.). Doing so would improve the quality and accuracy of the image information rendered for output and provide a means of accessing the information. Both inventions lie in the field of endeavor of image analysis using images from two cameras to generate a synthesized image while accounting for depth information . 07-21-aia AIA Claim (s) 20, 21, 22, 29, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), and in further view of Mao (US 20210365707 A1) . Regarding claims 20, Watts and Ouyang teach the apparatus of claim 19. Watts and Ouyang fail to teach: the at least one processor comprises an image signal processor (ISP). In a related art Mao teaches: the at least one processor comprises an image signal processor (ISP) ([0091] “The image processor 150 may include one or more processors, such as one or more image signal processors (ISPs) (including ISP 154)…”; FIG. 1: Image Processor 150 and ISP (image signal processor) 154) . It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the teachings of Watts and Ouyang to incorporate the teachings of Mao and use a well-known type of image signal processor (ISP) taught by Mao as at least one of the processors already taught by Watts and Ouyang in claim 11. Doing so would predictably enable the processing of a variety of image types or quality (e.g. raw data images) for real-time tasks like noise reduction, adjusting focus, and other tasks associated with creating and/or improving the quality of a synthesized image based on images from two different cameras found in the apparatus, thereby increasing the robustness and accuracy of the apparatus. All inventions are used to gain insight on motion in image sequences to identify meaning regions (e.g. moving foreground objects). Regarding claim 21, Watts and Ouyang teach the limitations found on lines 3-12 of claim 21. Limitations found on lines 3-12 of claim 21 equally mirror limitations found on lines 2-11 of claim 1 and lines 6-15 of claim 11. For sake of brevity, please refer back to 103 rejections of claims 1 and 11 for prior art taught by Watts and Ouyang and motivations to combine references. Watts further discloses : a computer-readable medium storing instructions that executed by a process that, when executed by a processor, cause the processor to perform operations (see Watts [0066]). Watts and Ouyang fail to explicitly disclose: A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations. In a related art, Mao teaches: A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations ([0309]; [0343]). It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to implement the computer-readable medium taught by Watts and Deng as a non-transitory computer readable recording medium, as taught by Fujiyama, since such implementation merely involves storing known executable instructions in a known medium, yielding predictable results. All inventions are used to gain insight on motion in image sequences to identify meaning regions (e.g. moving foreground objects). Regarding claim 22, Watts, Ouyang, and Mao teach the non-transitory computer-readable medium of claim 21. The remaining limitation(s) of claim 22 equally mirror(s) limitation(s) of claims 2 and 12 and is rejected based on the prior art of Watts and Ouyang taught in claims 2 and 12 of the present office action. Regarding claim 29, Watts, Ouyang, and Mao teach the non-transitory computer-readable medium of claim 21. The remaining limitation(s) of claim 29 equally mirror(s) limitation(s) of claim 9 and 19 and is rejected based on the prior art of Watts and Ouyang taught in claims 9 and 19 of the present office action. Regarding claim 30, Watts, Ouyang, and Mao teach the non-transitory computer-readable medium of claim 21, wherein the instructions, when executed by a processor, cause the processor to perform further operations. The remaining limitation(s) of claim 30 equally mirror(s) limitation(s) of claim 10 and is rejected based on the prior art of Watts and Ouyang taught in claim 10 of the present office action . 07-21-aia AIA Claim (s) 23-26 are rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), in further view of Mao (US 20210365707 A1), and in further view of Kim (US 20170104920 A1) . Regarding claim 23, Watts, Ouyang, and Mao teach the non-transitory computer-readable medium of claim 22. The remaining limitation(s) of claim 23 equally mirror(s) limitation(s) of claims 3 and 13 and is rejected based on the prior art of Watts, Ouyang, Mao, and Kim taught in claims 3 and 13 of the present office action. Regarding claim 24, Watts, Ouyang, Mao, and Kim teach the non-transitory computer-readable medium of claim 23. The remaining limitation(s) of claim 24 equally mirror(s) limitation(s) of claims 4 and 14 and is rejected based on the prior art of Watts, Ouyang, Mao, and Kim taught in claims 4 and 14 of the present office action. Regarding claim 25, Watts, Ouyang, Mao, and Kim teach the non-transitory computer-readable medium of claim 23. The remaining limitation(s) of claim 25 equally mirror(s) limitation(s) of claims 5 and 15 and is rejected based on the prior art of Watts, Ouyang, Mao, and Kim taught in claims 5 and 15 of the present office action. Regarding claim 26, Watts, Ouyang, Mao, and Kim teach the non-transitory computer-readable medium of claim 25. The remaining limitation(s) of claim 26 equally mirror(s) limitation(s) of claim 6 and 16 and is rejected based on the prior art of Watts, Ouyang, Mao, and Kim taught in claim 6 and 16 of the present office action . 07-21-aia AIA Claim (s) 27 is rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), in further view of Mao (US 20210365707 A1), in further view of Kim (US 20170104920 A1), and in further view of Li (US 20180293735 A1) . Regarding claim 27, Watts, Ouyang, Mao, and Kim teach the non-transitory computer-readable medium of claim 25. The remaining limitation(s) of claim 27 equally mirror(s) limitation(s) of claims 7 and 17 and is rejected based on the prior art of Watts, Ouyang, Mao, Kim, and Li taught in claim 7 and 17 of the present office action . 07-21-aia AIA Claim (s) 28 is rejected under 35 U.S.C. 103 as being unpatentable over Watts (US 20130169844 A1) in view of Ouyang (US 20190130532 A1), in view of Mao (US 20210365707 A1), and in further view of Pyojae Kim (US 20200051265 A1; referred to as “Pyojae Kim” below to distinguish from Kim reference used in earlier claims) . Regarding claim 28, Watts, Ouyang, and Mao teach the non-transitory computer-readable medium of claim 21 , wherein the instructions, when executed by a processor, cause the processor to perform further operations. The remaining limitation(s) of claim 28 equally mirror(s) limitation(s) of claims 8 and 18 and is rejected based on the prior art of Watts, Ouyang, and Pyojae Kim taught in claim 8 and 18 of the present office action. Relevant Art Not Relied Upon 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Nash (US 20170230585 A1) – teaches a multi-camera device including two asymmetric cameras to image a target scene, including transforming information from two images according to spatial information (e.g. alignment) to create a fused synthetic image. Kopf (US 20150248916 A1) – teaches outputting synthesized image frame based on a weight map that is processed using depths of pixels and motions of blur amount. Varekamp (US 20190356895 A1) – teaches an image filter unit using confidences values of a depth map and/or a motion confidence map based on a motion vector map to create composite images . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL DAVID BAYNES whose telephone number is (571)272-0607. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SDB/ Samuel Baynes Examiner | Art Unit 2665 /Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665 Application/Control Number: 18/729,081 Page 2 Art Unit: 2665 Application/Control Number: 18/729,081 Page 3 Art Unit: 2665 Application/Control Number: 18/729,081 Page 4 Art Unit: 2665 Application/Control Number: 18/729,081 Page 5 Art Unit: 2665 Application/Control Number: 18/729,081 Page 6 Art Unit: 2665 Application/Control Number: 18/729,081 Page 7 Art Unit: 2665 Application/Control Number: 18/729,081 Page 8 Art Unit: 2665 Application/Control Number: 18/729,081 Page 9 Art Unit: 2665 Application/Control Number: 18/729,081 Page 10 Art Unit: 2665 Application/Control Number: 18/729,081 Page 11 Art Unit: 2665 Application/Control Number: 18/729,081 Page 12 Art Unit: 2665 Application/Control Number: 18/729,081 Page 13 Art Unit: 2665 Application/Control Number: 18/729,081 Page 14 Art Unit: 2665 Application/Control Number: 18/729,081 Page 15 Art Unit: 2665 Application/Control Number: 18/729,081 Page 16 Art Unit: 2665 Application/Control Number: 18/729,081 Page 17 Art Unit: 2665 Application/Control Number: 18/729,081 Page 18 Art Unit: 2665 Application/Control Number: 18/729,081 Page 19 Art Unit: 2665 Application/Control Number: 18/729,081 Page 20 Art Unit: 2665 Application/Control Number: 18/729,081 Page 21 Art Unit: 2665 Application/Control Number: 18/729,081 Page 22 Art Unit: 2665 Application/Control Number: 18/729,081 Page 23 Art Unit: 2665 Application/Control Number: 18/729,081 Page 24 Art Unit: 2665 Application/Control Number: 18/729,081 Page 25 Art Unit: 2665 Application/Control Number: 18/729,081 Page 26 Art Unit: 2665 Application/Control Number: 18/729,081 Page 27 Art Unit: 2665 Application/Control Number: 18/729,081 Page 28 Art Unit: 2665 Application/Control Number: 18/729,081 Page 29 Art Unit: 2665
Prosecution Insights