CTNF 18/791,627 CTNF 101491 DETAILED ACTION 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Information Disclosure Statement The information disclosure statement (IDS) submitted on 10/25/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Priority The present application claims benefit of foreign priority application JP 2023-129515 with a filing date of 08/08/2023. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Specification 06-11 AIA The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. 06-11-01 AIA The following title is suggested: “Image Processing Method and Apparatus for Increasing Image Resolution Using the Detection of Change Across Image Frames” . Claim Objections 07-29-01 AIA Claim 8 objected to because of the following informalities: “for to” on line 2 should read “for”. For examination purposes, “for to” will be read as “for” . Appropriate correction is required. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. 07-30-06 This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an obtainment unit,” “a generation unit”, and “a detection unit” in independent claims 1 and 11, and dependent claims 2-9, and “a control unit” in claim 12. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claim 1-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claim 1, the limitation “generates a fourth image, as a high-resolution image based on the plurality of frames of first images, from the second image and an area of the third image corresponding to an area of the second image corresponding to the changed area” is unclear to a person of ordinary skill in the art which renders the claim indefinite. The limitation is ambiguous in how or what is used to generate the fourth image due to the inclusion of both phrases “based on” and “from the” when describing the reference images used to generate the fourth image because the phrases are directed to results from two different models. It is unclear if the fourth image is generated from one of the two models described in those scenarios, or whether the fourth images is supposed to be generated by some type of combination of the two models’ results. For example, it is unclear if the fourth image is generated (i) “based on the plurality of frames of first images”, (ii) “from the second image and an area of the third image corresponding to an area of the second image corresponding to the changed area” (iii) or if the “plurality of frames of first frame images” is being used to describe the “second image” in the latter statement of the limitation, as in “generates a fourth image, as a high-resolution image, from the second image and an area of the third image corresponding to an area of the second image corresponding to the changed area, wherein the second image is based on the plurality of frames of first images” (iv) or another logical meaning is meant to be derived from the limitation. Therefore, independent claim 1 and its dependent claims 2-9 are rendered indefinite and rejected under 35 U.S.C. 112(b). For examination purposes, Examiner will read the limitation as example (iii) detailed in the previous paragraph of the present office action. Regarding claims 10-13, independent claims 10 and 11 have limitations that are comparable to the indefinite limitation of claim 1 and is rejected under 35 U.S.C. 112(b) based on the same indefinite arguments detailed in the claim 1 rejection found hereinabove. Accordingly, both independent claims 10 and 11, and corresponding dependent claims 12-13 are rejected under 35 U.S.C. 112(b). In order to overcome the rejection for claim 1, 10, 11, and the corresponding dependent claims, Applicant is required to amend claim to provide greater clarity in how or what the fourth image is generated from. Regarding claim 8, the term “values” in line 3 is a relative term which renders the claim indefinite. The term “values” is not defined by the claim and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Upon reading the claim, one of ordinary skill in the art is left wondering if the values associated with the change area map is a pixel value associated with per-pixel coordinates and if so, what type of value (e.g. luminance, Color (RGB) value, Spectral value, categorical/class values, confidence values, Grayscale value, RGBA value, heatmap values, gradient values, depth values etc.), or is it an entirely different type of value based on a different structured or logical method for determining change. The present application’s specification describes yielding absolute values of luminance value differences on a per-pixel basis ([0070]) , but also states “The differences may be obtained on a per-pixel basis, or may be obtained on a per-block basis. Also, the difference image may be obtained for each of color planes of the color filters. Furthermore, the differences between elements other than luminance values may be obtained.” ([0069]) . While it is improper to read the specification into the claim (see MPEP 2111.01 subsection II), Examiner notes the specification fails to further clarify the “values” claim. Therefore, claim 8 is rendered indefinite and rejected under 35 U.S.C. 112(b). Further regarding claim 8, where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp. , 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The term “change area map” in claim 8 is not a known term in the art and it not clearly defined in the claim. While paragraph [0071] of the specification attempts to define the “change area map,” stating “It can be said that the changed area map 601 is an image indicating the position at which the changed area exists in the first high-resolution image,” one of ordinary skill is left wondering what type of image is used to represent the “change area map” (e.g. motion vector map, depth map, etc.). Further the claim fails to specify the “change area map” being presented as an image or any other variation of a formatted map (e.g. motion vector map, depth map, etc.). Thus, the term “change area map” is indefinite because the term is unclear to one of ordinary skill in the art and the claim does not clearly redefine the term. For examination purposes, “values” will be read to include any type of numerical associated with pixels, and “change area map” will be read to include any type of image representing a variation in pixels corresponding to “values” that differ. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. The examiner would like to point out that the various “units” (i.e. one or more processors that functions as an “obtainment unit”, a “generation unit”, and a “detection unit”) that correspond to instructions performed by a processor and identified in the Claim Interpretation section hereinabove are being interpreted under 35 U.S.C. 112(f) as a processor described in paragraph [0087] of the instant application specifications, and the “control unit” identified in the Claim Interpretation section hereinabove are being interpreted under 35 U.S.C. 112(f) as described in [0019] and FIG. 1 of the instant application specifications. Claims 1-4, 6-9, 10-11, and 13 are rejected under 35 U.S.C. 103 as being obvious over Nakagami et al. (US 20090010568 A1; hereinafter “Nakagami”) in view of El-Khamy (US 20180293707 A1). Regarding claim 1, as best understood based on the 35 U.S.C. 112(b) issue identified above, Nakagami teaches: An image processing apparatus (Abstract “An image processing device…”) comprising: one or more processors that execute a program stored in a memory and thereby function as (See Nakagami paragraph [0022] “a computer program that includes instructions that command a computer to perform the functions of the image processing device. Further, a recording medium is provided on which the program mentioned above is recorded.”; See hardware components, including a processor (more specifically, a CPU), explained in paragraphs [0108]-[0116] and seen in FIG. 3 and FIG. 8. Nakagami’s processor (i.e. CPU) for executing the subsequent/corresponding units found in claim one has been interpreted under 112(f) and corresponds to the processor described in paragraph [0087] of the instant application spec.) : an obtainment unit that obtains a plurality of frames of first images, wherein the first images differ from each other in viewpoint (Abstract “image acquisition unit that obtains video data that includes a plurality of consecutive frames”; Image frames are taken with respect to motion found in the plurality of video image data (i.e. “frames of first images”), which necessitates the first images differing from each other in viewpoint ([0037] “motion estimation and motion compensation are performed using substantially differing images.”; Abstract “uses the video data to detect a motion vector between the super resolution images”).) ; a generation unit that generates one frame of second image having a higher resolution than the first images by combining the plurality of frames of first images ([0066]-[0067] “Super resolution processing is a method using a plurality of low resolution picture LRPs with a distortion of one pixel or less, and the pixel values for corresponding pixels are combined to re-construct the pixels…the super resolution processing circuit 306 first gathers the data for a plurality of LRP frames corresponding to the video data and generates a single high resolution picture.” Examiner interprets Super Resolution Picture (SRP), taught by Nakagami, as corresponding to “a second image”.; [0040]) , and generates [uses] a third image having a same resolution as the second image from one of the plurality of frames of first images (Nakagami teaches using a high resolution picture (HRP) that corresponds to a frame from the video data, where the HRP and SRP generated from a plurality of low resolution pictures (LRPs) are each represented as high spatial resolution images, i.e. same resolution (see [0034], [0066]-[0067]). Nakagami further teaches using the HRP and SRP in subsequent processing to generate high resolution image data, referred to as “created picture (CP)” (see [0034] “intermittently recorded high spatial resolution data (hereinafter sometimes referred to as… the ‘high resolution picture (HRP)’… that corresponds to an appropriate frame (hereinafter sometimes referred to as the ‘low resolution picture (LRP)’) included in the video data, to generate the high resolution image data (hereinafter referred to as the ‘created picture (CP)’)” and [0069]). Examiner interprets High Resolution Picture (HRP), taught by Nakagami, as corresponding to “a third image”.) and a detection unit that detects a changed area that has changed among the plurality of frames of first images (Nakagami’s teaches performing motion estimation using SRP images generated from a plurality of first frame images (i.e., LRPs), including detecting motion vectors corresponding to temporal changes among the underlying plurality of first frames (LRPs are a plurality of first frame images used to create a SRP image, as previously detailed in a previous limitation (see Abstract and [0066]-[0067]);[0036] “the magnified picture MP (t) … and the magnified picture MP (t+h)… are used to detect the motion vector (MV) between the time t and the time t+h”; Paragraph [0039] teaches the disclosed embodiment performs motion estimation using the SRPs, rather than using magnified pictures (MPs) generated through FIR filter-based magnification, stating “according to an embodiment of the present invention does not perform motion estimation using a magnified picture MP generated using a finite impulse response (FIR) filter or the like, but performs motion estimation using a super resolution picture (SRP) generated by super resolution processing.”; [0073] “the image generation processing circuit 302 performs motion estimation using the super resolution picture SRP”). Under the broadest reasonable interpretation of “changed area”, such motion estimation using first image frames (i.e., the LRPs) corresponds to detecting changed area that has changed among the plurality of frames of first images.) , wherein the generation unit further generates a fourth image, as a high-resolution image based on the plurality of frames of first images, from the second image and an area of the third image corresponding to an area of the second image corresponding to the changed area (In addition to the detected changed areas taught by Nakagami in the previous limitation, Nakagami further teaches that “The image generation processing circuit 302 performs motion compensation based on the super resolution picture SRP … and the high resolution picture HRP … and generates the created picture CP” ([0069]), and based on the detected motion information, “a high resolution picture HRP(t) reference block corresponding to alow resolution picture LRP (t) reference block (BLK) is extracted,” where the extracted reference block corresponds to a region of the HRP associated with the motion estimated LRP block, and then the “reference block pixels…are merged…to generate the created picture CP” ([0036]). Thus, Nakagami teaches generating a fourth high-resolution image using: (1) the plurality of first image frames used to generate the SRP (i.e. second image), (2) detected motion/change information in the SRP (i.e. an area of the second image corresponding to the change area), and (3) corresponding regions of the HRP associated with the detected motion information (i.e. an area of the third image corresponding to an area of the second image corresponding to the changed area). Further, Nakagami’s teaches phase correction and alignment between differing image representation ([0069]-[0072]), which reinforces that corresponding areas between the SRP derived motion information and the HRP image regions are spatially aligned for motion compensation and picture creation) . While Nakagami teaches using a third image having a same resolution as the second image from one of the plurality of frames of first images, Nakagami fails to explicitly disclose: generating a third image having a same resolution as the second image from one of the plurality of frames of first image. In the interest of clarity, Nakagami fails to teach generating a high- resolution image (i.e. the same resolution as the second image) from a low resolution first frame image, rather Nakagami’s model already has access to a high resolution third image from one of the plurality of frames of first images. In a related art, El-Khamy teaches: a method for super resolution imaging, including using a processor to generate a high-resolution image from a single lower resolution image, (Abstract; [0043] “A super resolution imaging system is an electronic system configured to generate or recover a high resolution (HR) image based on a single low resolution (LR) image. Single image super resolution refers to producing a single HR image from one LR image.”) . It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to apply El-Khamy’s method of generating a high resolution image from a single low resolution image to Nakagami’s teachings because El-Khamy’s method would enable Nakagami’s framework to generate a third image having a same resolution as the second image from one of the plurality of frames of first images, thereby relying solely on a frame of the plurality of low resolution pictures frames captured through the video data as taught by (Nakagami [0034]) . Doing so would make the imaging processing apparatus more robust by enabling the creation of an intermediate high-resolution frame for subsequent detection and final high resolution image processing, and increase the image processing apparatus’ efficiency by eliminating a step of acquiring an initial high-resolution image for the subsequent processing tasks. Both inventions lie in the field of endeavor of image analysis and processing with super resolution solutions aimed at improving image resolution. Regarding claim 10, Nakagami and El-Khamy teach: An image processing method executed by an image processing apparatus, the image processing method comprising. The remaining limitations found in claim 10 equally mirror limitation(s) taught by Nakagami, in view of El-Khamy, in claim 1. For sake of brevity, refer back to claim 1’s 103 rejection hereinabove for corresponding teachings. Thus, claim 10 is rejected based on the prior art of Nakagami and El-Khamy in claim 1 of the present office action. Accordingly, the corresponding motivation to combine, found in claim 1 rejection above, is also applicable. Regarding claim 11, Nakagami and El-Khamy teach: An image capture apparatus (see Nakagami Abstract “…an image acquisition unit that obtains video data…”) , comprising: an image sensor (Nakagami [0044]-[0045] and FIG. 3) . The remaining limitations found in claim 11 equally mirror limitation(s) taught by Nakagami, in view of El-Khamy, in claim 1. For sake of brevity, refer back to claim 1’s 103 rejection hereinabove for corresponding teachings. Thus, claim 11 is rejected based on the prior art of Nakagami and El-Khamy in claim 1 of the present office action. Accordingly, the corresponding motivation to combine, found in claim 1 rejection above, is also applicable. Regarding claim 2, Nakagami and El-Khamy teach the image processing apparatus according to claim 1, including “a detection unit that detects a changed area that has changed among the plurality of frames of first images” (refer back to claim 1 rejection above for further explanation of Nakagami’s detection unit) . Nakagami and El-Khamy fail to explicitly disclose: wherein the detection unit detects the changed area based on a difference between the second image and the third image. However, it would have been obvious to one of skill in the art, prior to the effective filing date of the claimed invention, to have modified the detection model taught by Nakagami and El-Khamy to determine a corresponding change area between Nakagami’s SRP image (i.e. second image) and HRP image (i.e. third image) because Nakagami’s detection unit already teaches (1) generating both SRP (i.e. second image) and HRP (i.e. third image) of high resolutions (see Nakagami [0034] and [0066]-[0068]) , (2) using motion estimation to identify changes in areas (see [0036]) , (3) comparing different image representations and alignment processing of areas (Nakagami [0069]-[0072]) and doing so would predictably increase the accuracy of the subsequent/corresponding change area extraction processing and final high resolution image creation. Regarding claim 3, Nakagami and El-Khamy teach the image processing apparatus according to claim 1, including “a detection unit that detects a changed area that has changed among the plurality of frames of first images”. The detection unit taught by Nakagami, and explained in greater detail in claim 1 hereinabove (refer back to claim 1’s 103 rejection above), teaches the changed area is based on changes in motion vectors using SRP images generated from the plurality of LRPs (i.e. plurality of frames of first images), including detecting motion vectors corresponding to temporal changes among the underlying plurality of first image frames ([0036]; [0039]-[0040]; [0073]) , while the SRP images are based on a plurality of first frame LPR images (Abstract; [0066]-[0067]) . Therefore, under the broadest interpretation of the claim, Nakagami’s teachings of motion estimation identification across temporal frames using an SRP based on a plurality of first frame LPR images constitutes the limitation “ wherein the detection unit detects the changed area based on differences among the plurality of frames of first images.” Regarding claim 4, Nakagami and El-Khamy teach the image processing apparatus according to claim 3. El-Kamy also teaches, as detailed in claim 1 rejection: super resolution imaging, including generating or recovering a high resolution (HR) image from lower resolution (LR) image data (Abstract; [0043] “A super resolution imaging system is an electronic system configured to generate or recover a high resolution (HR) image based on a single low resolution (LR) image. Single image super resolution refers to producing a single HR image from one LR image.”) . Under the broadest interpretation, producing an HR image from LR image data through super resolution processing reasonably constitutes enlarging the lower resolution image data into a higher resolution image representation. So while, in claims 1 and 3, Nakagami and El-Kamay teach a detection unit, a plurality of frames of first images, a second image, using a plurality of frames of first images in the detection of the changed area, (Refer back to claims 1 and 3) and El-Kamay further teaches enlarging lower resolution image data into a higher resolution image representation, Nakagami and El-Kamay fail to explicitly disclose: the detection unit enlarges each of the plurality of frames of first images to the same resolution as the second image, and uses the enlarged plurality of frames of first images in the detection of the changed area. It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention to modify Nakagami and El-Kamay’s image processing apparatus framework and detection unit to enlarge each of the plurality of frames of first images to a spatial resolution corresponding to the second image (Nakagami teaches “second image” as SRP) using the super resolution techniques taught by El-Kamay. Matching the spatial resolutions of the image data used during motion estimation, corresponding area extraction, and motion compensation (i.e. tasks associated with detection of the changed area taught by Nakagami) would have predictably improved motion estimation, change area detection, and picture creation. Regarding claim 6, Nakagami and El-Khamy teach: The image processing apparatus according to claim 1. Nakagami and El-Khamy further teach: wherein the generation unit generates the third image from the one of the plurality of frames of first image (The limitation up until this point mirrors scope of limitation found on lines 8-9 of claim 1. For sake of brevity, refer back to Nakagami and El-Khamy’s teachings in claim 1’s 103 reject of the present office action.) with use of a machine learning model that has been trained to increase a resolution of an input image and output a resultant image. Nakagami and El-Khamy fail to explicitly disclose: wherein the generation unit generates the third image from the one of the plurality of frames of first image with use of a machine learning model that has been trained to increase a resolution of an input image and output a resultant image. However, El-Khamy further teaches: using a machine learning model that has been trained to increase a resolution of an input image and output a resultant image (El-Khamy teaches super resolution imaging (Abstract) and El-Khamy’s background discussion highlights super-resolution is used to generate or recover a high resolution (HR) image from a single low resolution (LR) image, further stating “The input is a blurred or LR image. The output is a high-resolution image” ([0002]). El-Khamy’s invention aims to solve the problem of increased time required to obtain the desired quality of image through traditional processing methods ([002]) by inputting a lower resolution image then using a series of convolutional neural networks (i.e. machine learning model) to ultimately output a high-resolution image (see FIG. 2 and [0005]-[0024]). Part of El-Khamy’s method includes an intermediate HR image that is output via a display, stating “The first individual network S.sub.1 is configured to receive an LR input image 202 (from the image source device 104) and generate an intermediate quality high resolution image (e.g., as a preview image) for providing to a display device and/or the image destination device 106.” ([0056]; see FIG. 2). It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to modify the previously established super resolution imaging processing apparatus and generation unit of Nakagami and El-Khamy to incorporate the machine learning techniques of El-Khamy when generating the third image from the one of the plurality of frames of first image in order to decrease the time required to obtain the desired quality of image (see El-Khamy [0002]) , thereby increasing the image processing apparatus’ efficiency. Regarding claim 7, Nakagami and El-Khamy teach the image processing apparatus according to claim 1. Nakagami and El-Khamy further teach: wherein the generation unit generates the second image by combining the plurality of frames of first images (The limitation up until this point mirrors scope of limitation found on lines 8-9 of claim 1. For sake of brevity, refer back to Nakagami and El-Khamy’s teachings in claim 1’s 103 reject of the present office action.) with use of pieces of information related to viewpoints that have been recorded respectively in association with the plurality of frames of first images (Nakagami teaches image frames are taken with respect to motion found in the plurality of video image data (i.e. “frames of first images”), which under the broadest reasonable interpretation of the claim corresponds to the plurality of first images with use of pieces of information related to viewpoints that have been recorded respectively in association with the plurality of frames of first images (See Nakagami Abstract “uses the video data to detect a motion vector between the super resolution images” and [0037]) . Regarding claim 8, as best understood based on the 35 U.S.C. 112(b) issues identified above, Nakagami and El-Khamy teach the image processing apparatus according to claim 1. Nakagami further teaches: wherein the detection unit generates a change area map for the plurality of frames of first images, the change area map having values that differ between the changed area and an unchanged area, and the generation unit generates the fourth image from the second image and the third image based on the change area map (Refer Nakagami’s FIG. 4 below, which has been annotated by the examiner for further clarity. FIG. 4 illustrates the generation of a change area map taught by Nakagami, including (see numbers on annotated FIG. 4 corresponding to the following examiner descriptions): This row represents the plurality of frames of first images , including black blocks representing values based on motion and white blocks representing no motion (i.e. change area map having values that differ between the changed area and unchanged areas). -Second image with respect to motion- compiled from the plurality of frames of first image and their corresponding change area maps. Third image of high resolution that is based on a frame of the first images and its corresponding change area map. Fourth image that is generated using the Second and Third images, and the change area maps from the plurality of frames from first image is used to create their respective maps. Specifically, the black squares in the Second image’s map shows overlapping/combined detected change areas from the plurality of frames from the first image, while the Third image is taken from a frame of the first image (as detailed in claim 1), next the Fourth image is created by replacing the area of the Second image corresponding to the changed area with the area of the Third image corresponding thereto. PNG media_image1.png 649 1159 media_image1.png Greyscale Regarding claim 9, Nakagami and El-Khamy teach the image processing apparatus according to claim 1. Nakagami further teaches: wherein the generation unit generates the fourth image by replacing the area of the second image corresponding to the changed area with the area of the third image corresponding thereto (Refer to Nakagami’s teachings in claim 8, specifically refer to FIG. 4, annotated Fourth image, and the examiner’s corresponding annotations found above the FIG. 4 that detail the generation of the Fourth image) . Regarding claim 13, Nakagami and El-Khamy teach the image capture apparatus according to claim 11, including generating a fourth image. El-Khamy further teaches : storing an intermediary high-resolution image, which corresponds to the third image generated in claim 11, in a buffer ([0023] “instructions further cause the processor to…store the intermediate high resolution image in a second buffer”) and teaches a memory for storing data (“The main memory unit 1522 may be one or more memory chips capable of storing data.”). Nakagami further teaches: storing low image video frames (i.e. frames of the plurality of first images) and other image data in a memory unit ([0016] “a memory unit that records the video data and the image data”). Nakagami and El-Khamy fail to explicitly disclose wherein the image capture apparatus records the fourth image generated by the image processing apparatus. However, it would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the image capture apparatus taught by Nakagami and El-Khamy to apply their further teachings of storing generated image data to the fourth image in order to make the system’s storage capabilities more robust by providing future access to the fourth generated image data, thereby enabling an assortment of additional future fourth image utilities (e.g. further training of machine learning models, providing users with access to the fourth image for further analysis opportunities, further image processing applications, etc.). Claims 5 and 12 are rejected under 35 U.S.C. 103 as being obvious over Nakagami et al. (US 20090010568 A1; hereinafter “Nakagami”) in view of El-Khamy (US 20180293707 A1), and in further view of Lee et al. ("Improving SEM Image Quality Using Pixel-Super Resolution."; copy provided by Examiner). Regarding claim 5, Nakagami and El-Khamy teach the image processing apparatus according to claim 1. Nakagami further teaches: wherein the generation unit generates the second image by combining the plurality of frames of first image so as to increase pixel density . Nakagami fails to explicitly disclose: wherein the generation unit generates the second image by combining the plurality of frames of first image so as to increase pixel density . In a related art, Lee teaches: generating a high-resolution image from multiple low-resolution images using super-resolution processing, including “mapping estimated information onto the high resolution (HR) pixel grid,” and generating “the effectively reconstructed HR image that has 4 times larger pixel density of 800 × 800 compared to the LR image” (Abstract; p. 6, lines 3-4; FIG. 4). Lee further teaches using “multiple sub-pixel shifted LR images” in order to “create the HR image of uniformly spaced HR sampling points” (p. 4, lines 5-18). Under the broadest reasonable interpretation, Lee’s teachings support increasing pixel density and spatial sampling density of the high-resolution image (i.e. second image) relative to the plurality of frames of first images. It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the teachings of Nakagami and El-Khamy to incorporate the super-resolution processing techniques taught by Lee in order to provide “visibility enhancement and resolved finer details of the sample by minimizing the drift distortion” (Lee p. 7, “Conclusions” section). All three references teach super resolution techniques for improving image resolution when motion is a factor. Regarding claim 12, Nakagami and El-Khamy teach the image capture apparatus according to claim 11. Nakagami further teaches: wherein the one or more processors of the image processing apparatus further function as a control unit (Nakagami teaches the CPU acts as a “control unit and controls all or part of the operations a controller that controls the of each structural member based on the various programs” ([0110]). Nakagami’s control unit includes a CPU, and ROM and RAM for storing program to be executed by the CPU; therefore, Nakagami’s control unit corresponds to the control unit of claim12 and interpreted under 112(f) as described in paragraph [0019] of the instant application spec.) that controls a position of the image sensor that causes the position of the image sensor to be different among times of shooting of the respective plurality of frames of first images (Nakagami further teaches an image sensor that records the frames of first images ([0044]; [0034] “series of continuously recorded image data”) and an image generating timing controls the timing of the image data to be obtained from the video data (i.e. a plurality of first image frames) ([0046] “An image generation timing controller 106 is a device that allows the operator to instruct that picture data be obtained while video data is being played back.”) and further teaches processing motion estimation and motion compensation in terms temporal position ([0091]). Under the broadest reasonable interpretation, Nakagami teaches “a control unit that controls a position of the image sensor that causes the position of the image sensor to be different among times of shooting of the respective plurality of frames of first images” because the control unit taught by Nakagami executes the program provided by the CPU ([0110]) which includes controlling an image sensor for motion capture and controlling timing positional parameters for motion capture of the respective plurality of frames of first images for super resolution processing.) . Nakagami does not explicitly disclose physically repositioning the image sensor between image capturing iterations. In a related art, Lee teaches: capturing differing relative sensor image sampling based on differing positions (see FIG. 1(b) and p. 2 lines ~19-20 “multiple LR images have the different sub-pixel shifts (SPS) with respect to the first one, as shown in Fig. 1(b), each LR image can capture slightly different information of the same imaging target.”) for subsequent reconstruction of high-resolution images (p. 3 line 5 “reconstruct a HR image of x from multiple sub-pixel shifted LR images”) . It would have been obvious to a person of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the control unit taught by Nakagami to incorporate sensor positional teachings for processing of low resolution images taught by Lee in order to make the framework for capturing first frame images more robust by allowing for enhanced controls of sensors, thereby increasing the accuracy of capturing frames experiencing motion and increasing subsequent super resolution processing of low resolution images. Relevant art not relied upon 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Kajimura (US 20170024856 A1) teaches: an image processing apparatus that uses a plurality of low resolution images and identifies areas of change throughout the plurality of images for further processing and creation of a high resolution image (see Kajimura’s Abstract “An image processing apparatus including: an image obtaining unit that obtains low-resolution (LR) images acquired in time series; a position alignment unit that aligns the LR images on the basis of a movement between the LR images to generate a high-resolution (HR) image; a correlation calculating unit that calculates correlation information between areas of the LR images, the areas corresponding to partial area in the HR image;… and an image compositing unit that composites the HR image and the image to be composited according to the ratio.”) . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL DAVID BAYNES whose telephone number is (571)272-0607. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /S.D.B./ Samuel Baynes Examiner, Art Unit 2665 /Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665 Application/Control Number: 18/791,627 Page 2 Art Unit: 2665 Application/Control Number: 18/791,627 Page 3 Art Unit: 2665 Application/Control Number: 18/791,627 Page 4 Art Unit: 2665 Application/Control Number: 18/791,627 Page 5 Art Unit: 2665 Application/Control Number: 18/791,627 Page 6 Art Unit: 2665 Application/Control Number: 18/791,627 Page 7 Art Unit: 2665 Application/Control Number: 18/791,627 Page 8 Art Unit: 2665 Application/Control Number: 18/791,627 Page 9 Art Unit: 2665 Application/Control Number: 18/791,627 Page 10 Art Unit: 2665 Application/Control Number: 18/791,627 Page 11 Art Unit: 2665 Application/Control Number: 18/791,627 Page 12 Art Unit: 2665 Application/Control Number: 18/791,627 Page 13 Art Unit: 2665 Application/Control Number: 18/791,627 Page 14 Art Unit: 2665 Application/Control Number: 18/791,627 Page 15 Art Unit: 2665 Application/Control Number: 18/791,627 Page 16 Art Unit: 2665 Application/Control Number: 18/791,627 Page 17 Art Unit: 2665 Application/Control Number: 18/791,627 Page 18 Art Unit: 2665 Application/Control Number: 18/791,627 Page 19 Art Unit: 2665 Application/Control Number: 18/791,627 Page 20 Art Unit: 2665 Application/Control Number: 18/791,627 Page 21 Art Unit: 2665 Application/Control Number: 18/791,627 Page 22 Art Unit: 2665 Application/Control Number: 18/791,627 Page 23 Art Unit: 2665 Application/Control Number: 18/791,627 Page 24 Art Unit: 2665 Application/Control Number: 18/791,627 Page 25 Art Unit: 2665 Application/Control Number: 18/791,627 Page 26 Art Unit: 2665
Prosecution Insights