CTNF 18/801,852 CTNF 87845 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 12-151 AIA 26-51 12-51 Status of Claims Claims 1- 20 are pending and are examined on the merits. Priority Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Republic of Korea on 09/15/2023. It is noted, however, that applicant has not filed a certified copy of the 10-2023-0123383 application as required by 37 CFR 1.55. The request for the USPTO to retrieve priority documents, filed on 08/13/2024, was unsuccessful on 2/15/2025. The examiner called the EBC customer support center number and requested the document tried to be retrieved again. The ticket number associated with the request is 2-00987328. As of the date of posting of this Office action, no priority documents have been received. Information Disclosure Statement The IDS(s) filed August 13, 2024 has been considered. An initialed copy of the form accompanies this Office action. 07-30-03-h AIA CLAIM INTERPRETATION 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "a directional component extractor configured to extract directional components of a target kernel..." recited in claims 1, 6-8, and 11; "an interpolation kernel determiner configured to determine an interpolation kernel based on the directional components" (claims 1 and 2) and the parallel determiner limitation in claim 11; and "a pixel interpolator configured to interpolate the target pixel using data included in the interpolation kernel" recited in claims 1, 10, and 11. The corresponding structure for each function is the algorithm disclosed in the specification, executed by the processor 1010 of computing device 1000 (Spec ¶¶109-116; FIG. 10): Extractor 121 – convolution between target kernel and each of a plurality of directional kernels ¶¶38, 46, 56-59, 107; FIGs. 2, 4A-4B; Determiner 122 – point-wise convolution between cost kernels and directional components, plus Arg max/Arg min selection (¶¶39-40, 47-48, 61-73, 91; FIGs. 5A- 5C); and Interpolator 123 – convolution between target kernel and selected interpolation kernel (¶¶41, 49, 75-91; FIGs. 6A-6B). Corresponding structure is sufficient under § 112(b) and § 112(a). Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claim 4, 13, and 20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 4 recites "the directional components are matched to channel," claim 13 recites the same, and claim 20 recites "matching the first directional components to channel." “Channel" lacks antecedent basis, and the singular form is inconsistent with the plurality of directional components recited earlier in each claim. For purposes of compact prosecution, the claims are interpreted as if reciting "matched to channels," i.e., each directional component is assigned to a respective channel of a multi-channel input. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim (s) 1, 9, 10, and 15 are is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2018/0232853 A1, hereinafter "Kim") . Claims 1 and 15. Kim teaches an image signal processor (Kim, Abstract, ¶42: "the image conversion device 100 … includes … a processing unit 120 … ") comprising: a directional component extractor configured to extract directional components of a target kernel by performing a convolution operation between the target kernel including a target pixel and each of a plurality of directional kernels (Kim, ¶48: "The processing unit 120 may use an edge filter including a vertical pixel conversion matrix and a horizontal pixel conversion matrix to determine the direction categories of the pixel groups."; ¶61: "the processing unit 120 can determine direction categories of edge direction components of the pixel groups by using the vertical pixel conversion matrix V and the horizontal pixel conversion matrix H while setting five direction categories."; ¶35 and FIG. 5. Each pixel group P (e.g., the 2x2 matrix [[P11,P12] ,[P21 ,P22]] in Eq. 1) reads on the target kernel; the V and H matrices read on the plurality of directional kernels; P ∘ V and P ∘ H are the convolution and the directional components, ¶¶64-65. Applicant’s specification at ¶56 defines the “convolution operation” as the element-wise multiply-and-sum of a same-size kernel against a same-size pixel neighborhood, with the worked example “0x1+1x1+0x0+4x1+0x0+0x0+3x0+0x0=5” producing a single scalar value. This is the same operation Kim performs in Eq. 1 (¶64) where P ∘ V and P ∘ H each compute the same element-wise multiply-and-sum between a same size pixel group and a same-size matrix. Whether labeled “convolution” (Adachi), “Frobenius inner product” (mathematics), or “ P ∘ V ” (Kim), the operation is identical, and the BRI of the “convolution operation” in light of the spec covers it.) ; an interpolation kernel determiner configured to determine an interpolation kernel based on the directional components (Kim, ¶69: "the direction category is determined to be the horizontal category 401 and the index is set to '1' ... the first diagonal category 403 and the index is set to '2' ... the vertical category 402 and the index is set to ' 3' ... the second diagonal category 404 and the index is set to '4'."; ¶72: "the processing unit 120 generates a high-resolution image ... by converting the plurality of low-resolution image patches by using a conversion kernel for each image patch corresponding to its image category among a plurality of prestored conversion matrices.") ; and a pixel interpolator configured to interpolate the target pixel using data included in the interpolation kernel (Kim, ¶73: "FIG. 7 shows the case of generating 2x2 polygonal high-resolution image patches by converting the 3x3 polygonal low-resolution image patches 501 using the kth conversion kernel Mk corresponding to the kth image category."; ¶78) . Examiner notes that Kim describes it’s kernel-based pixel-value generation as “conversion” rather than “interpolation,” and that Kim ¶27 characterizes Kim’s approach as operating “without an interpolation process” relative to a particular two-stage upsampling then quality improvement pipeline. Notwithstanding Kim’s choice of terminology and Kim’s disclaimer of a specific two-stage pipeline, Kim’s category-specific conversion-kernel application – using neighboring pixel information from the low-resolution patch (¶73, FIG. 7) to generate pixel values at higher-resolution grid positions that did not exist in the input (¶80, FIGs 8A-8C) – is a form of pixel-value interpolation under Applicant’s own definition of interpolation, see Spec ¶4 (“an operation of interpolating pixels having lost (or missed) information using information of the neighboring pixels”). Under the broadest reasonable interpretation in light of the Applicant’s specification, “interpolating the target pixel suing data included in the interpolation kernel” encompasses Kim’s conversion-kernel application. Accordingly, Kim teaches the limitation, and the rejection is presented under § 103 to acknowledge the terminology mismatch. Claim 9. Kim teaches the image signal processor according to claim 1, wherein the target kernel has the same size as the largest directional kernel from among the plurality of directional kernels (Kim’s pixel group P (¶64, Eq. 1) is the same size as the V and H matrices (FIG. 5).) . Claim 10. Kim teaches the image signal processor according to claim 1, wherein the pixel interpolator is configured to: match, to the target pixel, data located at a center of the interpolation kernel (Kim¶ 73, FIG. 7 – the kth conversion kernel Mk is applied to the 3x3 patch 501 centered on the target pixel.) ; and determine, as pixel data of the target pixel, a convolution resultant value between the target kernel and the interpolation kernel (Kim ¶¶ 73-74, and 78: "the processing unit 120 can calculate the pixel values of the high-resolution image patches by reading out the conversion kernels corresponding to the image categories from the external memory and performing the multiplying operation" – the high-resolution pixel value is computed as a convolution resultant value between the target-centered patch and Mk.) . 07-21-aia AIA Claim (s) 2-5, 11-14, and 18-20 are is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Kim et al. (US 2021/0065334 A1, hereinafter "Kim '334") . Claim 11. An image signal processor comprising: a directional component extractor configured to extract directional components of a target kernel by performing a first convolution operation between the target kernel and each of a plurality of first kernels, the target kernel including a target pixel (Kim ¶¶48, 61, 66-67, FIG. 5 – V and H are the plurality of first kernels; P ∘ V / P ∘ H is the first convolution operation; the V/H scalar outputs are the directional components.) ; an interpolation kernel determiner configured to obtain convolution resultant values by performing a second convolution operation between each of a plurality of second kernels and the directional components, and determine an interpolation kernel based on the convolution resultant values; and a pixel interpolator configured to interpolate the target pixel using data included in the interpolation kernel (Kim ¶¶73, 78, 80, FIGs. 8A-8C – same as claim 1; the disclaimer-rebuttal analysis at the claim 1 rejection above applies equally here.) . Kim does not teach the strikethrough portion; however, Kim '334 teaches "obtain convolution resultant values by performing a second convolution operation between each of a plurality of second kernels and the directional components, and determine an interpolation kernel based on the convolution resultant values" (Kim '334 ¶20: "The first convolution operator may include a depth-wise convolution operator and a point-wise convolution operator connected directly to the depth-wise convolution operator"; ¶142: "The PW convolution operator 223 may perform a convolution operation for each point. The PW convolution operator 223 may perform a PW convolution operation one or more times" – point-wise convolution applies a plurality of unit (1x1) kernels across channels of multi-channel feature data.) It would have been obvious to a POSITA before the effective filing date to perform Kim’s direction category selection using the point-wise convolution operation of Kim ‘334 – a known, hardware-efficient building block for combining multi-channel feature data – rather than Kim’s scalar arithmetic. Both references are overlapping KAIST inventors (Mun Churl Kim) directed to fixed coefficient kernel image processing, and a PHOSITA implementing Kim’s pipeline in a CNN architecture as taught by Kim ‘334 would have substitutes the disclosed point-wise operator for the scalar magnitude (m) and angle (d) computation (¶¶64-65, Eqs. 1-4). Claims 2 and 18. Kim in view of Kim '334 discloses the image signal processor according to claim 1, wherein the interpolation kernel determiner is configured to obtain convolution resultant values by performing a convolution operation between each of a plurality of cost kernels and the directional components, and determine the interpolation kernel based on the convolution resultant values Kim does not teach this; however Kim ‘334 teaches, "performing a convolution operation between each of a plurality of cost kernels and the directional components " (Kim '334 ¶¶20, 142 – pointwise convolution operator applying unit (1x1) kernels across channels of multi-channel feature data). Same rationale as claim 11 above. Claims 3 and 19. Kim in view of Kim '334 discloses the image signal processor according to claim 2, wherein each of the plurality of cost kernels includes a preset weight (Kim '334 ¶¶20, 142, FIG. 7- the point-wise operator 223 applies fixed (preset) per-channel weights. Kim independently uses fixed coefficients in V and H, Kim ¶64, Eq. 1) . Claims 4, 13 and 20. Kim in view of Kim '3 34 discloses the image signal processor according to claim 2, wherein the convolution resultant values include result values obtained by performing a point-wise convolution operation in which the directional components are matched to channel and each of the plurality of cost kernels is matched to a unit kernel (Kim '334 ¶20: "The first convolution operator may include a depth-wise convolution operator and a point-wise convolution operator connected directly to the depth-wise convolution operator"; ¶142: "The PW convolution operator 223 may perform a convolution operation for each point." The directional components from claim 1’s extraction step are the input channels to Kim ‘334’s PW operator (“directional components matched to channel”), and each cost kernel of the claim is implemented as a 1x1 weight vector applied by the PW operator (“cost kernels matched to a unit kernel”). Kim ‘334 FIG. 7 illustrates the PW operator 223 performing this multi-channel to point operation) . Claims 5 and 14. The image signal processor according to claim 2, wherein the interpolation kernel includes an interpolation kernel corresponding to a cost kernel that derives a maximum or minimum value from among the convolution resultant values (Kim ¶63: "when the magnitude value is greater than or equal to the threshold value, the direction category may be determined among the horizontal category 401, the vertical category 402, the first diagonal category 403, and the second diagonal category 404 depending on the angle value" – Kim selects the direction-category corresponding to the maximum-magnitude direction; in the combined system, this reads on the max/min selection from the per-channel point-wise outputs.) . Claim 12. Kim in view of Kim '334 discloses the image signal processor of claim 11, wherein the first convolution operation is performed in a state in which the target pixel is matched to data located at a center of each of the plurality of first kernels. (Kim ¶66, Eq. 1, FIG. 5 – Kim builds each pixel group P around a target pixel and applies V/H with the target pixel centered.) . 07-21-aia AIA Claim s 6-8, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2018/0232853 A1 – "Kim") in view of Hamilton et al. (US 5,629,734 – "Hamilton"). Claim 6. Kim in view of Hamilton discloses the image signal processor according to claim 1, wherein the directional component extractor is configured to extract the directional components by using pixel data of each pixel having a first color filter from among pixels included in the target kernel . Kim does not teach the strikethrough portion. However, Hamilton teaches "extract the directional components by using pixel data of each pixel having a first color filter from among pixels included in the target kernel" (Hamilton, Abstract: image sensor with "only one color value for each photosite location"; col. 2: "The object of this invention is to provide an improved apparatus for estimating the missing pixel values in a CFA"; col. 4, Block 5O, FIG. 3: "the absolute value of the Laplacian of the green plane is added to the absolute value of the gradient of the red or blue plane, depending on which was the corresponding color in the Bayer color filter array" – the Laplacian-of-green operation uses pixel data only of green-filter pixels.) It would have been obvious to a POSIT A to apply Kim's direction-category extraction to a Bayer-CFA image of the kind processed by Hamilton, using only first-color pixel data per Hamilton's plane-separation framework. The combination yields the predictable improvement in directional resolution at each color plane that Hamilton's framework validates as accurate per-color interpolation. Claim 7. Kim in view of Hamilton discloses the image signal processor according to claim 6, wherein the directional component extractor is configured to determine, as a particular value, pixel data of each pixel having a color filter different from the first color filter from among pixels included in the target kernel (Hamilton, col. 4, Block 5O, FIG. 3 – non-green pixels are excluded from the green-plane Laplacian; setting non-first-color pixel data to zero so they contribute nothing to the green-plane sum is the conventional implementation. Spec ¶107 expressly defines the "particular value" as zero.) . Claim 8. Kim in view of Hamilton discloses the image signal processor according to claim 6, wherein the directional component extractor is configured to determine, as a particular value, data corresponding to a position of each pixel having a color filter different from the first color filter from among data included in the plurality of directional kernels (Hamilton, col. 4, Block 5O, FIG. 3 – zeroing the directional-kernel coefficients at non-first-color positions is the algebraically equivalent implementation of Hamilton's plane-separated Laplacian; zeroing the input pixel data and zeroing the kernel coefficients produce the same convolution sum.) . Claim 16. Kim in view of Hamilton discloses the image signal processing method according to claim 15, further comprising: generating an interpolated target kernel by mixing an interpolated first color kernel with a second color kernel, the interpolated first color kernel including an interpolated first target pixel (Hamilton, col. 4-5, Blocks 44 and 46, FIGs. 4-5: "The green (luma) interpolation step (Block 44) has two parts . ... The first part (Block 80) averages the two luminance (green) values according to the selected interpolation method. The second part (Block 82) adds a correction factor based on either red or blue neighboring values"; "The red/blue (chroma) interpolation proceeds in a manner similar to the green (luma) interpolation described above" – the interpolated green image (interpolated first color kernel) mixed with the raw red/blue image (second color kernel) reads on the claimed mixed interpolated target kernel.) . It would have been obvious to a POSITA to substitute Kim's five-category directional classifier into the green/luma stage of Hamilton's established two-stage pipeline (and again into the chroma stage), preserving Hamilton's green-then-chroma operative principle while providing improved directional resolution at each stage. Claim 17. The image signal processing method according to claim 16, further comprising: extracting second directional components of the interpolated target kernel by performing a convolution operation between each of the directional kernels and the interpolated target kernel including a second target pixel; determining a second interpolation kernel based on the second directional components; and interpolating the second target pixel using data included in the second interpolation kernel (Hamilton, col. 4-5, Block 46 description, FIG. 5 – Hamilton's chroma interpolation step parallels the green/luma step on the green plus partial chroma intermediate image, computing classifier values, selecting the interpolation method corresponding to the smaller value, and interpolating the missing chroma value. Performing Kim's direction-category extraction and conversion-kernel interpolation as the second stage of Hamilton's pipeline reads on the limitation. Same rationale as claim 16.) . Conclusion The prior art made of record but not relied, yet considered pertinent to the applicant’s disclosure, is listed on the PTO-892 form. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Ross Varndell/Primary Examiner, Art Unit 2674 Application/Control Number: 18/801,852 Page 2 Art Unit: 2674 Application/Control Number: 18/801,852 Page 3 Art Unit: 2674 Application/Control Number: 18/801,852 Page 4 Art Unit: 2674 Application/Control Number: 18/801,852 Page 5 Art Unit: 2674 Application/Control Number: 18/801,852 Page 6 Art Unit: 2674 Application/Control Number: 18/801,852 Page 7 Art Unit: 2674 Application/Control Number: 18/801,852 Page 8 Art Unit: 2674 Application/Control Number: 18/801,852 Page 9 Art Unit: 2674 Application/Control Number: 18/801,852 Page 10 Art Unit: 2674 Application/Control Number: 18/801,852 Page 11 Art Unit: 2674 Application/Control Number: 18/801,852 Page 12 Art Unit: 2674 Application/Control Number: 18/801,852 Page 13 Art Unit: 2674 Application/Control Number: 18/801,852 Page 14 Art Unit: 2674 Application/Control Number: 18/801,852 Page 15 Art Unit: 2674 Application/Control Number: 18/801,852 Page 16 Art Unit: 2674 Application/Control Number: 18/801,852 Page 17 Art Unit: 2674 Application/Control Number: 18/801,852 Page 18 Art Unit: 2674
Prosecution Insights