DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant(s) Response to Official Action
The response filed on 6/17/2026 has been entered and made of record.
Response to Arguments/Amendments
No argument was found addressing the prior reference rejections of claims 1-7. A voicemail was left on 8/05/2026 to get clarification. No response was received.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 are rejected under 35 U.S.C. 103 as being unpatentable over Fan Chen et al. [US 20190026864 A1: already of record] in view of Matteo Tiezzi et al. [Foveated Neural Computation: already of record].
Regarding claim 1, Fan teaches:
1. A method (i.e. An embodiment of a semiconductor package apparatus may include technology to identify a region of interest portion of a first image, and render the region of interest portion with super-resolution. Other embodiments are disclosed and claimed- Abstract), comprising:
defining a plurality of regions of pixels (i.e. a camera may capture an image of a pupil and the system may determine where the user is looking (e.g., a focus area, depth, and/or direction). The camera may capture pupil dilation information and the system may infer where the user's focus area is based on that information- ¶0030 …One problem in foveated rendering is producing a smooth transition between the central vision with high pixel details (e.g., near the ROI) and the peripheral vision with less pixel details- ¶0050) in an image sensing pixel array (i.e. Turning now to FIGS. 11A to 11F, embodiments of regions of interest for foveated encoding may be represented by any of a variety of different shapes and sizes. An image area 110 may generally have a rectangular shape or a square shape. A focus area 112 may have any suitable shape such as circular (e.g., FIG. 11A), elliptical (e.g., FIGS. 11B and 11D), square or rectangular (e.g., FIG. 11E), a point (e.g., FIG. 11C), or arbitrary (e.g., FIG. 11F). A ROI 114 may have any suitable shape such as a square (e.g., FIGS. 11A, 11C, and 11E) or a rectangle (e.g., FIGS. 11B, 11D, and 11F). The size of the ROI 114 may be fixed or may be adjusted based on the size of the focus area 112. For example, the size of the ROI 114 may correspond to the size of the focus area 112 plus some delta X and delta Y (e.g., which may be different from each other). The ROI 114 may generally be bigger than the focus area 112 to provide a smooth transition between the relatively higher quality central vision region and the relatively lower quality peripheral vision region- ¶0058);
associating a plurality of filtering configurations with the plurality of regions respectively (i.e. The size of the ROI 114 may be fixed or may be adjusted based on the size of the focus area 112. For example, the size of the ROI 114 may correspond to the size of the focus area 112 plus some delta X and delta Y (e.g., which may be different from each other). The ROI 114 may generally be bigger than the focus area 112 to provide a smooth transition between the relatively higher quality central vision region and the relatively lower quality peripheral vision region- ¶0058);
generating, using the image sensing pixel array, image data representative of an image of a scene (i.e. a camera may capture an image of a pupil and the system may determine where the user is looking (e.g., a focus area, depth, and/or direction). The camera may capture pupil dilation information and the system may infer where the user's focus area is based on that information- ¶0030);
selecting first input data generated for the image by a first block of pixels in the image sensing pixel array located in a first region among the plurality of regions (i.e. For each input image to be rendered at block 101, an image for the central vision may be extracted at block 102 and provided to the trained super-resolution network at block 103- ¶0055);
performing, using a multiplier-accumulator unit, a dot product between the first weight matrix and the first image data to obtain first feature data representative of the first image data being filtered via a first kernel of a convolutional neural network (i.e. Some embodiments may provide a super-resolution technique/framework that may benefit from the foveated characteristic of the human vision. For example, some embodiments may train a super-resolution neural network with synthesized foveated rendering images, which may have high resolution details from super-resolution as well as a smooth transition from high-quality pixels to medium quality peripheral pixels. When the resulting foveated image is rendered on a VR display, for example, some embodiments may achieve less-visible boundaries between the central vision region and the peripheral vision regions to reduce or avoid artifact that may distract the viewer. An example of a suitable super-resolution neural network may include a convolutional neural network (CNN) such as super-resolution CNN (SRCNN) or a fast CNN such as a fast super-resolution CNN (FSRCNN)- ¶0053).
However, Fan does not teach explicitly:
a plurality of feature extraction filtering configurations… according to the feature extraction filtering configurations; identifying a first weight matrix associated with the first region in the plurality of feature extraction filtering configurations.
In the same field of endeavor, Matteo teaches:
a plurality of feature extraction filtering configurations (i.e. Fig. 1: Top-left: out-of-the-box FCLs. Bottom-left: example of R = 4 regions in the piecewise-defined kernel case, when the attention a is given. Right: three strategies (one-per-column) to implement a piecewise-defined kernel, with examples of spatial coverage of the 4 region-wise kernels (coordinates not covered due to dilation are blank). We report right after the strategy name further operations needed to fulfil the uniform spatial coverage assumption- page 5… This implies that all the region-defined filters share related semantics across the image plane, due to the shared nature of the learnable kj . A natural alternative to this model consists in using independent learnable filters in each region- page 7, ¶2) according to the feature extraction filtering configurations (i.e. In this paper we propose to go beyond such a scheme, introducing the notion of Foveated Convolutional Layer (FCL), that formalizes the idea of location-dependent convolutions with foveated processing, i.e., fine-grained processing in a given-focused area and coarser processing in the peripheral regions); identifying a first weight matrix associated with the first region in the plurality of filtering configurations (i.e. For each kernel kj :R2 → R, j = 1, . . . , F, the convolution between I and kj is defined as equation 1- page 4, ¶1).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan with the teachings of Matteo to efficiently handle the information in the peripheral regions, eventually avoiding the development of misleading biases (Matteo- Abstract).
Regarding claim 2, Fan and Matteo teach all the limitations of claim 1.
However, Fan does not teach explicitly:
wherein the plurality of filtering configurations identify a plurality of kernels of different kernel sizes for the plurality of regions respectively.
In the same field of endeavor, Matteo teaches:
wherein the plurality of filtering configurations identify a plurality of kernels of different kernel sizes for the plurality of regions respectively (i.e. The first two ones are based on the fact that the cost of convolution is directly proportional to the number of spatial components of the kernel, thus the computational burden can be controlled by reducing the size of the kernel defined in each region in function of ri. However, a smaller kernel size implies covering smaller receptive inputs, thus violating the previously introduced uniform spatial coverage assumption- page 6, ¶3)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan and Matteo with the teachings of Paolo allow the keypoints to be uniquely identified in each image (¶0077- Paolo).
Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Fan Chen et al. [US 20190026864 A1: already of record] in view of Matteo Tiezzi et al. [Foveated Neural Computation: already of record] and further in view of Paolo Di Febbo et al. [US 20180268256 A1: already of record].
Regarding claim 3, Fan and Matteo teach all the limitations of claim 2.
However, Fan and Matteo do not teach explicitly:
further comprising: selecting, according to the filtering configurations, second image data generated for the image by a second block of pixels in the image sensing pixel array located in a second region, different from the first region, among the plurality of regions; identifying a second weight matrix associated with the second region in the plurality of filtering configurations; and performing a dot product between the second weight matrix and the second image data to obtain second feature data representative of the second image data being filtered by a second kernel.
In the same field of endeavor, Paolo teaches:
further comprising: selecting, according to the filtering configurations, second image data generated for the image by a second block of pixels in the image sensing pixel array located in a second region, different from the first region, among the plurality of regions; identifying a second weight matrix associated with the second region in the plurality of filtering configurations; and performing a dot product between the second weight matrix and the second image data to obtain second feature data representative of the second image data being filtered by a second kernel (i.e. Because aspects of embodiments of the present invention are implemented using a convolutional neural network, and because the CNN processes each patch (having dimensions equal in size to the convolutional kernel, w×w) of its input image independently, the training of the CNN can be performed using patches that are selected from the response map and corresponding patches of the training images- ¶0129).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan and Matteo with the teachings of Paolo allow the keypoints to be uniquely identified in each image (¶0077- Paolo).
Regarding claim 4, Fan, Matteo and Paolo teach all the limitations of claim 3 and Fan further teaches:
wherein the first region is configured to capture a central region of the image; the second region is configured to capture a peripheral region of the image; and the second block of pixels has a size larger than the first block of pixel (i.e. see figs. 11).
Regarding claim 5, Fan, Matteo and Paolo teach all the limitations of claim 4.
However, Fan and Matteo do not teach explicitly:
wherein the central region of the image filtered using the first kernel but not the second kernel; and the peripheral region is filtered using the second kernel but not the first kernel.
In the same field of endeavor, Paolo teaches:
wherein the central region of the image filtered using the first kernel but not the second kernel; and the peripheral region is filtered using the second kernel but not the first kernel (i.e. Because aspects of embodiments of the present invention are implemented using a convolutional neural network, and because the CNN processes each patch (having dimensions equal in size to the convolutional kernel, w×w) of its input image independently, the training of the CNN can be performed using patches that are selected from the response map and corresponding patches of the training images- ¶0129).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan and Matteo with the teachings of Paolo allow the keypoints to be uniquely identified in each image (¶0077- Paolo).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Fan Chen et al. [US 20190026864 A1: already of record] in view of Matteo Tiezzi et al. [Foveated Neural Computation: already of record] further in view of Paolo Di Febbo et al. [US 20180268256 A1: already of record] and even further in view of Liao Jiping et al. [US 20220207764 A1: already of record].
Regarding claim 6, Fan, Matteo and Paolo teach all the limitations of claim 4.
However, Fan and Matteo do not teach explicitly:
wherein the plurality of filtering configurations further identify a plurality of stride lengths for filtering within the plurality of regions respectively; and the method further comprises: filtering the first region according to a first stride length; filtering the second region according to a second stride length larger than the first stride length.
In the same field of endeavor, Liao teaches:
wherein the plurality of filtering configurations further identify a plurality of stride lengths for filtering within the plurality of regions respectively; and the method further comprises: filtering the first region according to a first stride length; filtering the second region according to a second stride length larger than the first stride length (i.e. The convolution operator is also referred to as a kernel. In image processing, the convolution operator functions as a filter that extracts specific information from a matrix of an input image. The convolution operator may essentially be a weight matrix, and the weight matrix is usually predefined. In a process of performing a convolution operation on an image, the weight matrix usually processes pixels at a granularity level of one pixel (or two pixels, depending on a value of a stride) in a horizontal direction in the input image, to extract a specific feature from the image. A size of the weight matrix is related to a size of the image- ¶0094).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan and Matteo with the teachings of Liao to improve definition of the extended depth of field image (¶0017- Liao).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Fan Chen et al. [US 20190026864 A1: already of record] in view of Matteo Tiezzi et al. [Foveated Neural Computation: already of record] further in view of Paolo Di Febbo et al. [US 20180268256 A1: already of record] and even further in view of Liao Jiping et al. [US 20220207764 A1: already of record] and Sanghoon Lee et al. [Foveated Video Compression with Optimal Rate Control].
Regarding claim 7, Fan, Matteo, Paolo and Liao teach all the limitations of claim 6.
However, Fan, Matteo, Paolo and Liao do not teach explicitly:
wherein the plurality of filtering configurations further identify a plurality of quantization levels for filtering within the plurality of regions respectively; and the method further comprises: quantizing the first image data at a first precision level as an input to the multiplier-accumulator unit; and quantizing the second image data at a second precision level, lower than the first precision level.
In the same field of endeavor, Sanghoon teaches:
wherein the plurality of filtering configurations further identify a plurality of quantization levels for filtering within the plurality of regions respectively; and the method further comprises: quantizing the first image data at a first precision level as an input to the multiplier-accumulator unit; and quantizing the second image data at a second precision level, lower than the first precision level (i.e. The other is nonuniform quantization which maximizes the FSNR subject to a rate constraint over the curvilinear coordinates- page 982, ¶7… The area in Fig. 2(d) becomes Ac which is unchanged near the center of the foveation point and decreases from the foveation point toward the periphery relative to the area A0 in fig. 2b- page 981, ¶5… The QPs for coding macroblocks consist of a quantization state ={q1, q2,…, qM} vector
Q
-
={q1, q2,…, qM}- page 983, ¶1).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Fan, Matteo, Paolo and Liao with the teachings of Sanghoon to deliver high-quality video at reduced bit rates by seeking to match the nonuniform sampling of the human retina (Sanghoon - Abstract).
Allowable Subject Matter
Claims 8-12, 15-19 and 21-24 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CLIFFORD HILAIRE whose telephone number is (571)272-8397. The examiner can normally be reached 5:30-1400.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SATH V PERUNGAVOOR can be reached at (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CLIFFORD HILAIRE
Primary Examiner
Art Unit 2488
/CLIFFORD HILAIRE/Primary Examiner, Art Unit 2488
Prosecution Insights