DETAILED ACTION
This Action is in response to Applicant’s response filed on 04/06/2026. Claims 1-30 are still pending in the present application. This Action is made FINAL.
Response to Arguments
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection in view of Wang (CN111340696A).
Claim Objection
Correct “section region” in claims 17, 18, 21 and 22 to --second region--.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5 and 17-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US 2019/0026864 A1) in view of Wang (CN111340696A).
Regarding claims 1 and 23, Chen discloses an apparatus for processing image data, the apparatus comprising:
a memory; and (figure 5, memory 52)
one or more processors coupled to the memory, the one or more processors configured to: (figure 4, processor 51)
analyze an input image to determine that the input image includes a representation of an object, (paragraph 36; recognize objects in a scene) the input image including a first region and a second region, the first region and the second region having a first resolution; (paragraph 40; a resolution of the first image may be lower; medium quality image; paragraphs 40, 45-46 and 55: Chen discloses a first image having a ROI portion and other portions outside the ROI)
output a modified first region of the input image, the modified first region generated based on the first region using a first process and having a second resolution; (paragraphs 19, 52 and 55; Chen discloses extracting the ROI portion of the input image, providing it to a trained super-resolution network and outputting an enhanced central vision region with super-resolution.)
output a modified second region of the input image, the modified second region generated based on the second region using a second process and having the second resolution, wherein the second process is different from the first process. (paragraphs 19, 45 and 55; Chen discloses applying bi-cubic technique to generate a high resolution up-sampled image for the remaining peripheral regions outside the centra-vision region and fusing it with the super-resolution output.)
Chen fails to specifically disclose determining that the first region of the input image is more salient than the second region of the input image based on the first region including at least a portion of the representation of the object.
In related art, Wang discloses determining that the first region of the input image is more salient than the second region of the input image based on the first region including at least a portion of the representation of the object. (paragraph 19; A saliency detection method simulating the human visual attention mechanism is used to detect salient regions in remote sensing images. paragraph 52; For salient regions, an image super-resolution reconstruction method based on convolutional neural networks is used to perform super-resolution reconstruction. paragraph 77; For non-saliency regions, bicubic interpolation is used for super-resolution reconstruction.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Wang into the teachings of Chen to effectively remove image content that is not relevant and improve algorithm efficiency.
Regarding claims 2 and 24, Chen, as modified by Wang, discloses the claimed invention wherein the first process includes a super resolution process performed using a trained network. (paragraphs 40, 53 and 55)
Regarding claim 3, Chen, as modified by Wang, discloses the claimed invention wherein the trained network includes one or more trained convolutional neural networks. (paragraph 53: SRCNN, FSRCNN)
Regarding claims 4 and 25, Chen, as modified by Wang, discloses the claimed invention wherein the second process is an interpolation process. (paragraph 55: bi-cubic)
Regarding claim 5, Chen, as modified by Wang, discloses the claimed invention wherein the interpolation process includes at least one of nearest neighbor interpolation, linear interpolation, bilinear interpolation, trilinear interpolation, cubic interpolation, bicubic interpolation, tricubic interpolation, spline interpolation, lanczos interpolation, sine interpolation, Fourier-based interpolation, and edge-directed interpolation. (paragraph 55: bi-cubic)
Regarding claim 17, Chen, as modified by Wang, discloses the claimed invention wherein the second resolution is based on a resolution of a display, and wherein, to output the modified first region and to output the modified second region, the one or more processors are configured to display the modified first region and the modified second region on the display. (paragraphs 46 and 52: display panel 88)
Regarding claim 18, Chen, as modified by Wang, discloses the claimed invention wherein a display, wherein to output the modified first region and to output the modified second region, the one or more processors are configured to cause the modified first region and the modified second region to be displayed on the display. (paragraphs 46 and 52: display panel 88)
Regarding claim 19, Chen, as modified by Wang, discloses the claimed invention wherein an image sensor configured to capture the input image. (paragraph 24)
Regarding claim 20, Chen, as modified by Wang, discloses the claimed invention wherein the one or more processors are configured to: receive at least one user input; and modify at least one of the first region and the second region based on the at least one user input. (paragraphs 24-25, 28-31 and 56-60: Chen discloses user inputs used by the sensing pipeline including touch data and focus selection based on such information.)
Regarding claim 21, Chen, as modified by Wang, discloses the claimed invention wherein output a sequence of video frames, wherein the sequence of video frames includes a video frame that includes the modified first region and the modified second region. (paragraphs 31, 35 and 37: next frame/scene)
Regarding claim 22, Chen, as modified by Wang, discloses the claimed invention wherein display the modified first region and the modified second region in a preview stream. (paragraph 52)
Claims 6-7 and 26-27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Wang and in further view of Krahenbuhl (US 2014/0063275 A1).
Regarding claims 6 and 26, Chen, as modified by Wang, discloses the claimed invention except for wherein the one or more processors are configured to determine the first region of the input image is more salient than the second region of the input image based on a saliency map, the saliency map including one or more saliency values identifying the first region as more salient than the second region.
In related art, Krahenbuhl discloses determining the first region of the input image is more salient than the second region of the input image based on a saliency map, the saliency map including one or more saliency values identifying the first region as more salient than the second region. (abstract, paragraphs 15, 17, 23 and 48: Krahenbuhl discloses visual saliency estimation for images and video decomposing an image into elements and a per element saliency measure and producing a pixel-accurate saliency map)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Krahenbuhl into the teachings of Chen and Wang to effectively identify visually important regions for preferential processing.
Regarding claims 7 and 27, Chen, as modified by Wang and Krahenbuhl, discloses the claimed invention wherein generating the saliency map based on the input image at least in part by generating a respective saliency value of the one or more saliency values for each pixel of the input image. (Krahenbuhl: abstract, paragraphs 15, 17, 23 and 48)
Claim 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Wang and Krahenbuhl and in further view of Shaw (US 2017/0310979 A1).
Regarding claim 8, Chen, as modified by Wang and Krahenbuhl, discloses the claimed invention except for wherein a saliency value of the saliency map for a pixel of a plurality of pixels is based on a distance between the pixel and other pixels of the plurality of pixels.
In related art, Shaw discloses a saliency value of the saliency map for a pixel of a plurality of pixels is based on a distance between the pixel and other pixels of the plurality of pixels. (paragraph 26)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Shaw into the teachings of Chen, Wang to Krahenbuhl to effectively generate a saliency map that indicates the relative importance of each pixel in the corresponding frame based on its perceptual significance.
Claims 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Wang and Krahenbuhl and in further view of Jetley (US 2017/0308770 A1).
Regarding claim 9, Chen, as modified by Wang and Krahenbuhl, discloses the claimed invention except for wherein, to generate the saliency map, the one or more processors are configured to: apply an additional trained network to the input image.
In related art, Jetley discloses to generate the saliency map, the one or more processors are configured to: apply an additional trained network to the input image. (abstract)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Jetley into the teachings of Chen and Wang to Krahenbuhl to effectively predict saliency in an image.
Regarding claim 10, Chen, as modified by Wang, Krahenbuhl and Jetley, discloses the claimed invention wherein the additional trained network includes one or more trained convolutional neural networks. (Jetley: abstract)
Claims 11-16 and 28-30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Wang, Krahenbuhl, Jetley and in further view of Zhang (US 2017/0339431 A1).
Regarding claims 11 and 28, Chen, as modified by Wang, Krahenbuhl and Jetley, discloses the claimed invention except for wherein the one or more processors are configured to: partition the input image into a plurality of blocks.
In related art, Zhang partitioning the input image into a plurality of blocks. (paragraphs 24 and 52)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Zhang into the teachings of Chen, Wang, Krahenbuhl and Jetley to improve the functionality of the video processing of devices and systems executing the methods by providing higher video resolution using less processing and power.
Regarding claims 12 and 28, Chen, as modified by Wang, Krahenbuhl, Jetley and Zhang, discloses the claimed invention wherein each block of the plurality of blocks has a same shape and a same number of pixels as other blocks of the plurality of blocks. (Zhang: paragraphs 25, 33, 52 and 56)
Regarding claims 13 and 29, Chen, as modified by Wang, Krahenbuhl, Jetley and Zhang, discloses the claimed invention wherein the plurality of blocks include a first plurality of blocks and a second plurality of blocks, each block of the first plurality of blocks having a first shape and a first number of pixels, each block of the second plurality of blocks having a second shape and a second number of pixels, wherein the first plurality of blocks differs from the second plurality of blocks based on at least one of a number of pixels and shape. (Zhang: paragraphs 52 and 56)
Regarding claims 14 and 30, Chen, as modified by Wang, Krahenbuhl, Jetley and Zhang, discloses the claimed invention wherein the modified first region generated based on use of the first process to modify a first subset of the plurality of blocks corresponding to the first region of the input image from the first resolution to the second resolution. (Zhang: paragraphs 26-27 and 55)
Regarding claims 15 and 30, Chen, as modified by Wang, Krahenbuhl, Jetley and Zhang, discloses the claimed invention wherein the modified second region generated based on use of the second process to modify a second subset of the plurality of blocks corresponding to the second region of the input image. (Zhang: paragraphs 31, 35, 40 and 55)
Regarding claim 16, Chen, as modified by Wang, Krahenbuhl, Jetley and Zhang, discloses the claimed invention wherein the modified first region and the modified second region both generated based on modification of each of the plurality of blocks to increase the first resolution of each of the plurality of blocks to the second resolution. (Zhang: paragraphs 31-35, 40, 52 and 55)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BOBBAK SAFAIPOUR whose telephone number is (571)270-1092. The examiner can normally be reached Monday - Friday, 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BOBBAK SAFAIPOUR/Primary Examiner, Art Unit 2665
Prosecution Insights