DETAILED ACTION
Response to Arguments
Interpretation under 112(f) is withdrawn in light of amendment.
Applicant's arguments filed on page 3-4 have been fully considered but they are not persuasive.
Applicant argues in last paragraph, page 4, that “in Zhu, (1) the upscaling process 300 upscales the whole input image 100 ([0036]), and (2) Zhu block 340 crops expanded and up-scaled sub-pictures … which provides neither a description nor a suggestion of … in Claims 1, 17, and 19, which require (1) dividing the partial region into blocks as claimed and (2) then performing upscaling on the blocks”. The office respectfully disagrees. The limitation in argument is “an upscaled image of a partial region of an original image is to be generated … divide the partial region into a plurality of blocks at predetermined positions regardless…”. Zhu’s cropped blocks reads on the partial region, and upscaling those cropped blocks reads on the claimed “upscaled image of a partial region”. Therefore the argument is not persuasive .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 9-15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhu ( US 20200126184).
Regarding claim 1, Zhu teaches an image processing apparatus comprising:
one or more processors that execute a program stored in a memory ( 740, 710 in Fig. 7B) and thereby as the image processing apparatus to:
(1) divide an image into a plurality of blocks( 310 in Fig. 3) ;
(2) perform upscaling processing on each of the plurality of blocks ( 330 in Fig. 3); and
(3) combine upscaled images of the plurality of blocks into one upscaled image ( OUTPUT IMAGE in 360 in Fig. 3)
wherein, in a case where an upscaled image of a partial region of an original image is to be generated,
the one or more processors cause the image processing apparatus to divide the partial region into a plurality of blocks at predetermined positions regardless of a position of the partial region in the original image ( 340 in Fig. 3) .
Regarding claim 9, Zhu teaches the image processing apparatus according to claim 1, wherein the predetermined position is based on a multiple of a size of the block( 110, 120, 130, 140, 150, 160 in Fig. 1B).
Regarding claim 10, Zhu teaches the image processing apparatus according to claim 1, wherein the one or more processors cause the image processing apparatus to the upscaling processing using a trained machine learning model of which an input image is one of the plurality of blocks ( [0005], upscaling each sub-picture using an ML-based upscaler to produce an expanded upscaled sub-picture), and
divide the image into the plurality of blocks such that each of the plurality of blocks in a size determined in accordance with a size of an input image that the trained machine learning model can handle( 630 in Fig. 6).
Regarding claim 11, Zhu teaches the image processing apparatus according to claim 1, wherein the one or more processors cause the image processing apparatus to selectively divide the partial region into the plurality of blocks at positions based on the position of the partial region ( Fig. 2)
Regarding claim 12, Zhu teaches the image processing apparatus according to claim 11, wherein the one or more processors cause the image processing apparatus to divide the partial region into a plurality of blocks at positions based on the position of the partial region in case where a setting of the image processing apparatus or a dynamic factor satisfies a predetermined condition ( 132 in Fig. 2).
Regarding claim 13, Zhu teaches the image processing apparatus according to claim 1, wherein the one or more processors cause the image processing apparatus to divide the image into the plurality of blocks such that each of the plurality of blocks in a size selected from a plurality of sizes( [0027], the maximum size of the sub-picture that can be sent for training is 64×64 or 128×128 pixels).
Regarding claim 14, Zhu teaches the image processing apparatus according to claim 13, wherein the plurality of sizes are sizes of input images of a plurality of machine learning models that perform the upscaling processing([0027], Regarding how small a small-sized sub-picture needs to be depends on the memory size and the complexity of the neural network used in training).
Regarding claim 15, Zhu teaches the image processing apparatus according to claim 13, wherein in a case where a number of pixels that are not in the partial region in a block, among the plurality of blocks, that includes the partial region when dividing the image into a plurality of blocks in a first block size is smaller than that when dividing the image into a plurality of blocks in a second block size,
the one or more processors cause the image processing apparatus to select the first block size([0027], Regarding how small a small-sized sub-picture needs to be depends on the memory size and the complexity of the neural network used in training).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2-8, 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu in view of NOYES ( US20140198237).
Regarding claim 2, Zhu teaches the image processing apparatus according to claim 1, wherein, in the case where the upscaled image of the partial region of the original image is to be generated, the one or more processors cause the image processing apparatus to: divides all of the original image into the plurality of blocks at the predetermined positions ( 310 in Fig. 3).
Zhu does not expressly teach the processing not perform the upscaling processing on a block, among the plurality of blocks, that does not include the partial region.
However, NOYES teaches not perform the upscaling processing on a block, among the plurality of blocks, that does not include the partial region([0006], receiving cropped image data corresponding to the portion from the image sensor and upscaling the cropped image data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Zhu and NOYES, by cropping the image in Zhu following the teaching of NOYES before upscaling in Zhu, with motivation “to generate a zoomed image” ( NOYES, [0006]).
Regarding claim 3, Zhu in view of NOYES teaches the image processing apparatus according to claim 2, wherein the one or more processors further , the one or more processors cause the image processing apparatus to crop a part corresponding to the partial region from the one upscaled image (Zhu, [0005], cropping for the plurality of sub-pictures ; NOYES, [0006], receiving cropped image data).
Regarding claim 4, Zhu in view of NOYES teaches the image processing apparatus according to claim 1, wherein in the case where the upscaled image of the partial region of the original image is to be generated, the one or more processors cause the image processing apparatus to divide the partial region cropped from the original image into the plurality of blocks at the predetermined positions(Zhu, [0005], cropping for the plurality of sub-pictures ; NOYES, [0006], cropped image data).
Regarding claim 5, Zhu in view of NOYES teaches the image processing apparatus according to claim 4, wherein for a block, among the plurality of blocks, that includes a region in which pixels are not present, the one or more processors cause the image processing apparatus to the upscaling processing after supplementing pixels that are not present(Zhu, [0005], expanding each sub-picture of the plurality of small-sized sub-pictures using target padding pixels to produce an expanded sub-picture; upscaling each sub-picture).
Regarding claim 6, Zhu in view of NOYES teaches the image processing apparatus according to claim 5, wherein the one or more processors cause the image processing apparatus to supplement the pixels that are not present by shifting a position of the block that includes a region in which the pixels are not present within the partial region (Zhu, [0005], cropping the expanded upscaled sub-picture to an upscaled size equal to an original size of each sub-picture).
Regarding claim 7, Zhu in view of NOYES teaches the image processing apparatus according to claim 5, wherein for an upscaled image of the block that includes the supplemented pixels, the one or more processors cause the image processing apparatus to use, in the combining, only a region corresponding to a part that was included in the block before the pixels were supplemented(Zhu, [0005], cropping the expanded upscaled sub-picture to an upscaled size equal to an original size of each sub-picture).
Regarding claim 8, Zhu in view of NOYES teaches the image processing apparatus according to claim 5, wherein for a block, among the plurality of blocks, in which a percentage of the region in which pixels are not present is not greater than a threshold, the one or more processors cause the image processing apparatus not to supplement the pixels that are not present ( Zhu, [0033], the type A pad design is used for the left sub-region, the bottom sub-region, and the bottom left corner sub-region. Accordingly, the expanded sub-picture 132 shows those sub-regions (i.e., the left sub-region, the bottom sub-region, and the bottom left corner sub-region) being formed by extending the image area to the four direction) .
Claim 16 -18 and 19-20 recite similar apparatus for the apparatus and the method of claims 1-15, thus they are also rejected.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANGENG SUN whose telephone number is (571)272-3712. The examiner can normally be reached 8am to 5pm, EST, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Randolph Vincent can be reached at 571 272 8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIANGENG SUN
Examiner
Art Unit 2661
/Jiangeng Sun/Examiner, Art Unit 2671
