DETAILED ACTION
[1] Remarks
I. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
II. Claims 1-18 are pending and have been examined, where claims 1, 6-11, and 16-18 is/are rejected and claim 2-5 and 12-15 is/are objected to. Explanations will be provided below.
III. Inventor and/or assignee search were performed and determined no double patenting rejection(s) is/are necessary.
IV. Patent eligibility (updated in 2019) shown by the following: Claims 1-18 pass patent eligibility test because there is/are no limitation(s) or a combination of limitations amounting to an abstract idea. Also, the following limitation or the combinations of the limitations: “performing feature extraction on an input image to obtain a first feature map, wherein the input image comprises N image blocks, N is a positive integer, and N >2; aggregating at least two image blocks in the input image to obtain a second feature map, wherein positions of the at least two image blocks are correlated” effects a transformation or a reduction of a particular article to a different state or thing / adds a specific limitation(s) other than what is well-understood, routine and conventional in the field, or adding unconventional steps that confine the claim to a particular useful application and providing improvements to the technical field of deep learning, which recite additional elements that integrate the judicial exception into a practical application and amounting significant more.
[2] Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function. Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
Claim(s) 10-18 are not interpreted under 35 U.S.C. 112(f) or pre-AIA U.S.C. 112 6th paragraph because of the following reason(s): limitations are modified by sufficient structure or material for performing the claimed function.
Claim(s) 1-9 do not require 35 U.S.C. 112(f) or pre-AIA U.S.C. 112 6th paragraph interpretation because they are method claims and / or they are CRM claims.
Upon examination of the specification and claims, the examiner has determined, under the best understanding of the scope of the claim(s), rejection(s) under 35 U.S.C. 112(a)/(b) is not necessitated because of the following reasons: sufficient support are provided in the written description / drawings of the invention.
[3] Grounds of Rejection
Claim Rejections - 35 USC § 103
1. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claims 1, 6-11, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Coelho (US 20200092552) in view of Mabyalaht (US 20210166092).
Regarding claim 1, Coelho discloses an image processing method, wherein the method is implemented by using an image processing model, and the method comprises:
performing feature extraction on an input image to obtain a first feature map, wherein the input image comprises N image blocks, N is a positive integer, and N >2 (see figure 3, each frame is being divided into blocks, where the number of blocks divided is 16, see paragraph 4, using feature extraction layers of the CNN, first feature maps of the block, where each first feature map is of size S×S, see figure 10, output of 1004, 1008 or 1014);
aggregating at least two image blocks in the input image to obtain a second feature map, wherein positions of the at least two image blocks are correlated (see figure 10, the output of 1016 is read as the second feature map, the concatenating step is read as aggregating, where the image structure is divided into plurality of blocks, each adjacent block are correlated to each other):
PNG
media_image1.png
480
878
media_image1.png
Greyscale
.
Coelho is silent in disclosing obtaining an output image based on the first feature map and the second feature map. Mabyalaht discloses obtaining an output image based on the first feature map and the second feature map (see figure 5B illustration below, the output is a volume image, where any one of the images stacked together to form the volume feature map is read as the output image):
PNG
media_image2.png
380
678
media_image2.png
Greyscale
.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to include obtaining an output image based on the first feature map and the second feature map because the first feature map detects low-level details like edges, gradients, and simple color contrasts. Passing these into a second layer creates a second feature map that combines those edges into shapes, patterns, and object parts, which build a hierarchy of increasingly visual details improving image recognition.
Regarding claim 6, Mabyalaht discloses the method according to claim 1, wherein the obtaining an output image based on the first feature map and the second feature map comprises: fusing the first feature map and the second feature map to obtain the output image (see figure 5B illustration above, the output is a volume image, where any one of the images stacked together to form the volume feature map is read as the output image). See the motivation for claim 1.
Regarding claim 7, Mabyalaht discloses the method according to claim 1, wherein before the obtaining an output image based on the first feature map and the second feature map, the method further comprises: normalizing the second feature map to obtain a normalized second feature map (see figure 5b, conv 1 x 1 include a batch normalization layer, see paragraph 96); and wherein the obtaining an output image based on the first feature map and the second feature map comprises: obtaining the output image based on the first feature map and the normalized second feature map (see paragraph 96, the enriched map is reduced to an object with dimensions 1×1×C, one pixel with a large number C of channels, which goes through a 1×1 convolution to control the number of channels, a batch normalization and a non-linear layer ReLU). See the motivation for claim 1. Also normalizing a feature map ensures that its scale and distribution match the first or subsequent feature maps, preventing one dominant channel from overshadowing others.
Regarding claim 8, Coelho discloses the method according to claim 1, wherein the input image is a raw to-be-processed image, or the input image is a feature map obtained after feature extraction is performed on the raw to-be-processed image (see figure 3, the video stream is the raw data and is processed to divide each frame to N blocks).
Regarding claim 9, Coelho discloses an image processing method, wherein the method is implemented by using an image processing model, and the method comprises:
performing feature extraction on an input image to obtain a first feature map (see figure 3, each frame is being divided into blocks, where the number of blocks divided is 16, see paragraph 4, using feature extraction layers of the CNN, first feature maps of the block, where each first feature map is of size S×S, see figure 10, output of 1004, 1008 or 1014);
performing downsampling on the input image to obtain a downsampled input image, wherein the downsampled input image comprises P image blocks, P is a positive integer, and P > 2 (see paragraph 137, each time a filter is applied to a unit (e.g., the block 1002 or a feature map), the unit is downsized, i.e., down-sampled, in each dimension, by the size of the filter);
aggregating at least two image blocks in the downsampled input image to obtain a second feature map, wherein positions of the at least two image blocks are correlated (see figure 10, the output of 1016 is read as the second feature map, the concatenating step is read as aggregating, where the image structure is divided into plurality of blocks, each adjacent block are correlated to each other);
PNG
media_image3.png
480
878
media_image3.png
Greyscale
.
Coelho is silent in disclosing performing upsampling on the second feature map to obtain an upsampled second feature map; and obtaining an output image based on the first feature map and the upsampled second feature map.
Mabyalaht discloses
performing upsampling on the second feature map to obtain an upsampled second feature map (see figure 5a, top of the architect is read as the 2nd feature map, the 2nd feature map is upsampled, see the illustration below); and
obtaining an output image based on the first feature map and the second feature map (see figure 5a, see the illustration below, the 1st feature map is in the middle of the pipeline which is combined with the upsampled 2nd feature map, which output P5):
PNG
media_image4.png
234
666
media_image4.png
Greyscale
.
See the motivation for claim 1. In addition, performing upsampling on a feature map matches its spatial size, height and width, to another feature map so they can be combined, which improves network training.
Regarding claim 11 see the rationale and rejection for claim 9.
Regarding claim 10 see the rationale and rejection for claim 1. In addition, Coelho discloses an apparatus which includes a processor, storage, and display for performing all the limitations of claims 1 and 10.
Regarding claim 16 see the rationale and rejection for claim 6.
Regarding claim 17 see the rationale and rejection for claim 7.
Regarding claim 18 see the rationale and rejection for claim 8.
[4] Claim Objections
Claim(s) 2-5 and 12-15 is/are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
With regards to claim 2, the examiner cannot find any applicable prior art providing teachings for the following limitation(s): “the method according to claim 1, wherein the aggregating at least two image blocks in the input image to obtain a second feature map comprises: aggregating an ith image block in the input image and M image blocks other than the ith image block in the input image to obtain an ith feature block in the second feature map, wherein the M image blocks comprise image blocks forming a preset position relationship with the ith image block, M is a positive integer, N > M >1, and i = 1, ..., N”; in combination with the rest of the limitations of claim 1.
Coelho discloses the method according to claim 1, wherein the aggregating at least two image blocks in the input image to obtain a second feature map comprises: aggregating an ith image block in the input image and M image blocks other than the ith image block in the input image
PNG
media_image5.png
351
707
media_image5.png
Greyscale
.
Claim(s) 3-4 is/are objected as well because it is dependent on a claim with allowable subject matter.
Regarding claim 5, the examiner cannot find any applicable prior art providing teachings for the following limitation(s): the method according to claim 1, wherein the obtaining an output image based on the first feature map and the second feature map comprises: fusing the first feature map and the second feature map to obtain a third feature map; performing feature extraction on the third feature map to obtain a fourth feature map; and fusing the fourth feature map and the input image to obtain the output image; in combination with the rest of the limitations of claim 1.
Mabyalaht discloses the method according to claim 1, wherein the obtaining an output image based on the first feature map and the second feature map comprises: fusing the first feature map and the second feature map to obtain a third feature map (see figure 5B illustration below, the multiplication is the operation to fuse 1st and 2nd feature maps); performing feature extraction on the third feature map to obtain a fourth feature map (the addition operation to fuse 3rd and 2nd feature maps is an act to extract features):
PNG
media_image6.png
395
741
media_image6.png
Greyscale
.
Mabyalaht is silent in disclosing fusing the fourth feature map and the input image to obtain the output image.
Regarding claim 12 see the rationale and objection for claim 2. Claim(s) 13-14 is/are objected as well because it is dependent on a claim with allowable subject matter.
Regarding claim 15 see the rationale and objection for claim 5.
CONTACT INFORMATION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEX LIEW (duty station is located in New York City) whose telephone number is (571)272-8623 (FAX 571-273-8623), cell (917)763-1192 or email alexa.liew@uspto.gov. Please note the examiner cannot reply through email unless an internet communication authorization is provided by the applicant. The examiner can be reached anytime.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MISTRY ONEAL R, can be reached on (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEX KOK S LIEW/Primary Examiner, Art Unit 2674 Telephone: 571-272-8623
Date: 9/30/26
Prosecution Insights