DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “an obtaining module” and “a padding module” as recited by claim 8.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recites sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Objections
Claims 2-7, 10-15, and 17-22 are objected to for the following informalities:
The claims follow the general format of referring back to a previous step of an independent claim by explicitly spelling out said step word for word. The examiner instead suggests labeling the steps of the independent claims with indicators ‘(a)’, ‘(b)’, ‘(c)’ (etc.) such that the claims could refer back to these steps in a more concise manner. Such that claim 2, as an example, would read:
“The padding method of claim 1, before step (a), the padding method comprising…”
Appropriate correction is required.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-22 are rejected under 35 U.S.C. 101 because the claims do not appear directed to any of the four statutory categories, and instead appear directed to non-statutory subject matter, specifically signals per se. See MPEP § 2106.03(I), “Non-limiting examples of claims that are not directed to any of the statutory categories include…Transitory forms of signal transmission (often referred to as “signals per se”), such as a propagating electrical or electromagnetic signal or carrier wave”.
In the context of the flowchart illustrated in MPEP § 2106(III), claims 16-22 fail at Step 1 of the Subject Matter Eligibility test (Step 1: No). As relevant, page 7 of Applicant’s Specification does not explicitly disclaim transitory signals, stating only that the memory (Page 8, where the memory is the “computer storage medium”) “may be” a high-speed RAM memory, a disk memory, a storage apparatus, etc. Exemplary/non-limiting embodiments, even if generally understood to be ‘non-transitory’ in nature, do not serve to preclude an interpretation covering a transitory signal/medium embodiment, which does not fall within the definition of a process, machine, manufacture or composition of matter (In re Nujiten, 500 F.3d 1346, 1354, 84 USPQ2d 1495, 1500 (Fed. Cir. 2007)).
Accordingly, the claims are rejected.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 7-11, 15-18, and 22 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Kim et. al (US 20190164037 A1) (Hereinafter, “Kim”)
With respect to claim 1, Kim discloses:
A padding method ([0002]), the padding method comprising following steps:
when a calculation command is detected, obtaining a first coordinate set contained in a calculation window and a storage address set corresponding to an original picture ([0101] “In the present invention, when calculating an address, even though the address generator uses the K*K weight section index from 0 to K−1 for each direction, the address generator determines starting coordinates of the pixel group to calculate convolution with the weight by subtracting the value [K/2] corresponding to the amount of padding at that index”; [0128]-[0129])
and according to the first coordinate set and the storage address set, determining whether the calculation window contains a padding data position ([0012]; “and configures part of the plurality of adjacent pixels to padding based on a result of comparing the address information”; [0102] “If the calculated index (position of the input pixel groups to calculate the convolution) deviates from the address range of the original input feature map with respect to the width or height direction, the address generator regards this as a padding position, and fills it with 0”; [0128]-[0129])
and if the calculation window contains the padding data position, performing a padding operation on the padding data position according to a preset padding mode ([0044]; [0102]; [0128]-[0129] “pad with 0”)
With respect to claim 2, Kim discloses:
The padding method of claim 1, before the step of “when a calculation command is detected, obtaining a first coordinate set contained in a calculation window and a storage address set corresponding to an original picture ([0011] “The address generator obtains the address information of the input feature map, determines the second position based on the address information of the first position and the size of the first weight group among the address informaiton of the plurality of input pixels, and transmits the second position to the processor”), and according to the first coordinate set and the storage address set, determining whether the calculation window contains a padding data position” ([0054] “The CNN accelerator 230 may load the entire input data of the convolution neural network into the on-chip memory (not shown) of the CNN accelerator 230 and sequentially process the entire layer”; [0097] “When loading the input feature map into the SA_H feature map memory through the address generator, the CNN processor according to the present invention may not allocate the memory space for the padding area necessary for applying the KxK weight, and stores only the actual output feature map without padding the padding space, even if the convolution requires padding in the next layer”), the padding method comprising:
obtaining the original picture ([0098] “Therefore, when loading the input feature map from the address generator, the CNN processor uniformly distributes the height of the original feature map that does not add the padding area to SA_H banks, and when performing pooling, adjusts the output feature map on the same bank included in the same window”; [0128])
according to the number of rows and columns of the original picture, generating a second coordinate set corresponding to a valid data position in the original picture ([0098] wherein the “second coordinate set” corresponds to the original feature map; [0128]-[0129] setting of bw and bh), so as to obtain the storage address set corresponding to the original picture ([0013]; [0019])
and according to the storage address set, storing a valid data in the original picture into memory ([0054]; [0097])
With respect to claim 3, Kim discloses:
The padding method of claim 2, the step of “according to the first coordinate set and the storage address set, determining whether the calculation window contains a padding data position” comprising:
determining a boundary coordinate set of the original picture in the second coordinate set corresponding to the storage address set, and performing a boundary detection based on the first coordinate set and the boundary coordinate set ([0128]-[0129], 0, W, and last_bank_bank_height)
and if there is a first coordinate outside the boundary coordinate set in the first coordinate set, determining that the calculation window contains the padding data position ([0128]-[0129])
PNG
media_image1.png
310
646
media_image1.png
Greyscale
With respect to claim 7, Kim discloses:
The padding method of claim 1, after the step of “if the calculation window contains the padding data position, performing a padding operation on the padding data position according to a preset padding mode”, the padding method further comprising:
inputting a data set contained in the calculation window that has undergone the padding operation into a calculation module to obtain a target calculation result ([0062] “Each row of a plurality of processor units 334A-334P may process an input feature map provided by the feature map bank corresponding to the row to which it belongs. Each processor unit may receive an input feature map value and an instruction to process from a processor unit located on the left, receive a weight from a processor unit located on the top, and use the received weight and input feature map values to perform an operation corresponding to the command”; [0072] “The processor unit performs N*K*K operations of multiplying and accumulating the weight and the input feature map value to the value of K*K of N input feature maps corresponding to a position of a certain output feature map to be calculated for the convolution, and if necessary, applying batch normalization (subtract the average value, divide it by the standard deviation, and multiply the scale value again) to this value, adding a bias value corresponding to the output feature map, and selecting a maximum value among this plurality of adjacent values (e.g., 2×2) or calculating an average”; [00128]-[00129])
and performing the steps of obtaining the first coordinate set contained in the calculation window and the storage address set corresponding the original picture ([0128]-[0129])
and according to the first coordinate set and the storage address set, determining whether the calculation window contains the padding data position ([0128]-[0129] movement of fx and fy through boundary logic checks, with padding outputs)
PNG
media_image2.png
889
657
media_image2.png
Greyscale
With respect to claim 8, Kim discloses:
A padding apparatus (Fig. 2; [Abstract]; [0052]), the padding apparatus comprising:
an obtaining module ([0011]; [0052] Address Generator 220), configured to
when a calculation command is detected, obtain a first coordinate set contained in a calculation window and a storage address set corresponding to an original picture ([0011] “The address generator obtains the address information of the input feature map, determines the second position based on the address information of the first position and the size of the first weight group among the address informaiton of the plurality of input pixels, and transmits the second position to the processor”), and according to the first coordinate set and the storage address set, determine whether the calculation window contains a padding data position ([0012]; “The address generator obtains address information of the plurality of adjacent pixels, and configures part of the plurality of adjacent pixels to padding based on a result of comparing the address information”; [0102] “If the calculated index (position of the input pixel groups to calculate the convolution) deviates from the address range of the original input feature map with respect to the width or height direction, the address generator regards this as a padding position, and fills it with 0”; [0128]-[0129]); and
a padding module ([0062] address generator 220 in combination with processor units 334A-334P; [0102], configured to
if the calculation window contains the padding data position, perform a padding operation on the padding data position according to a preset padding mode ([0044]; [0062]; [0102]; [0128]-[0129] “pad with 0”)
With respect to claim 9, Kim discloses:
An AI chip padding system, the AI chip padding system (Fig. 2; [0051]) comprising:
a memory ([0051]), a processor ([0051]), an AI chip ([0051] CNN Accelerator 230; [0056] “As shown in Fig. 3, a CNN Accelerator 330 may be configured as a systolic array”), and a padding driver program stored in the memory and runnable on the processor to drive the AI chip to execute ([0052] “The network of the convolution neural network (CNN) may be composed of a plurality of layers, and first input data for a plurality of layers may be stored in the external memory 201. To use the CNN accelerator, the memory controller 210 may be connected to the external memory 201 to transfer data of the external memory 201 to the address generator 210”; [0065]; [0128]-[0129]); and
when driving the AI chip to execute, the padding driver program implementing the steps of the padding method of claim 1 ([0128]-[0129]; see rejection of claim 1)
With respect to claims 10-11 and 15, they are functionally parallel to claims 2-3 and 7 repectively, with the exception that the limitations are recited as steps to be performed by the AI chip padding system of claim 9. For reasons outlined in the rejection of claims 9, 2-3, and 7, the AI chip padding system of Kim is further configured to perform the steps of claims 2-3 and 7. Accordingly, the claims are rejected in the line with the analysis above.
With respect to claim 16, Kim discloses:
A computer readable storage medium ([0051]; [0054]), the computer readable storage medium storing a padding driver program thereon ([0051]-[0052]; [0128]-[0129]), and
when driving an AI chip to execute, the padding driver program implementing the steps of the padding method of claim 1 ([0052]; [0128]-[0129]; see rejection of claim 1)
With respect to claims 17-18 and 22, they are functionally parallel to claims 2-3 and 7 repectively, with the exception that the limitations are recited as steps to be performed by the instructinos of the computer readable storage medium of claim 16. For reasons outlined in the rejection of claims 16, 2-3, and 7, the computer readable storage medium of Kim stores instructions corresponding to the steps of claims 2-3 and 7. Accordingly, the claims are rejected in the line with the analysis above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Bradski et. al Adding borders to your images (Hereinafter, “Bradski”).
With respect to claim 4, Kim teaches the method of claim 1.
Kim does not explicitly teach the further limitations of claim 4.
However, Bradski, in the same field of endeavor of image padding, teaches:
the step of “if the calculation window contains the padding data position, performing a padding operation on the padding data position according to a preset padding mode” comprising:
if the calculation window contains the padding data position, obtaining and identifying the preset padding mode (Explanation [1] “int borderType”);
if the preset padding mode is a first padding mode, determining a padding rule according to the first padding mode, and performing the padding operation on the padding data position according to the padding rule (Code [2] wherein “BORDER_CONSTANT” is the “first padding mode”);
and if the preset padding mode is a second padding mode, reading a corresponding first data, and padding the first data to the padding data position (Code [2] wherein “BORDER_REPLICATE” is the “second padding mode”).
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Kim to include the limitations of padding modes, as taught by Bradski. Doing so would provide additional functionality in modes of operation. The systems readily integrate, as Bradski represents an additional variable the base method of Kim could readily be expected to process as well without undue experimentation.
With respect to claim 12, it is functionally parallel to claim 4, but performed by the AI chip system of claim 9. For reasons outlined in the rejections of claim 9 and 4 above, claim 12 is accordingly rejected.
With respect to claim 19, it is functionally parallel to claim 4, but performed by the instructions stored within the computer readable storage medium of claim 16. For reasons outlined in the rejections of claim 16 and 4 above, claim 19 is accordingly rejected.
Claims 5-6, 13-14, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Kim and Bradski in view of OpenCV Image Filtering (Hereinafter, “OpenCV”)
With respect to claim 5, Kim and Bradski teach the method of claim 4.
Kim and Bradski further teach:
the step of “if the preset padding mode is a first padding mode, determining a padding rule according to the first padding mode, and performing the padding operation on the padding data position according to the padding rule” comprising:
if the preset padding mode is the first padding mode, Bradski, Theory [3] “The row or column at the very edge of the original is replicated to the extra border”; Bradski, Results [9] “If you press ‘r’, the border will become a replica of the edge pixels”)
and padding the second data as a padding data to the padding data position (Bradksi, Theory [3]; Bradski, Results [9])
Kim and Bradski do not explicitly teach:
correcting a first storage address corresponding to the first coordinate of the padding data position
However, OpenCV, in the same field of endeavor of image border processing, teaches:
correcting a first storage address corresponding to the first coordinate of the padding data position (borderInterpolate, “This function computes and returns the coordinate of the donor pixel for a specified extrapolated pixel”)
PNG
media_image3.png
453
1621
media_image3.png
Greyscale
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Kim and Bradski to include the limitations of border interpolation, as taught by OpenCV. Doing so would predictably increase computational efficiency. The systems readily integrate, because as OpenCV states, the borderInterpolate function is used inside “copyMakeBorder()” which is also taught in Bradski, Explanation [7].
With respect to claim 6, Kim, Bradski and OpenCV teach the method of claim 5.
Kim, Bradski, and OpenCV further teach:
the step of “correcting a first storage address corresponding to the first coordinate of the padding data position” comprising:
according to the first padding mode and the storage address set corresponding to the original picture, determining a second storage address corresponding to a data to be padded in the padding data position (OpenCV, borderInterpolate, “Parameters”)
and correcting the first storage address corresponding to the first coordinate of the padding data position to the second storage address (OpenCV, copyMakeBorder; OpenCV, borderInterpolate “It is used inside FilterEngine and copyMakeBorder() to compute tables for quick extrapolation”; Bradski, copyMakeBorder)
With respect to claim 13-14, they are functionally parallel to claim 5-6, but performed by the AI chip system of claim 9. For reasons outlined in the rejections of claim 9 and 5-6 above, claims 13-14 are accordingly rejected.
With respect to claim 20-21, they are functionally parallel to claims 5-6, but performed by the instructions stored within the computer readable storage medium of claim 16. For reasons outlined in the rejections of claims 16 and 5-6 above, claims 20-21 are accordingly rejected.
Additional References
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH WILLIAM BOYAR whose telephone number is (571)272-8392. The examiner can normally be reached 8:30 – 5:00 EST, Monday – Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH W BOYAR/Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights