Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5-6, 8-10, 12-13, 15-17, 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Annapureddy (US Pub 2016/0321784 A1).
As to claim 1, Annapureddy discloses a processor, comprising: one or more circuits to adjust a resolution of information to be used by one or more neural networks based, at least in part, on one or more performance metrics of the one or more neural networks (Annapureddy, Fig. 5, ¶0062, Fig. 6A, ¶0077, “aspects of the present disclosure are directed to adjusting a DCN to operate on lower resolution images while the classification performance of the DCN remains greater than a performance threshold.” ¶0078, “the image resolution is dynamically reduced at various layers to adjust a complexity of a DCN. Still, in this configuration, a specific image resolution is selected so that the classification performance of the adjusted DCN is greater than a performance threshold. According to aspects of the present disclosure, the reduction factor refers to the ratio by which the input image resolution is reduced. For example, when a reduction factor of two is applied to a 28×28 image, the reduced image has a size of 14×14. As another example, if a reduction factor of three is applied to a 30×30 image, the reduced image has a size of 10×10.”).
As to claim 2, claim 1 is incorporated and Annapureddy discloses the resolution of the information is used to train one or more neural networks to generate one or more images (Annapureddy, ¶0047, “A DCN may be trained with supervised learning. During training, a DCN may be presented with an image 326, such as a cropped image of a speed limit sign, and a “forward pass” may then be computed to produce an output 328. The output 328 may be a vector of values corresponding to features such as “sign,” “60,” and “100.” The network designer may want the DCN to output a high score for some of the neurons in the output feature vector, for example the ones corresponding to “sign” and “60” as shown in the output 328 for a network 300 that has been trained.” ¶0055, ¶0102, ¶0109, “training data is used to obtain the weight matrices of the adjusted network. Specifically, the training data may be used to compute the higher-resolution input images that are supposed to be input to the convolution layer of the original DCN. Standard regression tools, such as least squares, may be specified to obtain a linear mapping from the retained pixels in the lower-resolution input images to the missing pixels. The missing pixels refer to pixels that were present in the higher-resolution image and are no longer present in the lower-resolution image. The linear mapping may be specified for the higher-resolution weight matrices to obtain the adjusted resolution weight matrices.” ¶0119.)
As to claim 3, claim 1 is incorporated and Annapureddy discloses the one or more performance metrics include one or more loss operations (Annapureddy, ¶0083, “For each adjusted DCN model, the inner-loop outputs the difference in classification performance between the original image resolution and the reduced image resolution. Additionally, the inner-loop outputs the computational complexity for each adjusted DCN model.” ¶0097, “Thus, for a given reduction factor, the outer-loop computes an energy reduction value based on a fraction of high energy components and a sum of high energy components and low energy components (e.g., E.sub.H/(E.sub.L+E.sub.H)). In the present configuration, if the energy reduction value is less than a threshold, then the reduction factor may be desirable because the reduction does not increase the loss of high energy components. Still, if the energy reduction value is greater than a threshold, then the reduction factor may not be desirable because the reduction may increase the loss of high energy components. Thus, according to an aspect of the present disclosure, for each layer, the outer-loop selects a reduction factor r based on whether the energy reduction value for each layer is less than a threshold. The energy components may be referred to as frequency components.” Or see claim 2-3 of Annapureddy.).
As to claim 5, claim 1 is incorporated and Annapureddy discloses the one or more neural networks include one or more transformer neural networks to perform bilinear interpolation when adjusting the resolution (Annapureddy, ¶0108, “The down sampling of the weight matrices may be accomplished using standard methods specified for image processing for image resizing, such as sync interpolation or bilinear interpolation.”).
As to claim 6, claim 1 is incorporated and Annapureddy discloses the resolution of the information is represented using a matrix of pixels of one or more images (Annapureddy, ¶0108, “FIG. 9, the adjusted DCN calculates an output 912 using fewer pixels in comparison to the original DCN.”).
As to claim 8, Annapureddy discloses a system comprising: one or more processors to adjust a resolution of information to be used by one or more neural networks based, at least in part, on one or more performance metrics of the one or more neural networks (See claim 1 for detailed analysis.).
As to claim 9, claim 8 is incorporated and Annapureddy discloses the resolution of the information is used to train one or more neural networks to generate one or more images (See claim 2 for detailed analysis.).
As to claim 10, claim 8 is incorporated and Annapureddy discloses the one or more performance metrics include one or more loss operations (See claim 3 for detailed analysis.).
As to claim 12, claim 8 is incorporated and Annapureddy discloses the one or more neural networks include one or more transformer neural networks to perform bilinear interpolation when adjusting the resolution (See claim 5 for detailed analysis.).
As to claim 13, claim 8 is incorporated and Annapureddy discloses the resolution of the information is represented using a matrix of pixels of one or more images (See claim 6 for detailed analysis.).
As to claim 15, Annapureddy discloses a method comprising: adjusting a resolution of information to be used by one or more neural networks based, at least in part, on one or more performance metrics of the one or more neural networks (See claim 1 for detailed analysis.).
As to claim 16, claim 15 is incorporated and Annapureddy discloses the resolution of the information is used to train one or more neural networks to generate one or more images (See claim 2 for detailed analysis.).
As to claim 17, claim 15 is incorporated and Annapureddy discloses the one or more performance metrics include one or more loss operations (See claim 3 for detailed analysis.).
As to claim 19, claim 15 is incorporated and Annapureddy discloses the one or more neural networks include one or more transformer neural networks to perform bilinear interpolation when adjusting the resolution (See claim 5 for detailed analysis.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4, 7, 11, 14, 18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Annapureddy (US Pub 2016/0321784 A1) in view of Dekel et al. (US Pub 2024/0419382 A1).
As to claim 4, claim 1 is incorporated and Annapureddy does not disclose the one or more neural networks include one or more encoders.
Dekel teaches the one or more neural networks include one or more encoders (Dekel, ¶0126, “An encoder-decoder model on graph, known as the graph U-Nets and based on gPool and gUnpool layers, is described in an article authored by Hongyang Gao and Shuiwang Ji published 2019 [arXiv: 1905.05178 [cs.LG]] entitled: “Graph U-Nets”, which is incorporated in its entirety for all purposes as if fully set forth herein.” ¶0198, “The best performing models also connect the encoder and decoder through an attention mechanism.” ¶0199-¶0203. ¶0207, “The Transformer Encoder 44 may include one or more layers. A layer (Lx) 46 of the Transformer Encoder 44 that may be part of, consists of, or may comprise, the transformer encoder 44”).
Annapureddy and Dekel are considered to be analogous art because all pertain to neural network. It would have been obvious before the effective filing date of the claimed invention to have modified Annapureddy with the features of “the one or more neural networks include one or more encoders.” as taught by Dekel. The suggestion/motivation would have been in order to generate high quality results in high resolution (Dekel, abstract).
As to claim 7, claim 1 is incorporated and Annapureddy the resolution of the information is adjusted to be increased (Annapureddy, Fig. 6A, ¶0048, “The gradient may indicate an amount that an error would increase or decrease if the weight were adjusted slightly. ¶0075, “Despite an increase in computational complexity, conventional DCNs may use higher resolution images to improve the classification of the image. In some cases, the DCN may have a task of determining whether the number three exists within a 32×32 input image with reduced detail. In this example, the size of the image may not improve the classification. In other cases, for images with increased detail, such as a landscape or an image with multiple objects, an increased image size is specified to improve the image classification.” ¶0110, “an original image size may be 28×28 and the filters may be 3×3. Furthermore, in the present example, if a reduction factor of two is applied to the image, such that the image size is 14×14, the filters at a given layer may be adjusted to accommodate the new 14×14 image.”).
Even though Annapureddy mostly focus on reduce resolution to improve performance, it would be obvious to increase resolution when the performance is adaptive.
Dekel teaches the resolution of the information is adjusted to be increased (Dekel, ¶0124, “These layers increase the resolution of the output, and a successive convolutional layer can then learn to assemble a precise output based on this information.” ¶0340, “These layers increase the resolution of the output” ¶0386, “Any ANN herein, may comprise, may use, or may be based on, a method, scheme or architecture such as U-Net, which is based on the fully convolutional network to supplement a usual contracting network by successive layers, where pooling operations are replaced by upsampling operators. These layers increase the resolution of the output, and a successive convolutional layer can then learn to assemble a precise output based on this information.”)
Annapureddy and Dekel are considered to be analogous art because all pertain to neural network. It would have been obvious before the effective filing date of the claimed invention to have modified Annapureddy with the features of “the resolution of the information is adjusted to be increased” as taught by Dekel. The suggestion/motivation would have been in order to generate high quality results in high resolution (Dekel, abstract).
As to claim 11, claim 8 is incorporated and the combination of Annapureddy and Dekel discloses the one or more neural networks include one or more encoders (See claim 4 for detailed analysis.).
As to claim 14, claim 8 is incorporated and the combination of Annapureddy and Dekel discloses the resolution of the information is adjusted to be increased (See claim 7 for detailed analysis.).
As to claim 18, claim 15 is incorporated and the combination of Annapureddy and Dekel discloses the one or more neural networks include one or more encoders (See claim 4 for detailed analysis.).
As to claim 20, claim 15 is incorporated and the combination of Annapureddy and Dekel discloses the resolution of the information is adjusted to be increased (See claim 7 for detailed analysis.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Liu et al. (US Pub 2022/0114702 A1) discloses Upsampling an Image Using One or More Neural Networks.
Kim et al. (US Pub 2021/0073945 A1) discloses as the zoom magnification increases, the number of pixels to be processed decreases, and therefore a super resolution neural network having higher complexity which is designed to focus on the processing performance rather than the processing time may be selected.
Kim et al. (US Pub 2020/0126186 A1) discloses the AI up-scaler may obtain the DNN setting information for generating the third image having quality and/or resolution, considering performance information of the display apparatus that is to reproduce the third image or the post-processed third image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights