DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/29/2025 was considered by the examiner.
Claim Objections
Claim 11 is objected to because of the following informalities: “The method according claim 10”. Appropriate correction to “The method according to claim 10” is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 10 and 20 recite the limitation “video processing module”. This limitation has been interpreted under 112(f) as a means plus function because of the combination of the non-structural, generic placeholder “video processing module”, as well as their respective functional languages “to process at least one input image frame among the plurality of input image frames” and is being interpreted as “terminal video processor 200” that corresponds to the structure found in the disclosure (Par. [0183] and Fig. 9B).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong).
Regarding Claim 1:
Tong teaches an image processing method, comprising: receiving an input image; and processing the input image by using a convolutional neural network to obtain an output image, wherein a definition of the output image is higher than a definition of the input image (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed); processing the input image by using the convolutional neural network to obtain the output image comprises: performing feature extraction on the input image to obtain a plurality of first images, and concatenating the input image and the plurality of first images to obtain a first image group, wherein the first image group comprises the input image and the plurality of first images (Tong: 3. Method; As shown in Figure 1, SRDenseNet can be decomposed into several parts: the convolution layer for learning low-level feature, the blocks of DenseNet for learning high level features, the deconvolution layers for learning upscal ing filters and the reconstruction layer for generating the HRoutput; 3.1. DenseNet blocks; After applying a convolution layer to the input LR im ages for learning low-level features, a set of DenseNet blocks are adopted for learning the high-level features; Consequently, the ith layer receives the feature maps of all preceding layers as input: Xi = max(0,wi ∗[X1,X2,...,Xi−1] +bi) (3) where [X1,X2,...,Xi−1] represents the concatenation of the feature maps generated in the preceding convolution layers 1,2,..., i − 1.); performing the feature extraction on the first image group to obtain a plurality of second images, fusing the plurality of second images and the plurality of first images to obtain a plurality of third images, and concatenating the input image and the plurality of third images to obtain a second image group, wherein the second image group comprises the input image and the plurality of third images (Tong: 3. Method; As shown in Figure 1, SRDenseNet can be decomposed into several parts: the convolution layer for learning low-level feature, the blocks of DenseNet for learning high level features, the deconvolution layers for learning upscal ing filters and the reconstruction layer for generating the HRoutput; 3.1. DenseNet blocks; After applying a convolution layer to the input LR im ages for learning low-level features, a set of DenseNet blocks are adopted for learning the high-level features; Consequently, the ith layer receives the feature maps of all preceding layers as input: Xi = max(0,wi ∗[X1,X2,...,Xi−1] +bi) (3) where [X1,X2,...,Xi−1] represents the concatenation of the feature maps generated in the preceding convolution layers 1,2,..., i – 1; 2.3. Contribution; We demonstrate that the deep CNN framework with the denseNet as basic blocks can achieve good recon struction performance and that the fusion of features at different levels through dense skip connections can further boost the reconstruction performance for SISR.); and performing the feature extraction on the second image group to obtain the output image (Tong: 3.3. Combination of feature maps; only the feature maps at the top layer are used as input for reconstructing the HR output; Further, a skip connection is introduced in the network as shown in Figure 1 (b) to concatenate the low-level and high-level features, which we term SR DenseNet HL.The concatenated feature maps are then used as input for deconvolution layers. In addition, we use dense skip connections to combine the feature maps produced at all convolution layers for SR reconstruction, and denote this method as SRDenseNet All.), wherein a count of convolution kernels, used for the feature extraction on the input image, in the convolutional neural network is N, 12 ≤ N ≤ 20, and N is an integer, a count of convolution kernels, used for the feature extraction on the first image group, in the convolutional neural network is M, 12 ≤ M ≤ 20, and M is an integer (Tong: 3.1. DenseNet blocks; Specifically, there are 8 convolution lay ers in one DenseNet block in our work. If each convolution layer produce k feature maps as output, the total number of feature maps generated by one DenseNet block is k ∗ 8, where k is refered to as growth rate. The growth rate k reg ulates how much new information each layer contributes to the final reconstruction. To prevent the network from grow ing too wide, the growth rate k is set to 16 in this study. This results in a total number of 128 feature maps from one DenseNet block.).
In regards to Claim 2, Tong further teaches the method according to claim 1, wherein N = M = 16 (Tong: 3.1. DenseNet blocks; Specifically, there are 8 convolution lay ers in one DenseNet block in our work. If each convolution layer produce k feature maps as output, the total number of feature maps generated by one DenseNet block is k ∗ 8, where k is refered to as growth rate. The growth rate k reg ulates how much new information each layer contributes to the final reconstruction. To prevent the network from grow ing too wide, the growth rate k is set to 16 in this study. This results in a total number of 128 feature maps from one DenseNet block.); an activation function used for the feature extraction in the convolutional neural network is y = max (0, x), where x represents an input of the activation function and y represents an output of the activation function (Tong: 3. Method; The ReLu activation function is ap plied element-wise. Let Xi−1 be the input, the output of ith convolution or deconvolution layer is expressed as: Xi = max(0,wi ∗Xi−1 +bi) where Wi and Bi are the weights and biases in the layer, and ∗ denotes either convolution or deconvolution opera tion for the convenience of formulation.).
In regards to Claim 3, Tong further teaches the method according to claim 2, wherein a count of convolution kernels, used for the feature extraction on the second image group, in the convolution neural network is 3; a size of the convolution kernels used for the feature extraction on the input image, a size of the convolution kernels used for the feature extraction on the first image group, and a size of the convolution kernels used for the feature extraction on the second image group are all 3×3 (Tong: 3.2. Deconvolution layers; In our work, two successive deconvolution layers with small 3×3 kernels and 256 feature maps are trained for upscaling; 3.4. Bottleneck and Reconstruction layers; Finally, the feature maps in the HR space are used to generate HR images via a reconstruction layer. The reconstruction layer is a convolution layer with 3×3 kernel and one channel of output.); and the input image comprises a red channel input image, a green channel input image, and a blue channel input image, and the output image comprises a red channel output image, a green channel output image, and a blue channel output image (Tong: Fig. 3; Super-resolution results for “img096” (top figure),“img099” (middle figure) and “img004” (bottom figure) from Urban100 with an upscaling factor of 4; 4.1. Datasets and metrics; PSNR and SSIM values are shown on the top of each sub-figure; showcases colored input and output images; Since SR was performed in the luminance chan nel in YCbCr colour space, the PSNR and SSIM were cal culated on the Y-channel of images.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4-8 are rejected under 35 U.S.C. 103 as being unpatentable over Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong) in view of Aguinaldo et al. (NPL: Compressing GANs using Knowledge Distillation, hereafter referred as Aguinaldo).
In regards to Claim 4, Tong fails to further teach the method according to claim 1, further comprising: training a second neural network to be trained based on a first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolution neural network, wherein parameters of the first neural network are more than parameters of the second neural network, the first neural network which is pre-trained is configured to transform an original image having a first definition, which is input to the first neural network that is pre-trained, into a new image having a second definition, the second definition is greater than the first definition, the second neural network which is trained is the convolutional neural network, a network structure of the second neural network to be trained is same as a network structure of the convolutional neural network, and parameters of the second neural network to be trained are different from parameters of the convolutional neural network.
Aguinaldo, like Tong, is directed to GANs for super resolution. Aguinaldo in combination with Tong does teach training a second neural network to be trained based on a first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolution neural network (Aguinaldo: Abstract; we propose a method to compress GANs using knowledge distillation techniques, in which a smaller “student” GAN learns to mimic a larger “teacher” GAN), wherein parameters of the first neural network are more than parameters of the second neural network (Aguinaldo: 3. Methods; The teacher (large, over-parameterized network) and student (small, few parameter network) GANs used either the original DCGAN architecture or a slightly modified DCGAN architecture (Radford et al., 2015), more closely resembling the WGAN (Arjovsky et al., 2017), referenced as the W DCGAN.), the first neural network which is pre-trained is configured to transform an original image having a first definition, which is input to the first neural network that is pre-trained, into a new image having a second definition, the second definition is greater than the first definition (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed), the second neural network which is trained is the convolutional neural network, a network structure of the second neural network to be trained is same as a network structure of the convolutional neural network, and parameters of the second neural network to be trained are different from parameters of the convolutional neural network (Aguinaldo: 3. Methods; The teacher (large, over-parameterized network) and student (small, few parameter network) GANs used either the original DCGAN architecture or a slightly modified DCGAN architecture (Radford et al., 2015), more closely resembling the WGAN (Arjovsky et al., 2017), referenced as the W DCGAN.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tong to the student GAN technique, as taught by Aguinaldo, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Aguinaldo, the proposed modification is able to decrease the tens of millions of parameters needed for GANS that make them expensive to deploy for applications in low SWAP (size, weight, and power) hardware, such as mobile devices, and for applications with real time capabilities (Aguinaldo: Abstract).
In regards to Claim 5, Tong as modified by Aguinaldo further teaches the method according to claim 4, wherein training the second neural network to be trained based on the first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolutional neural network, comprises: based on the first neural network which is pre-trained, the second neural network to be trained, and a discrimination network, alternately training the discrimination network and the second neural network to obtain the second neural network which is trained, so as to obtain the convolutional neural network (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):).
In regards to Claim 6, Tong as modified by Aguinaldo further teaches the method according to claim 5, wherein training the discrimination network comprises: inputting first sample data into the first neural network and the second neural network, respectively, so as to obtain first data output from the first neural network and second data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss.); setting the first data to have a true value tag, inputting the first data with the true value tag into the discrimination network to obtain a first discrimination result, setting the second data to have a false value tag, and inputting the second data with the false value tag into the discrimination network to obtain a second discrimination result (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):); calculating a first loss function based on the first discrimination result and the second discrimination result (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting parameters of the discrimination network according to the first loss function to obtain an updated discrimination network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.).
In regards to Claim 7, Tong as modified by Aguinaldo further teaches the method according to claim 6, wherein training the second neural network comprises: inputting second sample data into the first neural network and the second neural network, respectively, so as to obtain third data output from the first neural network and fourth data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss; obvious to one skilled in the art that the input of data can be repeated on various datasets); setting the fourth data to have a true value tag, and inputting the fourth data with the true value tag into the updated discrimination network to obtain a third discrimination result output from the discrimination network; calculating an error function based on the third data and the fourth data (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):), calculating a discrimination function based on the third discrimination result, and calculating a second loss function based on the error function and the discrimination function (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting the parameters of the second neural network according to the second loss function to obtain an updated second neural network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.), wherein the second loss function is a weighted sum of the error function and the discrimination function; a weight of the error function is in a range of 90 to 110, and a weight of the discrimination function is in a range of 0.5 to 2 (Aguinaldo: To combat the blurriness, we propose a joint loss function that supervises regular GAN training with MSE loss. Specifically, the joint loss train the student by solving the following optimization problem: Eq. 4; The α parameter controls the weight between the MSE loss and the regular GAN training.; obvious to one skilled in the art to tune the weights of the function based on the needs of the GAN); and the first sample data and the second sample data is image data obtained based on a plurality of videos having a same bitrate (Tong: 4.1. Datasets and metrics; 50,000 images were randomly selected from ImageNet for the training; obvious to one skilled in the art to change the input images based on the needs of the network).
In regards to Claim 8, Tong as modified by Aguinaldo further teaches the method according to claim 4, wherein training the second neural network to be trained based on the first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolutional neural network, comprises: inputting third sample data into the first neural network and the second neural network, respectively, so as to obtain fifth data output from the first neural network and sixth data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss; obvious to one skilled in the art that the input of data can be repeated on various datasets); calculating a third loss function based on the fifth data and the sixth data (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting the parameters of the second neural network according to the third loss function to obtain an updated second neural network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong) in view of Aguinaldo et al. (NPL: Compressing GANs using Knowledge Distillation, hereafter referred as Aguinaldo) and Ronneberger et al. (NPL: U-Net: Convolutional Networks for Biomedical Image Segmentation, hereafter referred as Ronneberger).
In regards to Claim 9, Tong as modified by Aguinaldo fails to further teach the method according to claim 4, wherein the first neural network comprises a plurality of stages of down-sampling units and a plurality of stages of up-sampling units corresponding to the plurality of stages of down-sampling units, an output of each stage of down-sampling unit serves as an input of a next stage of down-sampling unit, and an input of each stage of up-sampling unit comprises an output of a stage of down-sampling unit corresponding to the stage of up-sampling unit and an output of a previous stage of up-sampling unit of the stage of up-sampling unit.
Ronneberger, like Tong, is directed to image processing. Ronneberger does teach wherein the first neural network comprises a plurality of stages of down-sampling units and a plurality of stages of up-sampling units corresponding to the plurality of stages of down-sampling units, an output of each stage of down-sampling unit serves as an input of a next stage of down-sampling unit, and an input of each stage of up-sampling unit comprises an output of a stage of down-sampling unit corresponding to the stage of up-sampling unit and an output of a previous stage of up-sampling unit of the stage of up-sampling unit (Ronneberger: 2 Network Architecture and Fig. 1; The network architecture is illustrated in Figure 1. It consists of a contracting path (left side) and an expansive path (right side). The contracting path follows the typical architecture of a convolutional network. It consists of the repeated application of two 3x3 convolutions (unpadded convolutions), each followed by a rectified linear unit (ReLU) and a 2x2 max pooling operation with stride 2 for downsampling. At each downsampling step we double the number of feature channels. Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 convolution (“up-convolution”) that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3x3 convolutions, each fol lowed by a ReLU. The cropping is necessary due to the loss of border pixels in every convolution. At the final layer a 1x1 convolution is used to map each 64-component feature vector to the desired number of classes. In total the net work has 23 convolutional layers.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tong to utilize the down-sample and up-sampling technique, as taught by Ronneberger, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Ronnerberger, the proposed modification allows the strong use of data augmentation to use the available annotated samples more efficiently (Ronneberger: Abstract).
Claims 10-13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (C.N. Patent Pub No. 107197260 A, hereafter referred as Zhang) in view of Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong).
Regarding Claim 10:
Zhang further teaches a video processing method, comprising: obtaining an input video bitrate and an input video, the input video comprising a plurality of input image frames (Zhang: S2 the compressed video and the original video frame extraction, to obtain a plurality of frames, each said frame including a compressed video frame and an original video frame); and selecting, according to the input video bitrate, a video processing module corresponding to the input video bitrate to process at least one input image frame among the plurality of input image frames, so as to obtain at least one output image frame, wherein different input video bitrates correspond to different video processing modules (Zhang: S3, the frame extracting step S2 according to frame type and different quantization parameter divided into a plurality of groups; S4, building the convolutional neural network frame and initializing network parameter, using step S3 divided group respectively training the neural network to obtain a plurality of neural network model corresponding to the different quantization parameters and frame type; obtained in the all frames according to the frame type and a different quantization parameter QP divided into a plurality of groups, specifically dividing process such as: all frames to be divided according to different QP (32), then the frame of each QP according to frame types are divided into I frame, P frame and B frame so as to obtain a plurality of groups of the (obtained), in this example, frame of each group having the same frame type and the quantization parameter QP.), wherein denoising intensity of a neural network of a video processing module corresponding to a first input video bitrate is lower than denoising intensity of a neural network of a video processing module corresponding to a second input video bitrate, and the first input video bitrate is higher than the second input video bitrate (Zhang: the post filtering processing step comprises S5 and S6: S5 a plurality of neural network model obtained in step S4 is embedded to the post filtering process of the video encoder; S6 to be processed of the original video performing step S1 and S2 to obtain to-be-processed frame and processed frame according to the quantization parameter and frame type selecting corresponding neural network model to filter processing.).
Zhang fails to further teach wherein a definition of the at least one output image frame is higher than a definition of the at least one input image frame.
Tong, like Zhang, is directed to image processing. Zhang does teach wherein a definition of the at least one output image frame is higher than a definition of the at least one input image frame (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang to utilize the singe image super resolution technique, as taught by Tong, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Tong, the proposed modification would substantially reduces the number of parameters, enhancing the computational efficiency (Tong: Abstract).
In regards to Claim 11, Zhang as modified by Tong further teaches the method according claim 10, wherein the input image frame serves as an input image, the output image frame serves as an output image, and processing the input image frame to obtain the output image frame comprises (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed): processing the input image by using a convolutional neural network to obtain the output image, wherein processing the input image by using the convolutional neural network to obtain the output image comprises: performing feature extraction on the input image to obtain a plurality of first images, and concatenating the input image and the plurality of first images to obtain a first image group, wherein the first image group comprises the input image and the plurality of first images (Tong: 3. Method; As shown in Figure 1, SRDenseNet can be decomposed into several parts: the convolution layer for learning low-level feature, the blocks of DenseNet for learning high level features, the deconvolution layers for learning upscal ing filters and the reconstruction layer for generating the HRoutput; 3.1. DenseNet blocks; After applying a convolution layer to the input LR im ages for learning low-level features, a set of DenseNet blocks are adopted for learning the high-level features; Consequently, the ith layer receives the feature maps of all preceding layers as input: Xi = max(0,wi ∗[X1,X2,...,Xi−1] +bi) (3) where [X1,X2,...,Xi−1] represents the concatenation of the feature maps generated in the preceding convolution layers 1,2,..., i − 1.); performing the feature extraction on the first image group to obtain a plurality of second images, fusing the plurality of second images and the plurality of first images to obtain a plurality of third images, and concatenating the input image and the plurality of third images to obtain a second image group, wherein the second image group comprises the input image and the plurality of third images (Tong: 3. Method; As shown in Figure 1, SRDenseNet can be decomposed into several parts: the convolution layer for learning low-level feature, the blocks of DenseNet for learning high level features, the deconvolution layers for learning upscal ing filters and the reconstruction layer for generating the HRoutput; 3.1. DenseNet blocks; After applying a convolution layer to the input LR im ages for learning low-level features, a set of DenseNet blocks are adopted for learning the high-level features; Consequently, the ith layer receives the feature maps of all preceding layers as input: Xi = max(0,wi ∗[X1,X2,...,Xi−1] +bi) (3) where [X1,X2,...,Xi−1] represents the concatenation of the feature maps generated in the preceding convolution layers 1,2,..., i – 1; 2.3. Contribution; We demonstrate that the deep CNN framework with the denseNet as basic blocks can achieve good recon struction performance and that the fusion of features at different levels through dense skip connections can further boost the reconstruction performance for SISR.); and performing the feature extraction on the second image group to obtain the output image (Tong: 3.3. Combination of feature maps; only the feature maps at the top layer are used as input for reconstructing the HR output; Further, a skip connection is introduced in the network as shown in Figure 1 (b) to concatenate the low-level and high-level features, which we term SR DenseNet HL.The concatenated feature maps are then used as input for deconvolution layers. In addition, we use dense skip connections to combine the feature maps produced at all convolution layers for SR reconstruction, and denote this method as SRDenseNet All.), wherein a count of convolution kernels, used for the feature extraction on the input image, in the convolutional neural network is N, 12 ≤ N ≤ 20, and N is an integer, a count of convolution kernels, used for the feature extraction on the first image group, in the convolutional neural network is M, 12 ≤ M ≤ 20, and M is an integer (Tong: 3.1. DenseNet blocks; Specifically, there are 8 convolution lay ers in one DenseNet block in our work. If each convolution layer produce k feature maps as output, the total number of feature maps generated by one DenseNet block is k ∗ 8, where k is refered to as growth rate. The growth rate k reg ulates how much new information each layer contributes to the final reconstruction. To prevent the network from grow ing too wide, the growth rate k is set to 16 in this study. This results in a total number of 128 feature maps from one DenseNet block.).
In regards to Claim 12, Zhang as modified by Tong further teaches the method according to claim 11, wherein N = M = 16 (Tong: 3.1. DenseNet blocks; Specifically, there are 8 convolution lay ers in one DenseNet block in our work. If each convolution layer produce k feature maps as output, the total number of feature maps generated by one DenseNet block is k ∗ 8, where k is refered to as growth rate. The growth rate k reg ulates how much new information each layer contributes to the final reconstruction. To prevent the network from grow ing too wide, the growth rate k is set to 16 in this study. This results in a total number of 128 feature maps from one DenseNet block.); an activation function used for the feature extraction in the convolutional neural network is y = max (0, x), where x represents an input of the activation function and y represents an output of the activation function (Tong: 3. Method; The ReLu activation function is ap plied element-wise. Let Xi−1 be the input, the output of ith convolution or deconvolution layer is expressed as: Xi = max(0,wi ∗Xi−1 +bi) where Wi and Bi are the weights and biases in the layer, and ∗ denotes either convolution or deconvolution opera tion for the convenience of formulation.).
In regards to Claim 13, Zhang as modified by Tong further teaches the method according to claim 12, wherein a count of convolution kernels, used for the feature extraction on the second image group, in the convolution neural network is 3; a size of the convolution kernels used for the feature extraction on the input image, a size of the convolution kernels used for the feature extraction on the first image group, and a size of the convolution kernels used for the feature extraction on the second image group are all 3×3 (Tong: 3.2. Deconvolution layers; In our work, two successive deconvolution layers with small 3×3 kernels and 256 feature maps are trained for upscaling; 3.4. Bottleneck and Reconstruction layers; Finally, the feature maps in the HR space are used to generate HR images via a reconstruction layer. The reconstruction layer is a convolution layer with 3×3 kernel and one channel of output.); and the input image comprises a red channel input image, a green channel input image, and a blue channel input image, and the output image comprises a red channel output image, a green channel output image, and a blue channel output image (Tong: Fig. 3; Super-resolution results for “img096” (top figure),“img099” (middle figure) and “img004” (bottom figure) from Urban100 with an upscaling factor of 4; 4.1. Datasets and metrics; PSNR and SSIM values are shown on the top of each sub-figure; showcases colored input and output images; Since SR was performed in the luminance chan nel in YCbCr colour space, the PSNR and SSIM were cal culated on the Y-channel of images.).
Regarding Claim 20:
Zhang as modified by Tong further teaches a processing apparatus, comprising: a processor; and a memory, comprising one or a plurality of computer program modules, wherein the one or plurality of computer program modules are stored in the memory and configured to be executed by the processor, and the one or plurality of computer program modules are configured to execute a video processing method (Zhang: The invention relates to computer vision and video coding field, specifically claims a convolutional neural network-based video encoding post-filtering method.), the video processing method comprises: obtaining an input video bitrate and an input video, the input video comprising a plurality of input image frames (Zhang: S2 the compressed video and the original video frame extraction, to obtain a plurality of frames, each said frame including a compressed video frame and an original video frame); and selecting, according to the input video bitrate, a video processing module corresponding to the input video bitrate to process at least one input image frame among the plurality of input image frames, so as to obtain at least one output image frame (Zhang: S3, the frame extracting step S2 according to frame type and different quantization parameter divided into a plurality of groups; S4, building the convolutional neural network frame and initializing network parameter, using step S3 divided group respectively training the neural network to obtain a plurality of neural network model corresponding to the different quantization parameters and frame type; obtained in the all frames according to the frame type and a different quantization parameter QP divided into a plurality of groups, specifically dividing process such as: all frames to be divided according to different QP (32), then the frame of each QP according to frame types are divided into I frame, P frame and B frame so as to obtain a plurality of groups of the (obtained), in this example, frame of each group having the same frame type and the quantization parameter QP.), wherein a definition of the at least one output image frame is higher than a definition of the at least one input image frame (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed), wherein different input video bitrates correspond to different video processing modules, wherein denoising intensity of a neural network of a video processing module corresponding to a first input video bitrate is lower than denoising intensity of a neural network of a video processing module corresponding to a second input video bitrate, and the first input video bitrate is higher than the second input video bitrate (Zhang: the post filtering processing step comprises S5 and S6: S5 a plurality of neural network model obtained in step S4 is embedded to the post filtering process of the video encoder; S6 to be processed of the original video performing step S1 and S2 to obtain to-be-processed frame and processed frame according to the quantization parameter and frame type selecting corresponding neural network model to filter processing.).
Claims 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (C.N. Patent Pub No. 107197260 A, hereafter referred as Zhang) in view of Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong) and Aguinaldo et al. (NPL: Compressing GANs using Knowledge Distillation, hereafter referred as Aguinaldo).
In regards to Claim 14, Zhang as modified by Tong and Aguinaldo further teaches the method according to claim 11, further comprising: training a second neural network to be trained based on a first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolution neural network (Aguinaldo: Abstract; we propose a method to compress GANs using knowledge distillation techniques, in which a smaller “student” GAN learns to mimic a larger “teacher” GAN), wherein parameters of the first neural network are more than parameters of the second neural network (Aguinaldo: 3. Methods; The teacher (large, over-parameterized network) and student (small, few parameter network) GANs used either the original DCGAN architecture or a slightly modified DCGAN architecture (Radford et al., 2015), more closely resembling the WGAN (Arjovsky et al., 2017), referenced as the W DCGAN.), the first neural network which is pre-trained is configured to transform an original image having a first definition, which is input to the first neural network that is pre-trained, into a new image having a second definition, the second definition is greater than the first definition (Tong: 1. Introduction: we propose a novel super-resolution method termed SRDenseNet in which the dense connected convolutional networks were employed), the second neural network which is trained is the convolutional neural network, a network structure of the second neural network to be trained is same as a network structure of the convolutional neural network, and parameters of the second neural network to be trained are different from parameters of the convolutional neural network (Aguinaldo: 3. Methods; The teacher (large, over-parameterized network) and student (small, few parameter network) GANs used either the original DCGAN architecture or a slightly modified DCGAN architecture (Radford et al., 2015), more closely resembling the WGAN (Arjovsky et al., 2017), referenced as the W DCGAN.). The reasons for combining the references are the same as those discussed above in conjunction with Claim 4 mutatis mutandis.
In regards to Claim 15, Zhang as modified by Tong and Aguinaldo further teaches the method according to claim 14, wherein training the second neural network to be trained based on the first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolutional neural network, comprises: based on the first neural network which is pre-trained, the second neural network to be trained, and a discrimination network, alternately training the discrimination network and the second neural network to obtain the second neural network which is trained, so as to obtain the convolutional neural network (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):).
In regards to Claim 16, Zhang as modified by Tong and Aguinaldo further teaches the method according to claim 15, wherein training the discrimination network comprises: inputting first sample data into the first neural network and the second neural network, respectively, so as to obtain first data output from the first neural network and second data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss.); setting the first data to have a true value tag, inputting the first data with the true value tag into the discrimination network to obtain a first discrimination result, setting the second data to have a false value tag, and inputting the second data with the false value tag into the discrimination network to obtain a second discrimination result (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):); calculating a first loss function based on the first discrimination result and the second discrimination result (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting parameters of the discrimination network according to the first loss function to obtain an updated discrimination network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.).
In regards to Claim 17, Zhang as modified by Tong and Aguinaldo further teaches the method according to claim 16, wherein training the second neural network comprises: inputting second sample data into the first neural network and the second neural network, respectively, so as to obtain third data output from the first neural network and fourth data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss; obvious to one skilled in the art that the input of data can be repeated on various datasets; setting the fourth data to have a true value tag, and inputting the fourth data with the true value tag into the updated discrimination network to obtain a third discrimination result output from the discrimination network; calculating an error function based on the third data and the fourth data (Aguinaldo: 2.1. Generative Adversarial Networks; GAN was first proposed as a two player min-max optimization problem between a discriminator fw(.) and a generator gθ(.) as in (1) (Goodfellow et al., 2014). The generator is tasked with generating realistic examples that fool the discriminator while the discriminator learns how to differentiate between the real and the generated samples; 4.2. Frechet Inception Distance (FID); The Frechet Inception Distance assumes that when differing images are fed through the same network, their correspond ing values from the same activation layer will have different distributions. If the activation distributions of the generated images and the real images differ greatly, then it is likely that the generated images look significantly different from the real images and vice versa. Formally, Frechet Inception Dis tance measures the difference of the activation distributions with Frechet distance (Heusel et al., 2017):), calculating a discrimination function based on the third discrimination result, and calculating a second loss function based on the error function and the discrimination function (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting the parameters of the second neural network according to the second loss function to obtain an updated second neural network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.), wherein the second loss function is a weighted sum of the error function and the discrimination function; a weight of the error function is in a range of 90 to 110, and a weight of the discrimination function is in a range of 0.5 to 2 (Aguinaldo: To combat the blurriness, we propose a joint loss function that supervises regular GAN training with MSE loss. Specifically, the joint loss train the student by solving the following optimization problem: Eq. 4; The α parameter controls the weight between the MSE loss and the regular GAN training.; obvious to one skilled in the art to tune the weights of the function based on the needs of the GAN); and the first sample data and the second sample data is image data obtained based on a plurality of videos having a same bitrate (Zhang: in this example, frame of each group having the same frame type and the quantization parameter QP.).
In regards to Claim 18, Zhang as modified by Tong and Aguinaldo further teaches the method according to claim 14, wherein training the second neural network to be trained based on the first neural network which is pre-trained to obtain the second neural network which is trained, so as to obtain the convolutional neural network, comprises: inputting third sample data into the first neural network and the second neural network, respectively, so as to obtain fifth data output from the first neural network and sixth data output from the second neural network (Aguinaldo: 3.2.Training of Student Networks; This method uses the MSE as the student training loss function using a pre-trained teacher W-DCGAN. A schematic of the training framework is illustrated in Figure 3. The MSE loss minimizes the pixel level error between the images generated from the student and the teacher; The generated images tend to be slightly blurry when using the MSE loss, especially for the Celeb-A dataset. To combat the blurriness ,we propose a joint loss function that supervises regular GAN training with MSE loss; obvious to one skilled in the art that the input of data can be repeated on various datasets); calculating a third loss function based on the fifth data and the sixth data (Aguinaldo: 4.3. Variance of Laplacian (VoL); It thus highlights the regions of an image containing rapid intensity changes- the edges. A sharper image will have more well defined edges than a blurred image. The Variance of Laplacian metric quantifies the amount of edges in an image. Higher VoL corresponds to sharper images. We use this metric to compare the outputs of the student generator trained using the MSE loss versus those trained using the joint loss.); and adjusting the parameters of the second neural network according to the third loss function to obtain an updated second neural network (Aguinaldo: 5.1. MSE Loss Training; We compare the performance of the compressed GANs with the teacher WGAN and the regular WGANs of the same sizes. Ideally, the compressed GAN will perform close to the teacher WGAN and better than the regular WGANs of the same sizes. Again, because there currently does not exist an exact measure of visual quality, we use Inception Score and Frechet Inception Distance as proxies for performance; 3.2. Training of Student Networks; The α parameter controls the weight between the MSE loss and the regular GAN training.).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (C.N. Patent Pub No. 107197260 A, hereafter referred as Zhang) in view of Tong et al. (NPL: Image Super-Resolution Using Dense Skip Connections, hereafter referred as Tong), Aguinaldo et al. (NPL: Compressing GANs using Knowledge Distillation, hereafter referred as Aguinaldo), and Ronneberger et al. (NPL: U-Net: Convolutional Networks for Biomedical Image Segmentation, hereafter referred as Ronneberger).
In regards to Claim 19, Zhang as modified by Tong, Aguinaldo, and Ronneberger further teaches the method according to claim 14, wherein the first neural network comprises a plurality of stages of down-sampling units and a plurality of stages of up-sampling units corresponding to the plurality of stages of down-sampling units, an output of each stage of down-sampling unit serves as an input of a next stage of down-sampling unit, and an input of each stage of up-sampling unit comprises an output of a stage of down-sampling unit corresponding to the stage of up-sampling unit and an output of a previous stage of up-sampling unit of the stage of up-sampling unit (Ronneberger: 2 Network Architecture and Fig. 1; The network architecture is illustrated in Figure 1. It consists of a contracting path (left side) and an expansive path (right side). The contracting path follows the typical architecture of a convolutional network. It consists of the repeated application of two 3x3 convolutions (unpadded convolutions), each followed by a rectified linear unit (ReLU) and a 2x2 max pooling operation with stride 2 for downsampling. At each downsampling step we double the number of feature channels. Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 convolution (“up-convolution”) that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3x3 convolutions, each fol lowed by a ReLU. The cropping is necessary due to the loss of border pixels in every convolution. At the final layer a 1x1 convolution is used to map each 64-component feature vector to the desired number of classes. In total the net work has 23 convolutional layers.). The reasons for combining the references are the same as those discussed above in conjunction with Claim 9 mutatis mutandis.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENAE BITOR whose telephone number is (703)756-5563. The examiner can normally be reached Monday to Friday: 8:00 - 5:30 but off the 1st Friday of the biweek.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, GREG MORSE can be reached on (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RENAE A BITOR/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights