DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 6/11/2026 have been fully considered but they are not persuasive.
Claims 1-20 are pending in this application and have been considered below.
1st Argument:
The applicant argues that Xiao does not teach “obtain a plurality of difference maps between a first frame or a first feature map corresponding to the first frame, and a plurality of second feature maps corresponding to a second frame.”
Response:
Xiao shows the limitation "The computing system may generate a first feature map for the first frame and one or more second feature maps for the one or more second frames," par. 25 as referenced in the previous office action. Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009). Therefore, Xiao teaches to “obtain … a first feature map corresponding to the first frame, and a plurality of second feature maps corresponding to a second frame.”
2nd Argument:
The applicant argues that Song does not teach “obtain a plurality of third features amps and a plurality of fourth feature maps by performing a first pooling process based on a first size, and a second pooling process based on a second size, on the plurality of difference maps.”
Response:
Song shows the limitation "when an image having a size of 24×24 pixels is input in the neural network 1 of FIG. 1, the input image may be output as feature maps of 4 channels having a 20×20 size via a convolution operation between the input image and the kernel. Also, feature maps of 4 channels having a 10×10 size may be output by using only one or more of pixel values of the feature maps of 4 channels having the 20×20 size via a sub-sampling process. Methods for sub-sampling may include max-pooling," which teaches obtaining different-sized feature maps through a sub-sampling process like max-pooling applied to initial maps. The claim only requires "obtaining" feature maps by performing a pooling process, which Song accomplishes by giving an example of taking existing feature maps and pooling them to different sizes.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The IDSs dated 2/29/2024, 1/22/2025, and 12/12/2025 that have been previously considered remain placed in the application file.
1st Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 3, 4, 5, 6, 9, 10, 15, 16, 17, 18, 19, and 20 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.) in view of US Patent Publication 2019 0138898 A1, (Song et al.) and US Patent Publication 2021 0201071 A1, (Liao et al.).
Claim 1
Regarding Claim 1, Xiao et al. teach an image processing device comprising: at least one memory storing one or more instructions; ("memory 1304 includes main memory for storing instructions for processor 1302 to execute or data for processor 1302 to operate on," par. 77) and at least one processor configured to execute the one or more instructions to: ("memory 1304 includes main memory for storing instructions for processor 1302 to execute or data for processor 1302 to operate on," par. 77) obtain a plurality of difference maps between a first frame or a first feature map corresponding to the first frame, and a plurality of second feature maps corresponding to a second frame, ("The computing system may generate a first feature map for the first frame and one or more second feature maps for the one or more second frames," par. 25) obtain a plurality of modified difference maps by weighted-summing the plurality of third feature maps and the plurality of fourth feature maps; ("The computing system may generate, by the feature weighting module, a pixel-wise weighting map for each of the one or more up-sampled and warped second feature maps. The computing system may further multiply the pixel-wise weighting map with the corresponding up-sampled and warped second feature map to generate a reweighted feature map for the corresponding second frame," par. 47) identify any one collocated sample based on sizes of sample values of collocated samples of the plurality of modified difference maps corresponding to a current sample of the first frame; ("the feature reweighting module may be a 3-layer convolutional neural network, which may take the RGB-D of the zero-upsampled current frame as well as the zero-upsampled, warped previous frames as input, and generate a pixel-wise weighting map for each previous frame, with values between 0 and 10, where 10 is a hyperparameter," par. 47) and determine a motion vector of the current sample ("The determining may comprise identifying a motion vector for the corresponding second frame having the resolution lower than the target resolution and resizing the motion vector to the target resolution based on bilinear up-sampling," par. 44).
Xiao et al. does not explicitly teach all of wherein the plurality of difference maps indicates how much sample values of the first frame or the first feature map differ from sample values of the plurality of second feature maps at corresponding positions; to obtain a plurality of third feature maps and a plurality of fourth feature maps by performing a first pooling process based on a first size, and a second pooling process based on a second size, on the plurality of difference maps; and determine a filter kernel used to obtain one of the plurality of second feature maps corresponding to one of the plurality of modified difference maps comprising the identified collocated sample.
However, Song et al. teach to obtain a plurality of third feature maps and a plurality of fourth feature maps by performing a first pooling process based on a first size, and a second pooling process based on a second size, on the plurality of difference maps; ("when an image having a size of 24×24 pixels is input in the neural network 1 of FIG. 1, the input image may be output as feature maps of 4 channels having a 20×20 size via a convolution operation between the input image and the kernel. Also, feature maps of 4 channels having a 10×10 size may be output by using only one or more of pixel values of the feature maps of 4 channels having the 20×20 size via a sub-sampling process. Methods for sub-sampling may include max-pooling," par. 57) and determine a filter kernel used to obtain one of the plurality of second feature maps corresponding to one of the plurality of modified difference maps comprising the identified collocated sample ("As the convolution operation between the first feature map FM1 and the kernel is performed, a channel of the second feature map FM2 may be generated," par. 61).
Therefore, taking the teachings of Xiao et al. and Song et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al. to use the kernel division methods as taught by Song et al. The suggestion/motivation for doing so would have been that, “That is, semantic segmentation attempts to partition an image into semantically meaningful parts, and to classify the parts into classes. Semantic segmentation is a technique not only for identifying what is in the image, but also for precisely figuring out locations of objects in the image” as noted by the Song et al. disclosure in paragraph [0065], which also motivates combination because the combination would predictably have a higher accuracy as there is a reasonable expectation that the specific feature map generation method and the kernel division method would result in enhanced spatial resolution and more accurate boundary localization in the produced segmentation map; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Additionally, Liao et al. teach wherein the plurality of difference maps indicates how much sample values of the first frame or the first feature map differ from sample values of the plurality of second feature maps at corresponding positions ("The computation module 320 determines a content similarity based on a difference between the first feature map 312 and the second feature map," par. 49).
Therefore, taking the teachings of Xiao et al., Song et al., and Liao et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al. and the kernel division methods as taught by Song et al. to use the feature map difference indication as taught by Liao et al. The suggestion/motivation for doing so would have been that, “The corresponding pixels of the first feature map and the second feature map may be determined by aligning the grayscale source image 102 and the color reference image 104 (or the grayscale reference image 104.sub.1), which requires determination of a mapping between the two images. The mapping between one image and another image indicates correspondence between certain pixel locations in the one image and certain pixel locations in the other image, and thus may be referred to as correspondence of the images” as noted by the Liao et al. disclosure in paragraph [0049], which also motivates combination because the combination would predictably have a greater utility as there is a reasonable expectation that successfully mapping and correlating the pixel locations between the feature maps would yield predictable and accurate difference detection; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
The rejection of device claim 1 above applies mutatis mutandis to the corresponding limitations of method claim 15 while noting that the rejection above cites to both device and method disclosures. Claim 15 is mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 1 as noted above.
Xiao et al. do not explicitly teach all of wherein a first stride used in the first pooling process and a second stride used in the second pooling process are different from each other.
[AltContent: textbox (Figure 7 shows first kernel and second sub-kernel.)]
PNG
media_image1.png
422
592
media_image1.png
Greyscale
However, Song et al. teach wherein a first stride used in the first pooling process and a second stride used in the second pooling process are different from each other ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 3
Regarding claim 3, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 2 as noted above.
Xiao et al. do not explicitly teach all of wherein the first size and the first stride are greater than the second size and the second stride.
However, Song et al. teach wherein the first size and the first stride are greater than the second size and the second stride ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 4
Regarding claim 4, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 3 as noted above.
Xiao et al. do not explicitly teach all of wherein the first size and the first stride are k and k is a natural number, and wherein the second size and the second stride are k/2.
However, Song et al. teach wherein the first size and the first stride are k and k is a natural number, and wherein the second size and the second stride are k/2 ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 5
Regarding claim 5, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 1 as noted above.
Song et al. do not explicitly teach all of wherein the at least one processor is further configured to execute the one or more instructions to obtain, from a neural network, a first weight applied to the plurality of third feature maps, and a second weight applied to the plurality of fourth feature maps.
However, Xiao et al. teach wherein the at least one processor is further configured to execute the one or more instructions to obtain, from a neural network, a first weight applied to the plurality of third feature maps, and a second weight applied to the plurality of fourth feature maps ("this subnetwork may process each input frame individually and share weights across all frames except for the current frame … In particular embodiments, the initial feature map may be based on a first number of channels whereas each of the first feature map and the one or more second feature maps may be based on a second number of channels," par. 42).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 6
Regarding claim 6, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 1 as noted above.
Xiao et al. teach wherein the at least one processor is further configured to execute the one or more instructions to: ("memory 1304 includes main memory for storing instructions for processor 1302 to execute or data for processor 1302 to operate on," par. 77) obtain the plurality of modified difference maps by weighted-summing the plurality of third feature maps and the plurality of fourth feature maps, ("The computing system may generate, by the feature weighting module, a pixel-wise weighting map for each of the one or more up-sampled and warped second feature maps. The computing system may further multiply the pixel-wise weighting map with the corresponding up-sampled and warped second feature map to generate a reweighted feature map for the corresponding second frame," par. 47) based on a first preliminary weight and a second preliminary weight that are output from a neural network; ("this subnetwork may process each input frame individually and share weights across all frames except for the current frame … In particular embodiments, the initial feature map may be based on a first number of channels whereas each of the first feature map and the one or more second feature maps may be based on a second number of channels," par. 42) determine motion vectors corresponding to samples of the first frame, from the plurality of modified difference maps; ("In particular embodiments, the computing system may determine the motion estimation between the associated second time and the first time," par. 44) and motion-compensate the second frame based on the motion vectors, ("In particular embodiments, generating the reconstructed frame corresponding to the first frame may comprise combining the up-sampled first feature map and the reweighted feature maps associated with the one or more second frames," par. 49) and wherein the neural network is trained based on first loss information corresponding to a difference between the motion-compensated second frame and the first frame ("The training loss of our method, as given in Eq. (1), may be a weighted combination of the perceptual loss computed from a pretrained VGG-16 network and the structural similarity index (SSIM)," par. 51).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 9
Regarding claim 9, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 1 as noted above.
Xiao et al. teach wherein each of the first pooling process and the second pooling process comprises an average pooling process or a median pooling process ("The embodiments disclosed herein demonstrate the first learned super-sampling method that achieves significant 4×4 super-sampling with high spatial and temporal fidelity," par. 34).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 10
Regarding claim 10, Xiao et al., Song et al., and Liao et al. teach the image processing device of claim 1 as noted above.
[AltContent: textbox (Figure 4B shows the network and sub-network architecture.)]
PNG
media_image2.png
446
616
media_image2.png
Greyscale
Xiao et al. teach wherein the first feature map is obtained through first convolution processing on the first frame based on a first filter kernel, and wherein the plurality of second feature maps are obtained through second convolution processing on the second frame based on a plurality of second filter kernels ("FIG. 4A illustrates an example network architecture of our method. FIG. 4B illustrates example sub-networks of the example network architecture. The sub-networks may include the feature extraction, feature reweighting, and reconstruction networks. The numbers under each network layer represent the output channels at corresponding layers. The filter size is 3×3 at all layers," par. 42).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 15
Regarding Claim 15, Xiao et al. teach an image processing method performed by an image processing device, the image processing method comprising: obtaining a plurality of difference maps between a first frame or a first feature map corresponding to the first frame, and a plurality of second feature maps corresponding to a second frame ("The computing system may generate a first feature map for the first frame and one or more second feature maps for the one or more second frames," par. 25) obtaining a plurality of modified difference maps by weighted-summing the plurality of third feature maps and the plurality of fourth feature maps; ("The computing system may generate, by the feature weighting module, a pixel-wise weighting map for each of the one or more up-sampled and warped second feature maps. The computing system may further multiply the pixel-wise weighting map with the corresponding up-sampled and warped second feature map to generate a reweighted feature map for the corresponding second frame," par. 47) identifying any one collocated sample by considering sizes of sample values of collocated samples of the plurality of modified difference maps corresponding to a current sample of the first frame; ("the feature reweighting module may be a 3-layer convolutional neural network, which may take the RGB-D of the zero-upsampled current frame as well as the zero-upsampled, warped previous frames as input, and generate a pixel-wise weighting map for each previous frame, with values between 0 and 10, where 10 is a hyperparameter," par. 47) and determining a motion vector of the current sample ("The determining may comprise identifying a motion vector for the corresponding second frame having the resolution lower than the target resolution and resizing the motion vector to the target resolution based on bilinear up-sampling," par. 44).
Xiao et al. do not explicitly teach all of wherein the plurality of difference maps indicates how much sample values of the first frame or the first feature map differ from sample values of the plurality of second feature maps at corresponding positions; obtaining a plurality of third feature maps and a plurality of fourth feature maps by performing a first pooling process based on a first size, and a second pooling process based on a second size, on the plurality of difference maps; determining a filter kernel used to obtain one of the plurality of second feature maps corresponding to one of the plurality of modified difference maps comprising the identified collocated sample.
However, Song et al. teach obtaining a plurality of third feature maps and a plurality of fourth feature maps by performing a first pooling process based on a first size, and a second pooling process based on a second size, on the plurality of difference maps; ("when an image having a size of 24×24 pixels is input in the neural network 1 of FIG. 1, the input image may be output as feature maps of 4 channels having a 20×20 size via a convolution operation between the input image and the kernel. Also, feature maps of 4 channels having a 10×10 size may be output by using only one or more of pixel values of the feature maps of 4 channels having the 20×20 size via a sub-sampling process. Methods for sub-sampling may include max-pooling," par. 57) determining a filter kernel used to obtain one of the plurality of second feature maps corresponding to one of the plurality of modified difference maps comprising the identified collocated sample ("As the convolution operation between the first feature map FM1 and the kernel is performed, a channel of the second feature map FM2 may be generated," par. 61).
Additionally, Liao et al. teach wherein the plurality of difference maps indicates how much sample values of the first frame or the first feature map differ from sample values of the plurality of second feature maps at corresponding positions ("The computation module 320 determines a content similarity based on a difference between the first feature map 312 and the second feature map," par. 49).
Xiao et al., Song et al., and Liao et al. are combined as per claim 1.
Claim 16
Regarding claim 16, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 15 as noted above.
Xiao et al. do not explicitly teach all of wherein a first stride used in the first pooling process, and a second stride used in the second pooling process, are different from each other.
However, Song et al. teach wherein a first stride used in the first pooling process, and a second stride used in the second pooling process, are different from each other ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 15.
Claim 17
Regarding claim 17, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 16 as noted above.
Xiao et al. do not explicitly teach all of wherein the first size and the first stride are greater than the second size and the second stride.
However, Song et al. teach wherein the first size and the first stride are greater than the second size and the second stride ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 15.
Claim 18
Regarding claim 18, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 17 as noted above.
Xiao et al. do not explicitly teach all of wherein the first size and the first stride are k and k is a natural number, and wherein the second size and the second stride are k/2.
However, Song et al. teach wherein the first size and the first stride are k and k is a natural number, and wherein the second size and the second stride are k/2 ("The neural network apparatus may generate the sub-kernels 540 by dividing the second kernel 530 based on a stride value. According to an embodiment, the neural network apparatus may divide the second kernel 530 into the sub-kernels 540, the number of the sub-kernels 540 corresponding to a value obtained by squaring the stride value," par. 79).
Xiao et al., Song et al., and Liao et al. are combined as per claim 15.
Claim 19
Regarding claim 19, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 15 as noted above.
Song et al. do not explicitly teach all of obtaining, from a neural network, a first weight applied to the plurality of third feature maps, and a second weight applied to the plurality of fourth feature maps.
However, Xiao et al. teach obtaining, from a neural network, a first weight applied to the plurality of third feature maps, and a second weight applied to the plurality of fourth feature maps ("this subnetwork may process each input frame individually and share weights across all frames except for the current frame … In particular embodiments, the initial feature map may be based on a first number of channels whereas each of the first feature map and the one or more second feature maps may be based on a second number of channels," par. 42).
Xiao et al., Song et al., and Liao et al. are combined as per claim 15.
Claim 20
Regarding claim 20, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 15 as noted above.
Xiao et al. teach obtaining the plurality of modified difference maps by weighted-summing the plurality of third feature maps and the plurality of fourth feature maps, ("The computing system may generate, by the feature weighting module, a pixel-wise weighting map for each of the one or more up-sampled and warped second feature maps. The computing system may further multiply the pixel-wise weighting map with the corresponding up-sampled and warped second feature map to generate a reweighted feature map for the corresponding second frame," par. 47) based on a first preliminary weight and a second preliminary weight that are output from a neural network; ("this subnetwork may process each input frame individually and share weights across all frames except for the current frame … In particular embodiments, the initial feature map may be based on a first number of channels whereas each of the first feature map and the one or more second feature maps may be based on a second number of channels," par. 42) determining motion vectors corresponding to samples of the first frame, from the plurality of modified difference maps; ("In particular embodiments, the computing system may determine the motion estimation between the associated second time and the first time," par. 44) and motion-compensating the second frame based on the motion vectors, ("In particular embodiments, generating the reconstructed frame corresponding to the first frame may comprise combining the up-sampled first feature map and the reweighted feature maps associated with the one or more second frames," par. 49) wherein the neural network is trained based on first loss information corresponding to a difference between the motion-compensated second frame and the first frame ("The training loss of our method, as given in Eq. (1), may be a weighted combination of the perceptual loss computed from a pretrained VGG-16 network and the structural similarity index (SSIM)," par. 51).
Xiao et al., Song et al., and Liao et al. are combined as per claim 15.
2nd Claim Rejections - 35 USC § 103
Claim 7 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.), US Patent Publication 2019 0138898 A1, (Song et al.), and US Patent Publication 2021 0201071 A1, (Liao et al.) in view of US Patent Publication 2023 11604993 B1, (Tan).
Claim 7
Regarding Claim 7, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 6 as noted above.
Xiao et al. and Song et al. do not explicitly teach all of wherein the neural network is trained further based on second loss information indicating how much a sum of the first preliminary weight and the second preliminary weight differs from a predetermined threshold.
However, Tan teaches wherein the neural network is trained further based on second loss information indicating how much a sum of the first preliminary weight and the second preliminary weight differs from a predetermined threshold ("The difference may comprise a loss calculated between the target weights and weights specified by the filter. In some examples, the techniques may comprise adjusting parameters associated with a filter of a convolutional layer based at least in part on the second loss (e.g., adjusting weights of the filter to reduce a magnitude of the second loss)," col. 2, line 34).
Therefore, taking the teachings of Xiao et al., Song et al., Liao et al., and Tan as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al., the kernel division methods as taught by Song et al., and the feature map difference indication as taught by Liao et al. to use the weight difference threshold calculation as taught by Tan. The suggestion/motivation for doing so would have been that, “(e.g., adjusting weights of the filter to reduce a magnitude of the second loss). This may functionally drive parameters associated with the filter towards the target parameters” as noted by the Tan disclosure in paragraph [10], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that the optimized filter weights derived from the threshold calculation would result in faster convergence and improved accuracy of the neural network model; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
3rd Claim Rejections - 35 USC § 103
Claim 8 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.), US Patent Publication 2019 0138898 A1, (Song et al.), and US Patent Publication 2021 0201071 A1, (Liao et al.) in view of US Patent Publication 2022 0108183 A1, (Arpit).
Claim 8
Regarding Claim 8, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 6 as noted above.
Xiao et al. and Song et al. do not explicitly teach all of wherein the neural network is trained further based on third loss information indicating how small negative values of the first preliminary weight and the second preliminary weight are.
However, Arpit teaches wherein the neural network is trained further based on third loss information indicating how small negative values of the first preliminary weight and the second preliminary weight are ("The training aims to minimize the loss L(Enc,Dec;λ,τ,B,K) based on the theory above, where λ is the regularization weight, τ is the temperature hyperparameter, B is the mini-batch size, and K≥B is the number of samples used to estimate the negative component of the constructive loss," par. 33).
Therefore, taking the teachings of Xiao et al., Song et al., Liao et al., and Arpit as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al., the kernel division methods as taught by Song et al., and the feature map difference indication as taught by Liao et al. to use the weight regularization techniques as taught by Arpit. The suggestion/motivation for doing so would have been that, “ The negative component is minimized when the representations of the latent variables z_q are uniformly distributed over the unit hyper-sphere. Minimizing the negative component may minimize the contrastive loss” as noted by the Arpit disclosure in paragraph [0043], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that the resulting model would produce more uniformly distributed representations, thereby reducing contrastive loss and improving feature representation quality/efficiency; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
4th Claim Rejections - 35 USC § 103
Claim 11 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.), US Patent Publication 2019 0138898 A1, (Song et al.), and US Patent Publication 2021 0201071 A1, (Liao et al.) in view of US Patent Publication 2022 0292806 A1, (Su et al.).
Claim 11
Regarding Claim 11, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 10 as noted above.
Xiao et al. teach wherein a first distance between samples of the first frame on which a first convolution operation with the first filter kernel is performed, and a second distance between samples of the second frame on which a second convolution operation with the plurality of second filter kernels is performed ("generating the first feature map for the first frame and the one or more second feature maps for the one or more second frames may be based on one or more convolutional neural networks … The numbers under each network layer represent the output channels at corresponding layers. The filter size is 3×3 at all layers," par. 42).
Xiao et al. and Song et al. do not explicitly teach all a first distance between samples and a second distance between samples are greater than 1.
However, Su et al. teaches a first distance between samples and a second distance between samples are greater than 1 ("Accordingly, a dilation rate of three (D=3) 206 implies that between two kernel elements 208 there are two free spaces, and the size of the receptive field grows to 7×7," par. 61).
Therefore, taking the teachings of Xiao et al., Song et al., Liao et al., and Su et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al., the kernel division methods as taught by Song et al., and the feature map difference indication as taught by Liao et al. to use the dilated convolution techniques as taught by Su et al. The suggestion/motivation for doing so would have been that, “Dilated convolution, which means the convolution kernel has a dilation rate of at least two or higher, as illustrated in FIG. 2 may also have the effect of increasing the receptive field, while still capture well the local information” as noted by the Su et al. disclosure in paragraph [0060], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that the increased receptive field would allow for a broader context to be captured in the feature maps without a proportional increase in computational parameters or loss of resolution; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
5th Claim Rejections - 35 USC § 103
Claims 12 and 13 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.), US Patent Publication 2019 0138898 A1, (Song et al.), and US Patent Publication 2021 0201071 A1, (Liao et al.) in view of US Patent Publication 2018 0174031 A1, (Yang et al.).
Claim 12
Regarding Claim 12, Xiao et al., Song et al., and Liao et al. teach the image processing method of claim 10 as noted above.
Xiao et al. and Song et al. do not explicitly teach all of wherein, in the first filter kernel, a sample corresponding to the current sample of the first frame has a preset first value, and other samples of the first filter kernel have a value of 0.
However, Yang et al. teaches wherein, in the first filter kernel, a sample corresponding to the current sample of the first frame has a preset first value, and other samples of the first filter kernel have a value of 0 ("Each of the 3×3 kernels in the identity-value convolutional layer P.sub.1 2021 contains numerical value “0” except those kernels located on the diagonal of the N×N kernels. Each of the diagonal kernels 2022 contains numerical value “0” in each of the eight perimeter positions and “1” in the center position," par. 92).
Therefore, taking the teachings of Xiao et al., Song et al., Liao et al., and Yang et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al., the kernel division methods as taught by Song et al., and the feature map difference indication as taught by Liao et al. to use the filter kernels of an identity-value convolutional layer as taught by Yang et al. The suggestion/motivation for doing so would have been that, “As shown in FIG. 20C, the third replacement convolutional layer 2053 contains N×2N of 3×3 filter kernels formed by two identity-value convolutional layer P.sub.1 2021 each containing N×N of 3×3 filter kernels in a vertical stack. As a result, the third particular convolutional layer 2053 is configured for 2N-channel input and N-channel output” as noted by the Yang et al. disclosure in paragraph [0094], which also motivates combination because the combination would predictably have an additional utility as there is a reasonable expectation that the resulting convolutional layer would maintain structural integrity while effectively reducing computational complexity, enabling efficient processing of higher-channel inputs into lower-channel outputs; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 13
Regarding Claim 13, Xiao et al., Song et al., Liao et al., and Yang et al. teach the image processing method of claim 12 as noted above.
Xiao et al. and Song et al. do not explicitly teach all of the image processing device of claim 12, wherein, in the plurality of second filter kernels, any one sample has a preset second value, and other samples of the plurality of second filter kernels have a value of 0, and wherein positions of samples having the preset second value in the plurality of second filter kernels are different from each other.
PNG
media_image3.png
444
426
media_image3.png
Greyscale
[AltContent: textbox (Figure 20A shows the plurality of kernels in the identity-value layer.)]However, Yang et al. teaches the image processing device of claim 12, wherein, in the plurality of second filter kernels, any one sample has a preset second value, and other samples of the plurality of second filter kernels have a value of 0, and wherein positions of samples having the preset second value in the plurality of second filter kernels are different from each other ("Each of the 3×3 kernels in the identity-value convolutional layer P.sub.1 2021 contains numerical value “0” except those kernels located on the diagonal of the N×N kernels. Each of the diagonal kernels 2022 contains numerical value “0” in each of the eight perimeter positions and “1” in the center position," par. 92).
Xiao et al., Song et al., Liao et al., and Yang et al. are combined as per claim 12.
6th Claim Rejections - 35 USC § 103
Claim 14 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0366082 A1, (Xiao et al.), US Patent Publication 2019 0138898 A1, (Song et al.), US Patent Publication 2021 0201071 A1, (Liao et al.), and US Patent Publication 2018 0174031 A1, (Yang et al.) in view of US Patent Publication 2019 0205740 A1, (Judd et al.).
Claim 14
Regarding Claim 14, Xiao et al., Song et al., Liao et al., and Yang et al. teach the image processing method of claim 13 as noted above.
Xiao et al., Song et al., and Yang et al. do not explicitly teach all of wherein a sign of the preset first value and a sign of the preset second value are opposite to each other.
However, Judd et al. teaches wherein a sign of the preset first value and a sign of the preset second value are opposite to each other ("In the example of FIG. 3, the calculation of the complete filter would take one additional cycle, only the first three cycles are shown here. The elements of both filters have the same values with opposite signs,” par. 56).
Therefore, taking the teachings of Xiao et al., Song et al., Liao et al., Yang et al., and Judd et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify feature map generating methods as taught by Xiao et al., the kernel division methods as taught by Song et al., and the feature map difference indication as taught by Liao et al. to use the filter kernels of an identity-value convolutional layer as taught by Yang et al. to use the filter calculation as taught by Judd et al. The suggestion/motivation for doing so would have been that, the preset filter values as modified by having opposite signs can yield a predictable result of improved feature extraction since the oppositely signed filters allow for direct, simultaneous, or accelerated calculation of both positive and negative feature responses within a single pass, thereby reducing the total number of required filters and associated convolutional operations. Thus, a person of ordinary skill would have appreciated including in the preset filter values the opposite signs since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Karsten F. Lantz whose telephone number is (571)272-4564. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Karsten F. Lantz/Examiner, Art Unit 2664
Date: 7/29/2026
/PING Y HSIEH/Primary Examiner, Art Unit 2664
Prosecution Insights