Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on July 8, 2026, in which claims 1 and 2 are currently amended. Claims 3-8 are canceled. Claims 9-11 are newly added. Claims 1, 2, and 9-11 are currently pending.
Response to Arguments
The rejections to claims 1 and 2 under 35 U.S.C. § 112(b) are hereby withdrawn, as necessitated by applicant's amendments and remarks made to the rejections.
Applicant’s arguments with respect to rejection of claims 1 and 2 under 35 U.S.C. 103 based on amendment have been considered, however, are not persuasive.
With respect to Applicant's arguments on p. 6 of the Remarks submitted 7/8/2026 that "Jeon does disclose or suggest a super neuron model of a self-organized operational neural network in which a non-localized kernel operation uses a Kx x Ky kernel together with a composite nodal operator comprising a Qth-order Mac-Laurin-series nodal function", Examiner respectfully disagrees. A "self-organized operational neural network" is a coined term which is defined broadly in the instant specification and explicitly described as a superset of a convolutional neural network ([¶0013] "according to various embodiments, self-ONNs with super neurons may be a superset of CNNs with convolutional neurons") which encompasses the model in Jeon. The instant specification describes a non-localized kernel operation as introducing x/y spatial bias, shifting the previous layer output according to that bias, and then operating with the kernel ([¶0021] "Certain embodiments may perform the shift to obtain yk l(m+αk i, n+βk i), and then operate with the original Kx×Ky kernel, wik l+1") which reads very naturally on Jeon's architecture, where Jeon's abstract says "limited resources (e.g., mobile applications), heavy networks may not be usable. This study shows that naive convolution can be deconstructed into a shift operation and pointwise convolution. To cope with various convolutions, we propose a new shift operation called active shift layer (ASL) that formulates the amount of shift as a learnable function with shift parameters." In other words the instant specification describes non-localization in substantially the same functional manner that Jeon implements it. The linear CNN function in Jeon satisfies the mathematical definition of a Q=1 Maclaurin series and is interpreted as a nodal operator consistent with the instant specification ([¶0017] "as shown in FIG. 1, CNN and ONN neurons may have static nodal operators (linear and harmonic, respectively) [...] the generative-neuron can have any arbitrary nodal function, Ψ (including possibly standard functions such as linear and harmonic functions), for each kernel element of each connection"). In other words, the instant specification itself permits linear nodal functions and under ordinary mathematics, a linear nodal function can be represented by a first-order Maclaurin series (Q=1). For at least these reasons and those further detailed below, Examiner asserts that the interpretation of the combination of Jeon and Chung to cover the instant claims is very reasonable and should be maintained.
With respect to Applicant's arguments on p. 6 of the Remarks submitted 7/8/2026 that "Jeon also does not disclose parameters of such a composite nodal operator being customized during training for the kth output neuron connection", Examiner respectfully disagrees. The instant specification does not limit "customization" nor does it have ordinary meaning in the art that would limit the claim. Jeon's convolution is a learned (customized) pointwise-convolution weight matrix operating on shifted inputs, the shift component also having learned (customized) parameters.
With respect to Applicant's arguments on p. 6 of the Remarks submitted 7/8/2026 that "Jeon's active shift layer is not equivalent to the claimed Kx x Ky kernel operation. Claim 1 recites that the spatial bias is a deviation of a center of a Kx x Ky kernel from a pixel location and that the non-localized kernel operations are performed without altering kernel sizes. In contrast, Jeon deconstructs convolution into a shift operation and pointwise convolution. Jeon's channel-wise shift operation followed by pointwise convolution does not disclose or suggest the claimed non-localized Kx x Ky kernel operation using the recited composite nodal operator", Examiner respectfully disagrees. The claim does not require a physically monolithic Kx x Ky kernel to be moved as a block, nor does it require Kx or Ky to be greater than 1. A 1x1 kernel is a Kx x Ky kernel where Kx=Ky=1. Jeon shifts the spatial location from which that kernel receives its input and then applies the unchanged 1x1 kernel. That is precisely a non-localized kernel operation performed without changing kernel size. The instant specification itself describes the claimed operation by first shifting the previous layer output according to the spatial bias and then operating on the shifted information with the original Kx x Ky kernel ([¶0021] "perform the shift to obtain yk l(m+αk i, n+βk i), and then operate with the original Kx×Ky kernel, wik l+1"). For at least these reasons and those further detailed below, Examiner asserts that the interpretation of the combination of Jeon and Chung to cover the instant claims is very reasonable and should be maintained.
Claim Objections
Claim 1 objected to because of the following informalities: "Mac-Laurin-series" should read "Maclaurin series". Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 9, 10, and 11 are rejected under U.S.C. §103 as being unpatentable over the combination of Jeon (“Constructing Fast Network through Deconstruction of Convolution”, 2018) and Chung (US 20160379109 A1).
Regarding claim 1, Jeon teaches An apparatus comprising: at least one processor; and ([p. 1 §1] "facilitated by hardware developments such as graphics processing units (GPUs)" [p. 5] "Time is measured using an Intel i7-5930K CPU with a single thread and averaged over 100 repetitions")
[at least one memory storing] one or more super neuron models of a self-organized operational neural network with non-localized kernel operations, wherein([p. 1 §1] "this approach increases the inference and training times and consumes more memory" [p. 4] "a depthwise shift layer shifts each input channel [...] by removing both the spatial convolution and sparse memory access, it is possible to construct a network with only dense operations" [...] We called the new component the active shift layer (ASL)" The network with only dense operations is interpreted as the one or more super neuron models in view of the instant specification ([¶0022] "Various example embodiments may include a generative neuron model, including super neurons which may be an artificial neuron with a composite nodal-operator that can be generated during training without any restrictions" [¶0013] "according to various embodiments, self-ONNs with super neurons may be a superset of CNNs with convolutional neurons"). Depthwise shift layer interpreted as a non-localized kernel operation)
a set of additional parameters defines a spatial bias as the deviation of a center of a Kx x Ky kernel from the pixel location towards x- and y-direction ([p. 2] "We show that a convolution can be deconstructed into two components: 1×1 convolution and shift operation" [p. 4] "We formulated the shift values as a learnable function with the additional shift parameter θs that defines the amount of shift of each channel […] c is the index of the channel, and the parameters αc and βc define the horizontal and vertical amount of shift" Jeon explicitly formulates arbitrary spatial convolution and then shows it is the sum of 1x1 convolutions on shifted inputs. Thus Ks=Ky=1 is a species inside unrestricted Kx x Ky.)
for a kth output neuron connection to an input map for an ith neuron at layer l+1.([p. 3] "k is the spatial index of the kernel, which points from top-left to bottom-right of a spatial dimension of the kernel. Ik and jk are displacement values for the corresponding kernel index k" [p. 3] "shift function Sk that maps the original input to shifted input with integer-valued shift amounts for each kernel index k" [p. 3] "pointwise convolution is applied before and after the shift layer to make it invariant" [p. 6] "all layers in a network" Examiner notes that while the instant claims have not defined layer l, Jeon explicitly discloses pointwise convolution following the shift layer and that the neural network has multiple layers)
one or more generative neuron models configured to perform non-localized kernel operations without altering kernel sizes and with information flow diversity, ([p. 5] "ASL is independent of kernel size, and it can enlarge its receptive field by increasing the amount of shift" [p. 7] "Figure 3: Trained shift values of each layer. Shifted values are scattered to various positions. This enabled the network to cover multiple receptive fields" Shift layer which explicitly generates convolutional outputs interpreted as generative neuron model comprised by neural network model (super neuron model). Jeon's shift layer generates output activations via spatially shifted sampling (non-localized operation) and explicitly does so while remaining kernel-size independent (without altering kernel sizes) and explicitly yields scattered learning shifts covering multiple receptive fields (information flow diversity))
wherein at least one of the non-localized kernel operations uses the K, x K, kernel and a composite nodal operator comprising a Qth-order Mac-Laurin-series nodal function, and parameters of the composite nodal operator are customized during training for the kth output neuron connection,([Abstract] "limited resources (e.g., mobile applications), heavy networks may not be usable. This study shows that naive convolution can be deconstructed into a shift operation and pointwise convolution. To cope with various convolutions, we propose a new shift operation called active shift layer (ASL) that formulates the amount of shift as a learnable function with shift parameters" [p. 5] "shift values do not need to be assigned manually; instead, they are learned through backpropagation during training" a linear nodal function can be represented by a first-order Maclaurin series (Q=1) which is consistent with the instant specification ([¶0017] "as shown in FIG. 1, CNN and ONN neurons may have static nodal operators (linear and harmonic, respectively) [...] the generative-neuron can have any arbitrary nodal function, Ψ (including possibly standard functions such as linear and harmonic functions), for each kernel element of each connection"). Jeon explicitly learns (customizes) the shift parameters in training.)
wherein the one or more generative neuron models are configured to generate at least one particular pixel of a neuron in a layer associated with pixels of a larger area from at least one output map of at least one previous layer neuron,([p. 2] "we can simulate convolutions with large receptive fields such as a dilated convolution" [p. 6] "Large shift parameter values mean that a network can view a large receptive field" "Associated with pixels of a larger area from at least one output map of at least one previous layer neuron" interpreted as a large receptive field such as in dilated convolution. Jeon explicitly links learned shifts to the ability to simulate large receptive fields (including dilated convolution) which means each output activation ("pixel" in the output feature map) is computed using inputs drawn from a larger spatial area of the previous layer's feature map)
a location process of the one or more generative neuron models is based upon at least one randomly localized, uniformly distributed kernel within a bias range set for each layer; ([p. 2] "we do not need to assign shift values heuristically; instead, they can be trained from random initializations" [p. 8] "Training Real […] U[-1,1]" [p. 5] "For ASL, the shift parameters are randomly initialized with uniform distribution between-1 and 1" [p. 7] "Trained shift values of each layer" Jeon explicitly states that shifts can start from random initializations and explicitly reports uniform initialization for training the real-valued shift parameters which is interpreted as being based on a bias (shift) range set for each layer)
and at least one optimized location of each kernel during back-propagation,([p. 4] "We formulated the shift values as a learnable function with the additional shift parameter θs that defines the amount of shift of each channel […] c is the index of the channel, and the parameters αc and βc define the horizontal and vertical amount of shift" [Abstract] "This new layer can be optimized end-to-end through backpropagation and it can provide optimal shift values" [p. 4] "shifting is applied to each channel using shift parameters and they are optimized by training" Jeon repeatedly and explicitly states that the shift layer's shift parameters (locations) are optimized via backpropagation)
at least one randomly localized kernel is configured for the composite nodal operator with respect to at least one spatial bias initially set randomly, ([p. 2] "we do not need to assign shift values heuristically; instead, they can be trained from random initializations" [p. 8] "Training Real […] U[-1,1]" [p. 5] "For ASL, the shift parameters are randomly initialized with uniform distribution between-1 and 1" [p. 7] "Trained shift values of each layer" [p. 4] "Because the shifted input SC(X) goes through single 1×1 convolutions, this reduces the computational complexity by a factor of the kernel size K. More importantly, by removing both the spatial convolution and sparse memory access, it is possible to construct a network with only dense operations like 1×1 convolutions" Jeon explicitly states that shifts can start from random initializations for convolution. Convolution interpreted as a nodal operator.)
at least one location of each kernel is configured for the composite nodal operator and the spatial bias during at least one back-propagation procedure, ([p. 4] "We formulated the shift values as a learnable function with the additional shift parameter θs that defines the amount of shift of each channel […] c is the index of the channel, and the parameters αc and βc define the horizontal and vertical amount of shift" [Abstract] "This new layer can be optimized end-to-end through backpropagation and it can provide optimal shift values" [p. 4] "shifting is applied to each channel using shift parameters and they are optimized by training" Jeon explicitly optimizes the shift operator (spatial bias) for convolution (nodal operator) during back-propagation)
and at least one back-propagation procedure is performed based upon at least one non-integer bias value of at least one network parameter.([p. 4] "the value for non-integer shift can be calculated through interpolation").
However, Jeon does not explicitly teach at least one memory storing one or more super neuron models.
Chung, in the same field of endeavor, teaches at least one memory storing one or more super neuron models([¶0179] "Local memory 2308 stores configuration images associated with one or more other models (model 1, . . . , model n)." [¶0249] "Model loading component 3808 may be implemented with one or computer processors with memory store instructions").
Jeon as well as Chung are directed towards shifted convolutional neural networks. Therefore, Jeon as well as Chung are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Jeon with the teachings of Chung by storing and retrieving the model in Jeon from memory as disclosed in Chung. Chung provides as additional motivation for combination ([¶0187] “Memory controller 2520 may perform error correction as part of its services” [¶0188] “to exchange information with the local host component”).
Regarding claim 2, the combination of Jeon, and Chung teaches The apparatus of claim 1, wherein the spatial bias for a non-localized kernel for the ith neuron in layer l+1 associated with the kth output neuron connection in layer l comprises integer biases in x- and y-directions, αk i and βk i, respectively.(Jeon [p. 3] "the conventional convolution uses the usual shifted input corresponding to the kernel index k" [p. 4] "We formulated the shift values as a learnable function with the additional shift parameter θs that defines the amount of shift of each channel […] c is the index of the channel, and the parameters αc and βc define the horizontal and vertical amount of shift").
Regarding claim 9, the combination of Jeon, and Chung teaches The apparatus of claim 2, wherein the integer biases in the x- and y-directions are randomly set within a bias range for the layer l + 1.(Jeon [p. 2] "we do not need to assign shift values heuristically; instead, they can be trained from random initializations" [p. 5] "the shift parameters are randomly initialized with uniform distribution between-1 and 1").
Regarding claim 10, the combination of Jeon, and Chung teaches The apparatus of claim 1, wherein the parameters of the composite nodal operator and the spatial bias associated with the kth output neuron connection are optimized during back-propagation training.(Jeon [p. 5] "shift values do not need to be assigned manually; instead, they are learned through backpropagation during training").
Regarding claim 11, the combination of Jeon, and Chung teaches The apparatus of claim 10, wherein at least one of the x-direction displacement and the y-direction displacement comprises a non-integer value.(Jeon [p. 8] "The improvement by using ASL originated from expanding the domain of a shift parameter from integer to real and learning shifts" [p. 4] "We can relax the integer constraint and allow αc and βc to take real numbers, and the value for non-integer shift can be calculated through interpolation").
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yue (“Compact Generalized Non-local Network”, 2018) is directed towards non-local convolutional neural network operations involving Taylor expansion.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIDNEY VINCENT BOSTWICK/Examiner, Art Unit 2124
Prosecution Insights