Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8 are rejected under 35 U.S.C. 103 as being unpatentable over Geifman et al. (“SelectiveNet: A Deep Neural Network with an Integrated Reject Option” – hereinafter Geifman) and further in view of Jang et al. (“Categorical Reparameterization with Gumbel-SoftMax” – hereinafter Jang).
In regards to claim 1, Geifman discloses a method of training a selective network, wherein:
the selective network includes a selection node for selecting whether to make a prediction; (
wherein: (Geifman page 2 section 2 cites “A selective model (El-Yaniv &Wiener, 2010) is a pair (f,g), where f is a prediction function, and
g
:
χ
→
0,1
is a selection function, which serves as a binary qualifier for f as follows, (f,g)(x) = {f(x), if g(x) = 1 or don’t know, if g(x) = 0}. Thus, the selective model abstains from prediction at a point x iff g(x) = 0.” This teaches a selection node for selecting when to make a prediction or when to abstain from making a prediction.)
However, Geifman does not disclose during training, reparametrized as a differentiable function of learnable parameters acting on noise from a base distribution; and wherein the differentiable function approximates a sampling from a categorical distribution.
Jang discloses during training, reparametrized as a differentiable function of learnable parameters acting on noise from a base distribution; (Jang page 2-3 teaches
PNG
media_image1.png
328
554
media_image1.png
Greyscale
PNG
media_image2.png
237
550
media_image2.png
Greyscale
This teaches reparametrizing a differentiable function of learning parameters acting on noise and is supported by instant specification in para. [0024] wherein it states “The Gumbel-SoftMax reparameterization technique (Jang et al., 2017, Maddison et al., 2017) allows reparameterization
of a stochastic node that samples from a categorical distribution, again by replacing it with a differentiable function of learnable parameters, acting on noise from a base distribution.”.) and wherein the differentiable function approximates a sampling from a categorical distribution. (Jang Figure 1 and section 2.1 first paragraph teaches “Samples from Gumbel-SoftMax distributions are identical to samples from a categorical distribution as
τ
→
0
.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the Geifman with that of the Jang in order to allow for reparametrized a differentiable function as both references deal with training of models that make discrete decision. Geifman’s g(x) function performs selective prediction and replacing it with a differentiable stochastic gate of Jang allows it smoother optimization and gradient use creating a more accurate and efficient system.
In regards to claim 2, Geifman in view of Jang discloses the method of claim 1, wherein the base distribution is the Gumbel distribution. (Jang section 2 page 2 first paragraph teaches using a Gumbel-SoftMax distribution.)
In regards to claim 3, Geifman in view of Jang discloses the method of claim 2, further comprising:
during at least one forward pass of the network, using argmax to perform selection at the selection node; and (Jang page 3 section 2.2 cites “For scenarios in which we are constrained to sampling discrete values (e.g. from a discrete action space for reinforcement learning, or quantized compression), we discretize y using argmax but use our continuous approximation in the backward pass…” This teaches using argmax in the forward.)
during at least one backward pass of the network, using a SoftMax approximation of the argmax at the selection node to compute gradients. (Jang page 2 section 2 second paragraph cites “We use the SoftMax function as a continuous, differentiable approximation to argmax….” and page 3 section 2.2 cites “For scenarios in which we are constrained to sampling discrete values (e.g. from a discrete action space for reinforcement learning, or quantized compression), we discretize y using argmax but use our continuous approximation in the backward pass…” This teaches using SoftMax approximation is a continuous approximation and it is used in the backward pass. Also, page 1 section 1 bullet one teaches computing gradients.)
In regards to claim 4, Geifman in view of Jang discloses the method of claim 3, wherein the SoftMax approximation uses temperature annealing. (Jang page 3 second paragraph teaches using Gumbel-SoftMax and temperature annealing. Also, page 7 second paragraph gives a schedule for temperature annealing.)
In regards to claim 5, Geifman in view of Jang discloses the method of claim 1, wherein the noise is independent and identically distributed (i.i.d.) noise. (Jang page 2 teaches samples drawn from Gumbel (0,1) are i.i.d, wherein this includes the noise. )
In regards to claim 6, Geifman in view of Jang discloses the method of claim 1, wherein the prediction is a classification. (Geifman abstract cites “In contrast, SelectiveNet is trained to optimize both classification (or regression) and rejection simultaneously, end-to end.”, this teaches classification.)
In regards to claim 7, Geifman in view of Jang discloses the method of claim 1, wherein the prediction is a numerical value. (Geifman page 2 section 2 second paragraph teaches f is a prediction and
g
:
χ
→
0,1
is binary classifier for x, so it’s a numerical value. Also see page 4 “Concrete Compressive Strength” which using a dataset of 1030 instances with eight numerical features and one target value, and page 5 right column first paragraph teaches using tabular dataset.)
In regards to claim 8, Geifman in view of Jang discloses the method of claim 1, wherein the selective network is one of a convolutional network, a fully connected network, a residual network, and a recurrent network. (Geifman page 5 section 6.3 first paragraph teaches experiments for Convolutional neural network using well known VGG-16 architecture.)
Claims 9-24 are rejected under 35 U.S.C. 103 as being unpatentable over Geifman et al. (“SelectiveNet: A Deep Neural Network with an Integrated Reject Option” – hereinafter Geifman) in view of Jang et al. (“Categorical Reparameterization with Gumbel-SoftMax” – hereinafter Jang) and further in view Lee et al. (US 2023/0206114 A1 – hereinafter Lee).
In regards to claim 9, Geifman discloses a data processing system comprising:
Train a selective network, wherein the selective network includes a selection node for selecting whether to make a prediction; (Geifman page 2 section 2 cites “A selective model (El-Yaniv &Wiener, 2010) is a pair (f,g), where f is a prediction function, and
g
:
χ
→
0,1
is a selection function, which serves as a binary qualifier for f as follows, (f,g)(x) = {f(x), if g(x) = 1 or don’t know, if g(x) = 0}. Thus, the selective model abstains from prediction at a point x iff g(x) = 0.” This teaches a selection node for selecting when to make a prediction or when to abstain from making a prediction. Also, the abstract teaches SelectiveNet is trained.)
However, Geifman does not disclose during training, reparametrized as a differentiable function of learnable parameters acting on noise from a base distribution, wherein the differentiable function approximates a sampling from a categorical distribution; a processor; and memory coupled to the processor.
Jang discloses during training, reparametrized as a differentiable function of learnable parameters acting on noise from a base distribution; (Jang page 2-3 teaches
PNG
media_image1.png
328
554
media_image1.png
Greyscale
PNG
media_image2.png
237
550
media_image2.png
Greyscale
This teaches reparametrizing a differentiable function of learning parameters acting on noise and is supported by instant specification in para. [0024] wherein it states “The Gumbel-SoftMax reparameterization technique (Jang et al., 2017, Maddison et al., 2017) allows reparameterization
of a stochastic node that samples from a categorical distribution, again by replacing it with a differentiable function of learnable parameters, acting on noise from a base distribution.”.) and wherein the differentiable function approximates a sampling from a categorical distribution. (Jang Figure 1 and section 2.1 first paragraph teaches “Samples from Gumbel-SoftMax distributions are identical to samples from a categorical distribution as
τ
→
0
.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the Geifman with that of the Jang in order to allow for reparametrized a differentiable function as both references deal with training of models that make discrete decision. Geifman’s g(x) function performs selective prediction and replacing it with a differentiable stochastic gate of Jang allows it smoother optimization and gradient use creating a more accurate and efficient system.
However, Geifman in view of Jang does not disclose a processor and memory coupled to the processor.
Lee discloses a processor and memory coupled to the processor. (Lee discloses a selective network that determines with to make prediction/classification or when to abstain in para. [0032-0034] wherein it cites “[0032] One solution is to apply the principle of selective classification, which allows the classifier to abstain from making a decision when it is unsure in its prediction, deferring to a human agent. This is done by thresholding on a confidence value κ(x). When the confidence is a good measure of the probability of making a correct prediction, the minimum confidence threshold for making the prediction is increased (thus decreasing the coverage), and the risk on the classified samples should be seen to decrease or the accuracy over the classified samples should be seen to increase (coverage equals the fraction of samples which a prediction is made on). [0033] In particular, one sub-setting of fair classification which exhibits an interesting fairness-related phenomenon is that of selective classification. Generally speaking, selective classification is a variant of the classification problem where a model is allowed to abstain from making a decision. This has applications in settings where making a mistake can be very costly, but abstentions are not (e.g. if the abstention results in deferring classification to a human actor). [0034] In general, selective classification systems work by assigning some measure of confidence about their predictions, and then deciding whether or not to abstain based on this confidence, usually via thresholding.” It further teaches a processor coupled to memory to execute the method in para. [0009] wherein it cites “One or more embodiments of the invention or elements thereof can be implemented in the form of a computer program product including a computer readable storage medium with computer usable program code for performing the method steps indicated. Furthermore, one or more embodiments of the invention or elements thereof can be implemented in the form of a system (or apparatus) including a memory, and at least one processor that is coupled to the memory and operative to perform exemplary method steps.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the Geifman in view of Jang with that of the Lee in order to allow for using a processor coupled to memory to execute the abstract idea all the references deal with selective networks and using a computer with a processor and memory is well-known and widely used in artificial intelligence and it allows for fast and accurate implementation of various machine learning techniques.
In regards to claim 10, it is system embodiment of claim 2 with similar limitations and thus rejected using the same reasoning found in claim 2.
In regards to claim 11, it is system embodiment of claim 3 with similar limitations and thus rejected using the same reasoning found in claim 3.
In regards to claim 12, it is system embodiment of claim 4 with similar limitations and thus rejected using the same reasoning found in claim 4.
In regards to claim 13, it is system embodiment of claim 5 with similar limitations and thus rejected using the same reasoning found in claim 5.
In regards to claim 14, it is system embodiment of claim 6 with similar limitations and thus rejected using the same reasoning found in claim 6.
In regards to claim 15, it is system embodiment of claim 7 with similar limitations and thus rejected using the same reasoning found in claim 7.
In regards to claim 16, it is system embodiment of claim 8 with similar limitations and thus rejected using the same reasoning found in claim 8.
In regards to claim 17, it is computer program product embodiment of claim 9 with similar limitations and thus rejected using the same reasoning found in claim 9. The only difference is claim 9 claims a computer program product comprising a non-transitory tangible computer-readable medium, which is also disclosed in Lee in para. [0159] which cites “The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. … A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.”
In regards to claim 18, it is computer program product embodiment of claim 2 with similar limitations and thus rejected using the same reasoning found in claim 2.
In regards to claim 19, it is computer program product embodiment of claim 3 with similar limitations and thus rejected using the same reasoning found in claim 3.
In regards to claim 20, it is computer program product embodiment of claim 4 with similar limitations and thus rejected using the same reasoning found in claim 4.
In regards to claim 21, it is computer program product embodiment of claim 5 with similar limitations and thus rejected using the same reasoning found in claim 5.
In regards to claim 22, it is computer program product embodiment of claim 6 with similar limitations and thus rejected using the same reasoning found in claim 6.
In regards to claim 23, it is computer program product embodiment of claim 7 with similar limitations and thus rejected using the same reasoning found in claim 7.
In regards to claim 24, it is computer program product embodiment of claim 8 with similar limitations and thus rejected using the same reasoning found in claim 8.
Response to Arguments
Applicant's arguments filed 05 March 2026 have been fully considered but they are not persuasive.
The applicant argues that it would not have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Geifman with that of Jang as Geifman already solves the problem of non-differentiability of selection by using a relaxed function and would not need Jang and thus Geifman teaches away from other solutions.
The examiner respectfully traverses the applicant’s arguments as maintains that it would have been obvious to combine Geifman with Jang. Geifman page 2 section 2 cites teaching a selective model wherein it cites “A selective model (El-Yaniv &Wiener, 2010) is a pair (f,g), where f is a prediction function, and
g
:
χ
→
0,1
is a selection function, which serves as a binary qualifier for f as follows, (f,g)(x) = {f(x), if g(x) = 1 or don’t know, if g(x) = 0}. Thus, the selective model abstains from prediction at a point x iff g(x) = 0.”; this teaches a selection node for selecting when to make a prediction or when to abstain from making a prediction. Also, the abstract teaches SelectiveNet is trained. Further section 4.1 and 4.2 teaches optimizing the selective prediction objective which means it is trained, and section 4.1 cites “The architectures of these three heads can vary depending on the task type and complexity but it is always the case that the final layer of the selection head g(x) is a single neuron with a sigmoid activation. The final layer of f(x) depends on the application and could be softmax (classification) or linear (regression).”, meaning architecture and layers are application dependent and could be a softmax or linear regression, thus other solutions are considered. Additionally, the Jang et al. reference was used for the limitation of “during training, reparametrized as a differentiable function of learnable parameters acting on noise from a base distribution”. The Geifman reference did not mention reparametrizing the differentiable function of parameters acting on noise, thus the Jang reference was used. Jang page 2-3 teaches disclose reparameterizing the differentiable function of parameters acting on noise, wherein the noise is independent and identically distributed (i.i.d.) noise. Additionally, both Geifman and Jang disclose the use of Softmax functions and thus are analogous. The examiner maintains it would have been obvious to one of ordinary skill in the art to modify Geifman with teachings of Jang in as Geifman’s g(x) function performs selective prediction and replacing it with a differentiable stochastic gate of Jang allows it smoother optimization and gradient use creating a more accurate and efficient system by using backpropagation for training.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULINHO E SMITH whose telephone number is (571)270-1358. The examiner can normally be reached Mon-Fri. 10AM-6PM CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULINHO E SMITH/Primary Examiner, Art Unit 2127
Prosecution Insights