Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
This action is in response to the claims filed 12/15/2021:
Claims 1 – 20 are pending.
Claims 1 and 15 are independent.
Drawings
The drawings are objected to because FIGs 3, 4, and 21 are low quality scans containing illegible elements. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
Regarding claim 12, "low-quality" and "high-quality" are relative terms that have been interpreted broadly as simply compound pronouns “low quality data” and “high quality data” describing any type of data.
Claim Objections
Claim 1, 12, and 13 are objected to because of the following informalities:
Regarding claim 1, "at least to perform: organize" should read "at least to perform: organizing". Similarly, "to perform: [...] select" should read "to perform: [...] selecting".
Regarding claim 1, "a plurality of decoders side neural networks" should read "a plurality of decoder side neural networks".
Regarding claims 12 and 13, “for each candidate decoder side neural network perform: provide” should read “for each candidate decoder side neural network perform: providing”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 4, 6, 8-13, 16, 18, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 2 and 16, “the association” lacks antecedent basis. “An association” is recommended.
Regarding claims 4 and 18, “the task” lacks antecedent basis. Claims 1 from which claim 4 depends (and claim 15 from which claim 18 depends) introduces “one or more tasks” and claim 4 introduces “a task to be run on decoded data” such that it would be unclear to one of ordinary skill in the art which task was “the task”.
Regarding claims 6 and 20, “the one or more of the plurality of decoder side neural networks” lacks antecedent basis. “One or more of the plurality of decoder side neural networks” is recommended.
Regarding claims 6 and 20, "substantially lossless" is indefinite. One of ordinary skill in the art would not be able to reasonably determine the bounds of "substantially lossless" to limit the scope of the claim. In the interest of further examination any form of decoding is interpreted as substantially lossless.
Regarding claims 6 and 20, “the output of the decoder side neural network” lacks antecedent basis. “An output of the decoder side neural network” is recommended.
Regarding claim 7, "wherein the two or more decoder side neural networks are associated with different level of at least one" is indefinite. First there is a typo making it unclear if the language should read "a different level" or "different levels". It's grammatically unclear if the two or more decoder side neural networks have respective levels of complexity that are different from one another, or if the two or more decoder side neural networks are all associated with a "different level" of complexity as a group. Since these interpretations are contradictory the scope of the claim cannot be reasonably determined. In the interest of further examination this is interpreted as the group of two or more decoder side neural networks all being associated with a "different level" where the "different level" is a compound pronoun.
Regarding claim 8, "having characteristics that are same or substantially same" is indefinite. One of ordinary skill in the art would not be able to reasonably determine in what way characteristics are the same or substantially same such that the scope of the claim cannot be reasonably determined. In the interest of further examination any two characteristics are interpreted as being substantially the same.
Regarding claim 9, “to select the decoder side neural network, the apparatus is further caused to: […] select one or more decoder side neural networks” is circular and indefinite. This limitation make the antecedent basis for “the decoder side neural network” ambiguous and further it would be logically unclear to one of ordinary skill in the art how selecting a singular decoder side neural networks involves selecting one or more decoder side neural networks. In the interest of further examination this is interpreted as “to select the decoder side neural network, the apparatus is further caused to: […] select the decoder side neural network”.
Regarding claim 10, "the new task neural network" lacks antecedent basis. "A new task neural network" is recommended.
Regarding claim 10, “the bitstream” lacks antecedent basis. “A bitstream” is recommended.
Regarding claims 10, “the one or more of the plurality of decoder side neural networks” lacks antecedent basis. “One or more of the plurality of decoder side neural networks” is recommended.
Regarding claim 10, “the decoder side neural networks” lacks antecedent basis. “Decoder side neural networks” is recommended.
Regarding claims 11, “the one or more of the plurality of decoder side neural networks” lacks antecedent basis. “One or more of the plurality of decoder side neural networks” is recommended.
Regarding claim 13, “the high quality data” lacks antecedent basis. Claim 13 depends on claim 11 which does not provide antecedent basis for “high quality data”. “High quality data” is recommended.
Regarding claim 13, “the features extracted from the high quality data and the reconstructed data” lacks antecedent basis. “Features extracted from the high quality data and the reconstructed data” is recommended.
Regarding claim 13, “the input” lacks antecedent basis. “An input” is recommended.
Dependent claims 8 and 9 are rejected with respect to their dependence on rejected claim 2.
Dependent claim 12 is rejected with respect to its dependence on rejected claim 11.
Claim Rejections - 35 USC § 101
101 Rejection
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 and 11-20 are rejected under 35 USC § 101 because the claimed invention is directed to non-statutory subject matter.
Regarding Claim 1: Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to an apparatus, which is directed to a product, one of the statutory categories.
Step 2A Prong One Analysis: Claim 1 under its broadest reasonable interpretation is a series of mental processes. For example, but for the generic computer components language, the above limitations in the context of this claim encompass neural network processing, including the following:
organize a plurality of decoders side neural networks based on one or more task categories or one or more tasks (observation, evaluation, and judgement),
select a decoder side neural network based at least on the one or more task categories or the one or more task (observation, evaluation, and judgement)
Therefore, claim 1 recites an abstract idea which is a judicial exception.
Step 2A Prong Two Analysis: Claim 1 recites additional elements “An apparatus comprising at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform”. However, these additional features are computer components recited at a high-level of generality, such that they amount to no more than mere instructions to apply the judicial exception using a generic computer component. An additional element that merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, does not integrate the judicial exception into a practical application (See MPEP 2106.05(f)). Therefore, claim 1 is directed to a judicial exception.
Step 2B Analysis: Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the lack of integration of the abstract idea into a practical application, the additional elements recited in claim 1 amount to no more than mere instructions to apply the judicial exception using a generic computer component.
For the reasons above, claim 1 is rejected as being directed to non-patentable subject matter under §101. This rejection applies equally to independent claim 15, which recites a method, as well as to dependent claims 2-14 and 16-20. The additional limitations of the dependent claims are addressed briefly below:
Dependent claims 2 and 16 recite additional observation, evaluation, and judgement “associate one or more decoder side neural networks of the plurality of decoder side neural networks with the one or more task categories or the one or more tasks” and “select the decoder side neural network based on the association between the one or more tasks and the plurality of decoder side neural networks”
Dependent claims 3 and 17 recite additional observation, evaluation, and judgement “wherein the apparatus is further caused to select an optimal decoder side neural network based on one or more predetermined criteria.”
Dependent claims 4 and 18 recite additional instructions to apply the judicial exception using a generic computer component “wherein the plurality of decoder side neural networks comprise one or more shared decoder side neural networks, and wherein the one or more shared decoder side neural networks comprise one or more shared parameters and one or more non-shared parameters, and wherein the one or more shared parameters do not depend on a task to be run on decoded data, and wherein the one or more non-shared parameters depend at least on the task to be run on the decoded data” where an autoencoder having shared and unshared parameters broadly linked to a task is a generic computer component. Claim 4 also recites additional observation, evaluation, and judgement “to select a subset of the one or more non-shared parameters that are to be used by the decoder side neural network associated with the task”
Dependent claims 5 and 19 recite additional observation, evaluation, and judgement “to select an optimal decoder side neural network for a new task that was not known at a design phase”
Dependent claims 6 and 20 recite additional instructions to apply the judicial exception to generic computer components “decode a bitstream, received from an encoder side, by using a lossless or substantially lossless codec” and “run one or more decoder side neural network associated with at least one task of the one or more tasks” as well as additional insignificant extra-solution activity of gathering and outputting data (See MPEP 2106.05(g)) “provide the decoded bitstream to the one or more of the plurality of decoder side neural networks, based on the one or more tasks” and “provide an output or data derived from the output of the decoder side neural network to a task neural network associated with the task” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II)(i)).
Dependent claim 7 recites additional instructions to apply the judicial exception to generic computer components “to associate two or more decoder side neural networks with a task” as well as additional observation, evaluation, and judgement “wherein the two or more decoder side neural networks perform differently in terms of a rate-distortion trade-off or a trade-off between a rate and a task accuracy, and wherein the two or more decoder side neural networks are associated with different level of at least one of a computation complexity, a memory complexity, or a power complexity”
Dependent claim 8 recites additional observation, evaluation, and judgement “wherein a task category of the one or more task categories comprises one or more tasks having characteristics that are same or substantially same”
Dependent claim 9 recites additional observation, evaluation, and judgement “select one or more task neural networks based on one or more predetermined neural network tasks; select one or more decoder side neural networks associated with the one or more tasks” as well as additional instructions to apply the judicial exception using generic computer components “use the selected one or more decoder side neural networks to decode or process associated input data”
Dependent claim 11 recites additional observation, evaluation, and judgement “to select the decoder side neural network, the apparatus is further caused to evaluate performance of a new task neural network when applied on one or more sample data, wherein the new task neural network was not considered at codec design stage codec or the new task neural network is not associated with the one or more of the plurality of decoder side neural networks”
Dependent claim 12 recites additional observation, evaluation, and judgement “to measure the task performance on a low-quality data by using an approximated ground truth from high quality data, and wherein to measure the task performance, the apparatus is further caused to: derive the approximated ground truth from an output of the new task neural network, when an input data is the high quality data; determine one or more candidate decoder side neural network from the plurality of decoder side neural networks” and “compare at least one output of one or more outputs of the new task neural network with the approximated ground truth; and select a candidate decoder side neural network providing a predetermined accuracy value as the decoder side neural network” as well as additional insignificant extra-solution activity of gathering and outputting data (See MPEP 2106.05(g)) “provide the low-quality data as an input to the each candidate decoder side neural network; provide output of the each candidate decoder side neural network as an input to the new task neural network” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II)(i))
Dependent claim 13 recites additional observation, evaluation, and judgement “to measure a distance between the features extracted from the high quality data and the reconstructed data, and wherein to measure the distance, the apparatus is further caused to: derive the approximated ground truth from an output of a feature-extraction neural network when the input is the high-quality data; determine one or more candidate decoder side neural network from the plurality of decoder side neural networks” and “extract features based at least on the output of the each candidate decoder side neural network; and compare the features extracted from the output of the each candidate decoder side neural network with the approximated ground-truth; and select a candidate decoder side neural network providing a predetermined distance value or a lowest distance value as the decoder side neural network” as well as additional insignificant extra-solution activity of gathering and outputting data (See MPEP 2106.05(g)) “provide the low-quality data as an input to the each candidate decoder side neural network; provide output of the each candidate decoder side neural network as an input to the feature-extraction neural network” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II)(i))
Dependent claim 14 recites additional instructions to apply the judicial exception using generic computer components “the one or more tasks comprises one or more task-NNs”
Therefore, when considering the elements separately and in combination, they do not add significantly more to the inventive concept. Accordingly, claims 1-9 and 1-20 are rejected under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7-9, and 11-19 are rejected under U.S.C. §102(a)(1) as being anticipated by Meyerson (US20190130257A1) which incorporates ("Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing—and Back", 2018) by reference.
Regarding claim 1, Meyerson teaches An apparatus comprising at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:([¶0028] "one or more of the functional blocks (e.g., modules, processors, or memories) can be implemented in a single piece of hardware (e.g., a general purpose signal processor or a block of random access memory, hard disk, or the like) or multiple pieces of hardware. Similarly, the programs can be stand-alone programs, can be incorporated as subroutines in an operating system, can be functions in an installed software package, and the like")
organize a plurality of decoders side neural networks based on one or more task categories or one or more tasks; and([¶0035] "In FIG. 2, (c) shows supervision at custom depths which add output decoders at depths based on a task hierarchy. In FIG. 2, (d) shows universal representations which adapt each layer with a small number of task-specific scaling parameters")
select a decoder side neural network based at least on the one or more task categories or the one or more task.([0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks").
Regarding claim 2, Meyerson teaches The apparatus of claim 1, wherein the apparatus is further caused to: associate one or more decoder side neural networks of the plurality of decoder side neural networks with the one or more task categories or the one or more tasks; and(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder." [¶0186] "The method includes processing the accumulated output encoding through the selected decoder to produce classification scores for classes defined for the particular one of the classification tasks.")
select the decoder side neural network based on the association between the one or more tasks and the plurality of decoder side neural networks.(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder.").
Regarding claim 3, Meyerson teaches The apparatus of claim 1, wherein the apparatus is further caused to select an optimal decoder side neural network based on one or more predetermined criteria.(Meyerson [¶0169] "The system comprises a decoder selector. The decoder selector selects, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmits the accumulated output encoding produced for the final clone as input to the selected decoder." [¶0170] "The selected decoder processes the accumulated output encoding and produces classification scores for classes defined for the particular one of the classification tasks.").
Regarding claim 4, Meyerson teaches The apparatus of claim 1, wherein the plurality of decoder side neural networks comprise one or more shared decoder side neural networks, and wherein the one or more shared decoder side neural networks comprise one or more shared parameters and one or more non-shared parameters, and wherein the one or more shared parameters do not depend on a task to be run on decoded data, (Meyerson [Abstract] "The technology disclosed identifies parallel ordering of shared layers as a common assumption underlying existing deep multitask learning (MTL) approaches" [¶0021] "FIG. 4 depicts soft ordering of shared layers." [¶0047] "at each shared depth the applied weight tensors for each task are similar and compatible for sharing" [¶0066] " Each Fj in FIG. 4 includes a shared weight layer and a nonlinearity. This architecture enables the learning of layers that are used in different ways at different depths for different tasks." Meyerson explicitly teaches that the selected decoder is a soft ordering shared decoder with shared and unshared parameters where the shared parameters are not task dependent.)
and wherein the one or more non-shared parameters depend at least on the task to be run on the decoded data, (Meyerson [¶0066] "Soft ordering (Eq. 7) generalizes Eqs. 2 and 3 by learning a tensor S of task-specific scaling parameters. S is learned jointly with the Fj, to allow flexible sharing across tasks and depths. Each Fj in FIG. 4 includes a shared weight layer and a nonlinearity. This architecture enables the learning of layers that are used in different ways at different depths for different tasks." tensor S interpreted as non-shared task dependent parameter.)
and wherein the apparatus is further caused to select a subset of the one or more non-shared parameters that are to be used by the decoder side neural network associated with the task.(Meyerson [¶0068] "by s(i,σi(k),k)=1∀(i,k). S can be learned jointly with the other learnable parameters in the Wkεi, and Di via backpropagation. In training, all s(i,j,k) are initialized with equal values, to reduce initial bias of layer function across tasks" Meyerson explicitly selects subset s_(i,j,k) from set S where i corresponds to selected decoder Di.).
Regarding claim 5, Meyerson teaches The apparatus of claim 1, wherein the apparatus is further caused to select an optimal decoder side neural network for a new task that was not known at a design phase.(Meyerson [¶0032] "Learning a different soft ordering of layers for each task amounts to discovering a set of generalizable modules that are assembled in different ways for different tasks. This perspective points to future approaches that train a collection of layers on a set of training tasks, which can then be assembled in novel ways for future unseen tasks.").
Regarding claim 7, Meyerson teaches The apparatus of claim 1, wherein the apparatus is further caused to associate two or more decoder side neural networks with a task, (Meyerson [p. 1 §1] "when only a single task is available for training. The method is formalized as pseudo-task augmentation (PTA), in which a single task has multiple distinct decoders projecting the output of the shared structure to task predictions. By training the shared structure to solve the same problem in multiple ways, PTA simulates the effect of training towards distinct but closely-related tasks drawn from the same universe")
and wherein the two or more decoder side neural networks perform differently in terms of a rate-distortion trade-off or a trade-off between a rate and a task accuracy, and (Meyerson [p. 3 §3.1] "since each decoder induces a distinct model for a task, what matters is not the average over decoders, but the best performing decoder for each task, i.e. [...] Equation 5 is used for model validation, and to select the best performing decoder for each task from the final joint model" Meyerson explicitly chooses the best decoder which requires the decoders to perform differently in terms of a rate-distortion (loss/error).)
wherein the two or more decoder side neural networks are associated with different level of at least one of a computation complexity, a memory complexity, or a power complexity.(Meyerson [p. 1 §1] "pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems" Boosting performance interpreted as synonymous with being associated with a computational complexity.).
Regarding claim 8, Meyerson teaches The apparatus of claim 2, wherein a task category of the one or more task categories comprises one or more tasks having characteristics that are same or substantially same.(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder.").
Regarding claim 9, Meyerson teaches The apparatus of claim 2, wherein, to select the decoder side neural network, the apparatus is further caused to: select one or more task neural networks based on one or more predetermined neural network tasks;(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder." See also FIG. 1. Model 101 with selected decoder side neural network and encoder 102 is interpreted as a selected task neural network based on one or more predetermined neural network tasks.)
select one or more decoder side neural networks associated with the one or more tasks; and use the selected one or more decoder side neural networks to decode or process associated input data.(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder." Decoder interpreted as decoding input data by definition.).
Regarding claim 11, Meyerson teaches The apparatus of claim 5, wherein, to select the decoder side neural network, the apparatus is further caused to evaluate performance of a new task neural network when applied on one or more sample data, (Meyerson [¶0053] "In Eq. 3 above, σi is a task-specific permutation of size D, and σi is fixed before training. If there are sets of tasks for which joint training of the model defined by Eq. 3 achieves similar or improved performance over Eq. 2, then parallel ordering is not a necessary requirement for deep MTL. Of course, in this formulation, it is required that the wk can be applied in any order. See Section 6 for examples of possible generalizations.")
wherein the new task neural network was not considered at codec design stage codec or the new task neural network is not associated with the one or more of the plurality of decoder side neural networks.(Meyerson [¶0032] "Learning a different soft ordering of layers for each task amounts to discovering a set of generalizable modules that are assembled in different ways for different tasks. This perspective points to future approaches that train a collection of layers on a set of training tasks, which can then be assembled in novel ways for future unseen tasks.").
Regarding claim 12, Meyerson teaches The apparatus of claim 11, wherein to evaluate the performance of the new task neural network, the apparatus is further caused to measure the task performance on a low-quality data by using an approximated ground truth from high quality data, and wherein to measure the task performance, the apparatus is further caused to: derive the approximated ground truth from an output of the new task neural network, when an input data is the high quality data;(Meyerson [p. 3 §3] “Consider again the case where there are T distinct true tasks, but now let there be D decoders for each task. Then, the model for the dth decoder of the tth task is given by [See Eqn. 3] and the overall loss for the joint model from Eq. 2 becomes [See Eqn. 4]"” [p. 6 §4.1] “This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task”)
determine one or more candidate decoder side neural network from the plurality of decoder side neural networks;(Meyerson [0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks" [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5")
for each candidate decoder side neural network perform: provide the low-quality data as an input to the each candidate decoder side neural network;(Meyerson [¶0105] "The training, validation, and test splits provided by Liu et al. (2015b) were used. There are ≈160K images for training, ≈20K for validation, and ≈20K for testing" validation split interpreted as low-quality data from the full dataset (high-quality data) where the validation split comprises the ground truth examples.)
provide output of the each candidate decoder side neural network as an input to the new task neural network; and(Meyerson See FIG. 1)
compare at least one output of one or more outputs of the new task neural network with the approximated ground truth; and(Meyerson [p. 3 §3] “Consider again the case where there are T distinct true tasks, but now let there be D decoders for each task. Then, the model for the dth decoder of the tth task is given by [See Eqn. 3] and the overall loss for the joint model from Eq. 2 becomes [See Eqn. 4]"” [p. 6 §4.1] “This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task”)
select a candidate decoder side neural network providing a predetermined accuracy value as the decoder side neural network.(Meyerson [0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks" [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5").
Regarding claim 13, Meyerson teaches The apparatus of claim 11, wherein to evaluate the performance of the new task neural network, the apparatus is further caused to measure a distance between the features extracted from the high quality data and the reconstructed data, and wherein to measure the distance, the apparatus is further caused to: derive the approximated ground truth from an output of a feature-extraction neural network when the input is the high-quality data;(Meyerson [p. 3 §3] “Consider again the case where there are T distinct true tasks, but now let there be D decoders for each task. Then, the model for the dth decoder of the tth task is given by [See Eqn. 3] and the overall loss for the joint model from Eq. 2 becomes [See Eqn. 4]"” [p. 6 §4.1] “This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task”)
determine one or more candidate decoder side neural network from the plurality of decoder side neural networks;(Meyerson [0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks" [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5")
for each candidate decoder side neural network perform: provide the low-quality data as an input to the each candidate decoder side neural network;(Meyerson See FIG. 1)
provide output of the each candidate decoder side neural network as an input to the feature-extraction neural network;(Meyerson [p. 3 §3] “Consider again the case where there are T distinct true tasks, but now let there be D decoders for each task. Then, the model for the dth decoder of the tth task is given by [See Eqn. 3] and the overall loss for the joint model from Eq. 2 becomes [See Eqn. 4]"” [p. 6 §4.1] “This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task”)
extract features based at least on the output of the each candidate decoder side neural network; and(Meyerson [¶0035] "Underlying each of these approaches is the assumption of parallel ordering of shared layers. Also, each one requires aligned sequences of feature extractors across tasks.")
compare the features extracted from the output of the each candidate decoder side neural network with the approximated ground-truth; and(Meyerson [p. 3 §3] “Consider again the case where there are T distinct true tasks, but now let there be D decoders for each task. Then, the model for the dth decoder of the tth task is given by [See Eqn. 3] and the overall loss for the joint model from Eq. 2 becomes [See Eqn. 4]"” [p. 6 §4.1] “This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task”)
select a candidate decoder side neural network providing a predetermined distance value or a lowest distance value as the decoder side neural network.(Meyerson [0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks" [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5" [p. 3 §3.1] “Equation 5 is used for model validation, and to select the best performing decoder for each task from the final joint model. This decoder is then applied to future data”).
Regarding claim 14, Meyerson teaches The apparatus claim 1, wherein the one or more tasks comprises one or more task-NNs.(Meyerson [¶0185] "selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks and transmitting the accumulated output encoding produced for the final clone as input to the selected decoder.").
Regarding claims 15-19, claims 15-19 are directed towards the method performed by the apparatus of claims 1-5, respectively. Therefore, the rejections applied to claims 1-5 also apply to claims 15-19.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 6, 10, and 20 are rejected under U.S.C. §103 as being unpatentable over the combination of Meyerson and Hoogeboom (“Integer Discrete Flows and Lossless Compression”, 2019).
Regarding claim 6, Meyerson teaches The apparatus claim 1.
However, Meyerson doesn't explicitly teach, wherein the apparatus is further caused to: decode a bitstream, received from an encoder side, by using a lossless or substantially lossless codec;
provide the decoded bitstream to the one or more of the plurality of decoder side neural networks, based on the one or more tasks; run one or more decoder side neural network associated with at least one task of the one or more tasks; and
provide an output or data derived from the output of the decoder side neural network to a task neural network associated with the task.
Hoogeboom, in the same field of endeavor, teaches The apparatus claim 1, wherein the apparatus is further caused to: decode a bitstream, received from an encoder side, by using a lossless or substantially lossless codec;([p. 5 §3.3] "Lossless compression is an essential technique to limit the size of representations without destroying information. Methods for lossless compression require i) a statistical model of the source, and ii) a mapping from source symbols to bit stream [...] The mapping between symbols and bit streams may be provided by any entropy encoder")
provide the decoded bitstream to the one or more of the plurality of decoder side neural networks, based on the one or more tasks; run one or more decoder side neural network associated with at least one task of the one or more tasks; and([p. 2] "Figure 1: Overview of IDF based lossless compression. An image x is transformed to a latent representation z with a tractable distribution pZ(·). An entropy encoder takes z and pZ(·) as input, and produces a bitstream c. To obtain x, the decoder uses pZ(·) and c to reconstruct z. Subsequently, z is mapped to x using the inverse of the IDF." [p. 5] "the mapping f : x 7! z is defined by the IDF. Subsequently, z is encoded under the distribution pZ(z) to a bitstream c using an entropy encoder. Note that, when using factor-out layers, pZ(z) is also defined using the IDF. Finally, in order to decode a bitstream c, an entropy encoder uses pZ(z) to obtain z. and the original image is obtained by using the map f")
provide an output or data derived from the output of the decoder side neural network to a task neural network associated with the task.([p. 5] "Figure 4: Example of a 2-level flow architecture. The squeeze layer reduces the spatial dimensions by two, and increases the number of channels by four. A single integer flow layer consists of a channel permutation and an integer discrete coupling layer. Each level consists of D flow layers" See FIG. 4 which shows that inner decoder neural networks are inversely processed by providing decoder output to outer decoder layer(s).).
Meyerson as well as Hoogeboom are directed towards stacked decoder layer autoencoder architectures. Therefore, Meyerson as well as Hoogeboom are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Meyerson with the teachings of Hoogeboom by simply using both models sequentially (reconstructing bitstream using IDF decoder and then providing that reconstructed image to task-specific Meyerson decoder heads). Hoogeboom provides as additional motivation for combination ([p. 1 §1] “Every day, 2500 petabytes of data are generated. Clearly, there is a need for compression to enable efficient transmission and storage of this data [...] maximizing the log-likelihood (of data) is equivalent to minimizing the expected number of bits required per message”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 10, Meyerson teaches wherein the new task neural network was not used or known at codec design stage or the new task neural network is not associated with the one or more of the plurality of decoder side neural networks;(Meyerson [¶0032] "Learning a different soft ordering of layers for each task amounts to discovering a set of generalizable modules that are assembled in different ways for different tasks. This perspective points to future approaches that train a collection of layers on a set of training tasks, which can then be assembled in novel ways for future unseen tasks.")
extract features from the new task neural network;(Meyerson [¶0035] "Underlying each of these approaches is the assumption of parallel ordering of shared layers. Also, each one requires aligned sequences of feature extractors across tasks.")
extract features from the task neural network; and(Meyerson [¶0035] "Underlying each of these approaches is the assumption of parallel ordering of shared layers. Also, each one requires aligned sequences of feature extractors across tasks.")
compute distance metric between the extracted features; and(Meyerson [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5")
select a decoder side neural network comprising a predetermined distance metric or a lowest distance.(Meyerson [0185] "The method includes selecting, from among numerous decoders, a decoder that is specific to the particular one of the classification tasks" [¶0073] "For a representative two-task soft order experiment the layer-wise distance between scalings of the tasks increases by iteration, as shown by (b) in FIG. 5").
However, Meyerson doesn't explicitly teach The apparatus of claim 5, wherein, to select the decoder side neural network, the apparatus is further caused to: run at least part of the new task neural network on data derived from the bitstream received by the decoder,
for each task for which a known association with a decoder side neural network exists, perform: use the decoder side neural networks associated with the task to process data derived from the bitstream received by the decoder;
run at least part of the task neural network on data output by the decoder side neural network associated with the task or data derived from the output of the decoder side neural network.
Hoogeboom, in the same field of endeavor, teaches The apparatus of claim 5, wherein, to select the decoder side neural network, the apparatus is further caused to: run at least part of the new task neural network on data derived from the bitstream received by the decoder, ([p. 5 §3.3] "Lossless compression is an essential technique to limit the size of representations without destroying information. Methods for lossless compression require i) a statistical model of the source, and ii) a mapping from source symbols to bit stream [...] The mapping between symbols and bit streams may be provided by any entropy encoder" [p. 2 §1] "To obtain x, the decoder uses pZ(·) and c to reconstruct z. Subsequently, z is mapped to x using the inverse of the IDF")
for each task for which a known association with a decoder side neural network exists, perform: use the decoder side neural networks associated with the task to process data derived from the bitstream received by the decoder;([p. 5 §3.3] "Lossless compression is an essential technique to limit the size of representations without destroying information. Methods for lossless compression require i) a statistical model of the source, and ii) a mapping from source symbols to bit stream [...] The mapping between symbols and bit streams may be provided by any entropy encoder" [p. 2 §1] "To obtain x, the decoder uses pZ(·) and c to reconstruct z. Subsequently, z is mapped to x using the inverse of the IDF")
run at least part of the task neural network on data output by the decoder side neural network associated with the task or data derived from the output of the decoder side neural network;([p. 5] "Figure 4: Example of a 2-level flow architecture. The squeeze layer reduces the spatial dimensions by two, and increases the number of channels by four. A single integer flow layer consists of a channel permutation and an integer discrete coupling layer. Each level consists of D flow layers" See FIG. 4 which shows that inner decoder neural networks are inversely processed by providing decoder output to outer decoder layer(s).).
Meyerson as well as Hoogeboom are directed towards stacked decoder layer autoencoder architectures. Therefore, Meyerson as well as Hoogeboom are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Meyerson with the teachings of Hoogeboom by simply using both models sequentially (reconstructing bitstream using IDF decoder and then providing that reconstructed image to task-specific Meyerson decoder heads). Hoogeboom provides as additional motivation for combination ([p. 1 §1] “Every day, 2500 petabytes of data are generated. Clearly, there is a need for compression to enable efficient transmission and storage of this data [...] maximizing the log-likelihood (of data) is equivalent to minimizing the expected number of bits required per message”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 20, claim 20 is directed towards the method performed by the apparatus of claim 6. Therefore, the rejection applied to claim 6 also applies to claim 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Wang (“Neural-Network-Optimized Degree-Specific Weights for LDPC MinSum Decoding”, 2021) is directed towards a stacked autoencoder for lossless encoding.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax p