DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to pending claims 1-9, 11 filed 9/12/2025.
Claim Objections
The following claim(s) are objected to for formality issues:
In claim 1 ¶4 (a first learner ..), “configured to the one or more processors” should read “configured by the one or more processors”.
Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-3, 5-9, 11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. We analyze the claims according to the subject matter eligibility flowchart (MPEP 2106). As all claims recite statutory categories (device, method, computer media), step 1 is answered affirmatively and we proceed to step 2.
Representative claim 1 is directed to a image encoder and a partitioner that seeks to improve characteristic features of the encoder via random partitioning. This is analogous to a mental process where the mind may gain experience by exposing itself to random images in order to gain experience to better convert images into a vector. Likewise, converting images into a feature vector is a mental process analogous to using judgment and observation to determine image characteristics, and may be performed in the mind by considering, for example, presence or degree of presence of various attributes of the image, e.g., number of objects. Hence, the claims are directed to an abstract idea, that of mentally categorizing, determining the characteristics of, and classifying images.
The claims portions directed to the abstract idea and the additional elements (underlined below) are analyzed below:
For claim 1, Chai discloses: a conversion device that converts a given input vector representing an image to a feature vector of which a dimension is reduced by a conversion model and classifies the image by the feature vector (As above, such a device may be implemented mentally), the conversion device comprising a memory that stores program code and one or more processors configured to read the program code and operate as instructed by the program code including (These additional elements recite implementation on a general purpose computer and hence does comprises mere instructions to apply the exception on a computer and hence does not constitute an integration into a practical application (2a-2). Furthermore, the use of computers is WURC in the field of classification (2b).):
a partitioner code configured to cause the one or more processors to partition training vectors into groups randomly (partitioning randomly is a mental process, e.g., randomly deciding on batches for mental processing. See above for the analysis of the implementation on a general purpose computer via code);
a first classifier code configured to cause the one or more processors to classify feature vectors that are obtained by converting the training vectors with the conversion model, into any one of the groups by a first classification model (Classification, such as based on image attributes via a feature vector, is a mental process), and
a first learner code configured to the one or more processors to learn the conversion model and the first classification model by a first teacher data including the training vectors and the groups into which the training vectors are respectively partitioned (Learning patterns via training data and training vectors is a mental process), thereby sparsity of the feature vector being improved (improving sparsity, such as by reducing the number of image characteristics considered, can be performed mentally),
wherein:
the conversion model is implemented by an autoencoder comprising an encode part located in the former half of the autoencoder and a decode part located in the latter half of the autoencoder (Considering image characteristics of a feature vector, and considering whether such vectors correspond to the original image in order to assure correspondence, is a mental process. The additional elements recite implementation as an autoencoder, but such limitations comprise mere instructions to apply the mental process to a machine learning environment and hence does not constitute an integration into a practical application. Furthermore, the use of autoencoders is well-understood, routine, and conventional in the field of image processing),
an input to the conversion model is fed into the encode part (receiving input is a mental process),
the encode part produces an intermediate representation from the input (generating an encoded vector, such as by considering various image characteristics, is a mental process),
the intermediate representation is fed into the decode part (Considering whether the generated vector of image characteristics corresponds with the image is a mental process),
the decode part produces a result from the intermediate representation (Considering or imagining the image thereby generated is a mental process), and
the result is treated as an output from the conversion model (Considering the result of an imagined image is a mental process),
the given input vector is provided as the input to the conversion model (Classifying images based on image characteristics is a mental process),
the intermediate representation is treated as the feature vector (Considering image characteristics as a list is a mental process)), and
the conversion model is learned by minimizing a difference between the input to the conversion model and the output from the conversion model (Considering a difference between an imagined image and an original image based on a reconstruction or reduction to a list of essential image characteristics is a mental process).
For claim 2: the conversion device according to claim 1, wherein
the training vectors belong to classes, respectively,
the program code further includes (see above for analysis of implementation details on a general purpose computer):
a second code configured to cause the one or more processors to classify a given vector into any one of the classes by a second classification model (Classification, such as of image characteristics, may be performed mentally); and
a second learner code configured to cause the one or more processors to classify the second classification model by of a second teacher data including feature vectors that are obtained by converting the training vectors, and the classes to which the training vectors respectively belong, by the learned conversion model (Adjusting mental heuristics based on trained feature vectors, such as a listing of image characteristics, is a mental process),
when a new input vector representing a new image is given after the second classification model is learned, the conversion device converts the new input vector to a new feature vector by the learned conversion model, and the second classifier classifies the new feature vector into any one of the classes by the learned second classification model, thereby classifying the new input vector into which a class into the new feature vector is classified (conversion into a new feature vector and subsequent classification is similar to the training process described above and may be performed mentally).
For claim 3: the conversion device according to claim 2, wherein the feature vector has a dimension greater than the number of classes (This limitation imposed on the feature vector does not preclude performance in the mind).
For claim 5: the conversion device according to claim 3, wherein the feature vector has a dimension greater than the number of the groups (This limitation imposed on the feature vector does not preclude performance in the mind).
For claim 6: the conversion device according to claim 1, wherein the second classification model classifies the feature vector by logistic repression, ridge repression, lasso repression, support vector machine (SVM), random forest, or neural network (This merely recites the implementation of a general classification task on a machine learning model via a computer and hence does not constitute an integration into a practical application as it is a mere application and is extra solutional (2a-2). Furthermore, the use of machine learning models for classification is WURC in the field of classification (2b).).
For claim 7: the conversion device according to claim 1, wherein probabilities that the partitioner randomly partitions the training vectors into the groups, respectively, are not equal to each other (This limitation restricting the classification does not preclude its performance in the mind).
For claim 11: the conversion device according to claim 2, wherein the encode part is implemented by a first convolutional neural network with eight output layers, and the first classifier classifies the feature vector by a second convolutional neural network with eight output layers (This merely recites the implementation of a general classification task on a machine learning model via a computer and hence does not constitute an integration into a practical application as it is a mere application and is extra solutional (2a-2). Furthermore, the use of machine learning models, including those with varying numbers of output layers, for classification is WURC in the field of classification (2b).).
Claims 8, 9 recite methods and computer media analogous to the above and are hence rejected for the same reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-6, 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Chai ("A semi-supervised auto-encoder using label and sparse regularizations for classification", published 1/23/2019) in view of Pondenkandath ("Leveraging random label memorization for unsupervised pre-training", published 2018) in view of Okazaki (US 10803388 B2).
For claim 1, Chai discloses: a conversion device that converts a given input vector representing an image (image classification: p.207 col.2 ¶1, §4.1 ¶1, §5 ¶2) to a feature vector of which a dimension is reduced by a conversion model and classifies the image by the feature vector (fig.1 gives overview of the device including an input layer for accepting an input vector of a certain weight, a hidden encode layer of lower dimension, these two layers being an encoder conversion model, and finally a label layer and a classifier that classifies the image via the feature vector generated by the hidden layer):
a partitioner code configured to cause the one or more processors to partition training vectors into groups (p.201 col.1 ¶2 (“To improve ...”), §3 ¶1 contemplates using labeled data mixed with unlabeled data in a semi-supervised technique, hence, partitioning training vectors into corresponding groups via labeling as part of data preparation for input);
a first classifier code configured to cause the one or more processors to classify feature vectors that are obtained by converting the training vectors with the conversion model, into any one of the groups by a first classification model (fig.1: classifier, label layer), and
a first learner code configured to the one or more processors to learn the conversion model and the first classification model by a first teacher data including the training vectors and the groups into which the training vectors are respectively partitioned (fig.1 shows the overall trainer J-LSRAE operating as learner code, the trainer making adjustments to the classification model and the conversion model via the training or teacher data including the input vectors and groups), thereby sparsity of the feature vector being improved (J-sparse would improve the sparsity of the hidden layer activations, see §3 ¶1-2, hence, improving sparsity of feature vectors, for example, those vectors containing zeros in from the input layer will be preserved or have additional zeros due to the hidden layer sparsity),
wherein:
the conversion model is implemented by an autoencoder comprising an encode part located in the former half of the autoencoder and a decode part located in the latter half of the autoencoder (fig.1: input layer, hidden layer, output layer, §3 ¶1),
an input to the conversion model is fed into the encode part (fig.1: input layer, hidden layer),
the encode part produces an intermediate representation from the input (fig.1: output of the hidden layer),
the intermediate representation is fed into the decode part (fig.1: output of hidden layer to output layer),
the decode part produces a result from the intermediate representation (fig.1: output of the output layer, fed into J-AE (autoencoder loss)), and
the result is treated as an output from the conversion model (ibid),
the given input vector is provided as the input to the conversion model (fig.1: output of hidden layer into label layer, classifier),
the intermediate representation is treated as the feature vector (ibid, see also §3 ¶2 (“extract more local and informative features to represent the data)), and
the conversion model is learned by minimizing a difference between the input to the conversion model and the output from the conversion model (§3 ¶1: J-AE being the reconstruction error, see also eq.11 (p.208)).
Chai does not disclose: wherein the partitioning into groups occurs randomly.
Pondenkandath disclose: wherein the partitioning into groups occurs randomly (§2.4, §3-4: pre-training with random labels yielding improvement in results).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the device of Chai by incorporating the random labeling of Pondenkandath. Both concern the art of neural network training, and the incorporation would have, according to Pondenkandath, provide potential improvements in model performance (§4).
Chai modified by Pondenkandath does not disclose the remaining limitations.
Chai modified by Pondenkandath does not disclose the remaining limitations.
Okazaki (US 10803388 B2) discloses: a device comprising a memory that stores program code and one or more processors configured to read the program code and operate as instructed by the program code (fig.9, col.9 ¶2-5).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the device of Chai modified by Pondenkandath by incorporating computing hardware structure of Okazaki. Both concern the art of neural network training, and the incorporation would have, according to Okazaki, allowed for implementation on a general purpose computing devices (col.9 ¶2).
For claim 2, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 1, as described above. Chai further discloses: the training vectors belong to classes (p.201 col.1 ¶2 (“To improve ..”), §3 ¶1 contemplates using labeled data mixed with unlabeled data in a semi-supervised technique, hence, the training vector belonging to respective labeled classes), respectively,
the program code further includes:
a second classifier code configured to cause the one or more processors to classify a given vector into one of the classes by a second classification model (fig.1: classifier, label layer); and
a second learner that learns code configured to cause the one or more processors to classify the second classification model by a second teacher data including feature vectors that are obtained by converting the training vectors, and the classes to which the training vectors respectively belong, by the learned conversion model (fig.1: label layer, classifier, and respective learning functions (e.g., J-Label), these components being a second learner in contrast to the first learner of random labels),
when a new input vector representing a new image is given after the second classification model is learned, the conversion device converts the new input vector to a new feature vector by the learned conversion model, and the second classifier classifies the new feature vector into any one of the classes by the learned second classification model, thereby classifying the new input vector into which a class into the new feature vector is classified (ibid: passing the new image data through the entire encode-classify network, thereby generating a classification result).
For claim 3, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 2, as described above. Chai further discloses: wherein the feature vector has a dimension greater than the number of the classes (§4.1, table 1 (p.210) gives overview of dataset including number of classes, features).
For claim 5, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 3, as described above. Chai modified by Pondenkandath further discloses: wherein the feature vector has a dimension greater than the number of the groups (Chai §4.1 table 1, combination of the random labeling teaching of Pondenkandath would not change these metrics1).
For claim 6, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 2, as described above. Chai further discloses: wherein the second classification model classifies the feature vector by logistic regression, ridge regression, lasso regression, support vector machine (SVM), random forest, or neural network (fig.1: ELM constitutes a neural network).
Claims 8, 9 recite analogous methods and computer media and hence are rejected under the same rationale.
Claim(s) 7 are rejected under 35 U.S.C. 103 as being unpatentable over Chai ("A semi-supervised auto-encoder using label and sparse regularizations for classification", published 1/23/2019) in view of Pondenkandath ("Leveraging random label memorization for unsupervised pre-training", published 2018) in view of Okazaki (US 10803388 B2) in view of Bekker ("Training deep neural-networks based on unreliable labels", published 2016).
For claim 7, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 1, as described above. Chai modified by Pondenkandath does not discloses: wherein probabilities that the partitioner randomly partitions the training vectors into the groups, respectively, are not equal to each other.
Bekker discloses: wherein probabilities that the partitioner randomly partitions the training vectors into the groups, respectively, are not equal to each other (§3 ¶3: Bekker contemplates passage through a noisy channel, see §2 ¶1, hence, where randomness is induced based on the original distribution).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the device of Chai modified by Pondenkandath by incorporating the randomized labeling teachings of Bekker. Both concern the art of exploring the effects of relevancy via randomized labels on neural networks, and the incorporation would have, according to Bekker, to produce better results in noisy channels (§1 ¶1).
Claim(s) 11 are rejected under 35 U.S.C. 103 as being unpatentable over Chai ("A semi-supervised auto-encoder using label and sparse regularizations for classification", published 1/23/2019) in view of Pondenkandath ("Leveraging random label memorization for unsupervised pre-training", published 2018) in view of Okazaki (US 10803388 B2) in view of Yasrab ("ECRU: An encoder-decoder based convolution neural network (CNN) for road-scene understanding", published 2018).
For claim 11, Chai modified by Pondenkandath modified by Okazaki discloses the device of claim 2, as described above. Chai modified by Pondenkandath does not disclose the limitations of claim 11.
Yasrab discloses: wherein the encode part is implemented by a first convolutional neural network with eight output layers, and the first classifier classifies the feature vector by a second convolutional neural network with eight output layers (fig.1 shows encoding portion, with fig.2 showing a decoder, the encoder and the decoder having eight output layers).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the device of Chai modified by Pondenkandath by incorporating the CNN architecture of Yasrab. Both concern the art of neural networks for images, and the incorporation would have, according to Yasrab, allowed application to image segmentation (§1).
Response to Arguments
In the remarks, Applicant argues:
1. The claims are statutory based on the amendments.
---
Examiner respectfully disagrees; while the amended claims now recite statutory categories (step 1 of the 101 analysis), the claims are directed to an abstract idea without significantly more, as described in the rejection above.
2. The amended limitations overcome the art of record, as the claimed invention does not employ a cost function, but rather induces sparsity by an encoder and decoder. Furthermore, the application of Al-Qatf to Yamada is improper hindsight. Furthermore, Zhang does not overcome the deficiencies of the art and hence the remaining claims are allowable.
---
Examiner submits that applicant’s arguments are moot in view of newly applied art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Park (US 20200193269 A1) discloses a noise-augmented image decoder and classifier.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIANG LI whose telephone number is (303)297-4263. The examiner can normally be reached Mon-Fri 9-12p, 3-11p MT (11-2p, 5-1a ET).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The examiner is available for interviews Mon-Fri 6-11a, 2-7p MT (8-1p, 4-9p ET).
/LIANG LI/
Primary examiner AU 2143
Prosecution Insights