DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/11/2026 has been entered.
This action is in response to the arguments filed on 02/11/2026. Claims 1-20
are pending in the application and have been considered below.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 4, 6-7, 12, 17-18 and 20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-2, 11-12 and 20 of U.S. Patent No.:11,593,703 B2.
Claims 1, 4, 6-7, 12, 15, 17-18 and 20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 10 and 19 of U.S. Patent No.: 10,325,220 B2.
. Although the claims at issue are not identical, they are not patentably distinct from each other because each of the instant claim(s) is anticipated over the corresponding claims in the US Patent No.: US 11,593,703 and US Patent No.: US 10,325,220 B2.
Instant Application
US 11,593,703
US 10,325,220 B2
Claim 1
1. A method comprising:
training, by a computing device, a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level being trained using training data missing a number of applicable labels, the second level being trained using the first level's label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels missing from the training data used to train the first level;
generating, by the computing device, the digital content item's label prediction output
using the trained multi-level label prediction model, the digital content item's labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels; and
automatically generating, by the computing device, an annotation for the digital content item based on the digital content item's labeling prediction output.
Claim 1
1. A method comprising:
training, by a computing device using statistical modeling, a label prediction model to predict an applicability of each label of a plurality of labels to items of digital content, the label prediction model comprising multiple levels, the training comprising:
training an initial level of the multiple levels to generate a label prediction output predicting the applicability of each of the plurality of labels, the initial level being trained using a plurality of training
instances corresponding to a plurality of digital content items, each training instance corresponding to one digital content item of the plurality of digital content items and comprising features generated using content of the one digital content item and
comprising labeling data associated with the one digital content item, the labeling data associated with one or more digital content items of the plurality of digital content items missing a number of applicable labels from the plurality of labels; and
training a next level, of the multiple levels, of the prediction model using the label prediction output generated by the initial level of the label prediction model;
receiving, via the computing device, a digital content item and a set of labels associated with the received digital content item, each label in the set of labels associated
with the received digital content item is one of the plurality of labels;
generating, via the computing device, label prediction input for the received digital content item, the label prediction input comprising a feature set generated
using content of the received digital content item, the label prediction input further comprising the set of labels associated with the received digital content item;
using, via the computing device, the label prediction model and the label prediction input generated for the received digital content item to identify at least one
missing label that is applicable to the received digital content item and is missing from the set of labels, the at least one missing label is one of the plurality of labels; and
automatically annotating, via the computing device, the received digital content item using the at least one missing label, automatic annotation of the digital content augmenting the received set of labels with the at least one missing label.
Claim 1
1. A method comprising:
training, using a multimedia data storage and retrieval
system server, an initial level of a stacked model for use
in making a labeling prediction, the initial level being trained using feature information for each training
instance, corresponding to a content item, of a plurality
of training instances corresponding to a plurality of content items, at least one training instance of the
plurality is missing at least one label of a plurality of
labels, the feature information corresponding to a plurality of features associated with the training instance of the plurality;
generating, using the multimedia data storage and
retrieval system server, a labeling prediction for each
training instance of the plurality using the initial level
of the stacked model, the labeling prediction comprising a label applicability prediction for at least one label of the plurality of labels missing from the training instance's set of labels;
training, using the multimedia data storage and retrieval system server, one or more additional levels of the
stacked model using one or more previous levels of the
stacked model, the one or more previous levels comprising the initial level of the stacked model, each
additional level being trained using information for each training instance of the plurality, each training
instance's information comprising the labeling prediction, from a previous level of the stacked model, the feature information corresponding to the plurality to features, and information indicating the training instance's set of labels;
identifying, using the multimedia data storage and
retrieval system server, a plurality of labeling predictions for the plurality of content items using the stacked model, a labeling prediction, for a content item of the plurality, identifying for each label of the plurality whether the label is applicable to the content item, and
for each content item of the plurality that is missing at
least one label, its labeling prediction comprising a
prediction whether or not the at least one missing label
is applicable;
receiving, via the multimedia data storage and retrieval
system server and from a user computing device, a
content retrieval request comprising a label query;
identifying, via the multimedia data storage and retrieval system server, a number of content items, from the plurality of content items, the identification of the
number of content items comprising using the label
query in searching the plurality of labeling predictions, including each labeling prediction identified for each content item missing at least one label, determined
using the stacked model; and
serving, via the multimedia data storage and retrieval
system server and to a user computing device over an
electronic communications network, the number of
content items in response to the content retrieval request from a user computing device.
Claim 4
4. The method of claim 2, identifying the set of digital content items is responsive to
a received search request.
Claim 2
2. The method of claim 1, further comprising:
receiving, via the computing device, a digital content
search request comprising the at least one label missing
from the received set of labels; and
using, via the computing device, the at least one missing label missing from the received set of labels and
identified using the label prediction model, to retrieve
the received digital content item in response to the
digital content search request.
Claim 6.
6. The method of claim 1, the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels.
From claim 1:
training instances corresponding to a plurality of digital content items, each training instance corresponding to one digital content item of the plurality of digital content items… the labeling data associated with one or more digital content items of the plurality of digital content items missing a number of applicable labels from the plurality of labels.
From claim 1:
for each training instance, corresponding to a content item, of a plurality of training instances corresponding to a plurality of content items, at least one training instance of the plurality is missing at least one label of a plurality of labels, the feature
Claim 7
7. The method of claim 6, a training instance, of the plurality of training
instances comprising associated labeling data
and feature data.
From claim 1:
each training instance corresponding to one digital content item of the plurality of digital content items and comprising features generated
using content of the one digital content item and
comprising labeling data associated with the one
digital content item
From claim 1:
each training
instance's information comprising the labeling prediction, from a previous level of the stacked model, the feature information corresponding to the plurality to features, and information indicating the training instance's set of labels;
Claim 12
12. A computer readable non-transitory storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:
training a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level being trained using training data missing a number of applicable labels, the second level being trained using the first level's label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels missing from the training data used to train the first level;
generating the digital content item’s label prediction output using the trained multi-level label prediction model, the digital content item’s labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels; and automatically generating annotation for the digital content item based on the digital content item’s labeling prediction output.
Claim 11
11. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:
training, using statistical modeling, a label prediction model to predict an applicability of each label of a plurality of labels to items of digital content, the label prediction model comprising multiple levels, the training
comprising:
training an initial level of the multiple levels to generate a label prediction output predicting the applicability of each of the plurality of labels, the initial level being trained using a plurality of training
instances corresponding to a plurality of digital content items, each training instance corresponding to one digital content item of the plurality of digital content items and comprising features generated using content of the one digital content item and
comprising labeling data associated with the one digital content item, the labeling data associated with one or more digital content items of the plurality of digital content items missing a number of applicable labels from the plurality of labels; and
training a next level, of the multiple levels, of the prediction model using the label prediction output generated by the initial level of the label prediction model;
receiving a digital content item and a set of labels associated with the received digital content item, each label in the set of labels associated with the received digital content item is one of the plurality of labels;
generating label prediction input for the received digital content item, the label prediction input comprising a feature set generated using content of the received
digital content item, the label prediction input further comprising the set of labels associated with the received digital content item;
using the label prediction model and the label prediction input generated for the received digital content item to identify at least one missing label that is applicable to the received digital content item and is missing from
the set of labels, the at least one missing label is one of the plurality of labels; and
automatically annotating the received digital content item using the at least one missing label, automatic annotation of the digital content augmenting the received set
of labels with the at least one missing label.
Claim 19
19. A computer readable non-transitory storage medium having tangibly stored thereon processor-executable instructions, that when executed by a multimedia data storage and retrieval system server perform a method comprising:
training an initial level of a stacked model for use in
making a labeling prediction, the initial level being trained using feature information for each training instance, corresponding to a content item, of a plurality of training instances corresponding to a plurality of content items, at least one training instance of the plurality is missing at least one label of a plurality of labels, the feature information corresponding to a plurality of features associated with the training instance of the plurality;
generating a labeling prediction for each training instance of the plurality using the initial level of the stacked
model, the labeling prediction comprising a label applicability prediction for at least one label of the plurality
of labels missing from the training instance's set of
labels;
training one or more additional levels of the stacked model using one or more previous levels of the stacked the one or more previous levels comprising the initial level of the stacked model, each additional level
being trained using information for each training
instance of the plurality, each training instance's information comprising the labeling prediction from a previous level of the stacked model, the feature information corresponding to the plurality to features, and
information indicating the training instance's set of
labels;
identifying a plurality of labeling predictions for the 30
plurality of content items using the stacked model, a
labeling prediction, for a content item of the plurality,
identifying for each label of the plurality whether the
label is applicable to the content item;
receiving, from a user computing device, a content
retrieval request comprising a label query;
identifying a number of content items, from the plurality of content items, the identification of the number of content items comprising using the label query in
searching the plurality of labeling predictions, including each labeling prediction identified for each content item missing at least one label, determined using the stacked model; and
serving, to a user computing device over an electronic communications network, the number of content items
in response to the content retrieval request from a use;
computing device.
Claim 17
17. The computer readable non-transitory storage medium of claim 12, the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels.
From claim 11:
training instances corresponding to a plurality of digital content items, each training instance corresponding to one digital content item of the plurality of digital content items… the labeling data associated with one or more digital content items of the plurality of digital content items missing a number of applicable labels from the plurality of labels.
From claim 19:
for each training instance, corresponding to a content item, of a plurality of training instances corresponding to a plurality of content items, at least one training instance of the plurality is missing at least one label of a plurality of labels, the feature information corresponding to a plurality of features associated with the training instance of the plurality;
Claim 18
18. The computer readable non-transitory storage medium of claim 17, a training instance, of the plurality of training instances, comprising associated labeling data and feature data
From claim 11:
each training instance …comprising features generated using content of the one digital content item and comprising labeling data associated with the one digital content item
From claim 19:
for each training instance, corresponding to a content item, of a plurality of training instances corresponding to a plurality of content items, at least one training instance of the plurality is missing at least one label of a plurality of labels, the feature information corresponding to a plurality of features associated with the training instance of the plurality;
Claim 20
20. A system comprising:
a processor;
a storage medium for tangibly storing thereon program logic for execution by the processor, the stored logic comprising:
training logic executed by the processor for training a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level being trained using training data missing a number of applicable labels, the second level being trained using the first level's label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels missing from the training data used to train the first level;
generating logic executed by the processor for generating the digital content item's label prediction output using the trained multi-level label prediction model, the digital content item's labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels; and
generating logic executed by the processor for automatically generating annotation for the digital content item based on the digital content item's labeling prediction output.
Claim 20
20. A computing device comprising:
a processor;
a non-transitory storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:
training logic executed by the processor for training, using statistical modeling, a label prediction model to predict an applicability of each label of a plurality of labels to items of digital content, the label prediction model comprising multiple levels, the training
logic comprising:
training logic executed by the processor for training an initial level of the multiple levels to generate a label prediction output predicting the applicability of each of the plurality of labels, the initial level being trained using a plurality of training instances
corresponding to a plurality of digital content
items, each training instance corresponding to one digital content item of the plurality of digital content items and comprising features generated using content of the one digital content item and comprising labeling data associated with the one digital content item, the labeling data associated with one or more digital content items of the plurality of digital content items missing a number of applicable labels from the plurality of labels; and
training logic executed by the processor for training a next level, of the multiple levels, of the prediction model using the label prediction output generated by the initial level of the label prediction model;
receiving logic executed by the processor for receiving a digital content item and a set of labels associated with the received digital content item, each label in the set of labels associated with the received digital content item is one of the plurality of labels;
generating logic executed by the processor for generating label prediction input for the received digital content item, the label prediction input comprising a feature set generated using content of the received digital content item, the label prediction input further
comprising the set of labels associated with the received digital content item;
using logic executed by the processor for using the label prediction model and the label prediction input generated for the received digital content item to identify at least one missing label that is applicable to the received digital content item and is missing from the set of labels, the at least one missing label
is one of the plurality of labels; and
annotating logic executed by the processor for automatically annotating the received digital content item using the at least one missing label, automatic annotation of the digital content augmenting the received set of labels with the at least one missing label.
Claim 10
10. A multimedia data storage and retrieval system server
comprising:
a processor;
a non-transitory storage medium for tangibly storing
thereon program logic for execution by the processor,
the stored logic comprising:
training logic executed by the processor for training an
initial level of a stacked model for use in making a
labeling prediction, the initial level being trained using feature information for each training instance, corresponding to a content item, of a plurality of
training instances corresponding to a plurality of content items, at least one training instance of the
plurality is missing at least one label of a plurality of
labels, the feature information corresponding to a plurality of features associated with the training instance of the plurality;
generating logic executed by the processor for generating a labeling prediction for each training instance of the plurality using the initial level of the stacked model, the labeling prediction comprising a label applicability prediction for at least one label of the plurality of labels missing from the training instance's set of labels;
training logic executed by the processor for training one or more additional levels of the stacked model using one or more previous levels of the stacked model, the one or more previous levels comprising the initial level of the stacked model, each additional level being trained using information for each training instance of the plurality, each training instance's information comprising the labeling prediction from a previous level of the stacked model, the feature information corresponding to the plurality to features,
and information indicating the training instance's set of labels;
identifying logic executed by the processor for identifying
a plurality of labeling predictions for the plurality of content items using the stacked model, a labeling prediction, for a content item of the plurality, identifying for each label of the plurality whether the label is applicable to the content item;
receiving logic executed by the processor for receiving, from a user computing device, a content retrieval
request comprising a label query;
identifying logic executed by the processor for identifying
a number of content items, from the plurality of content items, the identification of the number of content items comprising using the label query in searching the plurality of labeling predictions,
including each labeling prediction identified for each
content item missing at least one label, determined using the stacked model; and
serving logic executed by the processor for serving, to a user computing device over an electronic communications network, the number of content items, in response to the content retrieval request from a user computing device.
The instant claims are anticipated by the reference claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1- 3, 6-14 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Montañés et al. ("Improving stacking approach for multi-label classification," hereinafter referred to as Montañés), in view of Bucak et al. ("Multi-label Learning with Incomplete Class Assignments," hereinafter referred to as Bucak).
As to claim 1, Montañés teaches a method comprising:
training, [by a computing device], a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level [using training data missing a number of applicable labels], the second level being trained using the first level’s label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels from the training data used to train the first level (see 2. Multi-label classification…We consider a multi-label classification task given by a training set S = f (x1; y1); (xn; yn)g, whose instances were independently and randomly obtained from an unknown probability distribution P(X;Y) on X _ Y, in which the output space Y is the power set of L, in symbols P(L)… 3. Stacking approaches ln the learning stage, the method builds two groups of stacked classifiers where the new features correspond to the outputs of the first-level classifiers. Hence, the outputs of the first-level classifiers, h1 (x), are only taken for obtaining the values of the augmented feature space of the second-level classifiers.; 3.2. Other related methods In training phase, CC orders the labels in a chain and builds m binary classifiers, one per each label, the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers.);
generating, [by the computing device], the digital content item's label prediction output using the trained multi-level label prediction model, the digital content item's labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels (see 1. Introduction, groups of stacked classifiers that the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack. The classifiers on the top of the stack determine the final prediction. The idea of the authors is to stack two levels of classifiers, where the first level is composed by independent classifiers for each label as those of BR, whose outputs are employed in the second level to build another group of classifiers that try to capture dependences among labels., 2. Multi-label classification …wherein multi-label classification correctly predicts the subset of labels).
However, Montañés fails to explicitly teach:
[by a computing device],
[training data missing a number of applicable labels];
automatically generating, by the computing device, an annotation for the digital content item based on the digital content item's labeling prediction output.
Bucak, in combination with Montañés, teaches:
by a computing device (page 2801, 1. Introduction – computer),
training data missing a number of applicable labels (page 2805, right column, first and second paragraphs; Figure 2 shows how different methods perform in finding the missing true labels for training examples, where only the underlined true labels. We observe that MLR-GL is able to find more missing labels than the other baselines);
automatically generating, by the computing device, an annotation for the digital content item based on the digital content item's labeling prediction output (see 1. Introduction…automatic image annotation…multi-label learning with incomplete class assignments, and the training data as incompletely labeled data… Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large; Fig. 1; section 4. Experimental Results…Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “automatic annotation” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to automatically detect the missed class assignment and consequentially improve the classification accuracy, as suggested by Bucak (Introduction).
As to claim 2, which includes the rejection of claim 1, Montañés fails to explicitly teach:
identifying, via the computing device, a set of digital content items using the digital content item's annotation.
Bucak, in combination with Montañés, teaches:
identifying, via the computing device, a set of digital content items using the digital content item's annotation (1. Introduction, Figure 1. Example images from ESP Game dataset and their annotations. The annotations highlighted by bold font, which are used to annotate the same concept/object in the corresponding images, are examples of label ambiguity problem; 4. Experimental Results Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets. The number of classes).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “an identification of a set of digital content items using the digital content item's annotation” to the combination system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to add the annotation of the multi-label classification to improve accuracy of the later classifications in the multi-labeling model, as suggested by Bucak (Introduction).
As to claim 3, which includes the rejection of claim 2, Montañés fails to explicitly teach:
each digital content item in the set of digital content items having a respective annotation generated using label prediction output from the trained multi-level label prediction model.
Bucak, in combination with Montañés, teaches:
each digital content item in the set of digital content items having a respective annotation generated using label prediction output from the trained multi-level label prediction model (see 1. Introduction, automatic image annotation, multi-label learning with incomplete class assignments, and the training data as incompletely labeled data, Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large; Fig. 1; section 4. Experimental Results Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a respective annotation generated using label prediction output from the trained multi-level label prediction model” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to the annotation of the multi-label classification to improve accuracy of the later classifications in the multi-labeling model, as suggested by Bucak (Introduction), as suggested by Bucak (Introduction).
As to claim 6, which includes the rejection of claim 1, Montañés fails to explicitly teach:
the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels.
Bucak, in combination with Montañés, teaches the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels (1. Introduction…we consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance. As an example, an instance whose true class assignment is (c1, c2, c3) is only presented with class c1 when it is used for training. Our goal is to learn a multi-labeling model from the training examples with incomplete class assignments).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “training instances” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance, as suggested by Bucak (Introduction).
As to claim 7, which includes the rejection of claim 6, Montañés fails to explicitly teach:
a training instance, of the plurality of training instances, comprising associated labeling data and feature data.
Bucak, in combination with Montañés, teaches a training instance, of the plurality of training instances, comprising associated labeling data and feature data (Introduction…we consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance. As an example, an instance whose true class assignment is (c1, c2, c3) is only presented with class c1 when it is used for training. Our goal is to learn a multi-labeling model from the training examples with incomplete class assignments).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a training instance” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance, as suggested by Bucak (Introduction).
As to claim 8, which includes the rejection of claim 7, Montañés fails to explicitly teach:
analyzing, a digital content item of the plurality of digital content items, and based on the analysis, generating the feature data for the analyzed digital content item.
Bucak, in combination with Montañés, teaches:
analyzing, a digital content item of the plurality of digital content items, and based on the analysis, generating the feature data for the analyzed digital content item (1. Introduction Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large, and it is only feasible for users to provide limited number of class labels for a given instance. Fig. 1 shows examples of annotated images from the ESP Game. We see some of the annotated words can cause ambiguity. For instance, the keywords baby, kid and boy can be used interchangeable, and this can be given as an example of label ambiguity. Note that these annotations are generated by collapsing annotated words from multiple users. It is thus very likely that each individual user only provides incomplete annotation with a few keywords).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a digital content item analysis” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to use a multi-label learning dealing with incompletely labeled data, as suggested by Bucak (Introduction).
As to claim 9, which includes the rejection of claim 1, Montañés teaches the number of levels of the multi-level label prediction model is empirically determined, a final level of the multi-level label prediction model being identified based on a determined convergence in the label prediction output generated by the final level and a potential next level (1. Introduction These kinds of approaches are based on building groups of stacked classifiers that the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack. The classifiers on the top of the stack determine the final prediction; 3. Stacking approaches In the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers).
As to claim 10, which includes the rejection of claim 1, Montañés fails to explicitly teach the digital content item is a document and the annotation comprises a number of words contained in the document.
Bucak, in combination with Montañés, teaches the digital content item is a document and the annotation comprises a number of words contained in the document (1. Introduction, Fig. 1 shows examples of annotated images from the ESP Game. We see some of the annotated words can cause ambiguity. For instance, the keywords baby, kid and boy can be used interchangeable, and this can be given as an example of label ambiguity. Note that these annotations are generated by collapsing annotated words from multiple users. It is thus very likely that each individual user only provides incomplete annotation with a few keywords).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “words contained in the document” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to have a user only provides incomplete annotation with a few keywords in the multi-label learning with incomplete labeled data, as suggested by Bucak (Introduction).
As to claim 11, which includes the rejection of claim 1, Montañés fails to explicitly teach
the digital content item comprises an image.
Bucak, in combination with Montañés, teaches the digital content item comprises an image (1. Introduction… Fig. 1 shows examples of annotated images from the ESP Game. We see some of the annotated words can cause ambiguity. For instance, the keywords baby, kid and boy can be used interchangeable, and this can be given as an example of label ambiguity. Note that these annotations are generated by collapsing annotated words from multiple users. It is thus very likely that each individual user only provides incomplete annotation with a few keywords…).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a digital content item’s image” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to verify the effectiveness of the proposed approach in handling incompletely labeled data, as suggested by Bucak (Introduction).
As to claim 12, Montañés teaches [a computer readable non-transitory storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device] perform a method comprising:
training, [by a computing device], a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level [using training data missing a number of applicable labels], the second level being trained using the first level’s label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels from the training data used to train the first level (see 2. Multi-label classification…We consider a multi-label classification task given by a training set S = f(x1; y1); ; (xn; yn)g, whose instances were independently and randomly obtained from an unknown probability distribution P(X;Y) on X _ Y, in which the output space Y is the power set of L, in symbols P(L)…; 3. Stacking approaches, ... ln the learning stage, the method builds two groups of stacked classifiers, where the new features correspond to the outputs of the first-level classifiers. Hence, the outputs of the first-level classifiers, h1 (x), are only taken for obtaining the values of the augmented feature space of the second-level classifiers.; 3.2. Other related methods In training phase, CC orders the labels in a chain and builds m binary classifiers, one per each label…In the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers.);
generating, [by the computing device], the digital content item's label prediction output using the trained multi-level label prediction model, the digital content item's labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels (see 1. Introduction, groups of stacked classifiers that the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack. The classifiers on the top of the stack determine the final prediction; 2. Multi-label classification, wherein multi-label classification correctly predicts the subset of labels).
However, Montañés fails to explicitly teach:
[a computer readable non-transitory storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device]
[using training data missing a number of applicable labels];
automatically generating, by the computing device, an annotation for the digital content item based on the digital content item's labeling prediction output.
Bucak, in combination with Montañés, teaches:
[a computer readable non-transitory storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device] (page 2801, 1. Introduction – computer), [using training data missing a number of applicable labels] (page 2805, right column, first and second paragraphs…Figure 2 shows how different methods perform in finding the missing true labels for training examples, where only the underlined true labels. We observe that MLR-GL is able to find more missing labels than the other baselines);
automatically generating, by the computing device, an annotation for the digital content item based on the digital content item's labeling prediction output (see 1. Introduction, automatic image annotation, multi-label learning with incomplete class assignments, and the training data as incompletely labeled data. Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large; Fig. 1; section 4. Experimental Results…Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “automatic annotation” to the combination system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to automatically detect the missed class assignment and consequentially improve the classification accuracy, as suggested by Bucak (Introduction).
As to claim 13, which includes the rejection of claim 12, Montañés fails to explicitly teach: identifying, via the computing device, a set of digital content items using the digital content item's annotation.
Bucak, in combination with Montañés, teaches:
identifying, via the computing device, a set of digital content items using the digital content item's annotation (1. Introduction, Figure 1. Example images from ESP Game dataset and their annotations. The annotations highlighted by bold font, which are used to annotate the same concept/object in the corresponding images, are examples of label ambiguity problem; 4. Experimental Results Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets. The number of classes).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “an identification of a set of digital content items using the digital content item's annotation” to the combination system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to add the annotation of the multi-label classification to improve accuracy of the later classifications in the multi-labeling model, as suggested by Bucak (Introduction).
As to claim 14, which includes the rejection of claim 13, Montañés fails to explicitly teach:
each digital content item in the set of digital content items having a respective annotation generated using label prediction output from the trained multi-level label prediction model.
Bucak, in combination with Montañés, teaches:
each digital content item in the set of digital content items having a respective annotation generated using label prediction output from the trained multi-level label prediction model (see 1. Introduction…automatic image annotation…multi-label learning with incomplete class assignments, and the training data as incompletely labeled data… Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large…; Fig. 1; section 4. Experimental Results…Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets…).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a respective annotation generated using label prediction output from the trained multi-level label prediction model” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to the annotation of the multi-label classification to improve accuracy of the later classifications in the multi-labeling model, as suggested by Bucak (Introduction).
As to claim 17, which includes the rejection of claim 12, Montañés fails to explicitly teach: the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels.
Bucak, in combination with Montañés, teaches the training data comprising a plurality of training instances corresponding to a plurality of digital content items, labeling data corresponding to at least one training instance is missing at least one applicable label from the plurality of labels (1. Introduction…we consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance. As an example, an instance whose true class assignment is (c1, c2, c3) is only presented with class c1 when it is used for training. Our goal is to learn a multi-labeling model from the training examples with incomplete class assignments.).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “training instances” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance, as suggested by Bucak (Introduction).
As to claim 18, which includes the rejection of claim 17, Montañés fails to explicitly teach a training instance, of the plurality of training instances, comprising associated labeling data and feature data.
Bucak, in combination with Montañés, teaches a training instance, of the plurality of training instances, comprising associated labeling data and feature data (1. Introduction…we consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance. As an example, an instance whose true class assignment is (c1, c2, c3) is only presented with class c1 when it is used for training. Our goal is to learn a multi-labeling model from the training examples with incomplete class assignments).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a training instance” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to consider the multi-label learning problem in which only a subset of the true class assignments is available for each training instance, as suggested by Bucak (Introduction).
As to claim 19, which includes the rejection of claim 18, Montañés fails to explicitly teach: analyzing, a digital content item of the plurality of digital content items, and based on the analysis, generating the feature data for the analyzed digital content item.
Bucak, in combination with Montañés, teaches: analyzing, a digital content item of the plurality of digital content items, and based on the analysis, generating the feature data for the analyzed digital content item (1. Introduction…Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large, and it is only feasible for users to provide limited number of class labels for a given instance. Fig. 1 shows examples of annotated images from the ESP Game. We see some of the annotated words can cause ambiguity. For instance, the keywords baby, kid and boy can be used interchangeable, and this can be given as an example of label ambiguity. Note that these annotations are generated by collapsing annotated words from multiple users. It is thus very likely that each individual user only provides incomplete annotation with a few keywords).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “a digital content item analysis” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to use a multi-label learning dealing with incompletely labeled data, as suggested by Bucak (Introduction).
As to claim 20, Montañés teaches a [system] comprising:
[a processor];
[a storage medium for tangibly storing thereon program logic for execution by the processor, the stored logic comprising:
training logic executed by the processor for] training a multi-level label prediction model to predict an applicability, to a digital content item, of each label of a plurality of labels, the multi-level label prediction model comprising a first level and a second level, the first level being trained [using training data missing a number of applicable labels], the second level being trained using the first level's label prediction output comprising a label applicability prediction for each of the plurality of labels including the number of applicable labels missing from the training data used to train the first level (see 2. Multi-label classification…We consider a multi-label classification task given by a training set S = f (x1; y1); (xn; yn)g, whose instances were independently and randomly obtained from an unknown probability distribution P(X;Y) on X _ Y, in which the output space Y is the power set of L, in symbols P(L)…; 3. Stacking approaches ln the learning stage, the method builds two groups of stacked classifiers where the new features correspond to the outputs of the first-level classifiers. Hence, the outputs of the first-level classifiers, h1 (x), are only taken for obtaining the values of the augmented feature space of the second-level classifiers.; 3.2. Other related methods In training phase, CC orders the labels in a chain and builds m binary classifiers, one per each label, the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers.);
[generating logic executed by the processor for] generating the digital content item's label prediction output using the trained multi-level label prediction model, the digital content item's labeling prediction output indicating an applicability, to the digital content item, of each label of the plurality of labels (see 1. Introduction, groups of stacked classifiers that the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack. The classifiers on the top of the stack determine the final prediction; 2. Multi-label classification, wherein multi-label classification correctly predicts the subset of labels).
However, Montañés fails to explicitly teach:
[system]
[a processor];
[a storage medium for tangibly storing thereon program logic for execution by the processor, the stored logic comprising:
training logic executed by the processor for];
[using training data missing a number of applicable labels];
[generating logic executed by the processor for] automatically generating annotation for the digital content item based on the digital content item's labeling prediction output.
Bucak, in combination with Montañés, teaches:
[system] (page 2801, 1. Introduction – computer)
[a processor] (page 2801, 1. Introduction – computer);
[a storage medium for tangibly storing thereon program logic for execution by the processor (page 2801, 1. Introduction – computer), the stored logic comprising:
training logic executed by the processor for] (1. Introduction – computer);
training data missing a number of applicable labels (page 2805, right column, first and second paragraphs; Figure 2 shows how different methods perform in finding the missing true labels for training examples, where only the underlined true labels. We observe that MLR-GL is able to find more missing labels than the other baselines);
[generating logic executed by the processor for] automatically generating annotation for the digital content item based on the digital content item's labeling prediction output (see 1. Introduction…automatic image annotation…multi-label learning with incomplete class assignments, and the training data as incompletely labeled data… Multi-label learning with incomplete class assignments is frequently encountered in automatic image annotation when the number of classes is very large; Fig. 1; section 4. Experimental Results…Datasets. Two multi-labeled datasets for automatic image annotation are used in our study: ESP Game [31] and MIR Flickr [16] datasets).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Montañés to add “automatic annotation” to the system of Montañés, as taught by Bucak above. The modification would have been obvious because one of ordinary skill would be motivated to be able to automatically detect the missed class assignment and consequentially improve the classification accuracy, as suggested by Bucak (Introduction).
Claims 4- 5 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Montañés et al. ("Improving stacking approach for multi-label classification," hereinafter referred to as Montañés), in view of Bucak et al. ("Multi-label Learning with Incomplete Class Assignments," hereinafter referred to as Bucak), and further in view of Yan et al. (“Active Learning from Crowds,” hereinafter referred to as Yan).
As to claim 4, which includes the rejection of claim 2, Montañés and Bucak fail to explicitly teach: identifying the set of digital content items is responsive to a received search request.
Yan, in combination with Montañés and Bucak, teaches:
identifying the set of digital content items is responsive to a received search request (Abstract…multiple labelers, with varying expertise, are available for querying…select both a sample and the annotator/s to query the labels from…; section 3. A Probabilistic Multi-Labeler Model…the labels from individual labelers may not be correct, may be missing, and may not be consistent with each other).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Montañés and Bucak to add “a digital content search request” to the combination system of Montañés and Bucak, as taught by Yan above. The modification would have been obvious because one of ordinary skill would be motivated to provide an optimization formulation that allows the selection of the most uncertain sample and the annotator to query the labels from for active learning, as suggested by Yan (Introduction).
As to claim 5, which includes the rejection of claim 4, Montañés and Bucak fail to explicitly teach: the received search request comprising information indicating the digital content item.
Yan, in combination with Montañés and Bucak, teaches the received search request comprising information indicating the digital content item (Abstract…multiple labelers, with varying expertise, are available for querying…select both a sample and the annotator/s to query the labels; section 3. A Probabilistic Multi-Labeler Model…the labels from individual labelers may not be correct, may be missing, and may not be consistent with each other; section 5.2. Text Data ..., wherein using the broadest reasonable interpretation, Examiner interprets "our multi-labeler could extract information available among the annotators and query the annotations from the most reliable annotator for incoming samples " to teach the limitation).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Montañés and Bucak to add “a digital content search request” to the combination system of Montañés and Bucak, as taught by Yan above. The modification would have been obvious because one of ordinary skill would be motivated to provide an optimization formulation that allows the selection of the most uncertain sample and the annotator to query the labels from for active learning, as suggested by Yan (Introduction).
As to claim 15, which includes the rejection of claim 13, Montañés and Bucak fail to explicitly teach: identifying the set of digital content items is responsive to a received search request.
Yan, in combination with Montañés and Bucak, teaches:
identifying the set of digital content items is responsive to a received search request (Abstract…multiple labelers, with varying expertise, are available for querying…select both a sample and the annotator/s to query the labels from…; section 3. A Probabilistic Multi-Labeler Model…the labels from individual labelers may not be correct, may be missing, and may not be consistent with each other).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Montañés and Bucak to add “a digital content search request” to the combination system of Montañés and Bucak, as taught by Yan above. The modification would have been obvious because one of ordinary skill would be motivated to provide an optimization formulation that allows the selection of the most uncertain sample and the annotator to query the labels from for active learning, as suggested by Yan (Introduction).
As to claim 16, which includes the rejection of claim 15, Montañés and Bucak fail to explicitly teach: the received search request comprising information indicating the digital content item.
Yan, in combination with Montañés and Bucak, teaches the received search request comprising information indicating the digital content item (Abstract, multiple labelers, with varying expertise, are available for querying…select both a sample and the annotator/s to query the labels from; section 3. A Probabilistic Multi-Labeler Model…the labels from individual labelers may not be correct, may be missing, and may not be consistent with each other; section 5.2. Text Data, wherein using the broadest reasonable interpretation, Examiner interprets " our multi-labeler could extract information available among the annotators and query the annotations from the most reliable annotator for incoming samples" to teach the limitation).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Montañés and Bucak to add “a digital content search request” to the combination system of Montañés and Bucak, as taught by Yan above. The modification would have been obvious because one of ordinary skill would be motivated to provide an optimization formulation that allows the selection of the most uncertain sample and the annotator to query the labels from for active learning, as suggested by Yan (Introduction).
Response to Applicant’s arguments
Applicant's arguments on file on 02/11/2026. with respect to prior art rejection of claims 1-20 have been considered and are not persuasive.
REMARKSDocket No.: 4952-575 Application No.: 15/956,894
Double Patenting Rejection
Argument (page 7)
None of the claims of the '703 Patent recites at least these elements nor is there any
evidence of record showing that the differences in the claims are obvious variants, rather they are patentably distinct inventions.
Accordingly, Applicant respectfully requests that this double patenting rejection be
withdrawn.
Examiner's response:
Examiner respectfully disagrees. All elements of independent claim 1 of the instant claims are recited in the ‘703 patent. Therefore, the instant claims are anticipated by the claims of the “703” and “220” Patents as shown in the rejection above. Therefore, Examiner maintains the “Double Patenting Rejection.”
The Art Rejections
Argument (pages 8-9)
Applicant appears to assert that unlike the claimed subject matter which trains a first level of a multi-level label prediction model using training data missing a number of applicable labels, Montanes merely discloses using training data without any missing labels (i.e., with all of the applicable labels) to train both levels of its classifiers (see, e.g., §§ 2 and 3.1 of Montanes).
Additionally, Montanes discloses that it does not use any prediction output from its first
level to train its second (or meta) level. See, e.g., § 3.1, last paragraph on page 3 of Montanes. Montanes expressly states that the first-level-classifier predictions are inadequate for its purposes. According to Montanes, it trains its second level of classifiers using the same training data set consisting of no missing labels used to train its first level of classifiers. See § 3 .1 of Montanes. This is an explicit teaching away from what is claimed.
This is in stark contrast to the claimed subject matter which recites that the claimed
second level of a multi-level label prediction model is trained using label prediction output generated by the trained first level that includes an applicability prediction for each of the number of applicable labels missing from the training data used to train the first level of the multi-level label prediction model.
Thus, according to Montanes: 1) its first level is trained using training data that includes
all of the applicable labels (i.e., its first level training data is not missing any label data); 2) its second (or meta) level training uses the same training data used to train the first level - i.e., training data that is not missing any labels; and 3) its second (or meta) level does not use any prediction output from its first level to train its second level.
This is clearly structurally and functionally different from the claimed subject matter,
which recites: 1) the claimed first level of the claimed muti-level label prediction model that is trained using training data that is missing a number of applicable labels; 2) the trained first level that generates prediction output comprising a label applicability prediction for each of the number of applicable labels missing from the first level' straining data; and 3) the claimed second level of the claimed multi-level label prediction model that is trained using the label prediction output generated by the claimed model's trained first level
Examiner's response:
Examiner respectfully disagrees.
“One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., Inc., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986)” (MPEP 2145 (IV).
1) its first level is trained using training data that includes all of the applicable labels (i.e., its first level training data is not missing any label data);
As stated in the Office Action, Montanes does teach multi-label classification based on stacking paradigm, wherein groups of stacked classifiers the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack.
However, Montanes does not teach “training data set that is missing a number of applicable labels.”
Bucak teaches “training data missing a number of applicable labels.”
Therefore, Bucak, in combination with Montañés, teaches the claimed first level.
2) its second (or meta) level training uses the same training data used to train the first level - i.e., training data that is not missing any labels.
Montanes does teach that “outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack.”
The outputs of the first level are transformed training data, therefore they are different from the original training data.
Therefore, the training data for the classifiers of the next level is not the same; it is the output of the previous level's classifiers, and it is typically generated using the original training data. Specifically, the outputs from the first-level classifiers (predictions on a portion of the original data) are used to train a second-level "meta-classifier."
3) its second (or meta) level does not use any prediction output from its first level to train its second level.
In section 3.2. Other related methods, Montanes does teach “In training
phase, CC orders the labels in a chain and builds m binary classifiers, one per
each label.
In the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers.
Therefore, Montanes does use prediction output from its first level to train its second level.
Argument (page 10)
Applicant respectfully disagrees. Since in fact, as is conceded by the Examiner, Montanes' training data set is not missing a number of applicable labels, the output of Montanes' first level of classifiers cannot correspond to the claimed first level's label prediction output comprising a label applicability prediction for each of the number of application labels missing from the training data set used to train the claimed first level. Respectfully, there is no technical support in the reference or in reality for the Examiner's presumption
Examiner's response:
In section 3.2. Other related methods, Montanes does teach “In training
phase, CC orders the labels in a chain and builds m binary classifiers, one per
each label.
In the prediction time, the classifiers are applied following the order of the chain in a way that the outputs of the previous classifiers are employed to augment the successive feature space of the next classifiers.
Therefore, Montanes does teach prediction output from its first level to train its second level.
However, Montanes does not teach “training data set that is missing a number of applicable labels.”
Bucak teaches “training data missing a number of applicable labels.”
Therefore, Bucak, in combination with Montañés, teaches the claimed first level.
Argument (page 11)
Applicant appears to assert that Montanes fails to disclose the annotation claim element. Applicant respectfully submits that Bucak fails to remedy the deficiencies of Montanes noted herein and by the Examiner.
Like Montanes, Bucak fails to disclose the claimed multi-level label prediction model.
Consequently, as with Montanes, Bucak cannot and does not disclose the claimed generation of an annotation for a digital content item based on labeling prediction output of the claimed multilabel prediction model.
In view of at least the foregoing, the Applicant submits that Montanes and Bucak do not
yield all of the elements in claim 1, and therefore Montanes and Bucak cannot form the basis of a proper§ 103 rejection. Furthermore, since Montanes and Bucak each fails to disclose each and every one of the elements of claim 1, none of them can form the basis of a proper § 102 rejection, and no such rejection is raised by the Office Action. Similar arguments are also applicable with independent claims 12 and 20. Since each of the dependent claims recites all of the elements of its independent base claim, the arguments made herein are equally applicable to each dependent claim.
Examiner's response:
Examiner respectfully disagrees. Bucak teaches multi-label learning where class assignments of training examples are incomplete.
Montanes teaches, in Introduction, where groups of stacked classifiers that the outputs of the classifiers of one level are used as inputs for the classifiers of the next level of the stack. The classifiers on the top of the stack determine the final prediction, and 2. Multi-label classification, wherein multi-label classification correctly predicts the subset of labels. 3. Stacking approaches…where the m new features correspond to the outputs of the first-level classifiers…
Montanes and Bucak do yield all of the elements in claims 1, 12 and 20.
Therefore, Montanes and Bucak form the basis of a proper§ 103 rejection (see office action above).
Argument (pages 13-14)
Applicant objects to the hypothetical combination proposed in the Office Action.
There is no motivation in any of the references to modify them to change the functionality of what is taught by the references into what is taught and claimed in the instant application, in other words how to use the system of Montanes with a training set that is missing labels. A person of skill would recognize it would not function and there is nothing in the prior art of record as how to remedy this deficiency. There is no teaching, suggestion, or motivation to combine the references to achieve what is taught in the instant application, and using the instant application as a roadmap to combine the art is impermissible hindsight reconstruction.
Examiner’s response:
Examiner respectfully provide an explanation for the rejection in the office action. In response to applicant's argument that there is no motivation in any of the references to modify them to change the functionality of what is taught by the references into what is taught and claimed in the instant application.
Examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007).
Per MPEP, "any judgment on obviousness is in a sense necessarily a reconstruction based on hindsight reasoning, but so long as it takes into account only knowledge which was within the level of ordinary skill in the art at the time the claimed invention was made and does not include knowledge gleaned only from applicant's disclosure, such a reconstruction is proper."
Examiner did rely on the teaching of the prior arts which is within the level of ordinary in
the art at the time the claimed invention was made and does not include knowledge
gleaned from applicant's disclosure (see Bucak et al. ("Multi-label Learning with Incomplete Class Assignments"), for the motivation).
Accordingly, combining Montanes and Bucak would have been obvious to one of ordinary skill in the art in order to teach the limitations of claim 1.
No further arguments were presented for the dependent claims. Therefore,
Montanes, Bucak and Yan would yield all of the elements in claims 4, 5, 15 and 16.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABABACAR SECK whose telephone number is (571)270-7146. The examiner can normally be reached Monday-Friday 8:00 A.M.-6:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lamardo Viker can be reached on 5712705871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABABACAR SECK/Examiner, Art Unit 2122
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights