DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/6/2024 has been entered.
Response to Arguments
Applicant's arguments filed 6/6/2024 have been fully considered but they are not persuasive.
Claims 1-20 are pending in this application and have been considered below.
Arguments:
The applicant argues that Iyer does not teach “training a generative adversarial network to provide contextual information.” Instead, Iyer disclose using generative model to incorporate an advertisement into the original content. Arguments p. 5.
Examiner’s Response:
The examiner does not see the distinction. Both the applicant and the examiner have stated that Iyer uses a generative model to incorporate an advertisement into the original content.1 Iyer’s generative neural network model that generates advertisement based on context must be trained, like any other generative neural network model. Iyer discloses this training process in ¶84 as disclosed in the final.2
Arguments:
The applicant argues Iyer does not provide the content of the claimed contextual information. Arguments p. 6.
Examiner’s Response:
Iyer disclose the contextual information is provided to the user in the form of advertisements. The claims are silent as to how the contextual information is provided. Therefore, the advertisements embedded in the video based on the a similar context as the watched context read on the claimed “contextual information to be embedded in said image or said video frames.”3
Arguments:
The applicant argues Iyer does not provide contextual information to be embedded in images or video frames. Arguments p. 6-8.
Examiner’s Response:
The applicant argues and examiner agrees that Iyer “us[ing] AI to intelligently place an advertised product or service into the watched content.” Arguments p. 8. The applicant argues the distinction that Iyer discloses intelligently placing an advertisement into the watched content as opposed to embedding contextual information in images or video frames. However, it has already been established that the advertisements are a form of contextual information. For instance, the modified video content takes in the context of the individual users to tailor the advertisements to individual users. Iyer ¶31. Therefore, the advertisements read on contextual information. Iyer provides multiple examples of products or people that can be replaced with advertisement information. It would not make sense to provide an out-of-context person in place of a product or a product in place of a person within the context of a video. Therefore, the scene’s context information as well as the context of the user watching, provides information to the content of the advertisement. In this way, advertising is contextual information.
Arguments:
The applicant argues that “The Examiner must provide a basis in fact and/or technical reasoning for concluding that the obtained premium video content from the content server, as taught in Iyer, corresponds to the claimed references. Ex parte Levy, 17 U.S.P.Q.2d 1461, 1464 (Bd. Pat. App. & Inter. 1990). That is, the Examiner must provide extrinsic evidence that must make clear that the obtained premium video content from the content server, as taught in Iyer, corresponds to the claimed references, and that it would be so recognized by persons of ordinary skill. In re Robertson, 169 F.3d 743, 745 (Fed. Cir. 1999).
Examiner’s Response:
The applicant answers their own arguments in the analysis of ¶¶35-36, 45-46 and 48-
49 by disclosing “Furthermore, Iyer discloses that using deep learning techniques, the system may analyze the watched content in real time, and/or may use cached video analytics from an offline or prior analysis of the watched content, to "learn" certain information about the content and what it contains … Iyer further discloses that certain video content is received from and delivered by a content producer, such as a movie, TV show, sporting event, and/or any
other type of premium or non-premium video content (emphasis added)” Arguments pp. 7-8.
Therefore, the argued limitations were written broad such that they read upon the cited references or are shown explicitly by the references. As a result, the claims stand as follows.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 11,636,682. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are virtually the same except the U.S. Patent discloses the additional features of “wherein said contextual information comprises text, sound and/or video frames that provides context to said image or said.”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 8-9, and 15-16 is/are rejected under 35 U.S.C. 102(a)(1) & (a)(2) as being anticipated by Iyer et al. (US 2020/0228880 A1 – hereinafter “Iyer”)
Claim 1:
Iyer discloses a computer-implemented (¶61) method for embedding contextual information in an image or video frames, the method comprising:
training a generative adversarial network to provide contextual information to be embedded in said image or said video frames (¶24 discloses “generative neural networks, such as
variational autoencoders (VAEs) and generative adversarial networks (GANs)”; ¶36 disclose “a machine learning model is trained based on certain video content and/or video genres, and based on that training, new content can then be dynamically generated”; ¶57; ¶84 discloses “the video advertisement content may be generated using a generative neural
network (GNN) model, such as a generative adversarial network (GAN) model”);
extracting references (¶56 discloses “obtains premium video content from a DRM content server 420”) containing contextual information associated with one or more images and/or one or more video frames (¶37 discloses “watched content … watched context (e.g., similar theme, environment, objects, people, and/or characters)”; theme i.e. topic and event, environment i.e. site, objects i.e. people; ¶65 discloses “contextual characteristics recognized in a scene, such as a theme, context, flow, actions, background vs. foreground, type or style of visual content ( e.g., movie, cartoon, lighting, ambiance), physical environment (e.g., indoor, outdoor), location, time of day, and so forth.”; actions i.e. event; ¶¶272-273, 275, 285)) identified in a database using said generative adversarial network (¶35 discloses AI techniques “are leveraged to personalize advertisements for users 110 based on the content they are currently watching … along with other information … such as user contexts”; ¶36 discloses “AI techniques, such as generative adversarial networks (GANs)”; ¶26 discloses “content distributors then leverage user profiles (e.g., user characteristics, preferences, viewing history) to selectively display the advertisements to targeted users who are currently viewing other media content.”; where, the user profiles are presumably stored in a database; ¶37 discloses “the advertising content may be generated as a separate or standalone advertisement with a similar context as the watched context (e.g., similar theme, environment, objects, people, and/or characters). (emphasis added)”; ¶59; ¶68 discloses “video analytics cache … video analytics datasets for a collection of videos”); and
augmenting said image or said video frames with said extracted references (¶56 discloses
“obtains premium video content from a DRM content server 420 … modifies/adapts the content … and then delivers the content to the corresponding user devices”).
Claims 2, 9, and 16:
Iyer discloses the method as recited in claim 1, wherein said training of said generative adversarial network comprises: receiving a stream of video frames (Iyer ¶66 discloses frames of the video content); and extracting features from said stream of video frames, wherein said extracted features comprise topics, events as well as sites and objects (Iyer ¶37 discloses “watched content … watched context (e.g., similar theme, environment, objects, people, and/or characters)”; theme i.e. topic and event, environment i.e. site, objects i.e. people; ¶65 discloses “contextual characteristics recognized in a scene, such as a theme, context, flow, actions, background vs. foreground, type or style of visual content ( e.g., movie, cartoon, lighting, ambiance), physical environment (e.g., indoor, outdoor), location, time of day, and so forth.”; actions i.e. event; ¶¶272-273, 275, 285).
Claim 8:
Iyer discloses the a computer program product for embedding contextual information in an image or video frames (¶56 discloses “modifies/adapts the content”), the computer program product comprising one or more computer readable storage mediums (¶¶ 244, 248) having program code embodied therewith, the program code (¶244) comprising programming instructions for …
Iyer discloses the remaining elements recited in claim 8 for at least the reasons discussed in claim 1 above.
Claim 15:
Iyer discloses a system, comprising: a memory (¶248) for storing a computer program (¶244) for embedding contextual information in an image or video frames (¶56 discloses “modifies/adapts the content”); and a processor (¶243) connected to said memory, wherein said processor is configured to execute program instructions of the computer program comprising …
Iyer discloses the remaining elements recited in claim 15 for at least the reasons discussed in claim 1 above.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 3-4, 7, 10-11, 14, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer in view of Mohandoss et al. (US 202/10158570 A1 – hereinafter “Mohandoss”).
Claims 3, 10, and 17:
The combination of Iyer and Mohandoss discloses the method as recited in claim 2, wherein said training of said generative adversarial network further comprises: (Iyer ¶84 discloses “(GNN) models may be trained to generate different types or genres of video content”).
Iyer discloses all of the subject matter as described above except for specifically
teaching “separating different events.” However, Mohandoss in the same field of endeavor
teaches “separating different events” (¶20 discloses “A pair of image frames is removed from the formed pairs within a cluster when the pair of image frames satisfies the content similarity threshold (e.g., the content of the two image frames are similar). Each remaining image pair in the training data set (e.g., each image pair with similar histograms, but different content)”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Iyer
and Mohandoss before the effective filing date of the claimed invention. The motivation for
this combination of references would have been to train a GAN using pairs of image frames to
generate context-sensitive information (Mohandoss ¶1). This motivation for the combination
of Iyer and Mohandoss is supported by KSR exemplary rationale (G) Some teaching, suggestion,
or motivation in the prior art that would have led one of ordinary skill to modify the prior art
reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP
2141 (III).
Claims 4, 11, and 18:
The combination of Iyer and Mohandoss discloses the method as recited in claim 3, wherein said training of said generative adversarial network further comprises: clustering images of said stream of video frames along with images of video frames (Mohandoss ¶19 discloses “the clustering may be based on a color similarity threshold”) stored in said database that are each associated with events within a threshold degree of similarity (Mohandoss ¶5 discloses “image frames that satisfy a content similarity threshold”; ¶¶19, 46, 56-57).
Claims 7 and 14:
Iyer discloses the method as recited in claim 1 further comprising: extracting features from said image or said video frames (¶37 discloses “watched content … watched context (e.g., similar theme, environment, objects, people, and/or characters)”; theme i.e. topic and event, environment i.e. site, objects i.e. people; ¶65 discloses “contextual characteristics recognized in a scene, such as a theme, context, flow, actions, background vs. foreground, type or style of visual content ( e.g., movie, cartoon, lighting, ambiance), physical environment (e.g., indoor, outdoor), location, time of day, and so forth.”; actions i.e. event; ¶¶272-273, 275, 285); and identifying said one or more images and/or said one or more video frames in said database using said generative adversarial network associated with features with a similarity to said extracted features from said image or said video frames that (¶35 discloses AI techniques “are leveraged to personalize advertisements for users 110 based on the content they are currently watching … along with other information … such as user contexts”; ¶36 discloses “AI techniques, such as generative adversarial networks (GANs)”; ¶26 discloses “content distributors then leverage user profiles (e.g., user characteristics, preferences, viewing history) to selectively display the advertisements to targeted users who are currently viewing other media content.”; where, the user profiles are presumably stored in a database; ¶37 discloses “the advertising content may be generated as a separate or standalone advertisement with a similar context as the watched context (e.g., similar theme, environment, objects, people, and/or characters). (emphasis added)”; ¶59; ¶68 discloses “video analytics cache … video analytics datasets for a collection of videos”)).
Iyer discloses all of the subject matter as described above except for specifically
teaching “exceeds a threshold value.” However, Mohandoss in the same field of endeavor teaches “exceeds a threshold value” (¶5 discloses “image frames that satisfy a content similarity threshold”; ¶¶19, 46, 56-57).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Iyer
and Mohandoss before the effective filing date of the claimed invention. The motivation for this combination of references would have been to train a GAN using pairs of image frames to generate context-sensitive information (Mohandoss ¶1). This motivation for the combination of Iyer and Mohandoss is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III). (III).
Allowable Subject Matter
Claims 5-6, 12-13, and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
This is a continuation of applicant's earlier Application No. 18/120187. All claims are drawn to the same invention claimed in the earlier application and could have been finally rejected on the grounds and art of record in the next Office action if they had been entered in the earlier application. Accordingly, THIS ACTION IS MADE FINAL even though it is a first action in this case. See MPEP § 706.07(b). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no, however, event will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached on M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2666
1 Arguments p. 5. Advisory Action p. 2.
2 Final p. 5.
3 Advisory Action p. 2. Final pp. 5-6.
Prosecution Insights