Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06 April 2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12 December 2025 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Claims 2, 11, 20, 29, and 53.
Claims 54-55 are canceled.
Claims 2-9, 11-26, 28-37, 40-49, 51, and 53 are presently pending.
Applicant’s arguments with respect to claims 2, 11, 20, and 29 have been considered but are moot in view of the new ground(s) of rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-9, 11-26, 28-37, 40-46, 48-49, 51, and 53 are rejected under 35 U.S.C. 103(a) as being unpatentable over Srinivasan et al. (US 2001/0023436 A1) (of record, hereinafter Srinivasan), in view Ackley et al. (US 2005/0022226 A1) (of record, hereinafter Ackley), and further in view of Candelore (US 2009/0041418 A1) (of record, hereinafter Candelore)
Regarding Claim 2, Srinivasan discloses a method [Figs. 7-8] comprising:
detecting a plurality of events in media content; [Figs. 7-8; 0047: where authoring stations may process any live or recorded content; 0051-52, 0089, 0094: video stream is received by one or more authoring stations 61 which may each track certain selected entities in the video to synchronize with associated annotations] and
sending, by a computing device and to an output device, a plurality of pieces of time-based metadata, [Figs. 7-8; 0091-92, 0094-97: authoring server 63 (computing device) receives annotation information from authoring stations 62, and outputs annotation stream 55 (plurality pieces of time-based metadata) to end users 48 (output device)]
wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, comprises a timestamp corresponding to an event, of the plurality of events, [0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data]
wherein each piece of time-based metadata is configured to facilitate output of interactive supplemental content by the output device during output of the event in the media content. [0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects]
Srinivasan fails to explicitly disclose sending, by a computing device and to an output device, in an order based on synchronization information received by the computing device, a plurality of pieces of time-based metadata. (Emphasis added to the particular elements of the limitations not explicitly disclosed by Srinivasan).
Ackley, in analogous art, teaches sending, by a computing device and to an output device, in an order based on synchronization information received by the computing device from the output device, a plurality of pieces of time-based metadata. [Figs. 1-4; 0014-15, 0028-29, 0034-36: wherein trigger data (such as the annotation information of Srinivasan above) is determined for locations/times during the video when or where a product is detected, and where such trigger data may be provided as some data/file that may be compiled as a list or table of triggers that exist for the associated video (where it would be implicitly understood that a compiled list of triggers based on timing information would necessarily comprise an order in which a plurality of events over the course of a video would occur); 0035-36, 0047: trigger data may point to additional data (i.e., time-based metadata) to play simultaneously with video which may be retrieved from some non-volatile storage or from the Internet; 0033, 0040: if trigger data is not present, video player may send a request to the remote location for trigger data associated with the movie on the disc (where some identifier for the movie – i.e., synchronization information – must necessarily be sent to correctly and appropriately identify the movie and associated trigger data to provide to the client device)]
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the method of Srinivasan with the teachings of Ackley to specify providing time-based metadata in an order based on received synchronization information as it is understood that trigger information for providing additional interactive content associated with underlying video content may be associated with specific times and locations, and where such information may be accessed from the Internet. [Ackley – ABST; 0033, 0040]
Srinivasan and Ackley fail to explicitly specify sending, by a computing device and to an output device, in an order based on audio information received by the computing device from the output device, a plurality of pieces of time-based metadata. (Emphasis added to the particular elements of the limitations not explicitly disclosed by Srinivasan and Ackley – namely that the transmission of the metadata is triggered based on audio information received from the output device).
Candelore, in analogous art, teaches sending, by a computing device and to an output device, in an order based on audio information received by the computing device from the output device, a plurality of pieces of time-based metadata. [Figs. 1-2, 6; 0015-16, 0020-26: display device 120 (e.g., a television) receives and renders media content retrieved by a networking device 110 from some network 105 (e.g., content provider), wherein the system generates one or more audio thumbnails/fingerprints (i.e., indication of playback of content) and routed to a database to identify matching audio thumbnails to associate the thumbnail with a given A/V program in order to receive related metadata for display supplemental/enhance content at pertinent times (such as the time-based metadata of Srinivasan and Ackley above)]
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the method of Srinivasan and Ackley with the teachings of Candelore to specify providing time-based metadata based on audio information received from the output device as it is understood that digital content may be identified by audio thumbnails of the output data so as to effectively receive interactive television metadata. [Candelore – ABST; 0006, 0015]
Regarding Claim 3, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, facilitates access to the interactive supplemental content associated with the event in the media content at a point in time indicated by the timestamp. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects; Ackley – Figs. 3-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet;]
Regarding Claim 4, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, causes the output device to output the interactive supplemental content associated with the event in the media content during output of the event in the media content. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects; Ackley – Figs. 3-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 5, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, causes the output device to retrieve the interactive supplemental content associated with the event in the media content at a point in time indicated by the timestamp from the computing device. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects; 0022, 0138: metadata may be downloaded as needed by user based on user interacting with entity in video stream; Ackley – Figs. 1-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet (i.e., a first computing device)]
Regarding Claim 6, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan discloses wherein the interactive supplemental content for each piece of time-based metadata, of the plurality of pieces of time-based metadata, is output by the output device based on a detection by the output device of the timestamp. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects;]
Regarding Claim 7, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the interactive supplemental content for each piece of time-based metadata, of the plurality of pieces of time-based metadata, comprises one or more of location information, audio information, or textual information associated with the event in the media content. [Srinivasan – 0044: wherein annotations may be icons, animated graphics, text, audio, etc.; Ackley – Figs. 3-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet;]
Regarding Claim 8, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the event in the media content for each piece of time-based metadata, of the plurality of pieces of time-based metadata, comprises one or more of an appearance of a character, an appearance of a product, or an indication of a particular location. [Srinivasan – 0042: entity may be any person or thing depicted in the video; Ackley – Figs. 3-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected product, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 9, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the interactive supplemental content for each piece of time-based metadata, of the plurality of pieces of time-based metadata, is output as an overlay during the output of the media content. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects; Ackley – Figs. 3-4; 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator (overlay) to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 11, Srinivasan discloses a method:
receiving, by an output device from by a computing device at least one piece of time-based metadata, [Figs. 7-8; 0091-92, 0094-97: authoring server 63 (computing device) receives annotation information from authoring stations 62, and outputs annotation stream 55 (plurality pieces of time-based metadata) to end users 48 (output device)]
wherein the at least one piece of time-based metadata comprises a timestamp corresponding to an event in the media content, [0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data] wherein the at least one piece of time-based metadata is configured to facilitate output of interactive supplemental content by the output device during output of the media content; [0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects] and
causing, after receiving the at least one piece of time-based metadata, output of the media content and the interactive supplemental content associated with the event in the media content during output of the media content. [0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects]
Srinivasan fails to explicitly disclose receiving, by an output device from by a computing device in an order based on synchronization information received by the computing device, at least one piece of time-based metadata. (Emphasis added to the particular elements of the limitations not explicitly disclosed by Srinivasan).
Ackley, in analogous art, teaches receiving, by an output device from by a computing device in an order based on synchronization information received by the computing device, at least one piece of time-based metadata. [Figs. 1-4; 0014-15, 0028-29, 0034-36: wherein trigger data (such as the annotation information of Srinivasan above) is determined for locations/times during the video when or where a product is detected, and where such trigger data may be provided as some data/file that may be compiled as a list or table of triggers that exist for the associated video (where it would be implicitly understood that a compiled list of triggers based on timing information would necessarily comprise an order in which a plurality of events over the course of a video would occur); 0035-36, 0047: trigger data may point to additional data (i.e., time-based metadata) to play simultaneously with video which may be retrieved from some non-volatile storage or from the Internet; 0033, 0040: if trigger data is not present, video player may send a request to the remote location for trigger data associated with the movie on the disc (where some identifier for the movie – i.e., synchronization information – must necessarily be sent to correctly and appropriately identify the movie and associated trigger data to provide to the client device)]
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the method of Srinivasan with the teachings of Ackley to specify providing time-based metadata in an order based on received synchronization information as it is understood that trigger information for providing additional interactive content associated with underlying video content may be associated with specific times and locations, and where such information may be accessed from the Internet. [Ackley – ABST; 0033, 0040]
Srinivasan and Ackley fail to explicitly specify receiving, by an output device from by a computing device in an order based on audio information received by the computing device from the output device, at least one piece of time-based metadata. (Emphasis added to the particular elements of the limitations not explicitly disclosed by Srinivasan and Ackley – namely that the transmission of the metadata is triggered based on audio information received from the output device).
Candelore, in analogous art, teaches sending, by a computing device and to an output device, in an order based on audio information received by the computing device from the output device, a plurality of pieces of time-based metadata. [Figs. 1-2, 6; 0015-16, 0020-26: display device 120 (e.g., a television) receives and renders media content retrieved by a networking device 110 from some network 105 (e.g., content provider), wherein the system generates one or more audio thumbnails/fingerprints (i.e., indication of playback of content) and routed to a database to identify matching audio thumbnails to associate the thumbnail with a given A/V program in order to receive related metadata for display supplemental/enhance content at pertinent times (such as the time-based metadata of Srinivasan and Ackley above)]
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the method of Srinivasan and Ackley with the teachings of Candelore to specify providing time-based metadata based on audio information received from the output device as it is understood that digital content may be identified by audio thumbnails of the output data so as to effectively receive interactive television metadata. [Candelore – ABST; 0006, 0015]
Regarding Claim 12, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 12 recites nearly identical limitations as Claim 3 and is rejected similarly as that claim.
Regarding Claim 13, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 13 recites nearly identical limitations as Claim 4 and is rejected similarly as that claim.
Regarding Claim 14, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 14 recites nearly identical limitations as Claim 5 and is rejected similarly as that claim.
Regarding Claim 15, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 15 recites nearly identical limitations as Claim 6 and is rejected similarly as that claim.
Regarding Claim 16, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 16 recites nearly identical limitations as Claim 7 and is rejected similarly as that claim.
Regarding Claim 17, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 17 recites nearly identical limitations as Claim 8 and is rejected similarly as that claim.
Regarding Claim 18, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein receiving the at least one piece of time-based metadata comprises receiving a package comprising the media content and the at least one piece of time-based metadata. [Srinivasan – Fig. 18; 0234-235: where metadata may be multiplexed and packaged with the video data, where the metadata may be inserted into the digital video; Ackley – 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 19, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 19 recites nearly identical limitations as Claim 9 and is rejected similarly as that claim.
Regarding Claim 20, the Claim recites an apparatus that performs the method of Claim 2. As such, Claim 20 is analyzed and rejected similarly as Claim 2, mutatis mutandis. (see also Srinivasan [Fig. 1] and [0048] and Ackley [Fig. 1])
Regarding Claim 21, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 21 recites nearly identical limitations as Claim 3 and is rejected similarly as that claim.
Regarding Claim 22, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 22 recites nearly identical limitations as Claim 4 and is rejected similarly as that claim.
Regarding Claim 23, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 23 recites nearly identical limitations as Claim 5 and is rejected similarly as that claim.
Regarding Claim 24, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 24 recites nearly identical limitations as Claim 6 and is rejected similarly as that claim.
Regarding Claim 25, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 25 recites nearly identical limitations as Claim 7 and is rejected similarly as that claim.
Regarding Claim 26, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 26 recites nearly identical limitations as Claim 8 and is rejected similarly as that claim.
Regarding Claim 28, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 28 recites nearly identical limitations as Claim 9 and is rejected similarly as that claim.
Regarding Claim 29, the Claim recites an apparatus that performs the method of Claim 11. As such, Claim 29 is rejected similarly as Claim 11, mutatis mutandis. (see also Srinivasan [Fig. 1] and [0048] and Ackley [Fig. 1])
Regarding Claim 30, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 30 recites nearly identical limitations as Claim 3 and is rejected similarly as that claim.
Regarding Claim 31, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 31 recites nearly identical limitations as Claim 4 and is rejected similarly as that claim.
Regarding Claim 32, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 32 recites nearly identical limitations as Claim 5 and is rejected similarly as that claim.
Regarding Claim 33, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 33 recites nearly identical limitations as Claim 6 and is rejected similarly as that claim.
Regarding Claim 34, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 34 recites nearly identical limitations as Claim 7 and is rejected similarly as that claim.
Regarding Claim 35, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 35 recites nearly identical limitations as Claim 8 and is rejected similarly as that claim.
Regarding Claim 36, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 36 recites nearly identical limitations as Claim 18 and is rejected similarly as that claim.
Regarding Claim 37, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 29, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 37 recites nearly identical limitations as Claim 9 and is rejected similarly as that claim.
Regarding Claim 40, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Ackley discloses receiving an indication of output of the media content, wherein sending the plurality of pieces of time-based metadata comprises sending the plurality of pieces of time-based metadata based on the indication of output of the media content. [Ackley – Figs. 2-3]
Regarding Claim 41, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 41 recites nearly identical limitations as Claim 40 and is rejected similarly as that claim.
Regarding Claim 42, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the interactive supplemental content comprises an advertisement. [Srinivasan – 0044: advertisements may be associated with tracked object; Ackley – 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 43, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, comprises the interactive supplemental content associated with the event in the media content. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes for display of the metadata along with the underlying video data, where additional information may be overlaid and interactable with the tracked objects; Ackley – 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet]
Regarding Claim 44, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, comprises a link to additional information associated with the interactive supplemental content associated with the event in the media content. [Srinivasan – 0081: interactive supplemental content may invoke a link to obtain additional information of the displayed entity Ackley – 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet, and may comprise a link to the additional data]
Regarding Claim 45, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan discloses wherein the timestamp corresponding to the event in the media content comprises a start time of an occurrence of the event in the media content and an end time of the occurrence of the event in the media content. [Srinivasan – 0017-0023, 0052: metadata is associated with tracked objects using PTSes of when the tracked object appears for display of the metadata along with the underlying video data]
Regarding Claim 46, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the interactive supplemental content output by the output device changes during output of the media content. [Srinivasan – col0017-0023, 0052: metadata is associated with tracked objects using PTSes of when the tracked object appears for display of the metadata along with the underlying video data; 0054-55, 0107: tracked objects may have various interactable shapes/images that follow the image of the object to allow users to acquire further information; (where it would also be implicitly understood that multiple objects appearing/leaving the image would cause supplemental content to change during output of the media, see MPEP 2144.01); Ackley – 0016-19, 0036, 0042-47: wherein trigger data may correspond to some interactive indicator to allow user to select/interact with a given object on display in the screen to allow the user to purchase or obtain more information on the detected object, and where the information may be retrieved from local memory, the video data, or from the Internet;]
Regarding Claim 48, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan discloses wherein each piece of time-based metadata, of the plurality of pieces of time-based metadata, is configured to facilitate output of the interactive supplemental content by the output device for a duration of the output of the event in the media content. [Srinivasan – col0017-0023, 0052: metadata is associated with tracked objects using PTSes of when the tracked object appears for display of the metadata along with the underlying video data (where it would be implicitly understood that tracking display of an image with timestamps would reasonably comprise the duration when a detected object/entity is present – see MPEP 2144.01)]
Regarding Claim 49, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Srinivasan and Ackley disclose wherein the detecting comprises detecting in the media content, at least one of a: an appearance of a character, an appearance of a product, an appearance of an object associated with the product, an appearance of audio content, an appearance of background supplemental content, or an indication of a particular location. [Srinivasan – 0042: entity may be any person or thing depicted in the video; Ackley – Figs. 1-4; 0014-15, 0028-29, 0034-36: wherein trigger data is determined for locations/times during the video when or where a product is detected, and where such trigger data may be provided as some data/file that may be compiled as a list or table of triggers that exist for the associated video (i.e., timeline data)]
Regarding Claim 51, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 20, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 51 recites nearly identical limitations as Claim 49 and is rejected similarly as that claim.
Regarding Claim 53, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Candelore discloses disclose wherein the audio information indicates a portion of the media content item that is being output. [Figs. 1-2, 6; 0015-16, 0020-26: display device 120 (e.g., a television) receives and renders media content retrieved by a networking device 110 from some network 105 (e.g., content provider), wherein the system generates one or more audio thumbnails/fingerprints (i.e., indication of playback of content) and routed to a database to identify matching audio thumbnails to associate the thumbnail with a given A/V program in order to receive related metadata for display supplemental/enhance content at pertinent times]
Claim 47 is rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Srinivasan, Ackley, and Candelore as applied to claim 2 above, and further in view of Heinley et al. (US 2008/0294663 A1) (of record, hereinafter Heinley).
Regarding Claim 47, Srinivasan, Ackley, and Candelore disclose all the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Srinivasan, Ackley, and Candelore fail to explicitly disclose wherein the plurality of pieces of time-based metadata are configured to facilitate output of the timeline of the interactive supplemental content.
Heinley, in analogous art, teaches wherein the plurality of pieces of time-based metadata are configured to facilitate output of the timeline of the interactive supplemental content. [0006-7, 0015, 0036, 0063-64, 0075, 0087: where visual systems may provide interactive timelines that provide a visual chronological view of time-based events on a timeline (such as the metadata of Srinivasan and Ackley), where such timelines may be based on media content such as movies or novels, etc., where users may utilize the timelines to obtain additional information associated with noted events on the timeline]
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the method of Srinivasan, Ackley, and Candelore with the teachings of Heinley to facilitate output of the timeline in order to provide users with a graphical view that details the chronological timeline of events that may occur in some content. [Heinley – 0006-7, 0036]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM J KIM whose telephone number is (571)272-2767. The examiner can normally be reached 9:30am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hadi Armouche can be reached at (571) 270-3618. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM J KIM/Primary Examiner, Art Unit 2409
Prosecution Insights