DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 11/11/2025 have been fully considered but they are not persuasive.
Applicant argues that Mickelsen does not teach “receiving, based on output of first content via a first device, user input indicating a command to output second content that modifies the output of the first content, the second content comprising the output of the first content modified to include an action occurring in the first content”. Applicant notes that the user input is unrelated to any “output of first content”. The Examiner respectfully disagrees and notes the claims do not recite that the user input corresponds to output of the first content, the claim broadly recites that the user input “indicating a command to output second content that modifies the output of the first content”. As taught by Mickelsen in Paragraph 0046, a user provides input that identifies the devices that will be used to output second content while the viewer is watching video content. This second content includes various sensory experience content outlined in Paragraphs 0041-0042. Therefore, to say the user input is unrelated to output of the first content is incorrect because the second content is output while the user is watching the first content and therefore represents a modification of the first content using the second content, wherein the user input identifies the devices that outputs the second content that modifies the first content.
In regards to the Heinz prior art reference, Applicant argues that because Heinz uses a second screen, Heinz fails to teach output of the first content. The Examiner respectfully disagrees and notes that Heinz discloses that objects in first content (pre-recorded video) can be modified. Although, Paragraph 0093 teaches a second screen, Paragraph 0093 clearly teaches that objects in the first content (pre-recorded video) can be modified and displayed to the user on the second screen to help explore the video content. Figure 4 Paragraphs 0089-0092 provide a more detailed explanation of this process. For example, Paragraph 0089 states manipulating/modifying an object in a scene in response to user input and playing second content which is the modified first content with the manipulated object. Therefore, Heinz teaches that the second content comprising the output of the first content is modified to include an action occurring in the first content, wherein the object, which in Paragraph 0088 is an automobile, computing device or entertainment system is assigned an action such as the purchase price of the object and the ability to order or purchase the object. Therefore, the rejection stands.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7, 34-46, 50-53 and 55 are rejected under 35 U.S.C. 103 as being unpatentable over Mickelsen et al. (U.S. Patent Application Publication 2014/0267906) in view of Heinz, II et al. (U.S. Patent Application Publication 2015/0264441).
Referring to claim 1, Mickelsen discloses receiving, based on output of first content via a first device, user input indicating a command to output second content that modifies the output of the first content (see Paragraph 0046 for the user inputting which devices available to provide additional content, which include personal viewer devices).
Mickelsen also discloses determining, based on the received user input (the output devices selected), an event indicating that output of the second content is associated with a point in time in the first content (see Paragraph 0007, 0011, 0015 and 0039).
Mickelsen also discloses sending instructions to output the second content via a second device in a user environment (see Paragraphs 0044 and 0047-0048).
Mickelsen fails to teach that the second content comprising the output of the first content modified to include an action occurring in the first content.
Heinz discloses that the second content comprising the output of the first content modified to include an action occurring in the first content (see Paragraph 0028 and 0093).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen, using the content modification functionality, as taught by Heinz, for the purpose of leveraging 2D or 3D graphics data to enable interactive exploration of 2D or 3D modeled worlds from scenes in pre-rendered, pre-recorded video (see Paragraph 0028 of Heinz).
Referring to claims 37 and 42, see the rejection of claim 1.
Referring to claim 55, Mickelsen discloses that determining, based on a correspondence between objects in the user environment and objects in an environment in the first content, that the action occurring in the first content is to be reproduced in the user environment using a third device in the user environment (see devices 16 and 18 and Paragraphs 0038-0044 for sending commands to these devices to enhance the viewing experience of the viewer by controlling objects in the user environment to correspond to the objects the viewer is viewing in the first content).
Claims 2-7, 34-36, 38-41, 43-46 and 50-53 are rejected under 35 U.S.C. 103 as being unpatentable over Mickelsen et al. (U.S. Patent Application Publication 2014/0267906) Heinz, II et al. (U.S. Patent Application Publication 2015/0264441) in further view of Carney et al. (U.S. Patent Application Publication 2013/0198642).
Referring to claim 2, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 2.
Carney discloses receiving information associated with the first content, wherein the event is associated with a set of facts in the information, wherein the at least a portion of metadata includes a set of rules associated with the first content (see Paragraphs 0072-0073 and Paragraph 0075 for an item database 601 which stores a set of facts associated with a user of the first and second device (e.g. keywords and platforms) and the metadata includes the time of the event in the program which is used in the comparison discussed in the rejection of claim 1).
Carney also discloses sending the instructions to initiate reproduction of the action occurring in the first content via the second device based at least in part on determining that the set of facts corresponds to a first rule of a set of rules associated with the first content (see Paragraph 0071 and 0075 for using the item database 601 and query engine 602 to use item data in the item database 601 by the query engine 602 to determine which items to send to the second device (further note Paragraphs 0086-0087)).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the rules functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 3, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 3.
Carney discloses that causing output of modified first content via the first device during the output of the interactive effects via the second device and causing the output of the modified first content comprises causing output of the first content via the first device to be delayed, altered or repeated (see Paragraph 0093 for delaying the primary content so the supplemental content can be digested and further adjusting the playback of the primary content using advance, fast-forward, rewind, or pause).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 4, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 4.
Carney discloses receiving the information associated with the first content (see the rejection of claim 1) and activating a rules engine based at least in part on information indicative of availability of the second device for output of secondary content (see Paragraphs 0125-0126 for the second device registering which then activates a rules engine that dictates which screen to provide to the second screen based on the user preferences entered during the registration process (see Paragraph 0135)).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 5, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 5.
Carney discloses receiving information indicative of availability of the second device for output of interactive effects (see Paragraphs 0125-0126 for the second device registering which then activates a rules engine that dictates which screen to provide to the second screen).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 6, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 6.
Carney discloses receiving information indicative of user-initiated event and adjusting a state of a rule engine based on the user-initiated event (see Paragraphs 0136-0137 for requesting for messages related to a particular piece of content, which returns messages related to “The Voice” program the user is viewing, therefore the state of the second screen manager is changed to provide messages using the conversation option 806b).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 7, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 7.
Carney discloses storing information indicative of a state of a rules engine, wherein the information stored is accessible based on information identifying a user of the first device and information identifying the first content (see Table 1 on Page 10 for storing item information that dictates the title of the programs, channel, keywords and platform used by the second screen manage to determine the type of information to send to the second screen (see Paragraphs 0071-0072)).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 34, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 34.
Carney further discloses wherein the sending the instructions causes output of a request for the second to receive one or more user inputs that one or more user inputs comprise controlling the second device (see Figure 4F for the running app having selectable “here” options as well as Figure 8 and Paragraphs 0089-0100 for the watch with me app data on the second screen device having multiple input options for the user to choose from).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 35, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 35.
Carney discloses that the first device is a television (see Paragraph 0003).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen, using the rules functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 36, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 36.
Carney discloses that the second device is a computing device or smartphone (see Paragraph 0003).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claims 37-41, see the rejection of claims 1, 34-35 and 3-4, respectively.
Referring to claims 43-46, see the rejection of claims 34-35 and 3-4, respectively.
Referring to claim 50, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 50.
Carney discloses that the interactive effects are an enhancement to the output of the first content and is associated with one or more user interactions with the second device, wherein the enhancement comprises an additional effect or additional audiovisual content to the first content, and where the sending the instructions causes output of a request for the second device to receive one or more inputs (see Paragraphs 0057-0058 and Figure 4H for activating the running app on a second screen device).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 51, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 51.
Carney discloses that the enhancement further comprises an additional interactive component (see Figure 4H for the running app having three selectable additional interactive components).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 52, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 52.
Carney discloses that the interactive effects are output via the second device based on a correspondence between one or more facts associated with the user environment and one or more rules of a set of rules associated with the first content (see the rejection of claim 2), the one more facts indicating that one or more devices of the plurality of devices are available in the user environment to reproduce the action occurring in the first content using the interactive effects via the one or more devices (see Paragraphs 0057-0058 and Figure 4H for activating the running app on a second screen device).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the rules functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Referring to claim 53, Mickelsen and Heinz discloses all of the limitations of claim 1, but fails to teach the limitations of claim 53.
Carney discloses that at least a portion of metadata comprises one or more rules that describe what events are to be triggered at certain points of the first content (see the bottom of Paragraph 0037 for the metadata described in the rejection of claim 1 indicating a rule in the form of a time within the program), wherein the information the information comprises one or more facts indicating one or more points in time in the output of the first content (see the bottom of Paragraph 0037 for the information containing a list of triggers, wherein the triggers indicate what supplemental content to display and the time to show the supplemental content (further note Paragraph 0058)), further comprising comparing the one or more facts (time in the program) to the one or more rules to detect the point in time in the first content that is associated with the event (triggers that indicate the supplemental content to display and the time to display the supplemental content).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the output of content to a second device, as taught by Mickelsen and Heinz, using the additional functionality, as taught by Carney, for the purpose of providing new and interesting ways to keep the viewer engaged (see the bottom of Paragraph 0002 of Carney).
Allowable Subject Matter
Claim 54 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON P SALCE whose telephone number is (571)272-7301. The examiner can normally be reached 5:30am-10:00pm M-F (Flex Schedule).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached at 571-272-1915. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jason Salce/Senior Examiner, Art Unit 2421
Jason P Salce
Senior Examiner
Art Unit 2421
February 16, 2026