DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/09/2026 has been entered.
Response to Arguments
Applicant’s arguments filed on 03/09/2026 with respect to claims 21-40 have been fully considered but they are not persuasive.
In re pages 7-15, Applicant states that “The Applicant respectfully disagrees with the Examiner's rejections. The Examiner Has the Burden to Show All Elements and an Impetus to Combine Them. To make a prima facie case under 35 U.S.C. § 103, the Examiner's burden includes (but is not limited to) citing references that teach or suggest all of the features of a claimed invention (In re Ochiai, 71 F.3d 1565, 1572 (Fed. Cir. 1995); In re Wada and Murphy, Appeal No. 2007-3733, slip op. at 7 (BPAI 2008)). The Applicant submits that the cited references, whether considered alone or in combination do not disclose at least the following elements of amended claim 21: retrieving the annotated timeline from the storage server, the annotated timeline comprising: a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references; at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams; [and...] modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline; The present application explains that during a live event, an operator may use a control station to receive content streams from various content sources (see e.g., para [0071] of the application as published). The operator can select different content streams and select how the various streams are combined. The control station then generates a derived media clip corresponding to the selection. The control station also generates an annotated timeline of the derived media clip (see e.g., paras [0083]-[0084] of the application as published). The annotated timeline includes a timebase reference annotated with various content identifiers associated with content streams. The annotated timeline provides a correlation between a time reference and content to be displayed at those times (see e.g., para [0035] of the application as published). FIG. 4, reproduced below, shows user interface 400 displaying an example annotated timeline 405. As shown, annotated timeline 405 includes timing references 410 and various channels 415. Channel 415b, for example, illustrates that audio stream from audio source 402f is selected between a first time 420e and a second time 420f. The annotated timeline 415 also includes transitioning operations (e.g., switching in and out operations 420) and combination operations in relation to the time references. The transitioning and combination operations can be associated with one or more of the content identifiers of content streams. Transitioning operations can also be fading, cross-fading or any other effects to transition from one source of content stream or object within a channel to another source of content stream or object (see e.g., para [0144] of the application as published). Combination operations can be muting, fading, video-in-video, image-in-video, etc. operations to combine various streams (see e.g., paras [0083] and [0143] of the application as published). The placement of the selected content streams, transitioning operations and combination operations on the timeline indicates the time and sequence (or order) of their occurrence in the derived media stream (see e.g., para [0084] of the application as published). As described in the Applicant's disclosure, the annotated timeline allows for instant editing of the derived media clip, for example, to correct any errors and/or improve the quality of the derived media clip (see e.g., para [0086] of the application as published). To edit the derived media clip using the annotated timeline, the annotated timeline is retrieved from the storage server. A first content identifier indicating the content to be swapped and a second content identifier indicating the new content (the second content stream) are then received. The new content may be available from content sources, from the storage server or from an external source (see e.g., para [0087] of the application as published). The annotated timeline may be edited to change or modify the transitioning operations and/or combination operations by replacing the first content identifier with the second content identifier (see e.g., para [0145] of the application as published) to obtain an annotated timeline. To derive the edited media clip, the new content stream corresponding to the content identifier is retrieved and the edited media clip corresponding to the annotated timeline, which includes the new content identifier and the modified transitioning and/or combination operation, is generated (see e.g., paras [0076]-[0077], [0145] of the application as published). The Applicant submits that none of Gupta, Ranjeet, Larsen nor any other cited prior art reference disclose the claimed features in amended claim 21. Gupta describes a system that allows annotations relating to one version of a multimedia presentation to be displayed on another version of the multimedia presentation. As explained in Gupta, educational presentations can be provided to a computer user as multimedia. To replace in-person interaction and classroom discussion, some multimedia presentations may allow students to comment on a particular point and another student, or a lecturer or assistant to respond to that student's comment. Gupta defines these comments as annotations (see e.g., para [0006] of Gupta). The Applicant submits that Gupta only retrieves individual annotation entries, not a timeline composed of content identifiers for content streams, and certainly not a timeline whose structure is governed by embedded timecode packets synchronized to a common timebase. In Gupta, the "timeline" is merely the playback time of the one selected version of the media stream to which the annotation corresponds. Gupta's "time range field 184" (See Fig. 5 of Gupta) is simply the temporal interval of a single annotation in a single stream. Gupta's system is built around normalizing annotation time ranges across different versions of the same content, not storing, editing, or manipulating multiple content streams in a unified annotated timeline. The Applicant agrees with the Examiner's statement at Page 11 of the Final Office Action that Gupta fails to teach at least one indication of a transitioning or combination operation in relation to the timecode references. Furthermore, Gupta does not disclose "replacing, in the annotated timeline, the first content identifier with the second content identifier" or "modifying... the transitioning or combination operation." Gupta also does not generate an "edited derived media clip" corresponding to an edited annotated timeline. Gupta never edits the media itself. It only streams existing media, overlays or presents annotations separately, and converts time references for annotations. The only media sent to the client is the original stream associated with the annotation, not a new derived clip. Ranjeet does not remedy the deficiencies of Gupta. Ranjeet describes techniques for selecting a view of a multi-view or 360-degree video based on viewport context, object-tracking, and scrub-bar interactions (see e.g., paras [0008] - [0009] and [0067] - [0074] of Ranjeet). As explained in Ranjeet, the disclosed system merely accesses a single multi-view video, initiates playback of that video in a viewport, and switches between camera views of that same video to keep an identified object in view (see e.g., paras [0053] - [0054] of Ranjeet). The scrub bar of Ranjeet provides only a visual representation of playback progression for that video, and user interaction with the scrub bar is used to generate thumbnail previews of frames of the same multi-view video (see e.g., para [0067] of Ranjeet). The Applicant submits that Ranjeet, like Gupta, does not disclose retrieving an annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams. Ranjeet does not disclose an annotated timeline at all. Instead, the "timeline" in Ranjeet corresponds only to the ordinary playback timeline of a single video file. The scrub bar of Ranjeet does not represent multiple content streams, does not contain start and end timecodes corresponding to different streams, and does not include any representation of embedded timecode packets synchronized to a common timebase. Rather, the scrub bar simply reflects the current position in a single video and allows the user to navigate within the video. Nowhere does Ranjeet suggest storing, retrieving, or editing a multi-stream annotated timeline as claimed in amended claim 21. Furthermore, Ranjeet does not disclose any transitioning or combination operation associated with a content identifier of a content stream. The switching described in Ranjeet is limited to switching the displayed camera view within a single multi-view video, such as when the system automatically changes from view 408(a) to view 408(b) to track an identified object (see e.g., paras [0053] - [0054] of Ranjeet). Ranjeet also does not disclose replacing a first content identifier with a second content identifier in an annotated timeline, nor modifying a transitioning or combination operation in response to such replacement. There is no concept of content identifiers, content streams, or a multi-stream timeline in Ranjeet. All content comes from a single multi-view video and selecting or switching a camera angle of that video does not constitute replacing a content stream with another. Nothing in Ranjeet corresponds to the claimed ability to retrieve a second content stream, substitute it for another content stream on the timeline, or modify transitioning or combination operations. In addition, Ranjeet does not disclose generating an edited derived media clip corresponding to an edited annotated timeline. Ranjeet never edits or generates any media clip. It merely displays different camera views of the same video and generates thumbnail previews. The underlying video remains unchanged in all cases. The Applicant submits that Ranjeet therefore fails to disclose or suggest generating an edited derived media clip in any form. Larsen does not remedy the deficiencies of Gupta and Ranjeet. Larsen describes a multimedia synchronization and presentation system in which various media resources, such as video, scrolling text, slides, images, comments, links, games and other supplemental assets, may be displayed in coordination with a primary video or audio stream (see e.g., paras [0017]). The system disclosed in Larsen focuses on aligning these auxiliary resources with the playback of a single primary video using timecode mapping and metadata so that, for example, scrolling transcript text, comments, or slides appear at the correct point during playback (see e.g., paras [0166] - [0167] of Larsen). Larsen does not disclose transitioning or combination operations of the type claimed in amended claim 21. The transitions in Larsen relate only to the appearance or disappearance of supplemental content items at specific timecodes during the playback of a single narrative. The Applicant notes that the transitions disclosed in Larsen are limited to the optional ability to apply slide transitions (e.g., fade, dissolve, wipe) when adding slides to a narrative, as shown for example in the slide authoring interface (see e.g., paras [0390] and [0425] of Larsen). These transitions apply only to the way in which a slide graphic itself is rendered as a supplemental asset when it appears relative to the video. They do not operate between content streams, nor do they associate with any "content identifier" corresponding to a content stream. The Examiner cites paragraph [0341] of Larsen as allegedly teaching the claimed transitioning or combination operation. Paragraph [0341] of Larsen states that "tapping/clicking on any index feature(s) takes the user to the associated timecode, scene, or reference point within the Narrative, and syncs one or more time-based feature(s) to that associated timecode, scene, or reference point." However, the Applicant submits that this disclosure is fundamentally different from the transitioning and combination operations recited in amended claim 21. The action described in paragraph [0341] of Larsen simply causes the system to update the display of supplemental time-based features (such as transcript text, slides, or other metadata) so that they correspond to the newly selected timecode. In other words, tapping an index entry performs a navigation operation: it instructs the system to jump the playback position of the primary video and then refresh the synchronized secondary materials to match that playback position. This "syncing" is therefore a context alignment function, not a transition or combination operation between content streams on a timeline. Furthermore, Larsen does not disclose replacing a first content identifier with a second content identifier in an annotated timeline and modifying a transitioning or combination operation. Because Larsen does not disclose multiple content streams on an annotated timeline, does not disclose any content identifiers corresponding to such streams, and does not disclose replacing one such identifier with another, it also cannot disclose modifying a transitioning or combination operation. Larsen also does not generate an edited derived media clip corresponding to an edited annotated timeline. As with Gupta and Ranjeet, the underlying video in Larsen is never altered or re-rendered. The system simply presents the primary video while synchronizing supplemental content items. Supplemental materials may appear or disappear at certain timecodes, but the primary video is never edited, nor is any derived media clip generated. Nothing in Larsen suggests generating a new media clip that integrates timeline edits, content swaps or modified transitioning or combination operations as claimed in amended claim 21. Critically, none of Gupta, Ranjeet or Larsen are directed toward or disclose to any degree editing an annotated timeline by replacing content items or modifying transitioning or combination operations. For completeness, the Applicant submits that Silvestri and Pedlow do not remedy the deficiencies of Gupta, Ranjeet, and Larsen. Therefore, claim 21 is non-obvious, even in view of Gupta, Ranjeet, Larsen and the common general knowledge of a person of skill in the art. Similar remarks apply to amended claim 31.”
(1) In response, the Examiner respectfully disagrees. It should be noted that the reference of Larsen is no longer needed. As a result, the Applicant’s arguments with regard to Larsen are moot. Thus, the Examiner will address the Applicant’s arguments with regard to GUPTA and Ranjeet. For instance, GUPTA discloses the following: First, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61. Second, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86. Third, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87. Also, see paragraphs 31, 59, 88. From the above passages, GUPTA indeed discloses the following amended claimed limitations of independent claim 21 that recites “at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams; modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline”. See actual claim rejections further below. Therefore, the user performs at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams as described in the above figures and paragraphs. Also, the user modifies in the annotated timeline the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline as described in the above figures and paragraphs. As a result the Applicant’s statements are unsupported by GUPTA.
In fact, GUPTA discloses the following claimed limitations of independent claim 21 that recites “- retrieving the annotated timeline from the storage server, the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references” (see ¶ 29 for retrieving the annotated timeline from the storage server (i.e. when a user of a client computer 15 desires to add or retrieve annotations, the client computer 15 contacts the annotation server 10 to perform the desired addition/retrieval as described in fig. 1 paragraph 31), the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70)); “– at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams” (see ¶s 31, 59 for at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Therefore, the user performs at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams as described in the above figures and paragraphs); “- receiving, from a first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams” (see ¶ 31 for receiving, from a first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraph 44, 47, 59); “- receiving, from a second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream, the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase” (see ¶ 31 for receiving, from a second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61), the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70). Also, see paragraph 37, 44, 47, 59); “- retrieving the second content stream associated with the second content identifier” (see ¶s 31, 59 for retrieving the second content stream associated with the second content identifier (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraphs 47-49, 70); “- modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline” (see ¶s 31, 59 for modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Thus, the user modifies in the annotated timeline the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline as described in the above figures and paragraphs); “and - generating the edited derived media clip corresponding to the edited annotated timeline” (see ¶s 59, 61 for generating the edited derived media clip corresponding to the edited annotated timeline (i.e. an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the segment specification includes the target identifier from target display 262 and the begin and end times from boxes 266 and 270, respectively, upon receipt of the segment specification from interface module 152, the browser communicates with media server 11 and requests the identified media segment using conventional HTTP requests, in response, media server 11 streams the media segment to client 15 for presentation to the user, this presentation allows, for example, the user to verify the portion of the media stream to which his or her annotation will correspond as described in fig. 8 paragraph 89). Also, see paragraph 87-88)
On the other hand, the reference of Ranjeet discloses the following claimed limitations of independent claim 21 that recites “- displaying, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface” (see figs. 1-2 ¶ 46 for displaying, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface (i.e. an example user interface 602 is depicted in accordance with one or more implementations for selecting a view of a multi-view video for a viewport, the user interface 602 includes a viewport 604, which is configured to display videos having multiple views of a real-world scene that were captured simultaneously, the viewport 604 may include a navigation control 606, which may be configured to allow manual navigation by a user to move amongst the multiple views of the video, additionally, the user interface 602 is depicted as having additional viewports 608, 610, and 612 as described in fig. 6 paragraph 62, furthermore, the display of the thumbnail preview may include additional information, such as a timestamp of the location of the thumbnail preview on the video playback timeline as described in fig. 10 paragraph 89). Also, see figs. 7-10 paragraphs 63-64, 67, 83-87)
Therefore, the combined teachings of the primary reference and the secondary reference do not destroy the primary reference; in fact, it enhances the operation of the primary reference since Ranjeet discloses in paragraphs 8-9 that selecting a view in a multi-view video, wherein a video is accessed via a video viewing application in a digital media viewing environment, wherein the video may include multiple views of a scene captured simultaneously by a camera system configured to capture images for the scene in multiple directions corresponding to the multiple views, such as a 360° video, wherein playback of the video is initiated in a viewport exposed by the viewing application, wherein the viewport is designed to selectively switch between presentation of the multiple views of the video at different times, wherein during playback of the video, a subject of interest is identified and tracked by automatically switching between the multiple views to maintain the at least one object in view within the viewport, additionally, a scrub bar of the viewport is provided to display a visual representation of a timeline of the video and the video's playback progress, wherein the scrub bar provides controls to navigate to different points in the video, wherein when user input is received at the scrub bar, a thumbnail preview of the video is generated and displayed, wherein the thumbnail preview is generated for a selected view of multiple available views of the video, wherein a correction may also be applied to account for image distortion that results from switching between multiple views in the viewport in order to ease the processing time and have a user friendly interaction. As a result, the combination of both references won’t change the principle of operation of the primary reference being modified, and then the teachings of the references are sufficient to render the claims prima facie obvious.
Thus, the combination of GUPTA and Ranjeet discloses all the claimed limitations of independent claim 21, and also, all the claimed limitations of independent claim 31 that recites similar features.
In re page 15, Applicant states that “Dependent claims 22 - 30 and 32 - 40 are non-obvious at least by virtue of their dependency on non-obvious claims 21 or 31. Withdrawal of rejections under 35 U.S.C. § 103 is respectfully requested.”
(2) In response, as discussed above in (1) with respect to independent claims 21 and 31 which is also applicable to the above Applicant’s arguments, the combination of GUPTA and Ranjeet discloses all the claimed limitations of independent claims 21 and 31.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-22, 24-27, 29-32, 34-37 and 39-40 are rejected under 35 U.S.C. 103 as being unpatentable over GUPTA et al. (US 2003/0196164 A1)(hereinafter GUPTA), and further in view of Ranjeet et al. (US 2017/0244959 A1)(hereinafter Ranjeet).
Re claim 21, GUPTA discloses a method for editing a derived media clip at an annotated timeline user interface of a control station, the media clip corresponding to an annotated timeline stored in a storage server, the method comprising: - retrieving the annotated timeline from the storage server, the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references (see ¶ 29 for retrieving the annotated timeline from the storage server (i.e. when a user of a client computer 15 desires to add or retrieve annotations, the client computer 15 contacts the annotation server 10 to perform the desired addition/retrieval as described in fig. 1 paragraph 31), the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70)); – at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams (see ¶s 31, 59 for at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Therefore, the user performs at least one indication of a transitioning or combination operation, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams as described in the above figures and paragraphs); - receiving, from a first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams (see ¶ 31 for receiving, from a first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraph 44, 47, 59); - receiving, from a second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream, the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase (see ¶ 31 for receiving, from a second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61), the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70). Also, see paragraph 37, 44, 47, 59); - retrieving the second content stream associated with the second content identifier (see ¶s 31, 59 for retrieving the second content stream associated with the second content identifier (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraphs 47-49, 70); - modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline (see ¶s 31, 59 for modifying, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Thus, the user modifies in the annotated timeline the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline as described in the above figures and paragraphs); and - generating the edited derived media clip corresponding to the edited annotated timeline (see ¶s 59, 61 for generating the edited derived media clip corresponding to the edited annotated timeline (i.e. an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the segment specification includes the target identifier from target display 262 and the begin and end times from boxes 266 and 270, respectively, upon receipt of the segment specification from interface module 152, the browser communicates with media server 11 and requests the identified media segment using conventional HTTP requests, in response, media server 11 streams the media segment to client 15 for presentation to the user, this presentation allows, for example, the user to verify the portion of the media stream to which his or her annotation will correspond as described in fig. 8 paragraph 89). Also, see paragraph 87-88)
GUPTA fails to explicitly teach – displaying, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface. However, the reference of Ranjeet explicitly teaches – displaying, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface (see figs. 1-2 ¶ 46 for displaying, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface (i.e. an example user interface 602 is depicted in accordance with one or more implementations for selecting a view of a multi-view video for a viewport, the user interface 602 includes a viewport 604, which is configured to display videos having multiple views of a real-world scene that were captured simultaneously, the viewport 604 may include a navigation control 606, which may be configured to allow manual navigation by a user to move amongst the multiple views of the video, additionally, the user interface 602 is depicted as having additional viewports 608, 610, and 612 as described in fig. 6 paragraph 62, furthermore, the display of the thumbnail preview may include additional information, such as a timestamp of the location of the thumbnail preview on the video playback timeline as described in fig. 10 paragraph 89). Also, see figs. 7-10 paragraphs 63-64, 67, 83-87)
Therefore, taking the combined teachings of GUPTA and Ranjeet as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (viewports) into the system of GUPTA as taught by Ranjeet.
One will be motivated to incorporate the above feature into the system of GUPTA as taught by Ranjeet for the benefit of selecting a view in a multi-view video, wherein a video is accessed via a video viewing application in a digital media viewing environment, wherein the video may include multiple views of a scene captured simultaneously by a camera system configured to capture images for the scene in multiple directions corresponding to the multiple views, such as a 360° video, wherein playback of the video is initiated in a viewport exposed by the viewing application, wherein the viewport is designed to selectively switch between presentation of the multiple views of the video at different times, wherein during playback of the video, a subject of interest is identified and tracked by automatically switching between the multiple views to maintain the at least one object in view within the viewport, additionally, a scrub bar of the viewport is provided to display a visual representation of a timeline of the video and the video's playback progress, wherein the scrub bar provides controls to navigate to different points in the video, wherein when user input is received at the scrub bar, a thumbnail preview of the video is generated and displayed, wherein the thumbnail preview is generated for a selected view of multiple available views of the video, wherein a correction may also be applied to account for image distortion that results from switching between multiple views in the viewport in order to ease the processing time and have a user friendly interaction (see ¶s 8-9)
Re claim 22, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the first user input device and the second user input device are the same user input device (see ¶ 31 for the first user input device and the second user input device are the same user input device (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37). Also, see paragraph 44, 47)
Re claim 24, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the second content stream is retrieved from a storage server (see ¶s 27, 31 for the second content stream is retrieved from a storage server (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45). Also, see paragraphs 47-49)
Re claim 25, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the second content stream is received from one of a content source or an external source (see ¶s 27, 31 for the second content stream is received from one of a content source or an external source (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45). Also, see fig. 11 paragraph 113)
Re claim 26, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the plurality of content streams are selected from a group consisting of a media stream, an audio stream and a data stream (see ¶s 27, 31 for the plurality of content streams are selected from a group consisting of a media stream, an audio stream and a data stream (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45). Also, see paragraphs 48-49, 51)
Re claim 27, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the plurality of content streams are included in corresponding content objects stored in a storage server and wherein the plurality of content identifiers identify a content source from which the associated content object originated (see ¶ 31 for the plurality of content streams are included in corresponding content objects stored in a storage server and wherein the plurality of content identifiers identify a content source from which the associated content object originated (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 5 paragraph 61, moreover, different media characteristics can be used to associate the annotation with a particular segment(s) of the media content, for example, annotations could be associated with (or "anchored" on) specific objects in the video content, or specific events in the audio content as described in paragraph 64, additionally, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70). Also, see paragraph 43-44, 47, 49)
Re claim 29, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the plurality of content identifiers are unique content identifiers (see ¶s 61, 69 for the plurality of content identifiers are unique content identifiers (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, annotation identifier field 194 stores data that uniquely identifies annotation entry 180, while related annotation identifier field 196 stores data that uniquely identifies a related annotation, an identifier of the first annotation would be stored in related annotation identifier field 196 of the second annotation, an identifier of the second annotation would be stored in related annotation identifier field 196 of the third annotation, and so on as described in fig. 5 paragraph 68, moreover, media content identifier field 200 contains data that uniquely identifies particular multimedia content as the content to which annotation entry 180 corresponds, media content identifier 200 comprises a media version table 204 that identifies each of the different streams of multimedia content (e.g., streams 160 of FIG. 4) to which annotation entry 180 corresponds as described in fig. 5 paragraph 70))
Re claim 30, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein the derived media clip corresponds to a live event and wherein the edited derived media clip is generated in real time (see ¶s 27, 59, 61 for the derived media clip corresponds to a live event and wherein the edited derived media clip is generated in real time (i.e. media server 11 selects a particular combination of a single audio stream and a single video stream to be the "base" version of the multimedia content, the audio and video streams having the speed factors and resolutions as the streams were originally created (or are received in the case of "live" streams) are selected as the base version of the multimedia content, the base version is used as a reference point to identify which segments of the media streams annotations correspond to as described in fig. 1 paragraph 51, furthermore, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, moreover, the segment specification includes the target identifier from target display 262 and the begin and end times from boxes 266 and 270, respectively, upon receipt of the segment specification from interface module 152, the browser communicates with media server 11 and requests the identified media segment using conventional HTTP requests, in response, media server 11 streams the media segment to client 15 for presentation to the user, this presentation allows, for example, the user to verify the portion of the media stream to which his or her annotation will correspond as described in fig. 8 paragraph 89). Also, see paragraph 87-88)
Re claim 31, GUPTA discloses a system for editing a derived media clip at an annotated timeline user interface of a control station, the media clip corresponding to an annotated timeline stored in a storage server, the system comprising: - a memory (i.e. system memory 22 as shown in fig. 2); - a first user input device (see ¶ 37 for a first user input device (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as shown in fig. 2)); - a second user input device (see ¶ 37 for a second user input device (i.e. a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as shown in fig. 2)); - a processor in communication with the memory, a display device, the first user input device, and the second user input device, the processor configured to (see ¶ 36 for a processor in communication with the memory, the display device, the first user input device, and the second user input device, the processor (i.e. computer 20 includes one or more processors or processing units 21, a system memory 22, and a bus 23 that couples various system components including the system memory 22 to processors 21 as described in fig. 2 paragraph 35, furthermore, a user may enter commands and information into computer 20 through input devices such as keyboard 40 and pointing device 42, other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like as described in fig. 2 paragraph 37)): - retrieve the annotated timeline from the storage server, the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references (see ¶ 29 for retrieve the annotated timeline from the storage server (i.e. when a user of a client computer 15 desires to add or retrieve annotations, the client computer 15 contacts the annotation server 10 to perform the desired addition/retrieval as described in fig. 1 paragraph 31), the annotated timeline comprising a plurality of content identifiers associated with a plurality of content streams and a start timecode and an end timecode associated with each of the plurality of content streams, wherein the plurality of content streams comprise a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to a common timebase and the start timecode and the end timecode are based on the timecode references (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70)); – at least one indication of a transitioning or combination operation in relation to the timecode references, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams (see ¶s 31, 59 for at least one indication of a transitioning or combination operation in relation to the timecode references, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Therefore, the user performs at least one indication of a transitioning or combination operation in relation to the timecode references, the transitioning or combination operation associated with at least a first content identifier corresponding to a first content stream of the plurality of content streams as described in the above figures and paragraphs); - receive, from the first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams (see ¶ 31 for receive, from the first user input device at the control station, a first user input including the first content identifier corresponding to the first content stream of the plurality of content streams (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraph 37, 44, 47, 59); - receive, from the second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream, the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase (see ¶ 31 for receive, from the second user input device at the control station, a second user input including a second content identifier corresponding to a second content stream (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61), the second content stream being separate from the plurality of content streams and comprising a plurality of media frames and one or more embedded timecode packets comprising timecode references associated with the plurality of media frames, wherein the embedded timecode references are synchronized to the common timebase (i.e. the annotation entry 180 includes a time range field 184, a time units field 186, a content field 192, an identifier field 194, a related annotation identifier field 196, a set identifier(s) field 198, a media content identifier field 200 as described in fig. 5 paragraph 59, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, time units field 186 contains data representing the units of time represented in time range field 184, together, time range field 184 and time units field 186 identify the relative time range of the annotation represented by annotation entry 180, this relative time range corresponds to a particular segment of the media content to which annotation entry 180 is associated as described in fig. 5 paragraph 61, moreover, data stored in identifier field 200 associates annotation entry 180 with particular media streams such that annotation server 10 can synchronize substantive content of annotation entry 180 with substantive content of the media streams as described in fig. 5 paragraph 70). Also, see paragraph 37, 44, 47, 59); - retrieve the second content stream associated with the second content identifier (see ¶s 31, 59 for retrieve the second content stream associated with the second content identifier (i.e. web browser 153 also provides the interface for a user to select particular media streams for presentation, the user can select which one of different versions of multimedia content he or she wishes to receive from media server 11 as described in fig. 1 paragraph 45, furthermore, time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61). Also, see paragraphs 47-49, 70); - modify, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline (see ¶s 31, 59 for modify, in the annotated timeline, the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline (i.e. time range field 184 contains data representing "begin" and "end" times defining a segment of media timeline to which annotation entry 180 is associated, the begin and end times for the annotation are provided by the user via interface 152 as described in fig. 3 paragraph 61, moreover, an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the annotation being created via dialog box 260 has a begin time and an end time, which together define a particular segment of the media stream, when "from" button 268 is selected, thumb 265 represents the begin time for the segment relative to the media stream, when "to" button 271 is selected, thumb 265 represents the end time for the segment relative to the media stream, alternatively, two different thumbs could be displayed, one for the begin time and one for the end time, the begin and end times are also displayed in an hours/minutes/seconds format in boxes 266 and 270, respectively as described in fig. 8 paragraph 87). Also, see paragraph 88. Thus, the user modifies in the annotated timeline the transitioning or combination operation by replacing the first content identifier with the second content identifier to obtain an edited annotated timeline as described in the above figures and paragraphs); and - generate the edited derived media clip corresponding to the edited annotated timeline (see ¶s 59, 61 for generate the edited derived media clip corresponding to the edited annotated timeline (i.e. an "add new annotation" dialog box 260 that results from user selection of add button 250 of FIG. 7 to create a new annotation, interface 150 assumes that the current media stream being presented to the user is the media stream to which this new annotation will be associated, the media stream to which an annotation is associated is referred to as the "target" of the annotation, an identifier of the target stream is displayed in a target specification area 262 of dialog box 260, alternatively, a user could change the target of the annotation, such as by typing in a new identifier in target area 262, or by selection of a "browse" button (not shown) that allows the user to browse through different directories of media streams as described in fig. 8 paragraph 86, furthermore, the segment specification includes the target identifier from target display 262 and the begin and end times from boxes 266 and 270, respectively, upon receipt of the segment specification from interface module 152, the browser communicates with media server 11 and requests the identified media segment using conventional HTTP requests, in response, media server 11 streams the media segment to client 15 for presentation to the user, this presentation allows, for example, the user to verify the portion of the media stream to which his or her annotation will correspond as described in fig. 8 paragraph 89). Also, see paragraph 87-88)
GUPTA fails to explicitly teach – display, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface. However, the reference of Ranjeet explicitly teaches – display, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface (see figs. 1-2 ¶ 46 for display, at a display device of the control station, the plurality of content streams in one or more corresponding content viewports of the annotated timeline user interface (i.e. an example user interface 602 is depicted in accordance with one or more implementations for selecting a view of a multi-view video for a viewport, the user interface 602 includes a viewport 604, which is configured to display videos having multiple views of a real-world scene that were captured simultaneously, the viewport 604 may include a navigation control 606, which may be configured to allow manual navigation by a user to move amongst the multiple views of the video, additionally, the user interface 602 is depicted as having additional viewports 608, 610, and 612 as described in fig. 6 paragraph 62, furthermore, the display of the thumbnail preview may include additional information, such as a timestamp of the location of the thumbnail preview on the video playback timeline as described in fig. 10 paragraph 89). Also, see figs. 7-10 paragraphs 63-64, 67, 83-87)
Therefore, taking the combined teachings of GUPTA and Ranjeet as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (viewports) into the system of GUPTA as taught by Ranjeet.
One will be motivated to incorporate the above feature into the system of GUPTA as taught by Ranjeet for the benefit of selecting a view in a multi-view video, wherein a video is accessed via a video viewing application in a digital media viewing environment, wherein the video may include multiple views of a scene captured simultaneously by a camera system configured to capture images for the scene in multiple directions corresponding to the multiple views, such as a 360° video, wherein playback of the video is initiated in a viewport exposed by the viewing application, wherein the viewport is designed to selectively switch between presentation of the multiple views of the video at different times, wherein during playback of the video, a subject of interest is identified and tracked by automatically switching between the multiple views to maintain the at least one object in view within the viewport, additionally, a scrub bar of the viewport is provided to display a visual representation of a timeline of the video and the video's playback progress, wherein the scrub bar provides controls to navigate to different points in the video, wherein when user input is received at the scrub bar, a thumbnail preview of the video is generated and displayed, wherein the thumbnail preview is generated for a selected view of multiple available views of the video, wherein a correction may also be applied to account for image distortion that results from switching between multiple views in the viewport in order to ease the processing time and have a user friendly interaction (see ¶s 8-9)
Re claim 32, the combination of GUPTA and Ranjeet as discussed in claim 22 above discloses all the claimed limitations of claim 32.
Re claim 34, the combination of GUPTA and Ranjeet as discussed in claims 24 and 31 above discloses all the claimed limitations of claim 34.
Re claim 35, the combination of GUPTA and Ranjeet as discussed in claims 25 and 31 above discloses all the claimed limitations of claim 35.
Re claim 36, the combination of GUPTA and Ranjeet as discussed in claim 26 above discloses all the claimed limitations of claim 36.
Re claim 37, the combination of GUPTA and Ranjeet as discussed in claim 27 above discloses all the claimed limitations of claim 37.
Re claim 39, the combination of GUPTA and Ranjeet as discussed in claim 29 above discloses all the claimed limitations of claim 39.
Re claim 40, the combination of GUPTA and Ranjeet as discussed in claim 30 above discloses all the claimed limitations of claim 40.
Claims 23 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over GUPTA et al. (US 2003/0196164 A1)(hereinafter GUPTA) as applied to claims 21-22, 24-27, 29-32, 34-37 and 39-40 above, and further in view of Ranjeet et al. (US 2017/0244959 A1)(hereinafter Ranjeet), and further in view of Silvestri et al. (US 2012/0317302 A1)(hereinafter Silvestri).
Re claim 23, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claim limitations with additional claimed feature taught by GUPTA wherein: - the plurality of content streams comprise live content streams (see ¶ 51 for the plurality of content streams comprise live content streams (i.e. streaming media server computer 11 has access to streaming media content in the form of different media streams, media streams might be supplied to the server on a "live" basis from other data source components through dedicated communications channels or through the Internet itself as described in fig. 1 paragraph 27))
GUPTA fails to explicitly teach and - one or more corresponding content viewports display the live content streams. However, the reference of Silvestri explicitly teaches and - one or more corresponding content viewports display the live content streams (see ¶s 176-178 for one or more corresponding content viewports display the live content streams (i.e. viewports 610 display live media streams, respectively as described in fig. 6 paragraph 179))
Therefore, taking the combined teachings of GUPTA, Ranjeet and Silvestri as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (viewports) into the system of GUPTA as taught by Silvestri.
One will be motivated to incorporate the above feature into the system of GUPTA as taught by Silvestri for the benefit of having user interface 600 which comprises one or more viewports, each of which may be generated by an application module, such as an application module 532, wherein in some cases, viewports may be arranged and rearranged in the user interface 600, based on user input, and in other cases, viewports may be arranged according to a predetermined pattern or template, which may be context-specific or user selectable, for example, in one context of metadata input, one viewport arrangement could display a viewport displaying a live media stream, another viewport displaying stored media streams (or media clips) and a metadata input interface viewport, and another viewport arrangement could display live media streams and an output stream viewport, wherein in the example shown, viewports 610 display live media streams, respectively, for example, the live media streams may correspond to separate media sources (e.g., television cameras) in order to have a user friendly interaction (see fig. 6 ¶s 177-179)
Re claim 33, the combination of GUPTA, Ranjeet and Silvestri as discussed in claim 23 above discloses all the claimed limitations of claim 33.
Claims 28 and 38 are rejected under 35 U.S.C. 103 as being unpatentable over GUPTA et al. (US 2003/0196164 A1)(hereinafter GUPTA) as applied to claims 21-22, 24-27, 29-32, 34-37 and 39-40 above, and further in view of Ranjeet et al. (US 2017/0244959 A1)(hereinafter Ranjeet), and further in view of Pedlow, JR. et al. (US 2005/0097614 A1)(hereinafter Pedlow).
Re claim 28, the combination of GUPTA and Ranjeet as discussed in claim 21 above discloses all the claimed limitations but fails to explicitly teach wherein one or more of the plurality of media frames are intracoded media frames. However, the reference of Pedlow explicitly teaches wherein one or more of the plurality of media frames are intracoded media frames (see ¶ 73 for one or more of the plurality of media frames are intracoded media frames (i.e. the content comprising intra-coded frames of video))
Therefore, taking the combined teachings of GUPTA, Ranjeet and Pedlow as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (intracoded media frames) into the system of GUPTA as taught by Pedlow.
One will be motivated to incorporate the above feature into the system of GUPTA as taught by Pedlow for the benefit of using the storage capacity of a normal play content file to store digital video content to facilitate trick play, the content comprising intra-coded frames of video, wherein storing a set of indices that relate the intra-coded frames in a first file with the intra-coded frames in a second file, such that playback of the second file simulates a fast-forward playback if played back in a first order and simulates a fast rewind if played back in a second order in order to improve efficiency (see fig. 3 ¶s 74-75)
Re claim 38, the combination of GUPTA, Ranjeet and Pedlow as discussed in claim 28 above discloses all the claimed limitations of claim 38.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE M MESA whose telephone number is (571)270-1706. The examiner can normally be reached Monday-Friday 8:30AM-6:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached on 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
06/09/2026
/JOSE M. MESA/
Examiner
Art Unit 2484
/THAI Q TRAN/ Supervisory Patent Examiner, Art Unit 2484
Prosecution Insights