DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim(s) 2-21 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-11 and 13 of U.S. Patent No. 11,094,351 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the claim of the instant application is either anticipated by, or the obvious variation of, the claim of U.S. Patent No. 11,094,351 B2, as shown in the table below.
Instant Application:
US 11,094,351 B2:
Claim 2. A method comprising:
receiving content comprising a plurality of frames;
condensing the content to determine a condensed visual representation and metadata for each frame of the plurality of frames;
determining, based on the condensed visual representation or metadata, a plurality of points in the content, each of the plurality of points comprising an insertion point in the content;
inserting supplemental content in at least one of the plurality of points; and
generating for display the content with the inserted additional content
Claim 1. A method comprising:
receiving content comprising a plurality of frames;
condensing the content to determine a condensed visual representation and metadata for each frame of the plurality of frames;
determining, based on the condensed visual representation, a plurality of points in the content, each of the plurality of points comprising a transition point or an insertion point in the content; and
causing simultaneous display of the condensed visual representation and identifiers of each of the plurality of points relative to the condensed visual representation.
Claim 9. The method of claim 1, further comprising inserting supplemental content in the content at one of the determined plurality of points in the content.
Claim 3. The method of claim 2, wherein condensing the content to determine the condensed visual representation and the metadata for each frame of the plurality of frames comprises:
aggregating, for each frame, a visual property of each of a plurality of pixels in at least one of a horizontal row and a vertical column to form a first representation of each frame; and
aligning the first representation of each frame according to an order of the plurality of frames in the content to determine the condensed visual representation.
Claim 2. The method of claim 1, wherein condensing the content to determine the condensed visual representation and the metadata for each frame of the plurality of frames comprises:
aggregating, for each frame, a visual property of each of a plurality of pixels in at least one of a horizontal row and a vertical column to form a first representation of each frame; and
aligning the first representation of each frame according to an order of the plurality of frames in the content to determine the condensed visual representation.
Claim 4. The method of claim 3, wherein aggregating the visual property of each of the plurality of pixels comprises averaging the color of each of the plurality of pixels.
Claim 3. The method of claim 2, wherein aggregating the visual property of each of the plurality of pixels comprises averaging the color of each of the plurality of pixels.
Claim 5. The method of claim 4, wherein the metadata for each frame comprises a standard deviation of the colors of the pixels in the frame from the average color.
Claim 4. The method of claim 3, wherein the metadata for each frame comprises a standard deviation of the colors of the pixels in the frame from the average color.
Claim 6. The method of claim 3, wherein the first representation of each frame is one pixel wide.
Claim 5. The method of claim 2, wherein the first representation of each frame is one pixel wide.
Claim 7. The method of claim 2, wherein condensing the content to determine the condensed visual representation and the metadata for each frame of the plurality of frames comprises:
grouping a plurality of pixels of each frame into a block;
aggregating, for each block, a visual property of each of the plurality of pixels; and
replacing the plurality of pixels of each block with the aggregated visual property.
Claim 6. The method of claim 1, wherein condensing the content to determine the condensed visual representation and the metadata for each frame of the plurality of frames comprises:
grouping a plurality of pixels of each frame into a block;
aggregating, for each block, a visual property of each of the plurality of pixels; and
replacing the plurality of pixels of each block with the aggregated visual property.
Claim 8. The method of claim 7, wherein grouping the plurality of pixels of each frame into the block comprises grouping the plurality of pixels of each frame into a predetermined number of blocks.
Claim 7. The method of claim 6, wherein grouping the plurality of pixels of each frame into the block comprises grouping the plurality of pixels of each frame into a predetermined number of blocks.
Claim 9. The method of claim 8, wherein the predetermined number of blocks is:
received as a user selection; or
automatically determined based on one of a dimension of the plurality of frames, an amount of visual information contained in each frame, and a visual characteristic of the content.
Claim 8. The method of claim 7, wherein the predetermined number of blocks is:
received as a user selection; or
automatically determined based on one of a dimension of the plurality of frames, an amount of visual information contained in each frame, and a visual characteristic of the content.
Claim 10. The method of claim 2, wherein the supplemental content comprises an advertisement.
Claim 10. The method of claim 9, wherein the supplemental content comprises a local advertisement.
Claim 11. The method of claim 2, wherein the metadata for each frame comprises a time stamp of the frame.
Claim 11. The method of claim 1, wherein the metadata for each frame comprises a time stamp of the frame.
Claim 12. The method of claim 2, wherein the condensed visual representation and identifiers of each of the plurality of points are generated for displayed along with the content and the supplemental content.
Claim 1
Claim 13. The method of claim 2, wherein the insertion point corresponds a scene cut in the content.
Claim 13. The method of claim 1, wherein the transition point comprises a scene cut in the content.
Claim 14
Claim 1, 9
Claim 15
Claim 3
Claim 16
Claim 5
Claim 17
Claim 7
Claim 18
Claim 8
Claim 19
Claim 9
Claim 20
Claim 10
Claim 21
Claim 13
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent Claim 2/14 recites the limitation "the inserted additional content" in las line. There is insufficient antecedent basis for this limitation in the claim. Other dependent claims are rejected due to their dependency on their corresponding independent claims.
For examination purpose, it is assumed the cited the inserted additional content is intended to be “the inserted supplemental content”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2-8,10-11, 13-18 and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Rayner (US 5,388,197) in view of Cotsaces et al. (Video Shot Detection and Condensed Representation, A review, IEEE Signal Processing March 2006) and Holcomb et al. (US 2005/0105883 A1).
Regarding Claim 2, Rayner discloses a method (col.2 lines57-61: It is a further object of the present invention to provide a method of video tape recorder position and velocity control that makes such control easier and more intuitive for the operator) comprising:
receiving content comprising a plurality of frames (Fig.3: notice video data are input to “VIDEO SOURCE INTERFACE 58”);
condensing the content to determine a condensed visual representation (col.9, line 15-26: Referring now to FIG. 3, the digital video and audio processor 58 produces a frame sample 17 for each video frame 19, and sends these frame samples to the local memory 60. In one embodiment, each frame sample 17 that is taken consists of eight individual pixels distributed vertically along the center of the video frame 19. In another embodiment, the frame sample 17 consists of eight composite pixels that each represent the average video values within eight different regions distributed vertically along the center of the video frame 19. In either case, these frame samples 17, when displayed in sequence as a video pictorial timeline, represent an "edge view" of the contents of the underlying video sequence Also see “Video Pictorial Timeline 10, 11” in Fig.3);
PNG
media_image1.png
453
695
media_image1.png
Greyscale
Rayner fails to disclose condensing the content to determine a condensed visual representation and metadata for each frame of the plurality of frames.
However Cotsaces, in the same field of endeavor, discloses a PHOSITA before the effective filing date of the claimed invention had already known that Condensed representation is the extraction of a characteristic set of either independent frames or short sequences from a video. This can be used as a substitute for the whole video for the purposes of indexing, comparison, and categorization. It is also especially useful for video browsing (p.2 left column lines 7-12) and The results of both shot boundary detection and condensed video representation do not need to be immediately directed to the above applications; they may instead be stored as metadata and used when they are needed (p.2 left column lines 16-19).
PNG
media_image2.png
243
380
media_image2.png
Greyscale
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Cotsaces into that of Rayner and to include the limitation of condensing the content to determine a condensed visual representation and metadata for each frame of the plurality of frames to provide an option for video editing to use metadata instead of using condensed visual representation.
Cotsaces discloses using condensed representation to detect shot change for video processing (p.1 first paragraph). But Rayner modified by Cotsaces fails to explicitly recite video editing includes determining, based on the condensed visual representation or metadata, a plurality of points in the content, each of the plurality of points comprising an insertion point in the content; inserting supplemental content in at least one of the plurality of points; and generating for display the content with the inserted additional content.
However Holcomb, in the same field of endeavor, discloses determining a plurality of points to indicate where a commercial can be inserted ([0035]: Described techniques and tools introduce conditions for a video frame or a video field to be a valid entry point in a bitstream. These conditions represent policies that a video encoder enforces during encoding of a video sequence to produce entry points in bitstreams (in order to enable "trick" modes, for example). The policies can produce entry points in the bitstream at different locations. For example, policies can direct entry points to be provided when a scene change occurs or to indicate where a commercial can be inserted to facilitate activities such as video summarization and splicing, respectively. The policies governing the insertion of entry points in the bitstream can also be designed such that entry points are provided at regular intervals and at specific time intervals (for example, to minimize latency upon tuning to the service)). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Holcomb into that of Rayner modified by Cotsaces and to include the limitation of determining, based on the condensed visual representation or metadata, a plurality of points in the content, each of the plurality of points comprising an insertion point in the content; inserting supplemental content in at least one of the plurality of points; and generating for display the content with the inserted additional content in order to assist video editors to locate places to insert advertisements or other commercial materials.
Regarding Claim 3, Rayner further teaches or suggests wherein condensing the content to determine the condensed visual representation and the metadata for each frame of the plurality of frames comprises:
aggregating, for each frame, a visual property of each of a plurality of pixels in at least one of a horizontal row and a vertical column to form a first representation of each frame (Fig.3: sample 17. Also see col.9, lines 22-26: these frame samples 17, when displayed in sequence as a video pictorial timeline, represent an "edge view" of the contents of the underlying video sequence); and
aligning the first representation of each frame according to an order of the plurality of frames in the content to determine the condensed visual representation (See “Video Pictorial Timeline 10, 11” in Fig.3).
Regarding Claim 4, Rayner further teaches or suggests wherein aggregating the visual property of each of the plurality of pixels comprises averaging the color of each of the plurality of pixels (col. 9 lines 19-22: In another embodiment, the frame sample 17 consists of eight composite pixels that each represent the average video values within eight different regions distributed vertically along the center of the video frame 19).
Regarding Claim 5, Rayner further teaches or suggests wherein the metadata for each frame comprises a standard deviation of the colors of the pixels in the frame from the average color (col. 9 lines 19-22: In another embodiment, the frame sample 17 consists of eight composite pixels that each represent the average video values within eight different regions distributed vertically along the center of the video frame 19. Note Cotsaces teaches metadata can be used instead of condensed representation for shot transitions see p.2 left column lines 18-19: they may instead be stored as metadata and used when they are needed).
Regarding Claim 6, Rayner wherein the first representation of each frame is one pixel wide (Fig.3, frame sample 17 is a vertical line at the center of frame. A skilled person would have recognized that a line is composed by one single pixel in vertical direction).
Regarding Claim 7, Rayner teaches or suggests grouping a plurality of pixels of each frame into a block (Fig.3: notice sample 17 8 blocks along center line); aggregating, for each block, a visual property of each of the plurality of pixels; and replacing the plurality of pixels of each block with the aggregated visual property (col.9 lines 19-22: In another embodiment, the frame sample 17 consists of eight composite pixels that each represent the average video values within eight different regions distributed vertically along the center of the video frame 19).
Regarding Claim 8, Rayner discloses wherein grouping the plurality of pixels of each frame into the block comprises grouping the plurality of pixels of each frame into a predetermined number of blocks (Fig.3: sample 17 notice the vertical line is grouped into 8 blocks).
Regarding Claim 10, Rayner as modified further teaches or suggests wherein the supplemental content comprises an advertisement (Holcomb [0035]: the policies can produce entry points in the bitstream at different locations. For example, policies can direct entry points to be provided when a scene change occurs or to indicate where a commercial can be inserted to facilitate activities such as video summarization and splicing, respectively).
Regarding Claim 11, Rayner as modified teaches metadata can be used to replace condensed video representation (Cotsaces p.2 left column lines 14-19). Since condensed video representation comprises time information, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to include a time stamp of the frame and to add the limitation of wherein the metadata for each frame comprises a time stamp of the frame in order to record time information within the metadata.
Regarding Claim 13, Rayner further teaches or suggests wherein the insertion point corresponds a scene cut in the content (col.4 lines 12-14: The background area 33 is used for basic editing that only involves a linear arrangement of video sequences with cuts or dissolves).
Regarding Claims 14, 15, 16-18, 20 and 21, Claims 14, 15, 16-18, 20 and 21 is/are similar to Claims 1, 3, 6-8, 10 and 13 except in the format of system. Therefore the same reason(s) for rejection is/are applied to Claims 1, 3, 6-8, 10 and 13 is/are also applied to Claims 14, 15, 16-18, 20 and 21.
Claims 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rayner (US 5,388,197) in view of Cotsaces et al. (Video Shot Detection and Condensed Representation, A review, IEEE Signal Processing March 2006) and Holcomb et al. (US 2005/0105883 A1) as applied to Claims 8 and 18 above, and further in view of Lightbody et al. (US 5,471,577).
Regarding Claim 9, Rayner teaches or suggests automatically determined based on one of a dimension of the plurality of frames, an amount of visual information contained in each frame, and a visual characteristic of the content (Rayner discloses grouping the pixels of each first representation into a predetermined number of blocks and replacing the pixels of each block with a pixel assigned by aggregating the visual property of each pixel in the block (col.9, lines 19-21).
But Rayner as modified fails to explicitly disclose wherein the predetermined number of blocks is: received as a user selection.
However, Lightbody discloses a video editing system which uses subsampling method to reduce the computation required for monitoring the video data and in subsampling video, each full frame is reduced to a small number of pixels, which is then displayed in a window on the display screen (col.1, lines 41-49 and col.4 lines 32-33: Subsampling is the process of reducing a full-size image to a reduced size image. Full-size video may be reduced to fit an arbitrary-size subsample window, for instance 160x120 pixels). Lightbody further discloses users may direct to re-size the subsample widow (col.8, lines 32-33). Therefore, it would have been obvious to one with ordinary skill in the art at the time of the invention to incorporate the teaching of Lightbody into that of Rayner as modified and to add a user interface to allow the predetermined number of block selected by a user by adding the limitation of wherein the predetermined number of blocks is: received as a user selection so that a user can decide the size of compression according to the video content and his/her editing needs and keep the video editing display updated.
Regarding Claim 19, Claim 19 is/are similar to Claim 9 except in the format of system. Therefore the same reason(s) for rejection is/are applied to Claim 9 is/are also applied to Claim 19.
Claim 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rayner (US 5,388,197) in view of Cotsaces et al. (Video Shot Detection and Condensed Representation, A review, IEEE Signal Processing March 2006) and Holcomb et al. (US 2005/0105883 A1) as applied to Claim 2 above, and further in view of Barbieri et al. (US 2003/0063130 A1).
Regarding Claim 12, Rayner as modified fails to explicitly wherein the condensed visual representation and identifiers of each of the plurality of points are generated for displayed along with the content and the supplemental content.
However Barbieri, in the same field of endeavor, discloses displaying the condensed visual representation ([0040]: Users associate a physical length in the slider to a fixed amount of time and this helps in navigating through the video content) and an identifier of the transition or the insertion point relative to the condensed visual representation ([0037]: Changes in different parts of video programs can be visualized effectively. The color slider enables a user to recognize at a glance the beginning of the match because it follows the headings and the report recorded in studio, which has another color in the slider bar. The color slider enables also the user to identify commercial breaks in a program. Often a commercial break is characterized by a high rate color change and the presence of black frames).
Therefore it would have been obvious to one ordinary person skilled in the art before the effective filing date of the claimed invention to incorporate the teaching of Barbieri into Rayner as modified and to add the limitation of wherein the condensed visual representation and identifiers of each of the plurality of points are generated for displayed along with the content and the supplemental content in order to determine the portion at which the marker is located, and to enable reproduction of the information unit at the position indicated by the determined portion as taught by Barbieri ([0001]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613
Prosecution Insights