DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to
www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
6. Claims 21, 30, and 36 are rejected under the judicially created doctrine of obviousness-type double patenting as being unpatentable over claims 4, 13, and 17 of U.S. Patent No. 12,394,201.
Although the conflicting claims are not identical, they are not patentably distinct from each other because Claims 4, 13, and 17 of Patent No. 12,394,201 contain every element of claims 21, 30, and 36 of the instant application and thus anticipate the claims of the instant application. Claims 21, 30, and 36 of the instant application therefore are not patently distinct from the earlier patent claims and as such are unpatentable over obvious-type double patenting.
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1,148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
8. Claims 21-27, 30-31, and 33-35 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Komar et al. (US Publication 2003/0079224) in view of Neely et al. (US Patent 7,796,155), and further in view of Polavarapu et al. (US Publication 2020/0265238).
Regarding claim 21, Komar discloses a method comprising:
receiving, by one or more computer processors, from a client device, an indication of a particular bounding area of a plurality of bounding areas (Komar, para. 0017-0024, the viewer can identify one of selectable display areas 141-142 having borders surrounding them, i.e., bounding areas; the viewer can select the identified selectable display area using the viewer input device. For example, the viewer could place cursor 145 over selectable display area 141 and select selectable display area 141 by double-clicking the left mouse key of mouse 162; display module 110 receives the location of cursor 145 on display device as viewer selection input, i.e., a particular bounding area. Similarly, a numbered list or numbered icons can be implemented to represent selectable display areas 141-142. In this case, the viewer could enter a number associated with the selected selectable display area using the keypad of remote control 161 or keyboard 163. The viewer's selection is transmitted to display module 110 as viewer input 135. Viewer input 135 can include the number selected if a viewer has entered a number using a keypad or the location of cursor 145 on display device 120 if a viewer has maneuvered cursor 145 to select one of selectable display areas 141-142; para. 0024, alternatively, in at least one embodiment, the information (display area action data 121-122) utilized by display module 110 is obtained from external data source 130. For example, external data source 130 could include a file server that is to provide 3-D images of advertiser's products (selectable display areas 141-142) included in scenes of video content 115. External data source 130 can include a variety of data sources, such as a networked data server. For example, the action associated with selectable display area 141 can include the retrieval of a web page from a web server on the Internet and the action associated with selectable display area 142 can include the retrieval of a video clip from the same web server. In this example, if a viewer selects selectable display area 141, display module 110 can send external data request 151 to the web server (external data source 130) to request the web page. The web server can then transmit the web page content (display area action data 121) to display module 110. Likewise, if a viewer selects selectable display area 142, display module 110 can send external data request 152 to the web server (external data source 130) to request the video clip (display area action data 122), which is then transmitted by the web server to display module 110; para. 0048, and 0037-0038, each of selectable display areas 141-142 (FIG. 1) has a corresponding selectable display area ID used by display module 110 (FIG. 1) to identify each of selectable display areas),
wherein: the client device is displaying video content of a sporting event on a graphical user interface to a user, and the particular bounding area is identified based at least in part on a value associated with one or more pixels associated with a portion of the graphical user interface selected by the user corresponding to a bounding area value associated with the particular bounding area (Komar, para’s 0017-0028, FIG’s 1 and 2, display to a user the video content on a user display interface; the displayed video content includes one or more scenes having one or more selectable display areas bounded by border “bounding areas”; para’s 0050 and 0069, selection analyzer 510 compares the coordinates “value” of cursor 145 to the center coordinates of selectable display areas “pixel areas” 141-142. If the coordinates “value” of cursor 145 are collocated within a circle defined by a specified radius and a center coordinate of one of selectable display areas “pixel areas” 141-142, selection analyzer 510 can assume that the selectable display area has been selected by the viewer; para. 0062, in one embodiment, the pattern recognition software searches each frame of video content 115 for pixel patterns representative of the selected selectable display area and marks its location when found);
determining, by the one or more computer processors, a particular object within a frame of the video content based at least in part on the particular bounding area (Komar, para. 0017, the selectable display area refers (to) an area within the display of a scene of video content representative of a particular object or item having additional information);
determining, by the one or more computer processors, a current augmentation state for the particular object (Komar, para. 0031, a selectable display area could have one or multiple associated actions; the multiple actions could be displayed as choices in a list in pop-up window 240 when the selectable display area is selected, i.e., augmentation state, and a viewer could select from the actions by scrolling through the list or entering a number associated with an action);
selecting, by the one or more computer processors, an augmentation image based at least in part on the current augmentation state for the particular object (Komar, para. 0031, multiple actions could be displayed as choices in a list in pop-up window 240, i.e., augmentation state, and a viewer could select an action from the multiple actions by scrolling through the list with action options).
Komar does not explicitly disclose:
wherein the particular object corresponds to at least one of a player or a non-player object in the sporting event; transmitting the augmentation image to the client device.
Neely discloses wherein the video content is of a sporting event and the particular object corresponds to at least one of a player or a non-player object that is in play in the sporting event (Neely, FIG. 13 illustrates an alternate embodiment of a model extraction processor 1300, where element 1300 corresponds to the previously described element 1102 and 1200. In this alternate embodiment of the present invention, suitable for an initial system roll-out, player geometry is limited to a 3D bounding box and technicians manually track the players in the video).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Neely’s features into Komar’s invention for enhancing viewer’s playback experience by providing augmentation video that corresponds to at least one of a player or a non-player object that in a sporting event.
Komar-Neely does not explicitly disclose but Polavarapu discloses:
transmitting the augmentation image to the client device (Polavarapu, para’s 0080-0084, upon a trigger of content augmentation on an object, rendering facility 324 may utilize any suitable data, such as tracking data from tracking data facility 310, to provide/transmit the augmentation content 602 at a current location of object 402-2 within rendered video frame 600 or at any other position relative to the current location of object 402-2 within rendered video frame 600).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Polavarapu’s features into Komar-Neely’s invention for enhancing viewer’s playback experience by effectively augmenting video content with additional information of objects depicted in the video content at specific locations in the video content.
Regarding claim 22, Komar-Neely-Polavarapu discloses the method of Claim 21, the method comprising: determining, by the one or more computer processors, video frame data and location data associated with the particular object; and transmitting the video frame data and the location data to the client device (Polavarapu, para’s 0080-0084, upon a trigger of content augmentation on an object, rendering facility 324 may utilize any suitable data, such as tracking data from tracking data facility 310, to provide/transmit the augmentation content 602 at a current location of object 402-2 within rendered video frame 600 or at any other position relative to the current location of object 402-2 within rendered video frame 600).
The motivation and obviousness arguments are the same as claim 1.
Regarding claim 23, Komar-Neely-Polavarapu discloses the method of Claim 22, wherein the client device is configured to display the augmentation image on the graphical user interface based at least in part on the video frame data and the location data (Komar, para’s 0041-0042, 0058-0060, display the generated video including scenes of the video content and additional information associated with the identified display area; para. 0025, information display field 150 can include a window within the display window used to display video content 115, such as the picture-in-picture format; Polavarapu, para’s 0080-0084, upon a trigger of content augmentation on an object, rendering facility 324 may utilize any suitable data, such as tracking data from tracking data facility 310, to provide/transmit the augmentation content 602 at a current location of object 402-2 within rendered video frame 600 or at any other position relative to the current location of object 402-2 within rendered video frame 600; para’s 0015, 0048, 0052, display a rendered video frame with augmentation content included within the rendered video frame).
The motivation and obviousness arguments are the same as claim 1.
Regarding claim 24, Komar-Neely-Polavarapu discloses the method of Claim 21, wherein selecting the augmentation image based at least in part on the current augmentation state for the particular object comprises: determining a next augmentation state for the particular object, the next augmentation state being located after the current augmentation state in a sequence of augmentation states identified for the particular object; and selecting the augmentation image based at least in part on the next augmentation state.
Regarding claim 25, Komar-Neely-Polavarapu discloses the method of Claim 21 further comprising receiving, by the one or more computer processors from the client device, a second indication of the current augmentation state (Komar, para. 0031, a selectable display area could have one or multiple associated actions; the multiple actions could be displayed as choices in a list in pop-up window 240 when the selectable display area is selected, i.e., augmentation state, and a viewer could select from the actions by scrolling through the list; the second item from the scrolling list is seen as the second indication of the augmentation state).
Regarding claim 26, Komar-Neely-Polavarapu discloses the method of Claim 21, wherein determining the current augmentation state for the particular object comprises determining that there is no current augmentation image associated with the particular object (Komar, para. 0017, additional information includes a 3-D image(s) of the soft drink; para. 0025, performing an action associated with one of selectable display areas includes displaying information, such as text, a web page, images, and/or a video clip; web pages and video clip each may be presented sequentially page by page and frame by frame respectively. Therefore, when the last page or last frame has been presented, there is no current augmentation image associated with the particular object; Polavarapu, para. 0057, object classification facility 304 may not detect an object 402 associated with the set of features within a region of interest).
Regarding claim 27, Komar-Neely-Polavarapu discloses the method of Claim 21 further comprising:
assigning, by the one or more computer processors, the plurality of bounding areas to a plurality of objects represented in the video content, wherein each bounding area of the plurality of bounding areas is associated with a respective object of the plurality of objects;
assigning, by the one or more computer processors, a respective bounding area value to each bounding area of the plurality of bounding areas; and
transmitting, to the client device, video data, wherein the video data is representative of the video content, the plurality of bounding areas, and the respective bounding area value for each bounding area of the plurality of bounding areas (Komar, para’s 0027-0028, FIG’s 1 and 2, display the video content on a user display interface; the displayed video content includes one or more scenes having one or more selectable display areas bounded by border “bounding boxes”; para’s 0050-0069, providing/transmitting to “client” display device video stream with one or more selectable display areas “bounding areas”; video stream 105 could include video content 115 already including selectable display area 141 with a blue border 250 and selectable display area 142 having relatively high-luminance pixels; each of selectable display areas has a corresponding selectable display area ID used by display module to identify each of selectable display areas; this disclosure indicates the assigned color value corresponding to each respective selectable display area was transmitted to the display device; selection analyzer 510 compares the coordinates “value” of cursor 145 to the center coordinates of selectable display areas “pixel areas” 141-142. If the coordinates “value” of cursor 145 are collocated within a circle defined by a specified radius and a center coordinate of one of selectable display areas “pixel areas” 141-142, selection analyzer 510 can assume that the selectable display area has been selected by the viewer; in one embodiment, the pattern recognition software searches each frame of video content 115 for pixel patterns representative of the selected selectable display area and marks its location when found; in addition, para. 0017, discloses identifying one or more of selectable display areas “bounding areas” within the display of a scene/frame(s) of video content representative of an object or item having additional information that could interest a viewer of the video; the disclosure above indicates determining assigning each bounding areas of the plurality of bounding areas with a respective object of a plurality of objects, assigning a respective bounding area value to each bounding area of the plurality of bounding areas; and transmitting, to the client device, video data, wherein the video data comprises the video content, the plurality of bounding areas, and the respective bounding area value for each bounding area of the plurality of bounding areas).
Claims 30-31 and 33-35 are rejected for the same reasons set forth in claims 21-23. Komar-Neely-Polavarapu further discloses processors, memory, and computer-readable medium (see Komar, para. 0074 and claim 71).
9. Claims 28 is rejected under AIA 35 U.S.C. 103 as being unpatentable over Komar-Neely-Polavarapu, as applied to claim 27 above, in view of Aksoy et al. (US Publication 2018/0225827, hereinafter Aksoy).
Regarding claim 28, Komar-Neely-Polavarapu discloses the method of Claim 27.
Komar-Neely-Polavarapu does not disclose but Aksoy discloses assigning an opacity value of zero to each bounding area of the plurality of bounding areas (Aksoy, para. 0058, setting opacity value of an image area to describe the transparency level, where 1 is not transparent at all, 0.5 is 50% see-through, and 0 is completely transparent as is well known in the art).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Aksoy’s features into Komar-Neely-Polavarapu’s invention for enhancing viewer’s playback experience by providing convenience for viewer to easily identify a pixel area containing an object of interest.
10. Claim 29, 32, and 36 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Komar-Neely-Polavarapu, as applied to claims 23, 31, and 34 above, in view of Sato et al. (US Publication 2005/0107159, hereinafter Sato).
Regarding claim 29, Komar-Neely-Polavarapu discloses the method of Claim 23, and further discloses presenting the one or more augmentation images on the graphical user interface based on the video frame data and the location data, and presenting the one or more augmentation images as the video content is presented on the graphical user interface as described above.
Komar-Neely-Polavarapu does not explicitly disclose but Sato discloses wherein the client device is configured to display the augmentation image on the graphical user interface based at least in part on the video frame data and the location data in conjunction with the video content so that, when the augmentation image is displayed, the augmentation image remains in a substantially fixed orientation relative to the particular object as the video content is presented on the graphical user interface (Sato, para. 0006, the player earning a high score is photographed with a video camera and the photographed player image is displayed on a display together with the scoring data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Sato’s features into Komar-Neely-Polavarapu’s invention for enhancing user’s viewing experience by visually showing a player and his or her scoring data augmented together.
Claims 32 and 36 are rejected for the same reasons set forth in claim 29.
11. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. These include:
Smith et al., US Patent 10,325,410
Siddique et al., US Publication 2013/0215116
Stojancic et al., US Publication 2019/0354764
Conclusion
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to LOI H TRAN whose telephone number is (571)270-5645. The examiner can normally be reached 8:00AM-5:00PM PST FIRST FRIDAY OF BIWEEK OFF.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached on 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LOI H TRAN/ Primary Examiner, Art Unit 2484
Prosecution Insights