DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 12/31/2025 have been fully considered but they are not persuasive.
Regarding claims 1, 14 and 30, applicant argues that Sarkar and Allegretti does not teach “a companion metadata file associated with the reaction atlas”, remarks pages 12-13.
However, the examiner respectfully disagrees with the applicant. Sarkar discloses that a social networking system 102 receives and relays a plurality of video clip reactions 312, e.g. reaction atlas, wherein the social networking system 102 presents the video clip for its entire duration (e.g., four seconds) within the video-graphical element 312 overlaid on the live-video-stream display 30; col. 13 lines 25-40 figures 3A-H.
Moreover, Sarkar recites that the social networking system 102 transmits data to the viewer device 114a that controls or adjusts the size of a video-graphical elements 418, 428, and 430 based on friends, followers, an affinity coefficient, etc.; col. 23 lines 1-20 figures 1And 4E. One or ordinary skill in the art would recognize that control data sent by the social networking system 102 clearly reads on the claimed companion metadata. Additionally, the examiner notes that the claims are silent to clearly define the meaning of the recited companion metadata. Therefore, Sarkar in view of Allegretti clearly disclose “a companion metadata file associated with the reaction atlas”, as currently claimed.
Allowable Subject Matter
Claims 11, 21 and 40 are allowed.
Claims 12 and 41 are held allowable since they depends from allowable claims 11 and 40, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4, 6, 14, 15, 17, 30, 32, 33 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar et al. (Patent No US 10,652,618) in view of Allegretti et al. (Pub No US 2016/0234551). Hereinafter, referenced as Sarkar and Allegretti, respectively.
Regarding claim 1, Sarkar discloses a method for delivering video reactions to a livestream display interface, comprising (or comprising computer instructions for):
receiving at least one of a reaction video or reaction animation by a server (Figure 1; e.g. social networking system 102), the at least one reaction video or reaction animation including video frames (Col. 27 lines 22-29 figures 5 and 6; receiving, from a viewer device of the plurality of viewer devices, a video clip of a reaction to the live video stream by a viewer of a plurality of viewers);
extracting a subset of or all of video frames from the at least one reaction video or reaction animation by the server to provide extracted frames (Col. 13 lines 25-40 figures 3A-H; social networking system 102 receives and relays the received video clip reactions 312, wherein the social networking system 102 presents the video clip for its entire duration (e.g., four seconds) within the video-graphical element 312 overlaid on the live-video-stream display 30);
generate a reaction atlas (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40) and a companion metadata file associated with the reaction atlas (Col. 23 lines 1-20 figures 1And 4E; social networking system 102 transmits data to the viewer device 114a that controls or adjusts the size of a video-graphical elements 418, 428, and 430 based on friends, followers, an affinity coefficient, etc.);
and transmitting the reaction atlas to a client device for insertion of the reaction atlas (e.g. reactions 428, 430, 418) into a computerized livestream display interface provided by the client device during the livestream (Col. 27 lines 6-56 figure 7; transmitting the live video stream 720 and the reaction video clip 740 to the viewers), the computerized livestream display interface including an arrangement of a plurality of reaction atlases from a corresponding plurality of client devices (Col. 22 lines 48-67, figures 4E-F; video reactions 428, 430, 418).
However, it is noted that Sarkar is silent to explicitly disclose compressing, by the server, the extracted frames to generate a reaction.
Nevertheless, in a similar field of endeavor Allegretti discloses compressing, by the server, the extracted frames to generate a reaction (Paragraphs [0118] figures 1 and 14; reaction server 26 may compress the file size of the viewer reactions, change in quality, bit rate or other metric, change in file type, or change in encoding standard).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar by specifically providing the elements mentioned above, as taught by Allegretti, for the predictable result of saving storage space and administering network bandwidth and resources in an effective manner.
Regarding claim 3, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses that the at least one reaction video or reaction animation is an animation video selected by a user (Col. 24 lines 49-67 figure 4F; user reactions may include reaction options such as a beating heart, thumbs up, etc., wherein user reaction, e.g. graphical element 312, may continuously move across the live-video-stream GUI 304 and toward an edge of the live-video-stream GUI 30; col. 24 lines 8-14).
Regarding claim 4, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses that the at least one reaction video or reaction animation is from a camera feed (Col. 20 lines 61-64; camera 402 of the viewer device 114a may capture a video clip of a reaction to the live video stream).
Regarding claim 6, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses that the server transmits the companion metadata file (Col. 23 lines 1-20 figures 1And 4E; social networking system 102 transmits data to the viewer device 114a that controls or adjusts the size of a video-graphical elements 418, 428, and 430 based on friends, followers, an affinity coefficient, etc.) associated with the reaction atlas to the client device (Col. 27 lines 6-56 figure 7; transmitting the live video stream 720 and the reaction video clip 740 to the viewers).
Regarding claims 14, 15, 17, Sarkar and Allegretti disclose all the limitations of claims 14, 15, 17; therefore, claims 14, 15, 17 are rejected for the same reasons stated in claims 1, 2 and 6, respectively.
Regarding claims 30, 32, 33 and 35, Sarkar and Allegretti disclose all the limitations of claims 30, 32, 33 and 35; therefore, claims 30, 32, 33 and 35 are rejected for the same reasons stated in claims 1, 3, 4, 5, respectively.
Claims 2, 13, 31 and 42 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar and Allegretti further in view of Neumeier et al. (Pub No US 2022/0270556). Hereinafter, referenced as Neumeier.
Regarding claim 2, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses display the reaction atlas (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40).
However, it is noted that Sarkar and Allegretti are silent to explicitly disclose that the client device is configured to display the reaction atlas at a framerate to simulate the at least one reaction video or reaction animation.
Nevertheless, in a similar field of endeavor Neumeier discloses that the client device is configured to display the reaction atlas at a framerate to simulate the at least one reaction video or reaction animation (Paragraph [0047]; Buffer 404 provides the video images, including user reactions, to the TV Display at the appropriate video frame rate for the region of operation).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Neumeier, for the predictable result of implementing significantly faster parallel processing, leading to smoothness, clarity, and realism of motion of the presented content objects.
Regarding claim 13, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses the reaction atlas (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40).
However, it is noted that Sarkar and Allegretti are silent to explicitly disclose that the client device stores the reaction in GPU memory, and a client device GPU processes the reaction atlas using a GPU application that moves or scales the textures.
Nevertheless, in a similar field of endeavor Neumeier discloses that the client device stores the reaction in GPU memory (e.g. GPU 403), and a client device GPU processes the reaction atlas using a GPU application that moves or scales the textures (Paragraphs [0044] [0047] figures 3A and 4; displaying user reactions at a user display device 101, which includes a memory 400, GPU 403, in order to scale and place the reaction picture in an appropriate location in the Video Frame Buffer 404).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Neumeier, for the predictable result of implementing significantly faster parallel processing, leading to enhanced performance in graphics-intensive applications.
Regarding claim 31, Sarkar, Allegretti and Neumeier disclose all the limitations of claim 31; therefore, claim 31 is rejected for the same reasons stated in claim 2.
Regarding claim 42, Sarkar, Allegretti and Neumeier disclose all the limitations of claim 42; therefore, claim 42 is rejected for the same reasons stated in claim 13.
Claims 5 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar and Allegretti further in view of Lanier (Patent No US 11,895,167). Hereinafter, referenced as Lanier.
Regarding claim 5, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses a plurality of reactions on a livestream display interface (Col. 22 lines 48-67, figures 4E-F; video reactions 428, 430, 418).
However, it is noted that Sarkar and Allegretti are silent to explicitly disclose that the server scales the extracted frames to fit in an arrangement with a plurality of reactions on a livestream display interface.
Nevertheless, in a similar field of endeavor Lanier discloses the server scales the extracted frames to fit in an arrangement with a plurality of reactions on a livestream display interface (Col. 6 lines 52-67 figures 1A-B; rendering user reaction in a session, wherein each rendering of each participant can be scaled to fit depicted seats, desk).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Lanier, for the predictable result of scaling different user reactions to fit a in virtual environment, enriching the user experience and increasing the realistic approach to the distributed reactions.
Regarding claim 34, Sarkar, Allegretti and Lanier disclose all the limitations of claim 34; therefore, claim 34 is rejected for the same reasons stated in claim 5.
Claims 7, 18 and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar and Allegretti further in view of Belaji et al. (Pub No US 2022/0353220). Hereinafter, referenced as Belaji.
Regarding claim 7, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses the reaction atlas for use with the computerized livestream display interface (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40).
However, it is noted that Sarkar and Allegretti are silent to explicitly disclose storing by the client device the reaction atlas for use or reuse with the computerized livestream display interface.
Nevertheless, in a similar field of endeavor Belaji discloses storing by the client device the reaction atlas for use or reuse with the computerized livestream display interface (Paragraph [0052] figure 4B; the viewer may add their own emoji-based “sticker packs”, which can be chosen by a participant and then incorporated into the set of available visual reactions, the user may upload their own reactions, and download premium reactions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Belaji, for the predictable result of allowing the user flexibility to add and collect their preferred visual reactions which may be utilized during a session.
Regarding claim 18, Sarkar, Allegretti and Belaji disclose all the limitations of claim 18; therefore, claim 18 is rejected for the same reasons stated in claim 7.
Regarding claim 36, Sarkar, Allegretti and Belaji disclose all the limitations of claim 36; therefore, claim 36 is rejected for the same reasons stated in claim 7.
Claims 8 and 37 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar, Allegretti and Belaji further in view of Yerli (Pub No US 2024/0179354). Hereinafter, referenced as Yerli.
Regarding claim 8, Sarkar, Allegretti and Belaji disclose the method of claim 7; moreover, Sarkar discloses the reaction atlas (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40). Moreover, Belaji discloses storing the reaction atlas (Paragraph [0052] figure 4B; the viewer may add their own emoji-based “sticker packs”, which can be chosen by a participant and then incorporated into the set of available visual reactions, the user may upload their own reactions, and download premium reactions).
However, it is noted that Sarkar, Allegretti and Belaji are silent to explicitly disclose that a reaction may be stored in WebGL memory.
Nevertheless, in a similar field of endeavor Yerli discloses that a reaction may be stored in WebGL memory (Paragraphs [0041] [0055] figure 1; displaying user reactions in a virtual environment 110 in memory 106, which may be based on WebGL).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar, Allegretti and Belaji by specifically providing the elements mentioned above, as taught by Yerli, for the predictable result of implementing interfaces that render animations directly on a web browser, implementing an accelerated graphics performance.
Regarding claim 37, Sarkar, Allegretti, Belaji and Yerli disclose all the limitations of claim 37; therefore, claim 37 is rejected for the same reasons stated in claim 8.
Claims 9, 10, 16, 19, 20, 38 and 39 are rejected under 35 U.S.C. 103 as being unpatentable over Sarkar and Allegretti further in view of Lerner et al. (Pub No US 2020/0225844). Hereinafter, referenced as Lerner.
Regarding claim 9, Sarkar and Allegretti disclose the method of claim 1; moreover, Sarkar discloses the reaction atlas (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40).
However, it is noted that Sarkar and Allegretti are silent to explicitly disclose that at least one of the client device or the server positions or sizes the reaction atlas in the arrangement in the computerized livestream display interface based upon a reaction signal associated with the reaction atlas received by the client device.
Nevertheless, in a similar field of endeavor Lerner discloses that at least one of the client device or the server positions or sizes the reaction atlas in the arrangement in the computerized livestream display interface based upon a reaction signal associated with the reaction atlas received by the client device (Paragraphs [0181] [0182] figure 16; displaying a reaction video 1615, and positioning a plurality of reactions 1620 reacting to the reaction video 1615).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Lerner, for the predictable result of visually differentiating the types of reactions based on the position of the reactions within the interface.
Regarding claim 10, Sarkar, Allegretti and Lerner disclose the method of claim 9; however, it is noted that Sarkar and Allegretti are silent to explicitly disclose that the reaction signal is at least one of a new reaction signal, a user is speaking signal, or another user is speaking signal.
Nevertheless, in a similar field of endeavor Lerner discloses that the reaction signal is at least one of a new reaction signal (e.g. reactions 1620), a user is speaking signal, or another user is speaking signal (Paragraphs [0181] [0182] figure 16; displaying a reaction video 1615, and positioning a plurality of reactions 1620 reacting to the reaction video 1615).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sarkar and Allegretti by specifically providing the elements mentioned above, as taught by Lerner, for the predictable result of visually differentiating the types of reactions based on the position of the reactions within the interface.
Regarding claim 16, Sarkar, Allegretti and Lerner disclose all the limitations of claim 16; therefore, claim 16 is rejected for the same reasons stated in claim 9.
Regarding claim 19, Sarkar, Allegretti and Lerner disclose all the limitations of claim 19; therefore, claim 19 is rejected for the same reasons stated in claim 9.
Regarding claim 20, Sarkar, Allegretti and Lerner disclose all the limitations of claim 20; therefore, claim 20 is rejected for the same reasons stated in claim 9.
Regarding claim 38, Sarkar, Allegretti and Lerner disclose all the limitations of claim 38; therefore, claim 38 is rejected for the same reasons stated in claim 9.
Regarding claim 39, Sarkar and Allegretti disclose the system of claim 38; moreover, Sarkar discloses that the reaction signal is at least one of a new reaction signal, a user is speaking signal, or another user is speaking signal (Col. 22 lines 48-67, figure 4E; present reaction video-graphical elements 418, 428, and 430 within the live-video-stream GUI 40).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUNIOR O MENDOZA whose telephone number is (571)270-3573. The examiner can normally be reached Mon-Fri 10am-6pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin Bruckart can be reached at 571-272-3982. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JUNIOR O. MENDOZA
Primary Examiner
Art Unit 2424
/JUNIOR O MENDOZA/Primary Examiner, Art Unit 2424