DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Amendment filed on 12/30/2025. Claims 1-20 are pending in the case.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8 and 11-18 are rejected under 35 U.S.C. 103 as being unpatentable over Thornton (US 2003/0212811 A1) in view of Brown (US 20140354664 A1) and further in view of Lawrence (US 20220222030 A1).
As to independent claim 1, Thornton teaches a computing device (compute 102), comprising:
a graphics processing circuitry including a buffer; and processing circuitry (“the computer 102 stores a video image or frame for each of the monitors 108A D. The computer 102 generates multiplexed video sequences” paragraph 0069) configured to:
determine an active screen of the computing device based on a user interaction event (“As used herein, the term "active frame" refers to a video frame that is associated with a display device that has, or is associated with, a higher activity level relative to the other display devices in a multiple display device system, where the activity level is determined based on specified (possibly user-defined) criteria. For example, in the embodiment of FIG. 7, the active frame refers to the frame that includes the cursor (e.g., the "mouse"). Metrics for determining the active frame may include, but are not limited to, user activity, e.g., user-driven peripheral activity, such as mouse movement, keyboard entry, touch screen events, user's viewing direction, i.e., whether the user is currently viewing a particular display, etc.; graphics activity, e.g., the amount or rate of change in image data; and application activity, e.g., which display device or frame is associated with an active application, e.g., a word processor program or media player, among others.” paragraph 0129);
McDowell does not appear to expressly teach identify a graphics rendering event occurring in the computing device, the graphics rendering event associated with a software application that executes on the computing device and is presented by the graphics processing circuity in the active screen; and
identify screen capture data in the buffer of the graphics processing circuitry that corresponds to the graphics rendering event.
Brown teaches identify a graphics rendering event occurring in the computing device, the graphics rendering event associated with a software application that executes on the computing device and is presented by the graphics processing circuity in the active screen (“FIG. 2 shows a typical computer graphics processing system. An application 1, such as a game, executing on host processor 10 will require graphics processing operations to be performed by an associated graphics processor 13. To do this, the application will generate API calls that are interpreted by a driver 12 for the graphics processor 13 that is running on the host processor 10 to generate appropriate commands to the graphics processor 13 to generate the graphics output required by the application 1.” Paragraph 0088, “FIG. 3 shows schematically an exemplary frame 20 to be generated by the graphics processor 13 together with the corresponding "dirty regions" map 21” paragraph 0099-0100, Fig. 2-4);
identify screen capture data in the buffer of the graphics processing circuitry that corresponds to the graphics rendering event (at step 30 in fig. 4, determine tiles in frame that could have changed and at step 33, the graphics processor 13 proceeds to render the tiles indicated as to be rendered by the tile map and stores those tiles in the frame buffer).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise identify a graphics rendering event occurring in the computing device, the graphics rendering event associated with a software application that executes on the computing device and is presented by the graphics processing circuity in the active screen. One would have been motivated to make such a combination to improve frame buffer operations in data processing systems.
McDowell does not appear to expressly teach communicate a contextual screen update event via an application programming interface, the contextual screen update event to cause a service to perform contextual processing of the screen capture data.
Lawrence teaches communicate a contextual screen update event via an application programming interface, the contextual screen update event to cause a service to perform contextual processing of the screen capture data (“At block 1110,…The example machine learning model controller circuitry 308 processes the input data and the application contextual data with the interactive context machine learning model to determine an interaction classification for the screen sharing event during the classification interval of the video conferencing meeting…. At block 1112, the example frame selection circuitry 312 identifies frames determined to correspond to the interaction classification… At block 1114, the example frame selection circuitry 312 removes the identified frames. In some examples, the frame selection circuitry 312 removes the identified video frames, and only the remaining generated video frames from the screen sharing event are included in the transport stream for the video conferencing meeting.” Paragraph 0090-0093).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise communicate a contextual screen update event via an application programming interface, the contextual screen update event to cause a service to perform contextual processing of the screen capture data. One would have been motivated to make such a combination to improve data rendering experiences by reducing streaming glitches for network bandwidth-constrained clients/users during video data spikes due to instances of large transitions of video (Lawrence [0116]).
As to dependent claim 2, Thornton teaches the computing device of claim 1, Thornton further teaches wherein the graphics rendering event is generated by the software application in a foreground of the active screen (“the analyzing may include determining respective levels of application activity associated with each display device. For example, a determination may be made as to which display device is displaying or is associated with an active application, e.g., the current active window.” Paragraph 0139).
As to dependent claim 3, Thornton teaches the computing device of claim 2, Thornton does not appear to expressly teach further teaches wherein the graphics rendering event is indicated via a DirectX Graphics Infrastructure (DXGI) platform, and wherein the graphics rendering event corresponds to a dirty rectangle update in the DXGI platform. However, Brown teaches FIG. 3 frame 20 to be generated by the graphics processor 13 together with the corresponding "dirty regions"” Paragraph 0099, examiner notes that using standard rendering platform like DirectX is considered obvious to a person of ordinary skill in the art.
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise wherein the graphics rendering event is indicated via a DirectX Graphics Infrastructure (DXGI) platform, and wherein the graphics rendering event corresponds to a dirty rectangle update in the DXGI platform. One would have been motivated to make such a combination to improve frame buffer operations in data processing systems.
As to dependent claim 4, Thornton teaches the computing device of claim 1, Thornton does not appear to expressly teach the processing circuitry further configured to:
determine an amount of a change between a first frame and a second frame of the screen capture data, based on the graphics rendering event;
wherein the contextual screen update event is generated based on the amount of the change.
Brown teaches determine an amount of a change between a first frame and a second frame of the screen capture data, based on the graphics rendering event (“FIG. 4 is a flowchart showing schematically the generation and use of the tile map 23. Thus as shown in FIG. 4, the driver 12 on the host processor 10 first determines the tiles in the frame that could have changed from the dirty region information provided by the compositor 4 (step 30), and then generates a tile map indicating the tiles to be rendered from the determined tiles in the frame that could have changed (step 31).” Paragraph 0100);
wherein the contextual screen update event is generated based on the amount of the change (“the changed sub-regions and the sub-region processing information (tile map) is determined and generated from a list of, e.g., dirty rectangles (dirty regions), provided by the host processor (compositor 4),” Paragraph 0107).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise determine an amount of a change between a first frame and a second frame of the screen capture data, based on the graphics rendering event; wherein the contextual screen update event is generated based on the amount of the change. One would have been motivated to make such a combination to improve frame buffer operations in data processing systems.
As to dependent claim 5, Thornton teaches the computing device of claim 1, Thornton further teaches wherein the computing device provides a plurality of screens including the active screen and at least one other screen (monitor #1-4 in fig. 2), and wherein to determine the active screen of the computing device is based in part on an application event provided from the software application that occurs in the active screen. (“As used herein, the term "active frame" refers to a video frame that is associated with a display device that has, or is associated with, a higher activity level relative to the other display devices in a multiple display device system, where the activity level is determined based on specified (possibly user-defined) criteria. For example, in the embodiment of FIG. 7, the active frame refers to the frame that includes the cursor (e.g., the "mouse"). Metrics for determining the active frame may include, but are not limited to, user activity, e.g., user-driven peripheral activity, such as mouse movement, keyboard entry, touch screen events, user's viewing direction, i.e., whether the user is currently viewing a particular display, etc.; graphics activity, e.g., the amount or rate of change in image data; and application activity, e.g., which display device or frame is associated with an active application, e.g., a word processor program or media player, among others.” paragraph 0129).
As to dependent claim 6, Thornton teaches the computing device of claim 1, Thornton further teaches wherein the computing device provides a plurality of screens including the active screen and at least one other screen (monitor #1-4 in fig. 2), and wherein to determine the active screen of the computing device is based in part on an interaction event, the interaction event originating from input of a human interface device operably coupled to the computing device that occurs in the active screen (“As used herein, the term "active frame" refers to a video frame that is associated with a display device that has, or is associated with, a higher activity level relative to the other display devices in a multiple display device system, where the activity level is determined based on specified (possibly user-defined) criteria. For example, in the embodiment of FIG. 7, the active frame refers to the frame that includes the cursor (e.g., the "mouse"). Metrics for determining the active frame may include, but are not limited to, user activity, e.g., user-driven peripheral activity, such as mouse movement, keyboard entry, touch screen events, user's viewing direction, i.e., whether the user is currently viewing a particular display, etc.; graphics activity, e.g., the amount or rate of change in image data; and application activity, e.g., which display device or frame is associated with an active application, e.g., a word processor program or media player, among others.” paragraph 0129).
.
As to dependent claim 7, Thornton teaches the computing device of claim 1, Thornton does not appear to expressly teach wherein the graphics processing circuitry comprises at least one graphics processing unit (GPU), wherein the application programming interface is provided by a driver installed in an operating system of the computing device, and wherein the driver provides an interface between the operating system and functions of the at least one GPU.
Lawrence teaches wherein the graphics processing circuitry comprises at least one graphics processing unit (GPU), wherein the application programming interface is provided by a driver installed in an operating system of the computing device, and wherein the driver provides an interface between the operating system and functions of the at least one GPU (“one or more GPUs, one or more DSPs, etc., and/or a combination thereof) and application programming interface(s) (API(s)) that may assign computing task(s) to whichever one(s) of the multiple types of the processing circuitry is/are best suited to execute the computing task(s).” paragraph 0022)..
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise wherein the graphics processing circuitry comprises at least one graphics processing unit (GPU), wherein the application programming interface is provided by a driver installed in an operating system of the computing device, and wherein the driver provides an interface between the operating system and functions of the at least one GPU. One would have been motivated to make such a combination to improve data rendering experiences by reducing streaming glitches for network bandwidth-constrained clients/users during video data spikes due to instances of large transitions of video (Lawrence [0116]).
As to dependent claim 8, Thornton teaches the computing device of claim 1, Thornton does not appear to expressly teach wherein the service is an AI inferencing engine, and wherein the contextual screen update event causes the AI inferencing engine to pause or resume processing of respective screen captures provided from the buffer.
Lawrence teaches wherein the service is an AI inferencing engine, and wherein the contextual screen update event causes the AI inferencing engine to pause or resume processing of respective screen captures provided from the buffer (“FIG. 5 illustrates an example of a process diagram 500 of the machine learning model of the interactive context frame selection circuitry 206 of FIG. 3. The example process diagram 500 is illustrative of further details of the interactive context machine learning model run by the example machine learning model controller circuitry 308 of FIG. 3 in relation to the example process diagram 400 of FIG. 4. … In some examples, depending on the application being shared in the screen sharing event, certain input sequences may be more relevant than others in the example interactive context machine learning model run at block 412…. At block 410 of FIG. 5, the example OS event determination circuitry 306 determined OS events that correspond to the HID inputs determined at block 408.” Paragraph 0060-0062, “In some examples, the frame selection circuitry 312 removes/excludes the identified video frames, and the remaining generated video frames from the screen sharing event are included in the transport stream for the video conferencing meeting. In some examples, the frame selection circuitry 312 removes/excludes the video frames (image frames) from the video data of the video conferencing meeting” 0054-0055, “In some examples, the frame selection circuitry 312 removes/excludes the video frames (image frames) from the video data of the video conferencing meeting and the corresponding audio data is included in the transport stream. In some examples, the identified video frames are treated as a pause such that the last video frame transmitted in the transport stream prior to the classification interval is repeated to avoid a video transition at the start of the classification interval.” Paragraph 0056).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise wherein the service is an AI inferencing engine, and wherein the contextual screen update event causes the AI inferencing engine to pause or resume processing of respective screen captures provided from the buffer. One would have been motivated to make such a combination to improve data rendering experiences by reducing streaming glitches for network bandwidth-constrained clients/users during video data spikes due to instances of large transitions of video (Lawrence [0116]).
Claims 11-18 are substantially the same as claims 1-8 and are therefore rejected under the same rationale.
Claims 9-10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Thornton in view of Brown, Lawrence and further in view of Wang (20230245651).
As to dependent claim 9, Thornton teaches the computing device of claim 8, Thornton does not appear to expressly teach wherein the AI inferencing engine is configured to generate tokens corresponding to contextual information in the respective screen captures.
Wang teaches wherein the AI inferencing engine is configured to generate tokens corresponding to contextual information in the respective screen captures (“The AI system then transforms the most relevant contextual information into textual form.” Paragraph 0005).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise wherein the AI inferencing engine is configured to generate tokens corresponding to contextual information in the respective screen captures. One would have been motivated to make such a combination for enabling contextually relevant communications (Wang [0004]).
As to dependent claim 10, Thornton teaches the computing device of claim 9, Thornton does not appear to expressly teach wherein the tokens corresponding to the contextual information are textual tokens usable in a transformer model of a generative AI service, and
wherein the graphics rendering event is used to determine a model used by the AI inferencing engine to convert graphical data of the screen capture data into the textual tokens.
Wang teaches wherein the tokens corresponding to the contextual information are textual tokens usable in a transformer model of a generative AI service (“The AI system then transforms the most relevant contextual information into textual form.” Paragraph 0005), and
wherein the contextual information is used to determine a model used by the AI inferencing engine to convert the data into the textual tokens (“Through iterative conversational interaction, the AI system and the user establish an understanding of the most relevant intent and objective. The AI system validates the understanding of the most relevant intent and objective with the user iteratively until the user agrees. Finally, the AI system utilizes the validated the most relevant intent and objective to facilitate user-centered, contextually relevant, and personalized conversational interaction.” Paragraph 0005).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Thornton to comprise wherein the tokens corresponding to the contextual information are textual tokens usable in a transformer model of a generative AI service, and wherein the graphics rendering event is used to determine a model used by the AI inferencing engine to convert graphical data of the screen capture data into the textual tokens. One would have been motivated to make such a combination for enabling contextually relevant communications (Wang [0004]).
Claims 19-20 are substantially the same as claims 9-10 and are therefore rejected under the same rationale.
Response to Arguments
Applicant's prior arts arguments have been fully considered but they are not persuasive. Applicant argues that Lawrence does not teach a “contextual processing” of screen capture data. Examiner respectfully disagrees. Lawrence teaches performing classification of the screen capture data, and it processes the determined classification to determine the contextual data to determine which frames have the relevant contextual data. Therefore, Lawrence expressly teaches performing contextual processing of the screen capture data in order to determine which frames to send.
Applicant’s remaining arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Canitz US 9779467 B2 teaches a graphics component, which is coupled to a graphics device and associated with an unsupported file type, is identified. The graphics component generates image data compromising the hardware accelerated graphics. When the web application requests a set of video frames, the image data is transformed into the set of video frames in accordance with the format. Then, the set of frames is communicated to a display device.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHELET SHIBEROU whose telephone number is (571)270-7493. The examiner can normally be reached Monday-Friday 9:00 AM-5:00 PM Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057 The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHELET SHIBEROU/Primary Examiner, Art Unit 2171
Prosecution Insights