DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Terminal Disclaimer
The terminal disclaimer filed on 7/2/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. Patent No. 11,620,731 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Response to Amendment
The amendments, filed 7/2/2026, have been entered and made of record. Claims 1-10 are pending.
Response to Arguments
Applicant's arguments filed 7/2/2026 have been fully considered but they are not persuasive.
In re pages 3-5, the applicant states “(1) Kohen does not disclose signatures associated with graphical objects. … The Examiner has not identified any disclosure in Kohen of: " obtaining a plurality of signatures; " associating signatures with graphical objects; " monitoring a communication stream for signatures; or " identifying graphical objects through detection of signatures. … (2) Metadata is not the claimed signature. … (3) Kohen does not inherently disclose the claimed signatures.”.
In response, the examiner respectfully disagrees. Kohen teaches a method for monitoring graphical content of a screen display, the method comprising: obtaining a plurality of signatures wherein each signature is associated with a graphical object(“The set of dynamic target information indicating value(s) displayed to a user on an application monitoring user interface (UI) may include, for example, a text, numeric, or graphical object, such as a still icon or picture or a moving image displayed on a user display or screen. In one embodiment, the set of dynamic target information may include some or all data in a specific coordinate region of a user display. For example, a user may enter coordinate values or select a region or objects on a screen to be monitored, e.g., by clicking, drawing, highlighting, or dragging with a mouse. In another embodiment, the set of dynamic target information may include a type of data, such as a high score list, which may be indicated by a user or programmer and automatically found on a screen and monitored. In some embodiments, the set of dynamic target information may include data that moves to different coordinate locations within a page or screen during application runtime.” in Para.[0029]);
monitoring a stream of data communicated to a display screen to capture at least one signature of the plurality of signatures(“A request to monitor a sub-region within a display page of a user interface of a second application may be accepted in a first application. The first application may iteratively capture data during the operation of the second application. The first application may determine that an instance of the captured data matches data representing the requested sub-region or page and in response, may extract content associated with the requested sub-region from the captured data. The first or second application may display the extracted content, for example, for monitoring the requested sub-region of the second application” in Abs, Para.[0029]);
cropping content of the display screen to obtain the graphical object associated with the captured signature(“The set of dynamic target information indicating value(s) displayed to a user on an application monitoring user interface (UI) may include, for example, a text, numeric, or graphical object, such as a still icon or picture or a moving image displayed on a user display or screen. In one embodiment, the set of dynamic target information may include some or all data in a specific coordinate region of a user display. For example, a user may enter coordinate values or select a region or objects on a screen to be monitored, e.g., by clicking, drawing, highlighting, or dragging with a mouse. In another embodiment, the set of dynamic target information may include a type of data, such as a high score list, which may be indicated by a user or programmer and automatically found on a screen and monitored. In some embodiments, the set of dynamic target information may include data that moves to different coordinate locations within a page or screen during application runtime.” in Para.[0029]); and
processing the cropped image to extract at least one of: text, and URL data presented within the cropped content(“According to embodiments of the invention embodiments, a separate monitoring application or module 160 may be implemented independently of application 170 to monitor dynamic target information 117 of application 170 displayed on user interface 195. In one embodiment, a modeler may request to monitor one or more target sub-regions within one or more target display pages of user interface 195 of application 170. For example, the modeler may draw a target sub-region such as a rectangle with a pointing device on user interface 195 and set the sub-region as dynamic target information 117. Monitoring module 160 may implement a capture mode during the runtime of application 170, in which content may be iteratively captured from user interface 195 of application 170. Monitoring module 160 may capture screen shots of user interface 195 or copy back buffers of a screen device (e.g., user display 144). In one embodiment, the entire screen shot is captured iteratively. In another embodiment, only a portion of the screen shot, e.g., the target sub-region, may be captured iteratively. Each iteration of screen capture may be triggered by an event, e.g., the lapse of a predetermined time interval or upon an event of action in application 170 such as receiving predetermined user input known to affect dynamic target information 117. For example, when a user shoots a weapon known to affect the score in a game and dynamic target information 117 is a user score sub-region, a user clicking or selecting designated shooting keys of an input device may trigger a screen capture. Application 170 may have multiple pages, some of which include the sub-region dynamic target information 117 and some of which do not. To determine whether or not to extract content from the captured screen shots of user interface 195, monitoring module 160 may determine whether or not the captured content are the target display pages and/or target sub-regions to be monitored. The captured content and the target pages and/or sub-regions selected by the modeler may include metadata 190. Monitoring module 160 may capture the entire page or sub-regions iteratively content and then compare metadata 190 of the captured content with metadata 190 of dynamic target information 117. If the respective metadata 190 substantially match, it may be determined that the content was captured from the target page or of the target sub-regions and therefore, data should be extracted from the captured content. Monitoring module 160 may execute page detection anchor 119 logic to further extract information from the captured content to generate dynamic target information 117 that may be displayed to a user for monitoring application 170, e.g., as described in further detail in reference to FIG. 5. If the respective metadata 190 of the captured content and dynamic target information 117 do not match, it may be determined that the captured content does not contain dynamic target information 117 and no data is extracted from the captured content” in Para.[0040], Para.[0029]).
In re pages 5-6, the applicant states “The Office has reject claims 3 and 8 under § 103 over Kohen in view of Revital. Applicant respectfully traverses. Claims 3 and 8 further recite: "using the signature as a watermark."”.
In response, the examiner respectfully disagrees.
Kohen is silent about further comprising: using the signature as a watermark.
Revital teaches further comprising: using the signature as a watermark (“The disclosed embodiments relate to systems and methods for watermarking content such as multimedia content. Disclosed embodiments allow for a watermark generation in multimedia content and for detection of the watermarked content. The multimedia content can include a video signal, an audio signal, an image, speech signals, 3D graphical objects, texts, software codes, network streams, XML data, or any other suitable binary or text data. The watermarking may use computer-based models such as machine-learning techniques, such as neural networks” in Para.[0031]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Kohen with the above teachings of Revital in order to enhance user experience beyond that of a conventional video streaming platform.
In re pages 6-7, the applicant states “The Office has rejected claims 4 and 9 under 35 U.S.C. §103 over Kohen in view of Grusd. Applicant respectfully traverses. Claims 4 and 9 require that an advertisement be associated with the claimed signature”.
In response, the examiner respectfully disagrees.
Kohen is silent about further comprising identifying an advertisement on the display screen, wherein the advertisement is associated with the signature.
Grusd teaches further comprising identifying an advertisement on the display screen, wherein the advertisement is associated with the signature(“Objects in a video are identified, and linked to ecommerce sites and advertisements. A user may select an object and be provided with information about the object. A video stream may be encoded with a separate metadata stream that contains the identity of each object on each frame of the video. The metadata channel may be extracted from a video stream and displayed to a user when the user clicks on the corresponding object. A device may take a video input, analyze it, and recognize the video. The device may look up said video in a database of tagged videos, output as an overlay or separate window to a video display device stream with information about the item. A user interface device may allow a user to select objects on a video screen and select and choose said object for integration with the other components of the system” in Abs).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Kohen with the above teachings of Grusd in order to enhance user experience beyond that of a conventional video streaming platform.
In re pages 7-8, the applicant states “Claims 5 and 10 further require: wherein the signature is associated with at least one parameter associated with at least one of: " the display screen; and " other content displayed on the display screen substantially concurrently.”.
In response, the examiner respectfully disagrees. Kohen teaches wherein the signature is associated with at least one parameter associated with at least one of: the display screen; and other content displayed on the display screen substantially concurrently(“Dynamic target information 117 identification. The modeler may define or select the information elements to be extracted from application 170, e.g., during an application simulation modeling phase. For example, the modeler may enter coordinate values or select, e.g., by clicking, drawing, or highlighting with a mouse, a region or objects on user interface 195 defining dynamic target information 117 to be monitored. In another embodiment, dynamic target information 117 may be identified by a visual or graphic feature, e.g., a color, texture, pattern, contrast setting, design, etc. A corresponding page detection anchor 119 may be generated for each element of dynamic target information 117. The modeler may also set a frequency, time interval, or trigger for iteratively capturing content from a running application 170. Otherwise, content may be captured at a default frequency” in Para.[0050]).
In re page 8, the applicant states “The References Teach Different Identification Technologies Applicant respectfully notes that the rejections rely upon three different identification mechanisms: " Kohen relies upon metadata and anchor regions. " Grusd relies upon object recognition and object metadata. " Revital relies upon digital watermarking. The Office Action treats these concepts as interchangeable with Applicant's claimed signatures. They are not Each reference employs its own identification mechanism for its own intended purpose. Nothing in the cited art suggests replacing Kohen's metadata architecture with Applicant's signature architecture or combining the disparate technologies in the manner proposed by the Office Action. Only Applicant's disclosure provides the roadmap for making such substitutions. The rejection therefore relies upon impermissible hindsight reconstruction”.
In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Kohen
Claims 1, 2, 5-7 and 10 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Kohen et al.(USPubN 2010/0138775; hereinafter Kohen).
As per claim 1, Kohen teaches a method for monitoring graphical content of a screen display, the method comprising: obtaining a plurality of signatures wherein each signature is associated with a graphical object(“The set of dynamic target information indicating value(s) displayed to a user on an application monitoring user interface (UI) may include, for example, a text, numeric, or graphical object, such as a still icon or picture or a moving image displayed on a user display or screen. In one embodiment, the set of dynamic target information may include some or all data in a specific coordinate region of a user display. For example, a user may enter coordinate values or select a region or objects on a screen to be monitored, e.g., by clicking, drawing, highlighting, or dragging with a mouse. In another embodiment, the set of dynamic target information may include a type of data, such as a high score list, which may be indicated by a user or programmer and automatically found on a screen and monitored. In some embodiments, the set of dynamic target information may include data that moves to different coordinate locations within a page or screen during application runtime.” in Para.[0029]);
monitoring a stream of data communicated to a display screen to capture at least one signature of the plurality of signatures(“A request to monitor a sub-region within a display page of a user interface of a second application may be accepted in a first application. The first application may iteratively capture data during the operation of the second application. The first application may determine that an instance of the captured data matches data representing the requested sub-region or page and in response, may extract content associated with the requested sub-region from the captured data. The first or second application may display the extracted content, for example, for monitoring the requested sub-region of the second application” in Abs, Para.[0029]);
cropping content of the display screen to obtain the graphical object associated with the captured signature(“The set of dynamic target information indicating value(s) displayed to a user on an application monitoring user interface (UI) may include, for example, a text, numeric, or graphical object, such as a still icon or picture or a moving image displayed on a user display or screen. In one embodiment, the set of dynamic target information may include some or all data in a specific coordinate region of a user display. For example, a user may enter coordinate values or select a region or objects on a screen to be monitored, e.g., by clicking, drawing, highlighting, or dragging with a mouse. In another embodiment, the set of dynamic target information may include a type of data, such as a high score list, which may be indicated by a user or programmer and automatically found on a screen and monitored. In some embodiments, the set of dynamic target information may include data that moves to different coordinate locations within a page or screen during application runtime.” in Para.[0029]); and
processing the cropped image to extract at least one of: text, and URL data presented within the cropped content(“According to embodiments of the invention embodiments, a separate monitoring application or module 160 may be implemented independently of application 170 to monitor dynamic target information 117 of application 170 displayed on user interface 195. In one embodiment, a modeler may request to monitor one or more target sub-regions within one or more target display pages of user interface 195 of application 170. For example, the modeler may draw a target sub-region such as a rectangle with a pointing device on user interface 195 and set the sub-region as dynamic target information 117. Monitoring module 160 may implement a capture mode during the runtime of application 170, in which content may be iteratively captured from user interface 195 of application 170. Monitoring module 160 may capture screen shots of user interface 195 or copy back buffers of a screen device (e.g., user display 144). In one embodiment, the entire screen shot is captured iteratively. In another embodiment, only a portion of the screen shot, e.g., the target sub-region, may be captured iteratively. Each iteration of screen capture may be triggered by an event, e.g., the lapse of a predetermined time interval or upon an event of action in application 170 such as receiving predetermined user input known to affect dynamic target information 117. For example, when a user shoots a weapon known to affect the score in a game and dynamic target information 117 is a user score sub-region, a user clicking or selecting designated shooting keys of an input device may trigger a screen capture. Application 170 may have multiple pages, some of which include the sub-region dynamic target information 117 and some of which do not. To determine whether or not to extract content from the captured screen shots of user interface 195, monitoring module 160 may determine whether or not the captured content are the target display pages and/or target sub-regions to be monitored. The captured content and the target pages and/or sub-regions selected by the modeler may include metadata 190. Monitoring module 160 may capture the entire page or sub-regions iteratively content and then compare metadata 190 of the captured content with metadata 190 of dynamic target information 117. If the respective metadata 190 substantially match, it may be determined that the content was captured from the target page or of the target sub-regions and therefore, data should be extracted from the captured content. Monitoring module 160 may execute page detection anchor 119 logic to further extract information from the captured content to generate dynamic target information 117 that may be displayed to a user for monitoring application 170, e.g., as described in further detail in reference to FIG. 5. If the respective metadata 190 of the captured content and dynamic target information 117 do not match, it may be determined that the captured content does not contain dynamic target information 117 and no data is extracted from the captured content” in Para.[0040], Para.[0029]).
As per claim 2, Kohen teaches further comprising: scanning the cropped image to extract relevant text using optical character recognition (OCR)(“ : The modeler may select one or more extraction modules or functionalities, e.g., from a set of options provided to the modeler. The modules may include optical character recognition (OCR) modules, image recognition modules, pattern recognition modules, or other modules that input page detection anchor 119 data and output corresponding dynamic target information 117 element(s).” in Para.[0055]).
As per claim 5, Kohen teaches wherein the signature is associated with at least one parameter associated with at least one of: the display screen; and other content displayed on the display screen substantially concurrently(“Dynamic target information 117 identification. The modeler may define or select the information elements to be extracted from application 170, e.g., during an application simulation modeling phase. For example, the modeler may enter coordinate values or select, e.g., by clicking, drawing, or highlighting with a mouse, a region or objects on user interface 195 defining dynamic target information 117 to be monitored. In another embodiment, dynamic target information 117 may be identified by a visual or graphic feature, e.g., a color, texture, pattern, contrast setting, design, etc. A corresponding page detection anchor 119 may be generated for each element of dynamic target information 117. The modeler may also set a frequency, time interval, or trigger for iteratively capturing content from a running application 170. Otherwise, content may be captured at a default frequency” in Para.[0050]).
As per claim 6, Kohen teaches a computer program product embodied on a non-transitory computer readable medium comprising computer code for(“a computer-program product for use in conjunction with a computer system, the computer-program product comprising a computer-readable storage medium and a computer-program mechanism embedded therein, the computer-program mechanism including instructions for performing any or all of the method steps described in (1)-(7) above and/or any or all of the method steps set forth in the appended claims or otherwise described above in this application” in Para.[0126]) and the other limitations in the claim 6 has been discussed in the rejection claim 1 and rejected under the same rationale.
As per claim 7, the limitations in the claim 7 has been discussed in the rejection claim 2 and rejected under the same rationale.
As per claim 10, the limitations in the claim 10 has been discussed in the rejection claim 5 and rejected under the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Kohen in view of Revital
Claims 3 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Kohen et al.(USPubN 2010/0138775; hereinafter Kohen) in view of Revital et al.(USPubN 2021/0067842; hereinafter Revital).
As per claim 3, Kohen teaches all of limitation of claim 1.
Kohen is silent about further comprising: using the signature as a watermark.
Revital teaches further comprising: using the signature as a watermark (“The disclosed embodiments relate to systems and methods for watermarking content such as multimedia content. Disclosed embodiments allow for a watermark generation in multimedia content and for detection of the watermarked content. The multimedia content can include a video signal, an audio signal, an image, speech signals, 3D graphical objects, texts, software codes, network streams, XML data, or any other suitable binary or text data. The watermarking may use computer-based models such as machine-learning techniques, such as neural networks” in Para.[0031]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Kohen with the above teachings of Revital in order to enhance user experience beyond that of a conventional video streaming platform.
As per claim 8, the limitations in the claim 8 has been discussed in the rejection claim 3 and rejected under the same rationale.
Kohen in view of Grusd
Claims 4 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Kohen et al.(USPubN 2010/0138775; hereinafter Kohen) in view of Grusd(USPubN 2014/0259056).
As per claim 4, Kohen teaches all of limitation of claim 1.
Kohen is silent about further comprising identifying an advertisement on the display screen, wherein the advertisement is associated with the signature.
Grusd teaches further comprising identifying an advertisement on the display screen, wherein the advertisement is associated with the signature(“Objects in a video are identified, and linked to ecommerce sites and advertisements. A user may select an object and be provided with information about the object. A video stream may be encoded with a separate metadata stream that contains the identity of each object on each frame of the video. The metadata channel may be extracted from a video stream and displayed to a user when the user clicks on the corresponding object. A device may take a video input, analyze it, and recognize the video. The device may look up said video in a database of tagged videos, output as an overlay or separate window to a video display device stream with information about the item. A user interface device may allow a user to select objects on a video screen and select and choose said object for integration with the other components of the system” in Abs).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Kohen with the above teachings of Grusd in order to enhance user experience beyond that of a conventional video streaming platform.
As per claim 9, the limitations in the claim 9 has been discussed in the rejection claim 4 and rejected under the same rationale.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUNGHYOUN PARK whose telephone number is (571)270-1333. The examiner can normally be reached M - Thur 6:00 am - 4 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI Q TRAN can be reached at (571)272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SUNGHYOUN PARK/Examiner, Art Unit 2484
Prosecution Insights