DETAILED ACTION
Continued Examination under 37 CFR § 1.114
A request for continued examination under 37 C.F.R. § 1.114, including the fee set forth in 37 C.F.R. § 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 C.F.R. § 1.114, and the fee set forth in 37 C.F.R. § 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 C.F.R. § 1.114. Applicant’s submission filed 23 October 2025 (hereinafter “Reply”) has been entered.
Status of the Claims
Claims 1, 3, 5-8, 21, and 23-28 are currently amended.
Claims 1-31 are pending.
Claims 9-20 are withdrawn.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of 35 U.S.C. § 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claim 23 is rejected under 35 U.S.C. § 112(a) as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention.
Regarding claim 23, there does not appear to be adequate support for the newly amended limitation: “wherein the text entity is located within a parent entity that is located within the retrieved contents, and wherein the entity visual cue is rendered in response to a system cursor being over the parent entity.”
Applicant has indicated that support for the amendment can be found in paragraphs 57 and 65-66 of the specification. Reply 9. However, the referenced portion of the specification does not disclose rendering an entity visual cue in response to a system cursor being over a parent entity. Rather, the specification appears to disclose highlighting a nested text entity in response to a discovery cursor moving over a parent image entity visual cue. Specification ¶ 67.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7, 21, 26, and 30-31 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Sharifi et al., US 9,762,651 B1.
Regarding claim 1, Sharifi discloses a method comprising:
Retrieving a contents [sic] of a display buffer of an active window of an application. Sharifi teaches a screen capture application capturing the current screen (i.e., active window) by copying or reading the contents of a device’s “frame buffer.” Sharifi 13:22-24.
Identifying a text entity within the retrieved contents. Sharifi teaches recognizing text items and finding candidate entities based on the recognized text items. Sharifi 17:34-47, fig. 5 (steps 510, 515).
Determining that the text entity is usable by an OS or third-party provided feature. Sharifi teaches identifying actionable content. Actionable content may include text entities that fit a template (e.g., street addresses). Sharifi 20:30-35. Each item of actionable content may be associated with a type of action (e.g., view on map). Id. 8:34-61, 30:35-36.
Rendering an overlay layer over the active window of the application, wherein the overlay layer includes an entity visual cue that appears next to, underneath, around, over, or partially over the text entity, wherein the overlay layer is rendered with participation of the application. Sharifi teaches an annotation engine that generates a user-interrace layer that offers additional information and/or actions across multiple applications. Sharifi 5:62-66. The additional information and/or actions may be accessed via visual cues that appear over text entities. Id. fig. 6B, 22:38-43. The visual cues are provided by the system across all applications. Id. 20:39-52. The annotation engine may be part of the OS. Id. 6:1-6.
Receiving, at the overlay layer, an indication of a selection of the entity visual cue; and invoking the OS or third-party provided feature. Sharifi teaches a selectable visual cue that initiates an associated action. Sharifi 22:52-64.
Regarding claim 2, which depends on claim 1, Sharifi discloses wherein invoking the OS or third-party feature comprises providing the OS or third-party provided feature with the text entity. Sharifi 22:9-18.
Regarding claim 3, which depends on claim 1, Sharifi discloses wherein the OS or third-party feature modifies the contents of the display buffer without participation of the application. Sharifi teaches adding an annotation to the current screen in the display buffer. Sharifi 8:14-19, 14:29-32.
Regarding claim 4, which depends on claim 1, Sharifi discloses wherein the contents of the display buffer is retrieved and the text entity is identified in response to previously unanalyzed contents of the display buffer coming into view. Sharifi teaches capturing the current screen (i.e., active window) by copying or reading the contents of a device’s frame buffer. Sharifi 13:22-24. The screen may be captured when the device transitions from displaying one application to another application. Id. 13:26-31.
Regarding claim 5, which depends on claim 1, Sharifi discloses wherein the entity visual cue is one of a plurality of entity visual cues that highlight the contents of the display buffer, wherein the plurality of entity visual cues are at least one of a plurality of entity visual cue types, and wherein entity visual cues having a same entity visual cue type are selected from the plurality of entity visual cues for display. Sharifi fig. 6B, 22:38-52.
Regarding claim 7, which depends on claim 1, Sharifi teaches wherein the contents of the active window of the application is retrieved from the display buffer after the contents has been drawn the content to the display buffer, wherein the display buffer comprises memory in a computing device storing image data that is read by a graphics processing unit for display. Sharifi teaches retrieving content by reading a frame buffer. Sharifi 13:18-48. A frame buffer is a portion of RAM containing a bitmap that drives a display. Modern video cards contain a frame buffer.
Claim 21 is drawn to a system that implements the method recited in claim 1. Accordingly, this claim is rejected for substantially the same reasons as indicated in the above rejection of the corresponding claim.
Claim 26 is drawn to instructions stored in a medium that implement the method recited in claim 1. Accordingly, this claim is rejected for substantially the same reasons as indicated in the above rejection of the corresponding claim.
Regarding claim 30, which depends on claim 26, Sharifi teaches wherein the contents of the display buffer includes [sic] non-image based content. Sharifi 13:22-48.
Regarding claim 31, which depends on claim 30, Sharifi teaches wherein the contents of the display buffer includes [sic] application-generated content. Sharifi 13:22-48.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6, 23, and 25 are rejected under 35 U.S.C. § 103 as being unpatentable over Sharifi et al., US 9,762,651 B1, in view of Bray et al., US 2012/0083294 A1.
Regarding claim 6, which depends on claim 1, Sharifi teaches wherein the entity visual cue is one of a plurality of entity visual cues derived from the contents of the display buffer, wherein two or more of the plurality of entity visual cues … are highlighted in the overlay layer. Sharifi teaches highlighting multiple visual cues. Sharifi fig. 6B, 22:38-52. In Sharifi, the visual cues are highlighted irrespective of their distance from a cursor. Nevertheless, Bray teaches visual cues that are located within a defined distance of a cursor are highlighted. Specifically, in Bray, a cursor placed over an entity (i.e., at a distance of 0) causes the entity visual cue to be highlighted. Bray ¶¶ 26-29.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Sharifi’s process of rendering entity visual cues with Bray’s process of rendering a highlighted entity visual cue. Such a modification would improve usability by allowing a user to better recognize that the entity visual cues are selectable.
Regarding claim 23, which depends on claim 21, Sharifi teaches wherein the text entity is located within a parent entity that is located within the retrieved contents. Sharifi teaches recognizing a text entity (e.g., street address) located within a parent entity (listing for an Italian restaurant). Sharifi 20:46-65, fig. 6B.
Sharifi does not disclose, but Bray teaches wherein the entity visual cue is rendered in response to a system cursor being over the parent entity. In Bray, a cursor placed over an entity causes an entity visual cue to be highlighted. Bray ¶¶ 26-29.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Sharifi’s process of rendering entity visual cues with Bray’s process of rendering a highlighted entity visual cue. Such a modification would improve usability by allowing a user to better recognize that the entity visual cues are selectable.
Regarding claim 25, which depends on claim 21, Sharifi does not disclose, but Bray teaches highlight[ing] a portion of the entity visual cue closest to a cursor. Bray teaches highlighting an entity visual cue closest to a cursor. Bray ¶¶ 26, 28, fig. 4B.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Sharifi’s process of rendering entity visual cues with Bray’s process of rendering a highlighted entity visual cue. Such a modification would improve usability by allowing a user to better recognize that the entity visual cues are selectable.
Claim 8 is rejected under 35 U.S.C. § 103 as being unpatentable over Sharifi et al., US 9,762,651 B1, in view of Ahuja et al., US 10,878,471 B1.
Regarding claim 8, which depends on claim 1, Sharifi alone does not disclose, but the combination of Sharifi with Ahuja renders obvious wherein the contents of the active window of the application comprises a visible portion of the active window and a non-visible portion of the active window. Sharifi teaches a screen capture application capturing the current screen (i.e., active window) by copying or reading the contents of a device’s “frame buffer.” Sharifi 13:22-24. Sharifi does not specifically mention non-visible portions of an active window. However, the use of an application window interface metaphor (including the concept of an active window) is prevalent throughout the personal computing industry. See, e.g., Ahuja 7:13-29. The application window interface metaphor allows for repositioning windows so that portions of the application window are moved from a non-visible area to a visible (i.e., displayed) area.
Accordingly, it would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Sharifi’s process of retrieving displayed content with an application window interface metaphor, including an active window displaying content. Such a modification would yield the predictable result of retrieving the content on which the user is currently focused.
Claim 22 is rejected under 35 U.S.C. § 103 as being unpatentable over Sharifi et al., US 9,762,651 B1, in view of Bai et al., US 2015/0095855 A1.
Regarding claim 22, which depends on claim 21, Sharifi alone does not disclose, but the combination of Sharifi with Bai renders obvious wherein the OS or third-party feature translates the text entity to a different language. Bai teaches a feature that translates text into a different language. Bai ¶ 18, figs. 2-3.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Sharifi’s process of invoking an OS or third-party feature with Bai’s feature of translating a text entity. Such a modification would increase the utility of the system by providing additional useful information efficiently and without disrupting the flow of the user. See Bai ¶ 1.
Allowable Subject Matter
Claims 24 and 27-29 contain allowable subject matter.
Claims 24 and 27-29 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
The prior rejections of the claims under §§ 112(a) and (b) have been withdrawn based on the corresponding amendment to the claims.
Regarding claim 1, Applicant alleges that Sharifi “makes no mention of displaying an overlay layer over the active window of the application, rendering the overlay layer without participation of the application, or receiving a selection of an entity visual cue at the overlay layer, as recited.” Reply 11. However, as explained above, Sharifi anticipates these limitations. Frist, Sharifi teaches an annotation engine that generates a user-interrace layer that offers additional information and/or actions across multiple applications. Sharifi 5:62-66. Second, the annotation engine may be part of the OS. Id. 6:1-6. Third, Sharifi teaches a selectable visual cue that initiates an associated action. Id. 22:52-64. Accordingly, Applicant’s allegation is without merit.
Regarding claim 26, Applicant provides no cogent argument. See Reply 11.
Regarding claim 6, Applicant argues that Bray alone does not teach/suggest the newly amended limitations. Reply 12. The arguments are moot because they do not apply to the combination of references being used in the current rejection to teach the newly amended/added limitations. Specifically, Sharifi teaches highlighting multiple visual cues. Sharifi fig. 6B, 22:38-52. Applicant is referred to the above detailed rejections for further explanation.
Regarding claim 23, Applicant merely alleges that “[n]one of the cited references teach or suggest nested entities.” Reply 13. The claim does not recite “nested entities.” Accordingly, Applicant’s allegation is irrelevant.
Regarding claim 24, Applicant’s arguments with respect to the novelty/nonobviousness have been fully considered and are persuasive. Accordingly, the prior rejection of this claim under §§ 102/103 has been withdrawn.
Regarding claim 25, Applicant merely alleges that “[n]one of the cited references teach or suggest highlighting a portion of a visual cue based on cursor position.” Reply 14. The amended claim does not recite “highlighting a portion of a visual cue based on cursor position.” Accordingly, Applicant’s allegation is irrelevant.
Conclusion
Although particular portions of the prior art may have been cited in support of the rejections, the specified citations are merely representative of the teachings. Other passages and figures in the cited prior art may apply. Accordingly, Applicant should consider the entirety of the cited prior art for potentially teaching all or part of the claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Asher D Kells whose telephone number is (571)270-7729. The examiner can normally be reached Mon. - Fri., 8 a.m. - 4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Asher D. Kells
Primary Examiner
Art Unit 2171
/Asher D Kells/ Primary Examiner, Art Unit 2171