Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 05/29/2026 have been entered. Claims 1-17 remain pending in the application. Applicant’s amendment to claim 17 has overcome the 35 USC § 112(b) rejection previous set forth in the final office action mailed 01/28/2026. Applicant’s amendments to the claims have not overcome the previously set forth 35 USC § 103 rejections and examiner’s response to the applicant’s arguments are provided below. Applicant’s amendments to the claims contain a minor informality which necessitates a claim objection described below.
Claim Objections
Claims 1 and 13 objected to because of the following informalities: There is a typographical error on the second page of claim 1, line 4, "to and", examiner is interpreting this as a mistake where the “and” was meant to be crossed out. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Soli (WO 2020/226846 A1) in view of Xia (US 2014/0363083 A1).
Regarding claim 1, Soli teaches
A computing device (Figure 1 #100, paragraph [0006]) for performing a modification on digital content of a document (Figures 12A-12AP, #1200, paragraph [0080]), comprising:
a display interface configured to display first digital content as handwritten input and second digital content as typeset input (paragraph [00167]);
an identification module configured to identify the handwritten input as digital ink and the typeset input as one or more typeset objects (paragraphs [00347]-[00351]);
a mode type selection module configured to select a mode type, wherein said selected mode type is pixel-mode or object-mode (Figure 12G #1212, 1212a, 1212b, paragraphs [00356], [00363], & [00380]);
an input surface configured to detect a modification gesture wherein one or more boundaries of the modification gesture are intersecting with at least part of the first and second digital content (Figures 2, 12K & 12R #1200, paragraph [0080]);
a content selection module configured to select, at least partially, the first and second digital content according to the selected mode type (Figures 12G-12I, #1204 #1212, paragraph [00372]), the content selection module comprising:
an object selection module configured, when the selected mode type is the object-mode, to:
recognize as ink objects at least the digital ink contained within the one or more boundaries of the detected modification gesture (Figures 12M-12W #1208a, #1208b, Figures 12AJ-12AP #1208c, 8F-8H #834, paragraph [00172],[00210],[00239]); and
select one or more objects of the ink objects and the typeset objects, wherein the selected one or more object are intersecting, at least partially, with the one or more boundaries of the detected modification gesture (Figures 12P-12W, 12AM-12AP, #1208, 1208a, 1208b) wherein the object selection module is configured to segment input strokes of the ink objects identified within the one or more boundaries of the detected modification gesture (Figures 12k & 12L, paragraph [00375] – pixel erase operation segments strokes “e.g. splits in two”) and
a pixel selection module configured, when the selected mode type is the pixel-mode, to:
interpret as typeset ink the one or more typeset objects of the digital content, intersecting, at least partially, with the one or more boundaries of the detected modification gesture (Figures 12I-12L & 12AF-12AJ, #1212a, paragraphs [00366] & [00371]); and
select one or more pixels of the digital ink and the typeset ink, wherein the selected one or more pixels are contained within the one or more boundaries of the detected modification gesture (Figures 12K-12L, 12AC-12AI, paragraphs [0064],[00366],[00371]); and
a content modification module configured to modify the selected digital content, the modification module comprising:
an object modification module configured to modify the selected one or more objects of the digital content (paragraph [00389]); and
a pixel modification module configured to modify the selected one or more pixels of the digital content (paragraph [00355]).
Soli fails to teach segment input strokes of the ink objects identified within the one or more boundaries of the detected modification gesture to re-render recognized elements as typeset ink on the display interface of the computing device (emphasis added);
However, Xia teaches to segment input strokes (Fig. 9A #912, 22A #2206, 22B #2228, paragraphs [0011], [0026], [0061]) of the ink objects identified within the one or more boundaries of the detected modification gesture to re-render recognized elements as typeset ink on the display interface of the computing device (Figures 11B-11K, 9B #922 [0106], [0211]-[0213] [0253], [0262]). Xia describes a method of handwriting recognition that segments the input of a user. This segmentation is repeated for additional user input as the character is further completed. As the user writes the character or word, Xia displays possible recognized characters. Xia is considered analogous to the claimed invention as it is in the same field of handwritten digital content. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to implement the handwritten text recognition of Xia, which shows improved recognition accuracy, with the teachings of Soli.
Method claim(s) 13 is/are drawn to the method of using the corresponding apparatus
claimed in claim(s) 1. Therefore, the method claim(s) 13 correspond(s) to the apparatus claim(s) 1, and is/are rejected for the same reasons of obviousness as used above.
Regarding claim 2, Soli view of Xia teaches the computing device of claim 1. Soli further teaches comprising: a modification type selection module configured to select a modification type, wherein the content modification module is configured to modify the selected digital content according to the selected modification type (Figures 2, 12G-12I, #112, #1212, paragraphs [0080] "modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and/or video images.").
Regarding claim 3, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein the one or more objects comprises characters, words, paragraphs or shapes (paragraph [00373).
Regarding claim 4, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein the digital content is interpreted as a plurality of segments and wherein the pixel selection module is configured to select one or more segments of the digital content intersecting within the one or more boundaries of the detected modification gesture, said pixel modification module being configured to modify the selected one or more segments (paragraph [00383], figures 12AH-12AL).
Regarding claim 5, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein it configured to format the selected content (paragraphs [00380],[00388]-[00391], figures 12P-12S, 12AL-12AP) and the pixel modification module is configured to change a rendering the selected one or more pixels when the selected mode type is object-mode and pixel-mode, respectively (paragraph [00380], [00389], [00355], figures 12K-12L, figures 12AH-12AL). Soli describes formatting (e.g. by highlighting) input, and modifying object content when the object mode is selected. Soli further describes changing the rendering of one or more pixels (by removing selected pixels).
Regarding claim 6, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein the object modification module is configured to erase the selected one or more objects (paragraph [00361], figures 12P-12S, 12AL-12AP) and the pixel modification module is configured to erase the selected one or more pixels, when the selected mode type is object-mode and pixel-mode, respectively (paragraph [00355], figures 12K-12L, 12AH-12AI).
Regarding claim 7, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein the mode type selection module is configured to select the mode type through interaction with a tool comprising at least one of a keyboard, UI buttons or a menu (Figures 12A-12AP, paragraph [00380]). The drawing palette described in Soli is a menu consisting of UI buttons.
Regarding claim 8, Soli view of Xia teaches the computing device of claim 1. Soli further teaches wherein the mode type selection module is configured to detect the selected mode type according to a selecting gesture detected on the input surface (paragraph [00148], [00353], [00391]). Soli describes detection of gestures for various inputs. It further describes using the result of this detected gesture input to select a mode of drawing or eraser mode (object or pixel).
Regarding claim 9, Soli view of Xia teaches the computing device of claim 8. Soli further teaches wherein the mode type selection module is configured to detect the selected mode type by assessing at least one characteristic of the selecting gesture (paragraphs [0064],[00148]). Soli describes various characteristics which are assessed for gesture determination such as speed, position or tap, swipe or drag motions.
Regarding claim 10, Soli view of Xia teaches the computing device of claim 9. Soli further teaches wherein said at least one characteristic of the selecting gesture comprises a first characteristic defined as a speed of the selecting gesture (paragraphs [00148],[00381]).
Method claim(s) 14 is/are drawn to the method of using the corresponding apparatus
claimed in claim(s) 10. Therefore, the method claim(s) 14 correspond(s) to the apparatus claim(s) 10, and is/are rejected for the same reasons of obviousness as used above.
Regarding claim 11, Soli view of Xia teaches the computing device of claim 9. Soli further teaches wherein said at least one characteristic of the selecting gesture comprises a second characteristic defined as a direction of the selecting gesture (paragraphs [00143]-[00145]).
Regarding claim 15, Soli view of Xia teaches the method of claim 13. Soli further teaches wherein: when the selected mode type is object-mode, modifying the selected digital content includes formatting or editing, such as highlighting or erasing the one or more selected objects (paragraphs [00388]-[00391]); and when the selected mode type is pixel-mode, modifying the selected digital content includes coloring or setting to a background color of the selected one or more pixels (paragraph [00380]).
Soli fails to teach, wherein the object modification module is configured to format the selected one or more objects.
However, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Soli to specify the formatting of content is done by the object modification module and done to the selected one or more objects. This is because the formatting is highlighting and thus it would make sense to perform this function as part of the object modification process.
Regarding claim 12, Soli view of Xia teaches the computing device of claim 8. Soli further teaches wherein the selecting gesture is the modification gesture (paragraph [0005], [00329], [00330], [00353]). Soli describes the use of both modification gestures and selecting gestures. Soli fails to teach that they are the same gesture. However, it would have been obvious to one of ordinary skill in the art to modify Soli to align the modification and selection gestures of Soli in order to improve the invention and allow for simultaneous modification and selection gesturing.
Method claim(s) 16 is/are drawn to the method of using the corresponding apparatus
claimed in claim(s) 12. Therefore, the method claim(s) 16 correspond(s) to the apparatus claim(s) 12, and is/are rejected for the same reasons of obviousness as used above.
Regarding claim 17, Soli view of Xia teaches the computing device of claim 3. Xia further teaches wherein the identification module is configured to recognize the segmented handwritten input strokes of the modified selected handwritten characters, words or paragraphs (Figs. 19A-D, paragraph [0300]) and generate modified characters, words or paragraphs (Fig. 22A, paragraph [0297]). Xia describes a method of segmenting input in order to recognize characters of handwritten input. Xia allows a user to edit the previously given input and generate the edited input as characters, words or sentences.
Response to Arguments
Applicant's arguments filed 05/29/2026 have been fully considered but they are not persuasive. Applicant argues on page 9 with respect to claim 1 that Xia does not remedy the described deficiencies of Soli. Applicant states “Xia appears to teach or suggest a standalone handwriting input interface having a dedicated handwriting input area (area 804) into which a user writes new strokes from scratch. Further, Xia's segmentation therefore operates prospectively on new, user-written strokes entering a blank, purpose-built input panel - not retrospectively on already-rendered ink that has been selected by a modification gesture within a mixed-content document.”. Examiner does not find this argument to be persuasive. The input of Xia is not necessarily from scratch and does teach modification of the typeset ink that is rendered in the recognition result area. This can be seen in Figures 11B and 11C where a user modifies the input by adding new input strokes which updates an already rendered preview area (1118). Additionally while Xia is not exclusively relied on for the limitation of segment input strokes of the ink objects identified within the one or more boundaries of the detected modification gesture, the limitation is taught. Xia segments users input strokes identified within the boundaries of a recognized character, as well as different input strokes representing different characters. The boundaries of the these characters, when an additional input is entered, can be considered boundaries of the detected modification gesture as the addition of strokes is a modification gesture. Applicant specifically argues against this saying that “Xia's recognition pipeline appears to be triggered by the act of writing new input, not by the detection of a modification gesture on existing document content”, examiner respectfully disagrees with this argument because under broadest reasonable interpretation, an action that changes the existing document content (for example adding a stroke that changes the recognized character), is a modification. Examiner also reminds applicant that Xia is not exclusively relied upon to reject this limitation and no argument has been provided against the provided citations of Soli. For the above reasons claims 1 and 13 remain rejected over Soli in view of Xia.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aidan W McCoy whose telephone number is (571)272-5935. The examiner can normally be reached 8:00 AM-5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN W MCCOY/Examiner, Art Unit 2611
/TAMMY GODDARD/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights