DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to claims in application 18/342,531 filed on 6/27/2023.
The instant application claims benefit to provisional application #63/459,995 with a priority date of 4/17/2023.
The Pre-Grant publication # 2017/0282014 is published on 10/5/2017.
Claims 1-20 are pending.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1,3-8,11-14,16,19 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by US 2017/0364167 B2 to Ribeiro et al. ( Ribeiro).
Claim 1. Ribeiro teaches a haptic stylus to provide haptic feedback to a user when using the haptic stylus to form letters on a writing surface (Para 0003 producing a writing experience with tactile feedback), comprising:
a controller(Fig.5A,5B process for controlling the actuator in the stylus depending on position information );
vibrator elements (Fig.1 elements 135 Actuator can be, for example, a piezoelectric actuator, an electromagnetic actuator, a linear resonant actuator LRA, an eccentric rotating mass motor ERM, or some other type of vibrating mechanism);
a computer readable storage medium comprising program instructions that, when executed by the controller, cause operations, the operations (Para 0103) comprising:
receiving a directive to write a selected letter (Para 0064 writing information; Para 0079 selected pattern of tactile feedback); and
generating vibration commands for the selected letter to control the vibrator elements to generate haptic feedback to guide the haptic stylus held by the user to form the selected letter on the writing surface when the user moves the haptic stylus on the writing surface (Para 0004, 0005 generated vibration command and tactile haptic feedback when the stylus is being moved across one area of the display screen, but not provide tactile feedback when the stylus is being moved across another area of the display screen determining its position and velocity on a writing surface).
Claim 3. Ribeiro teaches the haptic stylus of claim 1, further comprising: a microphone, wherein the program instructions are further executed to perform operations, the operations comprising: receiving audio input from the microphone; and processing the audio input to translate the audio input to the directive to write the selected letter ( Para 0142 audio interface headphones input are microphone).
Claim 4. Ribeiro teaches the haptic stylus of claim 1 in communication with a remote computing device, further comprising: a wireless transmitter to communicate with the remote computing device; and a camera, wherein the program instructions are further executed to perform operations, the operations comprising: receiving, from the remote computing device via the wireless transmitter, the directive to write the selected letter; capturing, by the camera, an image of the selected letter written with the haptic stylus in response to detecting completing writing the selected letter; and transmitting the captured image to the remote computing device (Para 0144 camera configuration for I/O operations).
Claim 5. Ribeiro teaches the haptic stylus of claim 1, further comprising: a camera, wherein the program instructions are further executed to perform operations, the operations comprising: processing a first image, captured by the camera, of the writing surface to determine a page and line layout on the writing surface; generating a scaled path to form the selected letter within the page and line layout, wherein the vibration commands are generated from the scaled path to guide the haptic stylus along the scaled path within a line of the page and line layout; and detecting from a second image, captured by the camera, the haptic stylus positioned on the line of the writing surface, wherein the vibration commands are executed in response to detecting the haptic stylus positioned on the line (Para 0051 linear guide to haptic stylus along scaled path as in figure 3A for a position determined within a line of a page; Para 0137 camera) .
Claim 6. Ribeiro teaches the haptic stylus of claim 5, wherein the operations further comprise: processing a third image, captured by the camera, to determine whether the haptic stylus is positioned a predetermined distance from a boundary of the page and line layout; and generating feedback to the user of the haptic stylus indicating that an end of the line has been reached in response to determining that the haptic stylus is positioned the predetermined distance from the boundary (Para 0064, 0182, 0188 predetermined features for haptic stylus position that may be depicting a page of notebook ; Para 0138 image processing ).
Claim 7. Ribeiro teaches the haptic stylus of claim 5, wherein the line comprises a first line, and wherein the operations further comprise: processing a third image, captured by the camera, to determine whether the user has positioned the haptic stylus on a second line of the page and line layout in response to determining that the haptic stylus was moved to an end of the first line; and generating feedback to the user of the haptic stylus indicating that the haptic stylus has not been positioned on the second line in response to determining that the haptic stylus has not been positioned on the second line vibration commands are executed in response to detecting the haptic stylus positioned on the line (Para 0051 linear guide to haptic stylus along scaled path for a position as determined by a line of a page; Para 0137 camera image).
Claim 8. Ribeiro teaches the haptic stylus of claim 1, wherein the operations further comprise: detecting that the user is moving the haptic stylus in a direction away from a path the haptic feedback is guiding the haptic stylus; and generating feedback to the user of the haptic stylus indicating that the user is not properly directing the haptic stylus in response to the detecting the user is moving the haptic stylus in a direction away from the path the haptic feedback is guiding the haptic stylus (Para 0023 stylus uses an electrostatic sensor to detect capacitively-coupled currents from signal while direction away from the path can be haptic feedback guiding a stylus) .
Claim 11. Ribeiro teaches a computer program product to provide haptic feedback to a user when using a haptic stylus to form letters on a writing surface, wherein the haptic stylus has vibrator elements, wherein the computer program product comprises a computer readable storage medium having program instructions embodied therewith that when executed cause operations, the operations comprising: receiving a directive to write a selected letter; and generating vibration commands for the selected letter to control the vibrator elements to generate haptic feedback to guide the haptic stylus held by the user to form the selected letter on the writing surface when the user moves the haptic stylus on the writing surface (Para 0003 producing a writing experience with tactile feedback; Fig.5A,5B process for controlling the actuator in the stylus depending on position information; Fig.1 elements 135 Actuator can be, for example, a piezoelectric actuator, an electromagnetic actuator, a linear resonant actuator LRA, an eccentric rotating mass motor ERM, or some other type of vibrating mechanism; Para 0103 computer-readable storage media; Para 0064 writing information; Para 0079 selected pattern of tactile feedback; Para 0004, 0005 generated vibration command and tactile haptic feedback when the stylus is being moved across one area of the display screen, but not provide tactile feedback when the stylus is being moved across another area of the display screen determining its position and velocity on a writing surface).
Claim 12 . Ribeiro teaches the computer program product program product of claim 11, wherein the haptic stylus includes a microphone, wherein the operations further comprise: receiving audio input from the microphone on the haptic stylus; and processing the audio input to translate the audio input to the directive to write the selected letter (Para 0142 audio interface headphones input are microphone).
Claim 13. . Ribeiro teaches the computer program product of claim 11, wherein the haptic stylus includes a camera, wherein the operations comprising: processing a first image, captured by the camera, of the writing surface to determine a page and line layout on the writing surface; generating a scaled path to form the selected letter within the page and line layout, wherein the vibration commands are generated from the scaled path to guide the haptic stylus along the scaled path within a line of the page and line layout; and detecting from a second image, captured by the camera, the haptic stylus positioned on the line of the writing surface, wherein the vibration commands are executed in response to detecting the haptic stylus positioned on the line (Ribeiro: Para 0033,0051 cognitive models; Para0070, 0078, 0102 writing areas and program updates; Para 0199 operation instructions and force sensors responses to change).
Claim 14. The computer program product of claim 11, wherein the operations further comprise: detecting that the user is moving the haptic stylus in a direction away from a path the haptic feedback is guiding the haptic stylus; and generating feedback to the user of the haptic stylus indicating that the user is not properly directing the haptic stylus in response to the detecting the user is moving the haptic stylus in a direction away from the path the haptic feedback is guiding the haptic stylus (Para 0023 stylus uses an electrostatic sensor to detect capacitively-coupled currents from signal while direction away from the path can be haptic feedback guiding a stylus).
Claim 16. A method for providing haptic feedback to a user when using a haptic stylus to form letters on a writing surface, comprising: receiving a directive to write a selected letter; and generating vibration commands for the selected letter to control vibrator elements to generate haptic feedback to guide the haptic stylus held by the user to form the selected letter on the writing surface when the user moves the haptic stylus on the writing surface (Para 0003 producing a writing experience with tactile feedback; Fig.5A,5B process for controlling the actuator in the stylus depending on position information; Fig.1 elements 135 Actuator can be, for example, a piezoelectric actuator, an electromagnetic actuator, a linear resonant actuator LRA, an eccentric rotating mass motor ERM, or some other type of vibrating mechanism; Para 0103 computer-readable storage media; Para 0064 writing information; Para 0079 selected pattern of tactile feedback; Para 0004, 0005 generated vibration command and tactile haptic feedback when the stylus is being moved across one area of the display screen, but not provide tactile feedback when the stylus is being moved across another area of the display screen determining its position and velocity on a writing surface) .
Claim 19. . Ribeiro teaches the computer method of claim 16, further comprising: detecting that the user is moving the haptic stylus in a direction away from a path the haptic feedback is guiding the haptic stylus; and generating feedback to the user of the haptic stylus indicating that the user is not properly directing the haptic stylus in response to the detecting the user is moving the haptic stylus in a direction away from the path the haptic feedback is guiding the haptic stylus (Para 0023 stylus uses an electrostatic sensor to detect capacitively-coupled currents from signal while direction away from the path can be haptic feedback guiding a stylus).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2,9,10,15,17,18,20 are rejected under 35 U.S.C. 103 as being unpatentable over Patent Application Publication Number US 2017/0364167 B2 to Ribeiro et al. ( Ribeiro) in view of Patent Number US 11086415 B1 Alfano et al. (Alfano).
Claim 2. Ribeiro teaches the haptic stylus of claim 1, wherein the writing surface is not explicitly comprises of paper that could further comprising of a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface. Alfano, however, teaches writing surface comprising of paper that could further have a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface (Fig. 3 element 240 a plurality of textured items e.g., notebook paper, construction paper for inking layer). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate writing surface comprising of paper that could further have a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface, as taught by Alfano, into the haptic stylus of Ribeiro , in order to additionally emulate a plurality of textured items.
Claim 9. Ribeiro in combination with Alfano teaches the haptic stylus of claim 1, further comprising: a camera; a cognitive model, implementing machine learning, wherein the operations further comprise: capturing an image, by the camera, of a handwritten letter formed by the user moving the haptic stylus guided by the haptic feedback in response to completing the generating the haptic feedback to guide the user to form the selected letter; inputting the image of the handwritten letter and a ground truth image of the selected letter into the cognitive model to output indication the handwritten letter sufficiently approximates the selected letter; and generating feedback to the user of the haptic stylus in response to determining that the handwritten letter does not sufficiently approximate the selected letter (Ribeiro: Para 0033,0051 cognitive models; Para 0070, 0078, 0102 writing areas and program updates; Para 0199 operation instructions and force sensors responses to changes; Alfano: Fig.2A a user may select a device configuration i.e., indicating a writing device to emulate from a plurality of writing devices e.g., ballpoint pen, fountain pen, pencil, chalk, and the like, provide input to display for a handwritten letter via haptic pen, and receive a haptic response from haptic pen emulating the selected writing device ).
Claim 10. Ribeiro in combination with Alfano teaches the haptic stylus of claim 9, wherein the operations further comprise: capturing a predetermined number of images of the handwritten letter determined to sufficiently approximate the selected letter; and saving a last of the captured predetermined number of images to use as the ground truth image of the selected letter to input into the cognitive model to determine whether the handwritten letter sufficiently approximates the selected letter ( Ribero: Para 0070 processor 115 then proceeds to operation 510 to obtain updated position information; Para 0070, 0078, 0102 writing areas and program updates; Para 0199 operation instructions and force sensors responses to changes; Alfano: Fig.2A a writing device to emulate from a plurality of writing devices e.g., ballpoint pen, fountain pen, pencil, chalk, and the like working on handwritten letter sufficiently approximates the selected letter).
Claim 15. Ribeiro in combination with Alfano teaches computer program product of claim 11, wherein the haptic stylus includes a camera and a cognitive model implementing machine learning, wherein the operations capturing an image, by the camera, of a handwritten letter formed by the user moving the haptic stylus guided by the haptic feedback in response to completing the generating the haptic feedback to guide the user to form the selected letter; inputting the image of the handwritten letter and a ground truth image of the selected letter into the cognitive model to output indication the handwritten letter sufficiently approximates the selected letter; and generating feedback to the user of the haptic stylus in response to determining that the handwritten letter does not sufficiently approximate the selected letter Ribero: Para 0070 processor 115 then proceeds to operation 510 to obtain updated position information ( Ribeiro: Para 0033,0051 cognitive models; Para 0070, 0078, 0102 writing areas and program updates; Para 0199 operation instructions and force sensors responses to changes; Alfano: Fig.2A a writing device to emulate from a plurality of writing devices e.g., ballpoint pen, fountain pen, pencil, chalk, and the like working on handwritten letter sufficiently approximates the selected letter.
Claim 17. Ribeiro teaches method of claim 16, wherein the writing surface do not comprises paper, further comprising: a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface. Alfano, however, teaches writing surface comprising of paper that could further have a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface (Fig. 3 element 240 a plurality of textured items e.g., notebook paper, construction paper for inking layer). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate writing surface comprising of paper that could further have a tip of the haptic stylus including an ink dispenser to dispense ink on the paper as the user moves the tip along the writing surface, as taught by Alfano, into the haptic stylus of Ribeiro , in order to additionally emulate a plurality of textured items.
Claim 18. Ribeiro in combination with Alfano teaches method of claim 16, further comprising: processing a first image, captured by a camera, of the writing surface to determine a page and line layout on the writing surface; generating a scaled path to form the selected letter within the page and line layout, wherein the vibration commands are generated from the scaled path to guide the haptic stylus along the scaled path within a line of the page and line layout; and detecting from a second image, captured by the camera, the haptic stylus positioned on the line of the writing surface, wherein the vibration commands are executed in response to detecting the haptic stylus positioned on the line ( Ribeiro: Para 0051 linear guide to haptic stylus along scaled path as in figure 3A for a position determined within a line of a page; Para 0137 camera; Alfano: Fig.2A,2B a writing device to emulate from a plurality of writing devices e.g., ballpoint pen, fountain pen, pencil, chalk, and texture database for information like working on letter sufficiently approximates the selected letter)
.
Claim 20. The method of claim 16, further comprising: capturing an image of a handwritten letter, by a camera, formed by the user moving the haptic stylus guided by the haptic feedback in response to completing the generating the haptic feedback to guide the user to form the selected letter; inputting the image of the handwritten letter and a ground truth image of the selected letter into a cognitive model to output indication the handwritten letter sufficiently approximates the selected letter; and generating feedback to the user of the haptic stylus in response to determining that the handwritten letter does not sufficiently approximate the selected letter (Ribeiro: Para 0033,0051 cognitive models with ground truth image input; Para0070, 0078, 0102 writing areas and program updates; Para 0199 operation instructions and force sensors responses to change and feedback; Alfano: Fig.2A,2B a writing device to emulate from a plurality of writing devices e.g., ballpoint pen, fountain pen, pencil, chalk, and texture database for information like working on letter sufficiently approximates the selected letter).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 12517595 B2 Dekel; Shoham et al.
Touch-sensitive input region for electronic stylus
US 10671186 B2 Ribeiro; Flavio Protasio et al. (camera)
Autonomous haptic stylus
US 9792038 B2 Gil; Erez Kikin et al.
Feedback via an input device and scribble recognition
US 11086415 B1 Alfano; Keith M. et al.
Haptic pen for an information handling system
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SADARUZ ZAMAN whose telephone number is (571)270-3137. The examiner can normally be reached M-F 9am to 5pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai can be reached at (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.Z/Examiner, Art Unit 3715 July 25, 2026
/XUAN M THAI/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights