DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 5, 7, 11, 12, 14, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0350150 to Powderly; in view of US 2024/0428473 to Vetencourt et al.
As per claim 1, Powderly et al. teach a wearable electronic device comprising:
a camera (paragraph 103, “the example wearable system 900 shown in FIG. 9 can include three pairs of cameras: a relative wide FOV or passive SLAM pair of cameras arranged to the sides of the user's face, a different pair of cameras oriented in front of the user to handle the stereo imaging process 940 and also to capture hand gestures and totem/object tracking in front of the user's face”);
a display (paragraph 40, “a display configured to present virtual reality, augmented reality, or mixed reality content to a user”);
a memory; and at least one processor including processing circuitry and operatively connected to the camera, the display, and the memory (Fig. 2, 260, paragraphs 51-52, “The display 220 can be operatively coupled, such as by a wired lead or wireless connectivity 250, to a local data processing module 260 which may be mounted in a variety of configurations, such as fixedly attached to the frame 230, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 210 … The local processing and data module 260 may include a hardware processor, as well as digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data.”),
wherein the memory stores instructions that, when executed by the at least one processor, individually or collectively, cause the wearable electronic device to:
obtain key information corresponding to each of a plurality of keys included in a keyboard (paragraph 283, “In some embodiments, receiving keyboard information can include selecting, downloading, or otherwise obtaining a virtual keyboard information file based on information associated with the physical keyboard, such as, for example, a number, shape, size, or layout of keys on the physical keyboard. In other embodiments, the keyboard information can be pre-programmed into the VR/AR/MR system for performing the method 2000.”);
in response to detecting a selection of a key from the keyboard, based on the image captured by the camera, identify key information corresponding to the selected key; and perform an operation corresponding to the selected key based on the identified key information (paragraphs 285-287, “At block 2030, information is received from the hand or finger placement sensor(s). … Hand or finger placement sensor information can include any information obtained by … external sensors (e.g., cameras, … Accordingly, the received sensor information can be indicative of a location of the user's hand(s) relative to the physical keyboard, a location of one or more of the user's fingers relative to the physical keyboard, a set of keys being depressed, a set of keys being touched by a finger of the user's hand, or other information derived from the sensors of the display system … Accordingly, as the user image is regenerated and redisplayed, the user image can appear to the user as animated hands moving and typing on the virtual keyboard consistent with actual movements of the user's hands on the physical keyboard”, paragraph 132, “Where the keyboard 1240 is a virtual keyboard, the HMD 1220 can detect a touch gesture or a swipe gesture on one or more keys for actuation of the one or more keys”).
Powderly et al. do not teach to obtain, from an image captured by the camera, a code image corresponding to a keyboard displayed on an external electronic device, and obtain the key information based on the code image.
Vetencourt et al. suggest to obtain, from an image captured by the camera, a code image corresponding to a keyboard displayed on an external electronic device, and obtain the key information based on the code image (paragraph 48, “In order to pair the XR device 205 and the separate computing device, the pairing module 218 can identify, in the video data, a pattern or a QR code being displayed by the screen of the separate computing device. For example, the separate computing device can display a QR code that can be detected by the XR device 205. The QR code uniquely identifies the separate computing device as a device to be paired to the XR device 205. After identifying the QR code, the pairing module 218 can create a communicative coupling between the XR device 205 and the separate computing device, which allows the separate computing device to send data about the screen currently being displayed on the separate computing device to the XR device 205 for display on the display of the XR device 205”).
It would have been obvious to one of ordinary skill in the art, to modify the device of Powderly et al., so as to obtain, from an image captured by the camera, a code image corresponding to a keyboard displayed on an external electronic device, and obtain the key information based on the code image, such as taught by Vetencourt et al., for the purpose of allowing seamless augmentation of mixed reality environments.
As per claim 4, Powderly and Vetencourt et al. teach the wearable electronic device of claim 1, wherein the instructions cause the wearable electronic device to: obtain an image corresponding to the external electronic device included in the image captured by the camera, while displaying a virtual reality screen through the display; and display the image corresponding to the external electronic device on the virtual reality screen (Powderly, Fig. 19, virtual keyboard 1840 is shown depicting real world keyboard 1820).
As per claim 5, Powderly and Vetencourt et al. teach the wearable electronic device of claim 4, wherein the instructions cause the electronic device to: magnify the image corresponding to the external electronic device; and display the magnified image on the virtual reality screen (Powderly, paragraphs 162-163, “causing the corresponding virtual key to appear closer and/or larger … The virtual key label may be enlarged to help the user to see which key the user is pressing”).
As per claim 7, Powderly and Vetencourt et al. teach the wearable electronic device of claim 1, wherein the instructions cause the wearable electronic device to: display text corresponding to the selected key on the display based on the identified key information (paragraph 130, “the HMD 1220 can render a QWERTY layout for a word processing application where a user types with 10 fingers”).
As per claim 11, Powderly and Vetencourt et al. teach the wearable electronic device of claim 1, further comprising a communication circuit, wherein the communication circuit is in a communication disconnection state with the external electronic device (Powderly, paragraph 334, “The input device may pair with the wearable display system through, for example, a wireless communication protocol (e.g., Bluetooth, ZigBee, etc.)”).
As per claim 12, it comprises similar limitations to those in claim 1 and it is therefore rejected for similar reasons.
As per claim 14, it comprises similar limitations to those in claim 4 and it is therefore rejected for similar reasons.
As per claim 18, it comprises similar limitations to those in claim 11 and it is therefore rejected for similar reasons.
As per claim 19, it comprises similar limitations to those in claim 1 and it is therefore rejected for similar reasons.
Allowable Subject Matter
Claims 2, 3, 6, 8-10, 13, 15-17 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE R SOTO LOPEZ whose telephone number is (571)270-5689. The examiner can normally be reached Monday-Friday, from 8 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at (571) 272-7603. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSE R SOTO LOPEZ/Primary Examiner, Art Unit 2622
Prosecution Insights