DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 1-18 are presented for examination.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 102(a)(1)/102(a)(2) as being anticipated by Chen (PG Pub NO 2018/0004478).
As in claim 1, Chen discloses An information prompt method, comprising:
receiving a first input, wherein the first input is used to enable a see-through function of a wearable device; (Fig 4c-5D and 6B step 675 and Par 0056, 0069) discloses user input to enable see-through function of a wearable device
and displaying target prompt information on an outer display screen of the wearable device in response to the first input, wherein the target prompt information is used to indicate that the see- through function of the wearable device is enabled. (Fig 4-5 and Par 0056, 0058, 0059) discloses displaying target prompt information (users eye/face) on an outer display screen (Fig 3 display 311) of the wearable device (HMD102); [0056] In some implementations, the non-HMD user may be able to view the HMD user's changing expressions during the time the HMD user is viewing/interacting with content rendering on the first display screen 310 for a period of time and then switch to viewing the HMD user's interaction with the content currently rendering on the first display screen 310 for a second period of time before switching back to the viewing the HMD user's expressions. In such implementations, the content provided to the second display screen 311 may be switched between images of the changing expressions of the HMD user and the images of the VR scene content. The switching in the content rendered on the second display screen 311 may be time-based or defined by the HMD user. The respective content (image of the user or images of the VR scene content) may be viewed by the non-HMD user without requiring the content to be displayed on a display screen that is outside of the HMD and without requiring the HMD user to remove his HMD.
PNG
media_image1.png
288
284
media_image1.png
Greyscale
As in claim 2, Chen discloses The information prompt method according to claim 1, wherein the target prompt information comprises at least one of the following:
text prompt information; a partial face image or a complete face image of a virtual object; a face image of a wearer of the wearable device, wherein the face image comprises a partial face image or a complete face image of the wearer; or a scene image, wherein the scene image is a two-dimensional scene image or a three- dimensional scene image of a real scene in which the wearable device is located. (Fig 4C-4F, 5C) discloses external display of HMD displaying target prompt information such as a face image of a wearer of the wearable device wherein the face image comprises a partial face image or scene image.
As in claim 3, Chen discloses The information prompt method according to claim 2, wherein when the target prompt information comprises the partial face image of the wearer of the wearable device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer; a face outer contour in the partial face image is the same as a face outer contour of the target face area of the wearer; and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the wearable device. (Fig 4C-4F, 5C and Par 0054, 0066) discloses the target prompt information comprises the partial face image of the wearer of the wearable device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer face outer contour in the partial face image is the same as a face outer contour and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the wearable device; [0054] The rendering includes superimposing the image overlay data over the existing image of the portion of the facial features of the user that is rendered on the second display screen 311, by matching the contours of the features captured in the two different images (image captured by the external camera 108 and the image overlay data).
As in claim 4, Chen discloses Chen discloses The information prompt method according to claim 2, wherein when the target prompt information comprises the face image of the wearer of the wearable device, the displaying target prompt information on an outer display screen of the wearable device comprises: updating the face image when it is detected that expression action information of the wearer changes. (Fig 4C-4F and Par 0041, 0066) discloses displaying face image of the wearer of the wearable device on the external display and updating the face image when it is detected that expression action information of the wearer changes. [0041] For example, if a user's eyes are determined to be looking in a specific direction, then the video rendering for that direction can be prioritized or emphasized, such as by providing greater detail or faster updates in the region where the user is looking. [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311.
PNG
media_image2.png
423
762
media_image2.png
Greyscale
As in claim 5, Chen discloses The information prompt method according to claim 4, wherein before the updating the face image when it is detected that expression action information of the wearer changes, the information prompt method further comprises: collecting a facial feature point of the wearer; and when it is detected that positions of at least some facial feature points of the wearer change, determining that the expression action information of the wearer changes. (Fig 4D-4F and Par 0059) discloses [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311. FIG. 4E illustrates another example emotion, a sadness emotion, captured by the inward facing camera and rendered on the second display screen 311 and FIG. 4F illustrates an image of the HMD user showing a surprise emotion. Each of the different emotions are detected by the inward facing camera or gaze detection camera and is provided for rendering the second display screen 311, so that the non-HMD user watching the HMD user can see the change in expression defining the different emotions that the HMD user goes through while interacting or viewing content on the first display screen of the HMD.
As in claim 6, Chen discloses The information prompt method according to claim 2, wherein when the target prompt information comprises the scene image, before the displaying target prompt information on an outer display screen of the wearable device, the information prompt method further comprises: collecting scene information of the real scene in which the wearable device is located; and generating the two-dimensional scene image or the three-dimensional scene image of the real scene based on the scene information; and wherein the displaying target prompt information on an outer display screen of the wearable device comprises: displaying the two-dimensional scene image or the three-dimensional scene image on the outer display screen of the wearable device. (Fig 5D and Par 0064) [0028] a camera 108 can be configured to capture image of the interactive environment in which the user 100 is located. These captured images can be analyzed to determine the location and movements of the user 100, the HMD 102, and the controller 104. [0064] FIG. 5D illustrates an expanded view of a VR scene that is rendered on the second display screen 311. As mentioned earlier, the VR scene content may be rendered on the entire second display screen 311 or in a portion of the second display screen 311. The HMD user is provided with a wider view of the VR scene 410 defined in the VR space as the display mechanisms provided in the HMD allow the HMD user to have a broader field of view. Of course, due to a limited size of the second display screen 311, only a portion 410a of the VR scene 410 viewed by the HMD user may be presented for rendering on second display screen 311, in some implementations. FIG. 5D illustrates one such implementation, wherein a portion 410a of the VR scene that is rendered on the second display screen 311 corresponds to the HMD user's gaze, while the HMD user is presented with a much broader view of the VR space 410. In some other implementations, the VR space 410 that is presented in the first display screen 310 may be scaled down to a size of the second display screen 311 and rendered at the second display screen 311.
As in claim 7, Chen discloses An electronic device, comprising a processor and a memory storing a program or an instruction that is capable of running on the processor, wherein the program or the instruction, when executed by the processor, causes the electronic device to perform:
receiving a first input, wherein the first input is used to enable a see-through function of the electronic device; (Fig 4c-5D and 6B step 675 and Par 0056, 0069) discloses user input to enable see-through function of a wearable device
and displaying target prompt information on an outer display screen of the electronic device in response to the first input, wherein the target prompt information is used to indicate that the see- through function of the electronic device is enabled. (Fig 4-5 and Par 0056, 0058, 0059) discloses displaying target prompt information on an outer display screen (Fig 3 display 311) of the wearable device (HMD102); [0056] In some implementations, the non-HMD user may be able to view the HMD user's changing expressions during the time the HMD user is viewing/interacting with content rendering on the first display screen 310 for a period of time and then switch to viewing the HMD user's interaction with the content currently rendering on the first display screen 310 for a second period of time before switching back to the viewing the HMD user's expressions. In such implementations, the content provided to the second display screen 311 may be switched between images of the changing expressions of the HMD user and the images of the VR scene content. The switching in the content rendered on the second display screen 311 may be time-based or defined by the HMD user. The respective content (image of the user or images of the VR scene content) may be viewed by the non-HMD user without requiring the content to be displayed on a display screen that is outside of the HMD and without requiring the HMD user to remove his HMD.
PNG
media_image1.png
288
284
media_image1.png
Greyscale
As in claim 8, Chen discloses The electronic device according to claim 7, wherein the target prompt information comprises at least one of the following: text prompt information; a partial face image or a complete face image of a virtual object; a face image of a wearer of the electronic device, wherein the face image comprises a partial face image or a complete face image of the wearer; or a scene image, wherein the scene image is a two-dimensional scene image or a three- dimensional scene image of a real scene in which the electronic device is located. (Fig 4C-4F, 5C) discloses external display of HMD displaying target prompt information such as a face image of a wearer of the wearable device wherein the face image comprises a partial face image or scene image.
As in claim 9, Chen discloses The electronic device according to claim 8, wherein when the target prompt information comprises the partial face image of the wearer of the electronic device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer; a face outer contour in the partial face image is the same as a face outer contour of the target face area of the wearer; and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the electronic device. (Fig 4C-4F, 5C and Par 0054, 0066) discloses the target prompt information comprises the partial face image of the wearer of the wearable device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer face outer contour in the partial face image is the same as a face outer contour and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the wearable device; [0054] The rendering includes superimposing the image overlay data over the existing image of the portion of the facial features of the user that is rendered on the second display screen 311, by matching the contours of the features captured in the two different images (image captured by the external camera 108 and the image overlay data).
As in claim 10, Chen discloses The electronic device according to claim 8, wherein when the target prompt information comprises the face image of the wearer of the electronic device, the displaying target prompt information on an outer display screen of the electronic device comprises: updating the face image when it is detected that expression action information of the wearer changes. (Fig 4C-4F and Par 0041, 0066) discloses displaying face image of the wearer of the wearable device on the external display and updating the face image when it is detected that expression action information of the wearer changes. [0041] For example, if a user's eyes are determined to be looking in a specific direction, then the video rendering for that direction can be prioritized or emphasized, such as by providing greater detail or faster updates in the region where the user is looking. [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311.
PNG
media_image2.png
423
762
media_image2.png
Greyscale
As in claim 11, Chen discloses The electronic device according to claim 10, wherein before the updating the face image when it is detected that expression action information of the wearer changes, the program or the instruction, when executed by the processor, causes the electronic device to further perform: collecting a facial feature point of the wearer; and when it is detected that positions of at least some facial feature points of the wearer change, determining that the expression action information of the wearer changes. (Fig 4D-4F and Par 0059) discloses [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311. FIG. 4E illustrates another example emotion, a sadness emotion, captured by the inward facing camera and rendered on the second display screen 311 and FIG. 4F illustrates an image of the HMD user showing a surprise emotion. Each of the different emotions are detected by the inward facing camera or gaze detection camera and is provided for rendering the second display screen 311, so that the non-HMD user watching the HMD user can see the change in expression defining the different emotions that the HMD user goes through while interacting or viewing content on the first display screen of the HMD.
As in claim 12, Chen discloses The electronic device according to claim 8, wherein when the target prompt information comprises the scene image, before the displaying target prompt information on an outer display screen of the electronic device, the program or the instruction, when executed by the processor, causes the electronic device to further perform: collecting scene information of the real scene in which the electronic device is located; and generating the two-dimensional scene image or the three-dimensional scene image of the real scene based on the scene information; and wherein the displaying target prompt information on an outer display screen of the electronic device comprises: displaying the two-dimensional scene image or the three-dimensional scene image on the outer display screen of the electronic device. (Fig 5D and Par 0064) [0028] a camera 108 can be configured to capture image of the interactive environment in which the user 100 is located. These captured images can be analyzed to determine the location and movements of the user 100, the HMD 102, and the controller 104. [0064] FIG. 5D illustrates an expanded view of a VR scene that is rendered on the second display screen 311. As mentioned earlier, the VR scene content may be rendered on the entire second display screen 311 or in a portion of the second display screen 311. The HMD user is provided with a wider view of the VR scene 410 defined in the VR space as the display mechanisms provided in the HMD allow the HMD user to have a broader field of view. Of course, due to a limited size of the second display screen 311, only a portion 410a of the VR scene 410 viewed by the HMD user may be presented for rendering on second display screen 311, in some implementations. FIG. 5D illustrates one such implementation, wherein a portion 410a of the VR scene that is rendered on the second display screen 311 corresponds to the HMD user's gaze, while the HMD user is presented with a much broader view of the VR space 410. In some other implementations, the VR space 410 that is presented in the first display screen 310 may be scaled down to a size of the second display screen 311 and rendered at the second display screen 311.
PNG
media_image3.png
556
745
media_image3.png
Greyscale
As in claim 13, Chen discloses A non-transitory readable storage medium storing a program or an instruction, wherein the program or the instruction, when executed by a processor (Fig 7), causes the processor to perform:
receiving a first input, wherein the first input is used to enable a see-through function of a wearable device; (Fig 4c-5D and 6B step 675 and Par 0056, 0069) discloses user input to enable see-through function of a wearable device
and displaying target prompt information on an outer display screen of the wearable device in response to the first input, wherein the target prompt information is used to indicate that the see- through function of the wearable device is enabled. (Fig 4-5 and Par 0056, 0058, 0059) discloses displaying target prompt information on an outer display screen (Fig 3 display 311) of the wearable device (HMD102); [0056] In some implementations, the non-HMD user may be able to view the HMD user's changing expressions during the time the HMD user is viewing/interacting with content rendering on the first display screen 310 for a period of time and then switch to viewing the HMD user's interaction with the content currently rendering on the first display screen 310 for a second period of time before switching back to the viewing the HMD user's expressions. In such implementations, the content provided to the second display screen 311 may be switched between images of the changing expressions of the HMD user and the images of the VR scene content. The switching in the content rendered on the second display screen 311 may be time-based or defined by the HMD user. The respective content (image of the user or images of the VR scene content) may be viewed by the non-HMD user without requiring the content to be displayed on a display screen that is outside of the HMD and without requiring the HMD user to remove his HMD.
PNG
media_image1.png
288
284
media_image1.png
Greyscale
As in claim 14, Chen discloses The non-transitory readable storage medium according to claim 13, wherein the target prompt information comprises at least one of the following: text prompt information; a partial face image or a complete face image of a virtual object; a face image of a wearer of the wearable device, wherein the face image comprises a partial face image or a complete face image of the wearer; or a scene image, wherein the scene image is a two-dimensional scene image or a three- dimensional scene image of a real scene in which the wearable device is located. (Fig 4C-4F, 5C) discloses external display of HMD displaying target prompt information such as a face image of a wearer of the wearable device wherein the face image comprises a partial face image or scene image.
As in claim 15, Chen discloses The non-transitory readable storage medium according to claim 14, wherein when the target prompt information comprises the partial face image of the wearer of the wearable device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer; a face outer contour in the partial face image is the same as a face outer contour of the target face area of the wearer; and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the wearable device. (Fig 4C-4F, 5C and Par 0054, 0066) discloses the target prompt information comprises the partial face image of the wearer of the wearable device, a facial feature point in the partial face image matches a facial feature point in a target face area of the wearer face outer contour in the partial face image is the same as a face outer contour and the target face area is a face area in which a face of the wearer is obstructed by the outer display screen of the wearable device; [0054] The rendering includes superimposing the image overlay data over the existing image of the portion of the facial features of the user that is rendered on the second display screen 311, by matching the contours of the features captured in the two different images (image captured by the external camera 108 and the image overlay data).
As in claim 16, Chen discloses The non-transitory readable storage medium according to claim 14, wherein when the target prompt information comprises the face image of the wearer of the wearable device, the displaying target prompt information on an outer display screen of the wearable device comprises: updating the face image when it is detected that expression action information of the wearer changes. (Fig 4C-4F and Par 0041, 0066) discloses displaying face image of the wearer of the wearable device on the external display and updating the face image when it is detected that expression action information of the wearer changes. [0041] For example, if a user's eyes are determined to be looking in a specific direction, then the video rendering for that direction can be prioritized or emphasized, such as by providing greater detail or faster updates in the region where the user is looking. [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311.
PNG
media_image2.png
423
762
media_image2.png
Greyscale
As in claim 17, Chen discloses The non-transitory readable storage medium according to claim 16, wherein before the updating the face image when it is detected that expression action information of the wearer changes, the program or the instruction, when executed by the processor, causes the processor to further perform: collecting a facial feature point of the wearer; and when it is detected that positions of at least some facial feature points of the wearer change, determining that the expression action information of the wearer changes. (Fig 4D-4F and Par 0059) discloses [0059] When one or more facial features change, such changes reflecting different emotions of the HMD user are rendered on the second display screen 311, as illustrated in FIGS. 4D-4F. For example, the emotion expressed by the HMD user has changed from plain emotion to an anger emotion and this change is reflected in the image rendered on the second display screen 311. FIG. 4E illustrates another example emotion, a sadness emotion, captured by the inward facing camera and rendered on the second display screen 311 and FIG. 4F illustrates an image of the HMD user showing a surprise emotion. Each of the different emotions are detected by the inward facing camera or gaze detection camera and is provided for rendering the second display screen 311, so that the non-HMD user watching the HMD user can see the change in expression defining the different emotions that the HMD user goes through while interacting or viewing content on the first display screen of the HMD.
As in claim 18, Chen discloses The non-transitory readable storage medium according to claim 14, wherein when the target prompt information comprises the scene image, before the displaying target prompt information on an outer display screen of the wearable device, the program or the instruction, when executed by the processor, causes the processor to further perform: collecting scene information of the real scene in which the wearable device is located; and generating the two-dimensional scene image or the three-dimensional scene image of the real scene based on the scene information; and wherein the displaying target prompt information on an outer display screen of the wearable device comprises: displaying the two-dimensional scene image or the three-dimensional scene image on the outer display screen of the wearable device. (Fig 5D and Par 0064) [0028] a camera 108 can be configured to capture image of the interactive environment in which the user 100 is located. These captured images can be analyzed to determine the location and movements of the user 100, the HMD 102, and the controller 104. [0064] FIG. 5D illustrates an expanded view of a VR scene that is rendered on the second display screen 311. As mentioned earlier, the VR scene content may be rendered on the entire second display screen 311 or in a portion of the second display screen 311. The HMD user is provided with a wider view of the VR scene 410 defined in the VR space as the display mechanisms provided in the HMD allow the HMD user to have a broader field of view. Of course, due to a limited size of the second display screen 311, only a portion 410a of the VR scene 410 viewed by the HMD user may be presented for rendering on second display screen 311, in some implementations. FIG. 5D illustrates one such implementation, wherein a portion 410a of the VR scene that is rendered on the second display screen 311 corresponds to the HMD user's gaze, while the HMD user is presented with a much broader view of the VR space 410. In some other implementations, the VR space 410 that is presented in the first display screen 310 may be scaled down to a size of the second display screen 311 and rendered at the second display screen 311.
PNG
media_image3.png
556
745
media_image3.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENYAM KETEMA whose telephone number is (571)270-7224. The examiner can normally be reached 9AM-5PM (M-F).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Temesghen Ghebretinsae can be reached at 571-272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENYAM KETEMA/Primary Examiner, Art Unit 2626
Prosecution Insights