DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/03/2026 has been entered.
Response to Amendment
1 This action is in response to the amendment filed on 06/03/2026. Claims 1, 8, and 15 have been amended. Claims 1-3, 5-10, 12-17, and 19-20 remain rejected.
Response to Arguments
2 Applicant’s arguments with respect to claims 1, 8, 15 filed on 06/03/2026, with respect to the rejection under 35 U.S.C. § 103 regarding that the prior art does not teach the following but not limited to “…entering a photographic capture mode in response to the trigger event, wherein, in the photographic capture mode, a virtual camera is displayed in a first color within the virtual environment and is configured to capture an image using the virtual camera in response to a first facial movement trigger event…” and “entering an orientation mode in which the virtual camera is displayed in a second color within the virtual environment and is configured to adjust a location, a distance, and an orientation of the virtual camera”. This argument has been considered, but are moot due to new grounds of rejection, with further clarifications regarding previously used prior art. More specifically, the applicant argued that prior art from Vandonkelaar does not teach the mentioned “first and second color(s)” based on the display of a photographic capture mode and an orientation mode, and instead relates to a spatial proximity. Although it may be true that Vandonkelaar teaches the specific color based on a specific spaces and tracking to avoid certain types of conflict, the specific color methods are still in a type of display in a virtual environment utilizing a type of capture and orientation as referenced below ([Abstract] reciting “One or more color cameras are used to view one or more spaces, and track positions and orientations of players and other objects according to the attached marker lights”; [0007] reciting “…if a second colored light, from among the at least one colored light, having the first color comes within a specified distance of the first colored light, the control system is further configured to change the assignment of color of one of the first colored light or the second colored lights to a color different that the first color.”; [0021] reciting “…analyze the vectors to determine a best fit for a position of each calibration marker light, and convert the detected orientation of the calibration object into a position and orientation respective to at each of the at least one color camera for use thenceforth in operation of the virtual reality.”).
3 Regarding claims 2-3, 5-7, 9-10, 12-17, and 19-20, they directly/indirectly depend on independent claims 1, 8, and 15 respectively. Applicant does not argue anything other than independent claims 1, 8, and 15. The limitations in those claims, in conjunction with combination, was previously established as explained.
Claim Rejections - 35 USC § 103
4 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6 Claim(s) 1, 3, 5-6, 8, 10, 12-13, 15, 17, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Black et al. (WO 2023156984 A1) in view of Novelli et al. (US 10842430 B1) and Vandonkelaar et al. (US 20170277940 A1).
7 Regarding claim 1, Black teaches a processor-implemented method ([Page 5, Lines 21-23; Page 6, Lines 1-2] reciting “In another aspect, a computer system comprising one or more computers having at least one processor and memory is programmed to perform any of the methods described herein.”), the method comprising:
the user is interacting with a virtual environment ([Page 7, Lines 21-23] reciting “In some embodiments, interactions such as conversation and collaboration between users in the virtual environments along with interactions with objects within the virtual environment are enabled.”);
entering a photographic capture mode, wherein, in the photographic capture mode, a virtual camera is displayed in a within the virtual environment and is configured to capture an image using the virtual camera ([Page 5, Lines 9-12] reciting “The computer system captures a video stream from the perspective of the virtual camera on the predetermined path. The video stream includes video of the user graphical representations in the positions arranged in the geometry.”)
entering an orientation mode in which the virtual camera is displayed in within the virtual environment and is configured to adjust a location, a distance, and an orientation of the virtual camera relative to the user based on a second movement , and
capturing one or more images using the virtual camera, while in the photographic capture mode ([Page 20, Lines 22-24] reciting “In the example illustrated in FIG. 3, users A-D access the virtual environment 312 through their corresponding client devices 310, wherein each user A-D has at least one camera 316 capturing, e.g., video data and/or image data…”), at the location, the distance, and the orientation ([Page 5, Lines 3-4] reciting “In some embodiments, the location or orientation of the movable virtual camera is configured to be controlled by at least one of the client devices.”; [Page 7, Lines 5-7] reciting “The virtual camera moves on a predetermined path that maintains a distance between the virtual camera and the positions arranged in the geometry for the user graphical representations.”)
8 Black does not explicitly teach detecting a trigger event performed by a user while the user is interacting with a virtual environment; entering a photographic capture mode in response to the trigger event, wherein, in the photographic capture mode, a virtual camera is displayed in a first color within the virtual environment and is configured to capture an image using the virtual camera in response to a first facial movement trigger event; entering an orientation mode in which the virtual camera is displayed in a second color within the virtual environment and is configured to adjust a location, a distance, and an orientation of the virtual camera relative to the user based on a second movement trigger event by the user, wherein the distance is confirmed by the user through a preconfigured series of facial actions and determined by a user eye size compared to a baseline user eye size and a time duration for which the user eye size is not equal to the baseline user eye size; and capturing one or more images using the virtual camera, while in the photographic capture mode, at the location, the distance, and the orientation based on the first facial movement trigger event.
9 Noveli teaches detecting a trigger event performed by a user while the user is interacting with a virtual environment; entering a photographic capture mode in response to the trigger event, wherein, in the photographic capture mode, a virtual camera is displayed in a within the virtual environment and is configured to capture an image using the virtual camera in response to a first facial movement trigger event; entering an orientation mode in which the virtual camera is displayed in trigger event by the user ([Page 17; Column 7, Lines 36-47] reciting “The system 100 can alert the user 104 if the user 104 is too close or too far from the screen of the mobile device. For example, the system 100 can trigger a distance-mitigation action. The distance mitigation action can be similar to the eye fatigue mitigation action or a different action. For example, eye-fatigue mitigation could cause the screen to blur, and the distance mitigation action could cause the screen to fade, with fade increasing in magnitude the closer the user 104 holds the mobile device to the pupil of the eye until turning dark when the distance from the device to the pupil is less than 0.6, 0.75, 0.8, 0.9, 1.0, or 1.1 times the width of the screen.”), wherein the distance is confirmed by the user through a preconfigured series of facial actions ([Page 15; Column 3, Lines 16-19] reciting “…and/or determining the eye fatigue mitigation action is warranted is based on a plurality of criteria including blink rate and one or more of color of the sclera, blink velocity, or distance of the user from the specified region.”) and determined by a user eye size compared to a baseline user eye size ([Page 14; Column 2, Lines 56-61] reciting “…estimate a distance from the camera to the pupil; calculate a distance from the pupil to the specified region based on the distance from the camera to the pupil; determine the pupil is too close to the specified region based on the distance from the pupil to the specified region being less than a predetermined value”) and a time duration for which the user eye size is not equal to the baseline user eye size ([Page 22; Column 2, Lines 22-26 & 31-35] reciting “…calculate a length of time during the time period that the gaze direction of the user is directed to a specified region corresponding to a display; determine that the length of time the gaze direction of the user is directed to the specified region exceeds a threshold value within the time period…determine a first blink rate of the user for a first duration based on a first portion of the plurality of visible light images; determine a second blink rate of the user for a second duration based on a second portion of the plurality of visible light images;”); and capturing one or more images using the virtual camera, while in the photographic capture mode, at the location, the distance, and the orientation based on the first facial movement trigger event (see above regarding the trigger event).
10 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black) to incorporate the teachings of Novelli to provide a method that can obtain the distance by using a type of facial actions (in this case blinking) utilizing a type of trigger event, as well as the distance to be determined by the eye size and the various time durations based on the distance modifications taught by Black. Doing so would account for residual eye fatigue as stated by Novelli ([Page 15; Column 4, Lines 8-9]).
11 Black in view of Novelli does not explicitly teach entering a photographic capture mode in response to the trigger event, wherein, in the photographic capture mode, a virtual camera is displayed in a first color within the virtual environment and is configured to capture an image using the virtual camera in response to a first facial movement trigger event; entering an orientation mode in which the virtual camera is displayed in a second color within the virtual environment and is configured to adjust a location, a distance, and an orientation of the virtual camera relative to the user based on a second movement trigger event by the user…
12 Vandonkelaar teaches entering a photographic capture mode in response to the trigger event, wherein, in the photographic capture mode, a virtual camera is displayed in a first color within the virtual environment and is configured to capture an image using the virtual camera in response to a first facial movement trigger event; entering an orientation mode in which the virtual camera is displayed in a second color within the virtual environment and is configured to adjust a location, a distance, and an orientation of the virtual camera relative to the user based on a second movement trigger event by the user ([Abstract] reciting “One or more color cameras are used to view one or more spaces, and track positions and orientations of players and other objects according to the attached marker lights”; [0007] reciting “…if a second colored light, from among the at least one colored light, having the first color comes within a specified distance of the first colored light, the control system is further configured to change the assignment of color of one of the first colored light or the second colored lights to a color different that the first color.”; [0021] reciting “…analyze the vectors to determine a best fit for a position of each calibration marker light, and convert the detected orientation of the calibration object into a position and orientation respective to at each of the at least one color camera for use thenceforth in operation of the virtual reality.”; [Claim 1] reciting “A system for operating a virtual reality environment including at least one space, the system comprising: at least one color camera configured to view the at least one space…”)…
13 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli) to incorporate the teachings of Vandonkelaar to provide a method that can determine different colors during types of modifications of the specific colors, while utilizing the virtual cameras and the virtual environments taught by Black in view of Novelli. Doing so would allow the cameras to be more synchronized, which ensures the plurality of cameras to be properly coordinated as stated by Vandonkelaar ([0040] recited).
14 Regarding claim 3, Black in view of Novelli and Vandonkelaar teaches the method of claim 1 (see claim 1 rejection above), wherein the location is determined based on a tracking of user head rotations in the virtual environment (Black; [Page 18, Lines 17-20] reciting “In some embodiments, the virtual camera location or orientation within the virtual environment can be adjusted based on user input, such as mouse input, keyboard input, controller input, touchscreen input, eye-and-head-tilting data, or head-rotation data, or a combination thereof.”) .
15 Novelli as previously mentioned in claim 1 can further teach the limitations, specifically …and a confirmation by the user through a preconfigured series of facial actions ([Page 15; Column 3, Lines 16-19] reciting “…and/or determining the eye fatigue mitigation action is warranted is based on a plurality of criteria including blink rate and one or more of color of the sclera, blink velocity, or distance of the user from the specified region.”).
16 As explained in the rejection of claim 1, the obviousness for combining of the confirmation by the user through a preconfigured series of facial actions of Novelli into Black (and by extension Vandonkelaar) is provided above.
17 Regarding claim 5, Black in view of Novelli and Vandonkelaar teaches the method of claim 1 (see claim 1 rejection above),
18 Novelli from claim 1 can further teach the limitations, specifically wherein detecting the user eye size is smaller than a baseline user eye size triggers an increase in the distance of the virtual camera from the user or a user avatar, and wherein detecting the user eye size is larger than the baseline user eye size triggers a decrease in the distance of the virtual camera from the user or the user avatar ([Page 14; Column 2, Lines 53-61] reciting “…based on the tracking the size of the pupil, where the determining that the eye fatigue mitigation action is warranted is based on the determining the size of the pupil has increased; estimate a distance from the camera to the pupil; calculate a distance from the pupil to the specified region based on the distance from the camera to the pupil; determine the pupil is too close to the specified region based on the distance from the pupil to the specified region being less than a predetermined value…”).
19 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli and Vandonkelaar) to incorporate additional teachings of Novelli to provide a method that decides an increase or a decrease of the distance of a camera based on the user’s eye size in a specific way, utilizing the eye size determination methods provided by Black in view of Novelli and Vandonkelaar. Doing so would account for residual eye fatigue as stated by Novelli ([Page 15; Column 4, Lines 8-9]).
20 Regarding claim 6, Black in view of Novelli and Vandonkelaar teaches the method of claim 1 (see claim 1 rejection above), wherein the orientation is determined based on a tracking of a user head rotation in the virtual environment (Black; [Page 18, Lines 17-20] reciting “In some embodiments, the virtual camera location or orientation within the virtual environment can be adjusted based on user input, such as mouse input, keyboard input, controller input, touchscreen input, eye-and-head-tilting data, or head-rotation data, or a combination thereof.”) .
21 Novelli as previously mentioned in claim 1 can further teach the limitations, specifically …and a confirmation by the user through a preconfigured series of facial actions ([Page 15; Column 3, Lines 16-19] reciting “…and/or determining the eye fatigue mitigation action is warranted is based on a plurality of criteria including blink rate and one or more of color of the sclera, blink velocity, or distance of the user from the specified region.”).
22 As explained in the rejection of claim 1, the obviousness for combining of the confirmation by the user through a preconfigured series of facial actions of Novelli into Black (and by extension Vandonkelaar) is provided above.
23 Claims 8 and 15 has similar limitations as of claim 1, therefore it is rejected under the same rationale as claim 1.
24 Claims 10 and 17 has similar limitations as of claim 3, therefore it is rejected under the same rationale as claim 3.
25 Claims 12 and 19 has similar limitations as of claim 5, therefore it is rejected under the same rationale as claim 5.
26 Claims 13 and 20 has similar limitations as of claim 6, therefore it is rejected under the same rationale as claim 6.
27 Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Black et al. (WO 2023156984 A1) in view of Novelli et al. (US 10842430 B1) and Vandonkelaar et al. (US 20170277940 A1) as of claim 1, further in view of Dehais et al. (US 20210142566 A1).
28 Regarding claim 2, Black in view of Novelli and Vandonkelaar teaches the method of claim 1 (see claim 1 rejection above),
29 Novelli from claim 1 can further teach the limitations, specifically wherein the virtual camera is initially displayed at a preconfigured location within the virtual environment, displayed at a preconfigured distance from the user or a user avatar ([Page 16; Column 5, Lines 26-31] reciting “In certain embodiments, a camera, configured to detect visible light, can be used to acquire a plurality of images. These images can be analyzed to track a location of a pupil of a user in relation to the camera over a time period (e.g., FIGS. 1-4). A gaze direction of the user can be determined based on the tracked location of the pupil.”; [Page 23; Column 19, Lines 5-10] reciting “Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how an initial configuration is obtained.”)…
30 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli and Vandonkelaar) to incorporate additional teachings of Novelli to provide a method that tells if the camera is displayed at a certain configuration including a type of display utilizing the camera display provided by Black in view of Novelli and Vandonkelaar. Doing so would account for residual eye fatigue as stated by Novelli ([Page 15; Column 4, Lines 8-9]).
31 Black in view of Novelli and Vandonkelaar does not explicitly teach … and oriented toward a geometric center of a face of the user or the user avatar.
32 Dehais teaches an oriented toward a geometric center of a face of the user or the user avatar ([0037] reciting “representing in a virtual space of the face of the individual by a virtual model generated beforehand, the virtual model, called an avatar, being positioned and oriented with respect to a virtual camera thanks to the real parameters determined beforehand…”).
33 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli and Vandonkelaar) to incorporate the teachings of Dehais to provide a way to orient the virtual camera to the face of the user or user avatar, utilizing the virtual cameras taught from Black in view of Novelli and Vandonkelaar. Doing so would allow a realistic positioning of a virtual frame as stated by Dehais ([Claim 2] recited).
34 Claims 9 and 16 has similar limitations as of claim 2, therefore it is rejected under the same rationale as claim 2.
35 Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Black et al. (WO 2023156984 A1) in view of Novelli et al. (US 10842430 B1) and Vandonkelaar et al. (US 20170277940 A1) as of claims 1 and 6, further in view of Nadler et al. (US 20160203646 A1) and Yeoh et al. (US 20180275410 A1).
36 Regarding claim 7, Black in view of Novelli and Vandonkelaar teaches the method of claim 6 (see claims 1 and 6 rejections above), but does not explicitly teach wherein the orientation is further determined by replicating an angular distance and a direction of the user head rotation, as measured from an original orientation of a user head to a current orientation of the user head after the user head rotation, at the location of the virtual camera.
37 Nadler teaches wherein the orientation is further determined by replicating an angular distance and a direction of the user head rotation ([0063] reciting “Optionally, the images further represent two distinctly different perspectives of a scene, alone or in combination with the offset images, created at an adjustable virtual camera distance.”; [0079] reciting “Moreover, according to an embodiment of the present disclosure, the user-wearable device includes an in-built motion and rotation sensor arrangement that is operable to sense a position and an angular orientation and/or turning angle of a head of the user when the user-wearable device is worn on the head.”).
38 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli and Vandonkelaar) to incorporate the teachings of Nadler to provide a way to determine the orientation using angular turning while utilize Black’s virtual cameras and head rotation methods. Doing so would allow the operation to compute hardware of the user devices to modulate camera and projection matrices as stated by Nadler ([0073] recited).
39 Black in view of Novelli, Vandonkelaar, and Nadler does not explicitly teach as measured from an original orientation of a user head to a current orientation of the user head after the user head rotation, at the location of the virtual camera.
40 Yeoh teaches as measured from an original orientation of a user head to a current orientation of the user head after the user head rotation, at the location of the virtual camera ([0296] reciting “Although a head-tracked virtual camera may be created and/or dynamically repositioned for each eye or eye socket based on information regarding the current position and orientation of the viewer's head, the position and orientation of such a head-tracked virtual camera may neither depend upon the position nor the orientation of each eye of the viewer relative to the respective eye socket of the viewer or the viewer's head.”; [0306] reciting “While the head-tracked render perspective 5010 may remain static or relatively static throughout the first and second time-sequential stages of FIGS. 26A-26D, in transitioning from the first stage to the second stage, the AR system may function to adjust the orientation of a fovea-tracked virtual camera in render space based on the change in gaze of the viewer's eye 210 from the first stage to the second stage. That is, the AR system may replace or reorient the fovea-tracked virtual camera as employed in the first stage to provide the fovea-tracked render perspective…”).
41 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Black in view of Novelli, Vandonkelaar, and Nadler) to incorporate the teachings of Yeoh to provide a way to determine the specific measurements between two different orientations that was obtained by Black’s virtual cameras and head-rotation methods. Doing so would allow the function to maintain the head-tracked virtual camera at the same position and orientation as described by Yeoh ([0306] recited).
42 Claim 14 has similar limitations as of claim 7, therefore it is rejected under the same rationale as claim 7.
Conclusion
43 The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Delamont et al. (US 20200368616 A1) teaches a type of trigger event that can be used for facial movement, as well as having a preconfigured location for their cameras.
Ryuma et al. (US 20220317769 A1) teaches an augmented device method for pausing based on eye movement with a head-wearable apparatus.
44 Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY TRAN LE whose telephone number is (571)272-5680. The examiner can normally be reached Mon-Thu: 7:30am-5pm; First Fridays Off; Second Fridays: 7:30am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHNNY T LE/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
