DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/04/2024 and 02/06/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 8-11, 14, 15 are rejected under 35 U.S.C. 103 as being unpatentable over KOBLIN et al. (US 2022/0395741 A1) in view of YOON et al. (US 2021/0279951 A1).
RE claim 1, Koblin teaches a computer-implemented method running a training program for a user wearing a VR headset, where the training program is modified based on the user’s activity [abstract]. Koblin teaches a method of generating and displaying interactive 3D content performed by a device, the method comprising:
(a)
receiving sensing data on a user's body from at least one sensor attached to the user's body;
Fig. 1, real-time user training virtual reality environment (100) [0152]. Person (102) uses headset (104) to view and interact with a virtual environment [0152]. Sensors (230) in the headset (104) and/or other sensors in the user’s environment may track the VR user’s actual movements [0156]. The user (102) may also wear or be connected to one or more physiological sensors (120) where the data is sent to the headset (104) [0162]. Data from the headset (104) may be provided in real-time from the VR fitness application (220) to the training program (310) [0157]. The movement/tracking mechanism(s) (314) may receive user data (116) and may determine and/or approximate, from that data, the user’s actual movements in the user’s real-world space/environment (112) [0189]. The user’s movements may be given relative to a 3D coordinate system (114) the user’s real-world space (112) [0189]. If the user also has handheld devices (110), the movement/tracking mechanism (314) may determine movement of one or both hands in the user’s real-world space (112). The user’s headset (104) may provide the user’s actual 3D coordinates in the real-world space (112) to the programs (310) on the system (108) [0189].
(b)
obtaining feature data related to the user's body and movement based on the sensing data;
The sensors (230) track the user’s actual movements (e.g., head movements, etc.) (said movement data) [0156]. Additionally, the one or more physiological sensors (120) can provide e.g., heartrate, temperature, sweat levels, breathing, etc. (said feature data) [0162].
(c)
obtaining a content level or difficulty matching the user by inputting feature data and 3D (dimension) content into an artificial intelligence (AI) model;
The VR fitness app (220) may present the user (102) with an activity or exercise routine for the user to perform [0179]. The monitoring/tracking mechanism(s) (224) may monitor and/or track the user’s performance of the activity, preferably in real-time, based on the manner in which the user is performing the activity [0180]. The VR fitness app (220) includes a real-time feedback system that may modify a user’s activity based on how the user is performing [0181]. The movement/tracking mechanism(s) (314) may determine or extrapolate aspects of the user’s movement based on machine learning (ML) or other models of user movement (said inputting feature data into AI model) [0190]. The training mechanism (316) may evaluate the user’s movements and compare the user’s movements to expected and/or desired movements for the activity the user is supposed to perform, i.e., determine how much the user’s movements differ or deviate from expected or desired movements (said obtaining a content level or difficulty matching user feature data) [0191]. An amount of difference or deviation of the user’s actual movements from the expected, may be used to suggest or apply one or more modifications to the user’s current and/or future activities [0192]. Koblin teaches presenting video images in a display of the VR headset corresponding to the activity of the user [0085]. As an example, the activity may comprise a game in which the user has to virtually hit objects (said content) in a virtual space. The objects may appear in the virtual space at certain locations and at a certain rate [0180].
Koblin teaches displaying object in virtual space [0180] and further teaches a 3D coordinate system (114) the user’s real-world space (112) [0189]. Koblin does not specifically term the object in virtual space as 3D content. Yoon teaches generating holographic images (said 3D content) in an environment [0055]. Yoon also teaches a user recognition sensor (310) that monitors a gesture of a user and may obtain information corresponding to the gesture [0122]. The user recognition sensor (310) may monitor a gesture corresponding to a hand action of the user to obtain gesture information [0123]. A shape of the hologram image may be changed based on the gesture information [0124]. It would have been obvious before the effective filing date of the claimed invention to render the virtual objects displayed on the VR headset of Koblin as 3D objects as taught by Yoon. In doing so, holographic display technology may enable the observation of different images on the basis of the movement of a view [0004]. Yoon further mentions that 3D display technology can cause dizziness and the sense of fatigue on the eyes caused by the problem of an accommodation-convergence mismatch being done [Yoon: 0003]. The holographic display of Yoon helps avoid the strain on the eyes.
(d)
setting a level or difficulty of the 3D content according to the obtained content level or difficulty; and
The system of Koblin may also use the physiological data to determine one or more modifications (said according to obtained content difficulty) [0193]. Modification of the user’s activity may include, without limitation, increasing or decreasing a degree of difficulty (said setting a difficulty) [0195]. Koblin provides the example of virtually hitting objects in virtual space [0180]. If the user keeps missing objects, then the monitoring/tracking mechanism(s) (224) may adjust the activity to have objects appear at a lower rate and/or closer to each other (said setting difficulty of 3D content). On the other hand, if the user seldom misses an object, then the monitoring/tracking mechanism(s) (224) may adjust the activity to make it harder (e.g., by making objects appear at a higher rate and/or faster and/or in a wider area) (said setting difficulty of 3D content) [0180]. Yoon is relied upon as teaching the 3D content. See claim 1(c).
(e)
uploading the 3D content to a display terminal worn by the user.
Koblin teaches presenting video images in a display of the VR headset corresponding to the activity of the user [0085]. Yoon is relied upon as teaching the 3D content. See claim 1(c).
RE claim 2, Koblin teaches wherein:
(a)
the sensing data includes at least one of coordinates, velocity, acceleration, or angular velocity of a body part of the user, and
Koblin teaches the sensors (230) track the user’s actual movements (e.g., head movements, etc.) (said movement data) [0156]. The user (102) may also have one or two handheld devices (110-1, -2). Hand movement information from the handheld device(s) (110) may be provided to the application (220) [0158]. The user’s movements may be given relative to a 3D coordinate system (114) of the user’s real-world space (112) (said sensing data includes at least one of coordinates). If the user also has handheld devices (110), the movement/tracking mechanism(s) (314) may also determine movement of one or both of the user’s hands in the user’s real-world space (112) [0189].
the obtaining the feature data comprises:
(b)
obtaining characteristic data including a level of physical ability of the user and a characteristics of the user's movement through at least one of the coordinates, velocity, acceleration, or angular velocity of a body part of the user.
Fig. 4a, Koblin teaches the program (310 selects a user activity/routine based on the user’s choices and sends instructions for that activity/routine (118) to the headset (104) [0200]. The user begins their activity, and the monitoring/tracking mechanism(s) (224) of the headset (104) track/monitor the activity and the user’s performance (408) (said obtaining characteristic data) [0203]. Fig. 4c, Fig. 3a, programs (310) analyzes the user’s data (426) to try to recognize and analyze the user’s movement [0215-0216]. The programs (310) try to determine whether and by how much the user’s movements (said coordinates) or activity deviates from the expected/desired activity (said obtaining characteristic data). The fitness/training program(s) (310) may also determine or evaluate other factors (e.g., user’s heartrate, goals, prior activities, etc.) [0216]. Based on the deviation of the user’s activity/movements from those expected or desired (said level of physical ability), the fitness/training program(s) (310) determines whether modification to the user’s activities is needed, and if so, modifies the activities [0217].
RE claim 3, in further view of Yoon, Yoon teaches further comprising:
(a)
obtaining a color image and a depth map for at least one object; and
Yoon teaches multi-view color images may be easily obtained by a camera module equipped in a mobile device [0038]. Furthermore, Yoon teaches the CNN may be used to extract a depth map from the multi-view color images [0040]. The input image data may be image data for providing a selected scene or 3D information about a selected object [0050]. The input image may be an RGB color image [0051]. Fig. 2, depth map generator (110) generates depth map image data (11) from input image data (10) [0059].
(b)
generating 3D data for the at least one object based on each of the color images and the depth map.
Fig. 1 of Yoon, hologram processing device (100) may be a device which generates hologram data (said 3D data) from the input image data (said based on each of the color images and depth map) and provides (or uploads) the hologram data to the display terminal (200) [0049]. Fig. 2, hologram generator (120) generates hologram data (120) by using the color image data (10) and the depth map image data (11) [0059].
The same motivation to combine as taught in the rationale of claim 1 is incorporated herein.
RE claim 4, in further view of Yoon, Yoon teaches further comprising:
(a)
preprocessing 3D data for at least one object;
Yoon teaches undesired data or noise may be removed from the selected image region [0079-0080].
(b)
generating at least one hologram data using the preprocessed 3D data for at least one object; and
Yoon teaches the hologram generator (120) may calculate a complex value hologram on the basis of the color image data and the depth image data (11) generated by the depth map generator (1120) to generate the hologram data (12) [0084]. The hologram data (12) is generated based on the CGH calculation algorithm, the hologram generator (120) may perform a processing process, such as space noise minimization (said using the preprocessed 3D data) [0090].
(c)
correcting the at least one hologram data based on information on the display terminal.
Yoon teaches encoder (130 may perform correction on the basis of a characteristic of the display terminal (200) [0093].
The same motivation to combine as taught in the rationale of claim 1 is incorporated herein.
RE claim 8, claim 8 recites similar limitations as claim 1 but in system form. Therefore, the same rationale used for claim 1 is applied. Koblin further teaches the system comprising:
(i)
at least one memory; and
Fig. 6, main memory (606), real-only memory (608), removable storage media (610), mass storage (612) [0250].
(ii)
at least one processor,
Fig. 6, one or more processors (604) [0250].
RE claim 9, claim 9 recites similar limitations as claim 2 but in system form. Therefore, the same rationale used for claim 2 is applied.
RE claim 10, claim 10 recites similar limitations as claim 3 but in system form. Therefore, the same rationale used for claim 3 is applied.
RE claim 11, claim 11 recites similar limitations as claim 4 but in system form. Therefore, the same rationale used for claim 4 is applied.
RE claim 14, Koblin teaches wherein:
(a)
the display terminal comprises a head mounted display, and
Fig. 1, headset (104) [0152, 0167-0170].
(b)
the at least one operating device comprises a wearable device capable of interacting with the at least one hologram data included in the 3D content.
The movement/tracking mechanism(s) (224) may monitor and/or track the user’s performance of the activity/routine, and may modify aspects thereof, preferably in real-time, based on the manner in which the user is performing the activity. For example, the activity may comprise a game in which the user has to virtually hit objects in a virtual space (said interacting with 3D content) [0180]. Koblin additionally teaches the user can also hold handheld devices (110) (said wearable device capable of interacting) where the movement/tracking mechanism(s) (314) may also determine movement of one or both of the user’s hands in the user’s real-world space (112) [0189].
As modified by Yoon, Yoon is relied upon as teaching hologram data as discussed in the rationale of claim 1(c)
The same motivation to combine as taught in the rationale of claim 1 is incorporated herein.
RE claim 15, Koblin teaches a computer-implemented method running a training program for a user wearing a VR headset, where the training program is modified based on the user’s activity [abstract]. Koblin teaches a system for creating and displaying interactive 3D content, the system comprising:
(i)
a device; and
Fig. 1, VR device or headset (104) is connected to a fitness/training system (108) [0152].
(ii)
a display terminal,
The VR headset (104) presents the VR user (102) with a view (122) (on the headset’s display (204) [0159]. Headset (103) includes display (204) [0167].
wherein the device is configured to:
(a)
receive sensing data on a user's body from at least one sensor attached to the user's body;
Fig. 1, real-time user training virtual reality environment (100) [0152]. Person (102) uses headset (104) to view and interact with a virtual environment [0152]. Sensors (230) in the headset (104) and/or other sensors in the user’s environment may track the VR user’s actual movements [0156]. The user (102) may also wear or be connected to one or more physiological sensors (120) where the data is sent to the headset (104) [0162]. Data from the headset (104) may be provided in real-time from the VR fitness application (220) to the training program (310) [0157]. The movement/tracking mechanism(s) (314) may receive user data (116) and may determine and/or approximate, from that data, the user’s actual movements in the user’s real-world space/environment (112) [0189]. The user’s movements may be given relative to a 3D coordinate system (114) the user’s real-world space (112) [0189]. If the user also has handheld devices (110), the movement/tracking mechanism (314) may determine movement of one or both hands in the user’s real-world space (112). The user’s headset (104) may provide the user’s actual 3D coordinates in the real-world space (112) to the programs (310) on the system (108) [0189].
(b)
obtain feature data related to the user's body and movement based on the sensing data;
The sensors (230) track the user’s actual movements (e.g., head movements, etc.) (said movement data) [0156]. Additionally, the one or more physiological sensors (120) can provide e.g., heartrate, temperature, sweat levels, breathing, etc. (said feature data) [0162].
(c)
obtain a content level or difficulty matching the user by inputting feature data and 3D (dimension) content into an artificial intelligence (AI) model;
The VR fitness app (220) may present the user (102) with an activity or exercise routine for the user to perform [0179]. The monitoring/tracking mechanism(s) (224) may monitor and/or track the user’s performance of the activity, preferably in real-time, based on the manner in which the user is performing the activity [0180]. The VR fitness app (220) includes a real-time feedback system that may modify a user’s activity based on how the user is performing [0181]. The movement/tracking mechanism(s) (314) may determine or extrapolate aspects of the user’s movement based on machine learning (ML) or other models of user movement (said inputting feature data into AI model) [0190]. The training mechanism (316) may evaluate the user’s movements and compare the user’s movements to expected and/or desired movements for the activity the user is supposed to perform, i.e., determine how much the user’s movements differ or deviate from expected or desired movements (said obtaining a content level or difficulty matching user feature data) [0191]. An amount of difference or deviation of the user’s actual movements from the expected, may be used to suggest or apply one or more modifications to the user’s current and/or future activities [0192]. Koblin teaches presenting video images in a display of the VR headset corresponding to the activity of the user [0085]. As an example, the activity may comprise a game in which the user has to virtually hit objects (said content) in a virtual space. The objects may appear in the virtual space at certain locations and at a certain rate [0180].
Koblin teaches displaying object in virtual space [0180] and further teaches a 3D coordinate system (114) the user’s real-world space (112) [0189]. Koblin does not specifically term the object in virtual space as 3D content. Yoon teaches generating holographic images (said 3D content) in an environment [0055]. Yoon also teaches a user recognition sensor (310) that monitors a gesture of a user and may obtain information corresponding to the gesture [0122]. The user recognition sensor (310) may monitor a gesture corresponding to a hand action of the user to obtain gesture information [0123]. A shape of the hologram image may be changed based on the gesture information [0124]. It would have been obvious before the effective filing date of the claimed invention to render the virtual objects displayed on the VR headset of Koblin as 3D objects as taught by Yoon. In doing so, holographic display technology may enable the observation of different images on the basis of the movement of a view [0004]. Yoon further mentions that 3D display technology can cause dizziness and the sense of fatigue on the eyes caused by the problem of an accommodation-convergence mismatch being done [Yoon: 0003]. The holographic display of Yoon helps avoid the strain on the eyes.
(d)
set a level or difficulty of the 3D content according to the obtained content level or difficulty; and
The system of Koblin may also use the physiological data to determine one or more modifications (said according to obtained content difficulty) [0193]. Modification of the user’s activity may include, without limitation, increasing or decreasing a degree of difficulty (said setting a difficulty) [0195]. Koblin provides the example of virtually hitting objects in virtual space [0180]. If the user keeps missing objects, then the monitoring/tracking mechanism(s) (224) may adjust the activity to have objects appear at a lower rate and/or closer to each other (said setting difficulty of 3D content). On the other hand, if the user seldom misses an object, then the monitoring/tracking mechanism(s) (224) may adjust the activity to make it harder (e.g., by making objects appear at a higher rate and/or faster and/or in a wider area) (said setting difficulty of 3D content) [0180]. Yoon is relied upon as teaching the 3D content. See claim 1(c).
(e)
upload the 3D content to a display terminal worn by the user, and
Koblin teaches presenting video images in a display of the VR headset corresponding to the activity of the user [0085]. Yoon is relied upon as teaching the 3D content. See claim 1(c).
wherein the display terminal is configured to:
(f)
display the 3D content in a 3D space
Koblin teaches a VR environment [0229]. It is well known in the art that a virtual reality environment generates a 3D digital space. In further view of Yoon, Yoon teaches generating hologram data from input image data to display on terminal (200) [0049]. Yoon further teaches the display terminal (200) may be a device which reconstructs (or displays) a hologram image in a 3D space [0055].
(g)
apply a user command input through at least one operating device to the 3D content.
The movement/tracking mechanism(s) (224) may monitor and/or track the user’s performance of the activity/routine, and may modify aspects thereof, preferably in real-time, based on the manner in which the user is performing the activity. For example, the activity may comprise a game in which the user has to virtually hit objects in a virtual space (said apply user command input to 3D content) [0180]. Koblin additionally teaches the user can also hold handheld devices (110) (said operating device) where the movement/tracking mechanism(s) (314) may also determine movement of one or both of the user’s hands in the user’s real-world space (112) [0189].
As modified by Yoon, Yoon is relied upon as teaching hologram data (said 3D content) as discussed in the rationale of claim 15(c)
Claims 5-7, 12, 13 are rejected under 35 U.S.C. 103 as being unpatentable over KOBLIN et al. (US 2022/0395741 A1) in view of YOON et al. (US 2021/0279951 A1) as applied claims 1 and 8 respectively above, and in further view of VEMBAR et al. (US 2020/0160609 A1).
RE claim 5, in further view of Yoon, Yoon teaches further comprising:
(a)
the information on the display terminal includes a size of the display terminal, a performance of the display terminal, a type of an operating device connected to the display terminal, and a type of the display terminal, and
Yoon teaches the encoder (130) may perform correction on the basis of a characteristic of the display terminal (200) [0093]. The characteristic of the display terminal (200) (said information on the display termination) may include a source light wavelength (said performance of display terminal) and a resolution (said size of said display terminal), which are used in the display terminal (200), and a modulation schedule of data transmitted to the display terminal (200) (said performance of display terminal) [0093]. Additionally, Koblin teaches the display terminal can be a heads-up display [0268]. It would have been obvious before the effective filing date of the claimed invention to indicate to the system the type of display since Koblin permits different types (said type of display terminal). This would allow the system to adjust pending on the different characteristics of the display terminal. However, Koblin in view of Yoon fail to discuss the information including a type of operating device.
Vembar is made of record as teaching a hybrid rendering system that includes a wearable display tethered to a computer [abstract]. Overall, the method/system of Vembar determines which processing system, i.e., local processing at the HMD or remote processing with the tethered computer, should be used to compute the display [0032]. The tethered computer comprises a more powerful graphics pipeline to perform rendering and therefore transmits the rendering data to the HMD [0032]. The compute workload is portioned between the low power graphics processor on the HMD (102) and the workstation (110) that is sent over connection (108) [0033]. The tethered computer has a higher powered CPU and a more powerful GPU, so it can render the virtual world at higher fidelity, including resolution and frame rates [0033]. The system of Vembar teaches using a time warp process in which the tethered computer renders a larger frame than the HMD [0040]. The larger frame allows for a margin or border around the displayed frame that is not displayed [0040]. The HMD can modify the display based on the user’s change of orientation etc., as long as the user does not go beyond the extra margin around the frame [0042]. Thus, the method/system of Vembar notes the different processing systems, local HMD or remote computer, to determine the types of processing (said a type of an operating device connected to the display terminal).
It would have been obvious before the effective filing date of the claimed invention to utilize the hybrid system of Vembar with the modified system of Koblin in view of Yoon. Making use of the remote and local processing allows for a more immediate response to the user’s interaction, and therefore a more realistic experience [Vembar: 0044].
(b)
a size and arrangement of the at least one hologram data are corrected based on the size of said display terminal, the performance of the display terminal, the type of at least one operating device connected to the display terminal, and the type of the display terminal.
Yoon teaches the encoder (130) may perform correction on the basis of a characteristic of the display terminal (200) [0093]. The characteristic of the display terminal (200) may include a source light wavelength and a resolution (said based on the size of said display terminal), which are used in the display terminal (200), and a modulation schedule of data transmitted to the display terminal (200) [0093].
RE claim 6, Koblin teaches wherein:
(a)
the AI model is trained to output a content level or difficulty level that matches the user based on at least one of the corrected hologram data, the level of physical ability of the user, and the characteristics of the user's movements.
Koblin teaches the movement/tracking mechanism(s) (314) may determine or extrapolate aspects of the user’s movement based on machine learning (ML) or other models of user movement (said AI model is trained) [0190]. The training mechanism(s) (316) may evaluate the user’s movements, and then compare the user’s movements to expected and/or desired movements for the activity that the user is supposed to be performing, i.e., how much the user differs or deviates from expected or desired movements [0191]. An amount of difference or deviation of the user’s actual movements may be used by the training mechanism(s) (316) to suggest or apply one or more modifications to the user’s current and/or future activities (said output a content level or difficult level that matches the user based on the characteristics of the user’s movements) [0192].
RE claim 7, Koblin teaches wherein:
(a)
the display terminal comprises a head mounted display, and
Fig. 1, headset (104) [0152, 0167-0170].
(b)
the at least one operating device comprises a wearable device capable of interacting with the at least one hologram data included in the 3D content.
The movement/tracking mechanism(s) (224) may monitor and/or track the user’s performance of the activity/routine, and may modify aspects thereof, preferably in real-time, based on the manner in which the user is performing the activity. For example, the activity may comprise a game in which the user has to virtually hit objects in a virtual space (said interacting with 3D content) [0180]. Koblin additionally teaches the user can also hold handheld devices (110) (said wearable device capable of interacting) where the movement/tracking mechanism(s) (314) may also determine movement of one or both of the user’s hands in the user’s real-world space (112) [0189].
As modified by Yoon, Yoon is relied upon as teaching hologram data as discussed in the rationale of claim 1(c)
The same motivation to combine as taught in the rationale of claim 1 is incorporated herein.
RE claim 12, claim 12 recites similar limitations as claim 5 but in system form. Therefore, the same rationale used for claim 5 is applied.
RE claim 13, claim 13 recites similar limitations as claim 6 but in system form. Therefore, the same rationale used for claim 6 is applied.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE L SAMS:
direct telephone number:
(571) 272-7661
email:
michelle.sams@uspto.gov
The examiner is currently part time and can be reached Mon.-Fri. 5:30am-9:30am.
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee M. Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE L SAMS/
Primary Examiner, Art Unit 2611
24 June 2026
Prosecution Insights