Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
RESPONSE TO AMENDMENT
Acknowledgment is made of the amendment filed 05/08/2026, in which:claims 1, 4-5, 10, and 19-20 are amended; claims 7-8, 12-14, and 18 are cancelled; new claims 21-25 are added; and the rejections of the claims are traversed. Claims 1-6, 9-11, 15-17, 19-20 and 21-25 are currently pending and an Office Action on the merits follows.
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 , if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1-6, 9-11, 15-17, 19-22, and 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Mallinson US 20190236836 in view of Son et al. US 20190171280.
Consider claim 1. Mallinson discloses an information processing apparatus comprising one or more processors and/or circuity fig. 1A HMD 102 [0035] client device 100 may be an HMD configured to:
execute a Head Mounted Display HMD information obtainment processing that obtains HMD configuration information indicating a relationship between components of an HMD and a visually induced motion sickness fig. 1A see data from the client 100 being sent to the discomfort classification model [0046] data collected from VR content engine 111 including metadata. Baseline VR content is associated with predetermined an expected discomfort and or sickness reactions;
execute an estimation processing [0050] discomfort and/or sickness state a user is predicted to experience as determined by the deep learning engine 190. [0117][0133] present invention is embodied in the form of hardware and software and the hardware maybe a general processor that estimates whether or not a user using the HMD experiences the visually induced motion sickness based on an induction degree of the visually induced motion sickness obtained by using the HMD configuration information [0041-0042] physiological data (active or passive) is collected and monitored to produce reliable correlations between discomfort and sickness patterns of the VR content; and
execute an output processing that outputs an estimation result obtained in the estimation processing [0050] Given a set of inputs the output nodes indicate the level of discomfort and/or sickness state a user is predicted to experience as determined by the deep learning engine 190.
Mallinson however does not explicitly disclose components of an HMD.
Son however disclose components of an HMD fig. 4 human factor parameters or VR sickness-inducing factors[Wingdings font/0xE0] device factor. Also see fig. 5 which shows HMD component related human factor parameters and their associated cyber sickness symptoms. also see fig. 6 resolution and movement of HMD camera.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the information processing apparatus of Mallinson to include components of an HMD, as taught by Son, to help build machine learning based VR motion sickness prediction model for virtual reality content [0002].
Consider claim 2. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the one or more processors and/or circuitry is further configured to execute a measurement obtainment processing that obtains biological effect information in which biological effects of the visually induced motion sickness on the user have been indicated as results of psychological or physiological measurements Mallinson fig. 1A [0042] passive sensors 125 measure heart rate brain activity etc. Son [0032] detect EEG ECG PPG or galvanic skin response; and
execute a model generation processing that generates a trained model by using the HMD configuration information and the biological effect information as learning data to perform training, and in the estimation processing Mallinson fig. 1A biometric data from passive sensors are sent to the deep learning engine to create a model along with HMD related data. also see Son, fig. 4 device factor and personal factor are both used in the machine learning model, the induction degree of the visually induced motion sickness is obtained by inputting the HMD configuration information into the trained model Son fig. 5 [0053] . Referring to FIG. 5, it is confirmed that the correspondence of each item in the list of VR sickness symptoms to each VR sickness-inducing factor and the magnitude of each item in the list of VR sickness symptoms are acquired, as a result of performing the machine learning.
Motivation to combine is similar to motivation in claim 1.
Consider claim 3. Mallinson as modified by Son disclose the information processing apparatus according to claim 2, wherein the one or more processors and/or circuitry is further configured to execute an operation obtainment processing that obtains input information indicating whether or not the user is experiencing the visually induced motion sickness based on an operation input performed by the user Mallinson [0040-0041] active or passive monitoring of motion sickness. when a user is feeling discomfort and or sickness the user may actively engage the actuator. the user may also be prompted to enter the degree of discomfort and or sickness. ; and
execute a model update processing that updates the trained model by using the input information Mallinson [0050] the deep learning engine’s parameters are modified and refined to iteratively determine which VR content or patterns of VR content induce discomfort by comparing predetermined and true discomfort and sickness reactions.
Consider claim 4. Mallinson as modified by Son disclose the information processing apparatus according to claim 2, wherein the biological effect information includes information indicating the result of the measurement of at least one of a Simulator Sickness Questionnaire SSQ score, a body temperature, a heart rate, the number of times of eye closures, a skin moisture content, brain waves, and a mismatch between an eye movement and a video image Mallinson [0042] heart rate, brain activity, among many others.
Consider claim 5. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the one or more processors and/or circuitry is further configured to execute a content information obtainment processing Mallinson fig. 1a 115 VR content is send to the classification modeler that obtains content information indicating a relationship between an eXtended Reality XR content displayed on the HMD and the visually induced motion sickness Mallinson [0038] the VR content 115, various data is monitored, wherein the data is associated with the VR content 115 (e.g., rendered images, etc.) and the user 105. The data may be monitored and collected by client device 100, or directly by the discomfort classification modeler 120, and in the estimation processing, the induction degree of the visually induced motion sickness is obtained by also using the content information. Mallinson [0045-0046] VR content patterns. Also see Son [0045-0047] machine learning is used to predict correlation between VR sickness inducing factors and the list of VR sickness symptoms for any VR content fig. 5.
Motivation to combine is similar to motivation in claim 1.
Consider claim 6. Mallinson as modified by Son disclose the information processing apparatus according to claim 5, wherein the one or more processors and/or circuitry is further configured to execute a measurement obtainment processing that obtains biological effect information in which biological effects of the visually induced motion sickness on the user have been indicated as results of psychological or physiological measurements Mallinson fig. 1A [0042] passive sensors 125 measure changes in physiological data e.g. heart rate brain activity etc. which indicate discomfort and or sickness. Son [0032] detect EEG ECG PPG or galvanic skin response; and execute a model generation processing that generates a trained model by using the HMD configuration information, the content information, and the biological effect information as learning data to perform training, and in the estimation processing Mallinson fig. 1A 120 receives biometric data from 125 metadata 117 and VR content data. Also see Son fig. 4 device factor content factor and personal factor, the induction degree of the visually induced motion sickness is obtained by inputting the HMD configuration information and the content information into the trained model see Son fig. 4 device factor content factor and personal factor [0045].
Motivation to combine is similar to motivation in claim 1.
Consider claim 9. Mallinson as modified by Son disclose the information processing apparatus according to claim 5, wherein the content information includes information indicating a relationship between the visually induced motion sickness, and at least one of a frame rate, a degree of a video image shake, a viewpoint type, a movement method of an avatar, a moving speed of the avatar, a moving acceleration of the avatar, and presence or absence of a visually induced motion sickness reduction function when the avatar moves Son [0035] VR content which causes visually induced motion sickness (VIMS) a change in intrinsic characteristics of the content or extrinsic characteristics of the content (for example, movement of an object, movement of a camera). Mallinson [0039] if the same pattern of VR content (e.g., moving horizon, avatar linear and rotational accelerations, etc.), or patterns of actions, consistently induces discomfort and/or sickness in the test users, then those patterns can be labeled as producing discomfort and/or sickness reactions in consumer users who interact with that VR content.
Motivation to combine is similar to motivation in claim 1.
Consider claim 10. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the one or more processors and/or circuitry is further configured to execute a content information obtainment processing Mallinson fig. 1a 115 VR content is send to the classification modeler that obtains content information indicating a relationship between an XR content displayed on the HMD and the visually induced motion sickness Mallinson [0038] the VR content 115, various data is monitored, wherein the data is associated with the VR content 115 (e.g., rendered images, etc.) and the user 105. The data may be monitored and collected by client device 100, or directly by the discomfort classification modeler 120,; and execute a user information obtainment processing that obtains user information indicating a relationship between the user and the visually induced motion sickness Mallinson [0042] head movement. Son fig. 4 Task factor, and in the estimation processing, the induction degree of the visually induced motion sickness is obtained by also using the content information and the user information Mallinson [0075] fig. 3B predicting discomfort as the VR content including FOV throttling as the head of the user is moving in time.
Consider claim 11. Mallinson as modified by Son disclose the information processing apparatus according to claim 10, wherein the one or more processors and/or circuitry is further configured to execute a measurement obtainment processing that obtains biological effect information in which biological effects of the visually induced motion sickness on the user have been indicated as results of psychological or physiological measurements Mallinson fig. 1A [0042] passive sensors 125 measure changes in physiological data e.g. heart rate brain activity etc. which indicate discomfort and or sickness. Son [0032] detect EEG ECG PPG or galvanic skin response; and execute a model generation processing that generates a trained model by using the HMD configuration information, the content information, the user information, and the biological effect information as learning data to perform training Mallinson fig. 1A 120 receives biometric data from 125 metadata 117 and VR content data. Also see Son fig. 4 device factor content factor and personal factor and task factor, and in the estimation processing, the induction degree of the visually induced motion sickness is obtained by inputting the HMD configuration information, the content information, and the user information into the trained model see Son fig. 4 device factor content factor and personal factor task factor [0045]. Also see Son fig. 5 [0053]
Motivation to combine is similar to motivation in claim 1.
Consider claim 15. Mallinson as modified by Son disclose the information processing apparatus according to claim 10, wherein the user information includes information indicating a relationship between the visually induced motion sickness, and at least one of a head movement amount, a head rotation amount, an age, a race, a health condition, a use frequency of the HMD, and a use time of the HMD Mallinson [0042] head movement. Son fig. 4 Task factor.
Consider claim 16. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the one or more processors and/or circuitry is further configured to execute a user information obtainment processing that obtains user information indicating a relationship between the user and the visually induced motion sickness Mallinson [0042] head movement. Son fig. 4 Task factor, and in the estimation processing, the induction degree of the visually induced motion sickness is obtained by also using the user information Mallinson [0075] fig. 3B predicting discomfort as the VR content including FOV throttling as the head of the user is moving in time.
Consider claim 17. Mallinson as modified by Son disclose the information processing apparatus according to claim 16, wherein the one or more processors and/or circuitry is further configured to execute a measurement obtainment processing that obtains biological effect information in which biological effects of the visually induced motion sickness on the user have been indicated as results of psychological or physiological measurements Mallinson fig. 1A [0042] passive sensors 125 measure changes in physiological data e.g. heart rate brain activity etc. which indicate discomfort and or sickness. Son [0032] detect EEG ECG PPG or galvanic skin response; and execute a model generation processing that generates a trained model by using the HMD configuration information, the user information, and the biological effect information as learning data to perform training Mallinson fig. 1A 120 receives biometric data from 125 metadata 117 and VR content data. Also see Son fig. 4 device factor, content factor, personal factor, and task factor are used in building the machine learning model, and in the estimation processing, the induction degree of the visually induced motion sickness is obtained by inputting the HMD configuration information and the user information into the trained model see Son fig. 4 device factor content factor and personal factor task factor [0045]. Also see Son fig. 5 [0053].
Motivation to combine is similar to motivation in claim 1.
Claim 19 is rejected mutatis mutandis for the reasons set forth in claim 1.
Claim 20 is rejected mutatis mutandis for the reasons set forth in claim 1. [0117][0133] present invention is embodied in the form of hardware and software and the hardware maybe a general processor.
Consider claim 21. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein estimation processing estimates whether or not a user currently using the HMD experiences the visually induced motion sickness see Mallinson [0097] receiving an input indicating discomfort in the user. The method includes applying a discomfort reduction filter effect to VR content currently displayed purposes of reducing potential discomfort, such as applying FOV throttling to the VR content generated for display. For example, FOV throttling may be applied as an overlay to the rendered VR content that is generated for display.
Consider claim 22. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the HMD configuration information includes information indicating at least one of a display viewing angle, a refresh rate, a display resolution, a weight, a motion-to-photon delay, and a photon-to-photon delay of the HMD. Son [0004-0005] VR latency. VR rendering depends on system specification. Things that influence VR latency include resolution, field of view binocular. disparity and the life. [0035] VR human facto parameters (which include device factors se fig. 4-6) include resolution binocular parallax angle of view. son fig,. 4-6 Device factors
Consider claim 24. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the estimation processing includes comparing the induction degree with a predetermined threshold value and estimating that the user experiences the visually induced motion sickness when the induction degree exceeds the predetermined threshold value Son fig. 5 . [0045-0046] where sickness symptoms has different magnitudes and below which motion sickness is not registered.
Consider claim 25. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, wherein the estimation processing is repeatedly executed at predetermined time intervals. Mallinson fig. 1A-1B build discomfort recognition model through training (iterative process) implement by deep learning engine.
2. Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Mallinson US 20190236836 in view of Son et al. US 20190171280 and further in view of Leiby et al. US 20200043223.
Consider claim 23. Mallinson as modified by Son disclose the information processing apparatus according to claim 1, but does not disclose wherein the HMD configuration information includes information indicating at least one of tracking performance of the HMD, presence or absence of a function to set an interpupillary distance of the HMD, strength of lens distortion, a display afterglow property, and presence or absence of a visually induced motion sickness reduction function.
Leiby however discloses wherein the HMD configuration information includes information indicating at least one of tracking performance of the HMD, presence or absence of a function to set an interpupillary distance of the HMD, strength of lens distortion, a display afterglow property, and presence or absence of a visually induced motion sickness reduction function [0002] user can become nauseous if the application fails to made frame rate and there is no re projection to compensate for missing frames [0033] the compositor can also monitor the rendering performance of the application at 118 by tracking the frames.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the information processing apparatus of Mallinson to include wherein the HMD configuration information includes information indicating at least one of tracking performance of the HMD, as taught by Leiby, to prevent the user from experiencing nausea [0002].
III. RESPONSE TO ARGUMENTS
Applicant's arguments have been fully considered but are moot in view of the new grounds of rejection.
Applicant argues (pages 12-14) that the cited references do not teach the invention of claims in view of the new claims 21-25. The Applicant argues that the distinction between the prior art and instant invention is that the application uses information of hardware components of an HMD that relates to visually induced motion sickness. In contrast, the applicant contends, Mallinson and Son use physiological response modeled through machine learning.
The Office however respectfully disagrees and has updated the rejection to address the new claims 21-25( see rejection above). Mallinson as modified by Son also clearly disclose that training machine learning models uses device, content, personal, and task factors (Son fig. 4) to determine degree of motion sickness (see rejection above for more details).
for at least these reasons the cited references read on the claimed invention.
IV. CONCLUSION
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM A KHAN whose telephone number is (571)270-7998. The examiner can normally be reached on 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nitin Patel can be reached on 571-272-7677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
IBRAHIM A. KHAN
Primary Examiner
Art Unit 2628
/IBRAHIM A KHAN/ 06/24/2026Primary Examiner, Art Unit 2628
Prosecution Insights