Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM FOR TRACKING EMOTIONS IN A VIRTUAL ENVIRONMENT”.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-8, and 10-15 are rejected under 35 U.S.C. 102(a)(1) or 102(a)(2) (or both) as being anticipated by Tzvieli (Pub No. US 20180116528 A1).
As per claim 1, Tzvieli teaches the claimed:
1. An information processing system comprising at least one processor, the at least one processer carrying out: (Tzvieli [0047]: “The following figures show various examples of HMSs equipped with head-mounted cameras. FIG. 1a illustrates various inward-facing head-mounted cameras coupled to an eyeglasses frame 15. Cameras 10 and 12 measure regions 11 and 13 on the forehead, respectively. Cameras 18 and 36 measure regions on the periorbital areas 19 and 37, respectively. The HMS further includes an optional computer 16, which may include a processor, memory, a battery and/or a communication module. FIG. 1b illustrates a similar HMS in which inward-facing head-mounted cameras 48 and 49 measure regions 41 and 41, respectively. Cameras 22 and 24 measure regions 23 and 25, respectively. Camera 28 measures region 29. And cameras 26 and 43 measure regions 38 and 39, respectively”).
a task information acquisition process for acquiring task information that is related to a task carried out by a user in a virtual space; (Tzvieli [0245]: “The eye tracker tracks the user's gaze while viewing items. Optionally, the user's gaze is indicative of attention of the user in the items. Optionally, the user's gaze is indicative of attention the user paid to each of the items. In one embodiment, the user's gaze is tracked while the user performs some work related task, such as performing office work (e.g., if the user is a clerk) or examining a patient (e.g., if the user is a doctor). Optionally, the user's gaze is tracked while the user views video related to the task the user is performing. In one example, the video is presented via the HMD, which may be a virtual reality display or an augmented reality display.”).
a line-of-sight information acquisition process for acquiring line-of-sight information that is related to a line of sight of the user; (Tzvieli [0214]: “Thus, using the data generated by the eye tracker, attention levels of the user in at least some of the items can be determined (e.g., by the computer). In one example, an attention level of the user in an item is indicative of the amount of time the user spent focusing on the item (e.g., before moving to another item). In another example, the attention level of the user in an item is indicative of the number of times the user's sight was focused on the item during certain duration. In still another example, the attention level of the user is a relative value, which indicates whether the user paid more attention or less attention to the item compared to the other items in the video.”).
an emotion information acquisition process for acquiring emotion information that is related to an emotion of the user; (Tzvieli [0174]: “What corresponds to an “irregular physiological response” may vary between different embodiments. The following are some examples of criteria and/or ways of determining whether a physiological response is considered an “irregular physiological response”. In one example, the irregular physiological response involves the user experiencing stress that reaches a certain threshold. Optionally, for most of the time the user wears the HMD, the stress level detected for the user does not reach the certain threshold. In another example, the irregular physiological response involves the user experiencing at least a certain level of one or more of the following emotions: anxiety, fear, and anger. Optionally, for most of the time the user wears the HMD, the extent to which the user experiences the one or more emotions does not reach the certain level. In yet another example, an irregular physiological response corresponds to atypical measurement values. For example, if a probability density function is generated based on previously taken TH.sub.ROI of the user, values with a low probability, such as a probability value that is lower than the probability of 97% of the previously taken TH.sub.ROI, may be considered atypical.”).
and a relevance information output process for outputting information indicating relevance between a change in the task information, a change in the line-of-sight information, and a change in the emotion information, which are associated with a progress of the task.
(Tzvieli fig. 16 shows the light of sight information based on what is visible through the glasses. Tzveili teaches the line-of-sight information based on the attention level of a user to a specific item. Tzvieli [0214]: “Thus, using the data generated by the eye tracker, attention levels of the user in at least some of the items can be determined (e.g., by the computer). In one example, an attention level of the user in an item is indicative of the amount of time the user spent focusing on the item (e.g., before moving to another item). In another example, the attention level of the user in an item is indicative of the number of times the user's sight was focused on the item during certain duration. In still another example, the attention level of the user is a relative value, which indicates whether the user paid more attention or less attention to the item compared to the other items in the video.”). The change of the user’s attention to another item is the change in line-of-sight information. The relative value of attention level shows the changes in line-of-sight over time. Fig. 18 shows output of the physical characteristics which indicate the emotional information. Fig. 19b shows the task information with the objects involved. These comprise the relevance information.).
As per claims 14 and 15, these claims are similar in scope to limitations recited in claim 1, and thus, are rejected under the same rationale. Tzvieli teaches a non-transitory computer-readable medium to perform the instructions for the above. Tzvieli [0233]: “The following is a description of steps involved in one embodiment of a method for identifying when video includes an item that agitates a user. The steps described below may be performed by running a computer program having instructions for implementing the method. Optionally, the instructions may be stored on a computer-readable medium, which may optionally be a non-transitory computer-readable medium. In response to execution by a system including a processor and memory, the instructions cause the system to perform operations of the method. In one embodiment, the method includes at least the following steps:”
As per claim 2, Tzvieli teaches the claimed:
2. The information processing system according to claim 1, wherein the emotion information includes information indicating a magnitude of a predetermined emotion. (Tzvieli [0174]: “Optionally, for most of the time the user wears the HMD, the stress level detected for the user does not reach the certain threshold. In another example, the irregular physiological response involves the user experiencing at least a certain level of one or more of the following emotions: anxiety, fear, and anger. Optionally, for most of the time the user wears the HMD, the extent to which the user experiences the one or more emotions does not reach the certain level. In yet another example, an irregular physiological response corresponds to atypical measurement values. For example, if a probability density function is generated based on previously taken TH.sub.ROI of the user, values with a low probability, such as a probability value that is lower than the probability of 97% of the previously taken TH.sub.ROI, may be considered atypical.” The certain level of emotion is the information indicating a magnitude.).
As per claim 3, Tzvieli teaches the claimed:
3. The information processing system according to claim 1, wherein the line-of- sight information includes information indicating a virtual object disposed on the line of sight in the virtual space. (Tzvieli [0105]: “In one embodiment, the system may include a head-mounted display (HMD) that presents digital content to the user and does not prevent CAM from measuring the ROI. In another embodiment, the system may include an eye tracker to track the user's gaze, and an optical see through HMD that operates in cooperation with the following components: a visible-light camera that captures images of objects the user is looking at, and a computer that matches the objects the user is looking at with the detected stress levels. Optionally, the eye tracker is coupled to a frame worn by the user. In yet another embodiment, the system may include a HMD that presents video comprising objects, and an eye tracker. The computer utilizes data generated by the eye tracker to match the objects the user is looking at with the detected stress levels. It is to be noted that there may be a delay between being affected by a stressor and a manifestation of stress as a reaction, and this delay may be taken into account when determining what objects caused the user stress.” The object the user is looking at is the object in the line of sight.).
As per claim 4, Tzvieli teaches the claimed:
4. The information processing system according to claim 1, wherein the task information includes information indicating a degree of appropriateness of an operation associated with the task, or start, end, or type of the task. (Tzvieli [0143]: “When a user is affected by one or more potential stressors, in some embodiments, the stress level of the user may depend on quantitative aspects of the potential stressors. In some examples, the degree to which a potential stressor affects the user's stress level may depend on the amount of time the potential stressor affected the user (e.g., the duration the user spent at a certain location) and/or the magnitude of the potential stressor (e.g., the extent to which an argument was heated—which may be expressed by the level of noise in peoples shouting). In some embodiments, the indications 490 include values that quantify how much at least some of the potential stressors affected the user” The duration of the work relates to the start and end time, and can produced the potential stressors. The tacked stress level can affect whether the job is being persormed well enough with the right amount of attention being paid. Tzvieli [0139]: “In one embodiment, one or more of the potential stressors may relate to various locations the user was at (e.g., work, school, doctor's office, in-laws house, etc.) and/or to various activities the user partakes in (e.g., driving, public speaking, operating machinery, caring for children, choosing clothes to wear, etc.)”. Tzvieli [0241]: “In yet another embodiment, the method may optionally include a step involving identifying that an item from among the items is a suspicious item based on the stress level reaching a first threshold and a difference between the attention the user paid to the item and the expected attention to the item reaching a second threshold.” This can relate to the degree of appropriateness, which the examiner is interpreting as the ability of the user to properly perform the operations of the task, some of which may require more care than others. This is reflected in applicant’s specification [0117]: “(Step S104B)
In step S104B, the task information acquisition unit 11B of the server lOB acquires task information on the basis of the
operation information received. The task information includes information indicating a degree of appropriateness of an operation associated with a task, or start, end, or type of a task.
A specific example of a process for acquiring task information that includes the start, end, or type of the task is as described
in step S104. Here, a specific example of a process for acquiring task information that includes the degree of appropriateness of an operation will be described. The task information acquisition unit 11B calculates the degree of appropriateness of an operation associated with a task, by collating, with the correct answer operation pattern included in the content data
DT2, the operation information received. ”. ).
As per claim 5, Tzvieli teaches the claimed:
5. The information processing system according to claim 1, wherein in the relevance information output process, the at least one processor outputs, as the information indicating the relevance,
information including a graph that indicates the change in the emotion information with respect to an elapsed time of the task, (Tzvieli [0027]: “FIG. 19b illustrates generation of a graph of the stress level of the child detected at different times while different movie scenes were viewed;” The stress level is the emotional information.).
and the task information or the line-of-sight information that was acquired at any elapsed time point indicated in the graph. (Tzvieli [0104]: “FIG. 19a and FIG. 19b illustrate one scenario of detecting a user's stress level. FIG. 19a illustrates a child watching a movie while wearing an eyeglasses frame 570 with at least five CAMs. FIG. 19b illustrates the at least five CAMs 571, 572, 573, 574, and 575, which measure the right and left periorbital areas, the nose, and the right and left cheeks, respectively (the different ROIs are designated by different patterns). The figure further illustrates how the system produces a graph of the stress level detected at different times while different movie scenes were viewed.”).
As per claim 6, Tzvieli teaches the claimed:
6. The information processing system according to claim 1, wherein
in the relevance information output process, the at least one processor outputs,
as the information indicating the relevance, information including a moving image in which an image of the progress of the task is captured by a virtual camera disposed in the virtual space, and the task information, (Tzvieli [0105]: “In one embodiment, the system may include a head-mounted display (HMD) that presents digital content to the user and does not prevent CAM from measuring the ROI. In another embodiment, the system may include an eye tracker to track the user's gaze, and an optical see through HMD that operates in cooperation with the following components: a visible-light camera that captures images of objects the user is looking at, and a computer that matches the objects the user is looking at with the detected stress levels. Optionally, the eye tracker is coupled to a frame worn by the user. In yet another embodiment, the system may include a HMD that presents video comprising objects, and an eye tracker. The computer utilizes data generated by the eye tracker to match the objects the user is looking at with the detected stress levels. It is to be noted that there may be a delay between being affected by a stressor and a manifestation of stress as a reaction, and this delay may be taken into account when determining what objects caused the user stress.” The video related to the eye tracker is the moving image. Tzvieli teaches tracing the object the user is looking at, which is the line-of-sight information in the virtual space, and the stress levels, which are the emotional information. Tzvieli [0245]: “The eye tracker tracks the user's gaze while viewing items. Optionally, the user's gaze is indicative of attention of the user in the items. Optionally, the user's gaze is indicative of attention the user paid to each of the items. In one embodiment, the user's gaze is tracked while the user performs some work related task, such as performing office work (e.g., if the user is a clerk) or examining a patient (e.g., if the user is a doctor). Optionally, the user's gaze is tracked while the user views video related to the task the user is performing. In one example, the video is presented via the HMD, which may be a virtual reality display or an augmented reality display”).
the emotion information, or the line-of-sight information that was acquired at any elapsed time point of the task corresponding to any reproduction position of the moving image. (The video and eye tracker pertain the line of sight, and represent emotional information including stress levels, as described above. The reproduction position is the virtual reality display of the task, which is reproduced since it was previous performed to record for the virtual reality.).
As per claim 7, Tzvieli teaches the claimed:
7. The information processing system according to claim 1, wherein: in the relevance information output process, the at least one processor outputs,
to the virtual space in which the progress of the task carried out in the past is being replicated, the information indicating the relevance; (Tzvieli [0245]: “Optionally, the user's gaze is tracked while the user views video related to the task the user is performing. In one example, the video is presented via the HMD, which may be a virtual reality display or an augmented reality display.” A machine learning model is trained on measurements taken from previous instances of the task. Tzvieli [0056]: “Utilizing measurements taken of a long period (e.g., measurements taken on “different days”) may have an advantage, in some embodiments, of contributing to the generalizability of a trained model. Measurements taken over the long period likely include measurements taken in different environments and/or measurements taken while the measured user was in various physiological and/or mental states (e.g., before/after meals and/or while the measured user was sleepy/energetic/happy/depressed, etc.). Training a model on such data can improve the performance of systems that utilize the model in the diverse settings often encountered in real-world use (as opposed to controlled laboratory-like settings). Additionally, taking the measurements over the long period may have the advantage of enabling collection of a large amount of training data that is required for some machine learning approaches (e.g., “deep learning” This indicates the task has been done many times in the past in the virtual environment and can be replicated.).
and the information indicating the relevance indicates relevance between a change in the task information, a change in the emotion information and a change in the line-of-sight information, which were acquired when the task being currently replicated was carried out in the past. (Tzvieli [0069]: “In addition to feature values that are generated based on thermal measurements, in some embodiments, at least some feature values utilized by a computer (e.g., to detect a physiological response or train a mode) may be generated based on additional sources of data that may affect temperatures measured at various facial ROIs. Some examples of the additional sources include: (i) measurements of the environment such as temperature, humidity level, noise level, elevation, air quality, a wind speed, precipitation, and infrared radiation; (ii) contextual information such as the time of day (e.g., to account for effects of the circadian rhythm), day of month (e.g., to account for effects of the lunar rhythm), day in the year (e.g., to account for seasonal effects), and/or stage in a menstrual cycle; (iii) information about the user being measured such as sex, age, weight, height, and/or body build. Alternatively or additionally, at least some feature values may be generated based on physiological signals of the user obtained by sensors that are not thermal cameras, such as a visible-light camera, a photoplethysmogram (PPG) sensor, an electrocardiogram (ECG) sensor, an electroencephalography (EEG) sensor, a galvanic skin response (GSR) sensor, or a thermistor.” The physiological signals from the user are the emotional information. Tzveili teaches different attention levels to different objects in an environment. This reflects the change in line-of-sight information. Tzvieli [0214]: “…Optionally, determining which items were presented in the video may involve utilizing various image processing algorithms, which can identify items (e.g., objects and/or people) in the video and/or define the boundaries of the items in the video at various times. Thus, using the data generated by the eye tracker, attention levels of the user in at least some of the items can be determined (e.g., by the computer). In one example, an attention level of the user in an item is indicative of the amount of time the user spent focusing on the item (e.g., before moving to another item). In another example, the attention level of the user in an item is indicative of the number of times the user's sight was focused on the item during certain duration. In still another example, the attention level of the user is a relative value, which indicates whether the user paid more attention or less attention to the item compared to the other items in the video.” Tzvieli teaches tracking the gaze during previous viewings of the environment and tasks. Tzvieli [0218]:“The expected attention levels in the items may be based, in some embodiments, on tracking gazes during previous viewings of the video and/or previous viewings of similar videos in order to determine attention in the items. In one embodiment, the attention levels in the items are based on attention levels detected when the same video was viewed by other people. For example, the attention levels may represent the average attention paid by the other users to each item. In another embodiment, the attention levels in the items are based on attention levels of the user to items in similar videos. For example, if the video depicts a scene (e.g., a person filling out a loan application), then a similar video may include the same scene, possibly with slightly different items (e.g., a different person filling out a loan application).” The attention levels for different items are part of the task information since the task will involve working with items. The gathering of attention levels is based on the task being done in the past, which is being replicated. The different items are the change in task information.).
As per claim 8, Tzvieli teaches the claimed:
8. The information processing system according to claim 1, wherein: in the relevance information output process, the at least one processor outputs,
to the virtual space in which the user is carrying out the task, the information indicating the relevance;
and the information indicating the relevance indicates relevance between a change in the task information, a change in the emotion information, and a change in the line-of- sight information, which were acquired when a task identical to the task was carried out in the past (All of the following is done while interacting with the virtual environment and measures the information that corresponds to the emotional information. Tzvieli [0174]: “What corresponds to an “irregular physiological response” may vary between different embodiments. The following are some examples of criteria and/or ways of determining whether a physiological response is considered an “irregular physiological response”. In one example, the irregular physiological response involves the user experiencing stress that reaches a certain threshold. Optionally, for most of the time the user wears the HMD, the stress level detected for the user does not reach the certain threshold. In another example, the irregular physiological response involves the user experiencing at least a certain level of one or more of the following emotions: anxiety, fear, and anger. Optionally, for most of the time the user wears the HMD, the extent to which the user experiences the one or more emotions does not reach the certain level. In yet another example, an irregular physiological response corresponds to atypical measurement values. For example, if a probability density function is generated based on previously taken TH.sub.ROI of the user, values with a low probability, such as a probability value that is lower than the probability of 97% of the previously taken TH.sub.ROI, may be considered atypical.” The system is trained on previous states of both vision and emotion. It measures if emotions have reached a certain level. This detects a change. Tzvieli [0180]: “In another embodiment, the feature values generated by the computer may include feature values that describe one or more of the following factors: an emotional state of the user while accessing the certain sensitive data, a condition of the environment in which the user accessed the certain sensitive data, and the type of the certain sensitive data. Thus, the factors mentioned above may be considered when the determination is made regarding whether the user experienced an irregular physiological response. In one example, the computer receives values indicative of the user's emotional state while being exposed to the certain sensitive data, and utilizes a machine learning-based model to detect whether the user experienced the irregular physiological response based on the certain TH.sub.ROI. Optionally, in this example, the machine learning-based model was trained based on previous TH.sub.ROI taken while the user was in a similar emotional state. In another example, the computer is receives values indicative of the environment the user was in while being exposed to the certain sensitive data (e.g., temperature and/or humidity level), and utilizes a machine learning-based model to detect whether the user experienced the irregular physiological response based on the certain TH.sub.ROI. Optionally, in this example, the machine learning-based model was trained based on previous TH.sub.ROI taken while the user was in a similar environment.”).
As per claim 10, Tzvieli teaches the claimed:
10. The information processing system according to claim 1, wherein in the relevance information output process, the at least one processor outputs,
as the information indicating the relevance, information that satisfies a condition related to at least one selected from the group consisting of the task information,
the line- of-sight information, the emotion information, and the user. (Tzvieli [0172]: “In one embodiment, the certain TH.sub.ROI are taken during a certain window of time that depends on the type of the irregular physiological response (e.g., a certain stress level and/or a certain emotional response). Optionally, the window is at least five seconds long, at least thirty seconds long, at least two minutes long, at least five minutes long, at least fifteen minutes long, at least one hour long, or is some other window that is longer than one second. Optionally, during the time the user is exposed to sensitive data, TH.sub.ROI from multiple windows may be evaluated (e.g., using a sliding window approach), which include a window that contains a period during which the certain TH.sub.ROI were taken.”).
As per claim 11, Tzvieli teaches the claimed:
11. The information processing system according to claim 1, wherein: the task information includes the task information that is related to a plurality of users; (Tzvieli [0110]: “Optionally, the method further includes generating feature values based on TH.sub.ROI1 and TH.sub.ROI2, and using a model for detecting the stress level based on the feature values. The model was trained based on previous TH.sub.ROI1 and TH.sub.ROI2 of one or more users, taken during different days, which comprise: a first set of measurements taken while the one or more users had a first stress level according to a predetermined stress scale, and a second set of measurements taken while the one or more users had a second stress level according to the predetermined stress scale.”).
the emotion information includes the emotion information that is related to the plurality of users; (Tzvieli [0188]: “In some embodiments, a baseline may be calculated utilizing a predictor, which receives input comprising feature values describing various values such as characteristics of user (e.g., age, gender, weight, occupation), the sensitive data, the environment in which the user is in, and/or the emotional state of the user. The predictor utilizes a machine learning-based model to calculate, based on the feature values, the baseline which may be, for example, a value of thermal measurements, a stress level, or an extent of expressing a certain emotion. Optionally, the model was trained based on measurements of the user. Optionally, the model was trained based on measurements of other users.”).
the line-of-sight information includes the line-of-sight information that is related to the plurality of users; (Tzvieli [0214]: “In one example, an attention level of the user in an item is indicative of the amount of time the user spent focusing on the item (e.g., before moving to another item). In another example, the attention level of the user in an item is indicative of the number of times the user's sight was focused on the item during certain duration. In still another example, the attention level of the user is a relative value, which indicates whether the user paid more attention or less attention to the item compared to the other items in the video.” As established above, there can be a plurality of users.).
and in the relevance information output process, the at least one processor outputs, in a comparable manner, the information that indicates the relevance and that is related to each of the users. (Tzvieli [0184]: “Detecting the irregular physiological response may involve utilization of one or more baselines. Optionally, a baseline may be indicative of typical values for the user, such as typical thermal measurements when exposed to sensitive data, the extent to which a user is typically stressed when exposed to sensitive data, and/or the extent the user typically expresses one or more of the following emotions when exposed to sensitive data: anxiety, fear, and anger. Optionally, a baseline may correspond to the user, i.e., it may represent expected values of the user. Additionally or alternatively, a baseline may correspond to multiple users, and represent expected values of other users (e.g., a general response).” The measurement information which corresponds to relevance information applies to multiple users.).
As per claim 12, Tzvieli teaches the claimed:
12. The information processing system according to claim 1, wherein:
in the relevance information output process, the at least one processor outputs, as the information indicating the relevance,
information including the task information or the line-of-sight information which was acquired at an elapsed time point at which the change in the emotion information associated with the progress of the task is larger than changes in the emotion information before and after the change in the emotion information. (Tzvieli [0174]: “What corresponds to an “irregular physiological response” may vary between different embodiments. The following are some examples of criteria and/or ways of determining whether a physiological response is considered an “irregular physiological response”. In one example, the irregular physiological response involves the user experiencing stress that reaches a certain threshold. Optionally, for most of the time the user wears the HMD, the stress level detected for the user does not reach the certain threshold. In another example, the irregular physiological response involves the user experiencing at least a certain level of one or more of the following emotions: anxiety, fear, and anger. Optionally, for most of the time the user wears the HMD, the extent to which the user experiences the one or more emotions does not reach the certain level. In yet another example, an irregular physiological response corresponds to atypical measurement values. For example, if a probability density function is generated based on previously taken TH.sub.ROI of the user, values with a low probability, such as a probability value that is lower than the probability of 97% of the previously taken TH.sub.ROI, may be considered atypical.” This is a change in a physiological response that reaches a certain level and is atypical, meaning that before and after the change in level the emotion is usually lower.).
As per claim 13, Tzvieli teaches the claimed:
13. The information processing system according to claim 1, wherein:
in the relevance information output process, the at least one processor outputs, as the information indicating the relevance,
information including information indicating a cause due to which the change in the emotion information associated with the progress of the task is larger than changes in the emotion information before and after the change in the emotion information. (Tzvieli [0176]: “In one embodiment, the computer compares one or more values derived from the certain TH.sub.ROI to a certain threshold, and determines whether the threshold is reached (which is indicative of an occurrence of the irregular physiological response). Optionally, the threshold is determined based on previously taken TH.sub.ROI of the user (e.g., taken when the user had an irregular physiological response). Optionally, the threshold is determined based on baseline thermal measurements of the user, and the threshold represents a difference of a certain magnitude relative to the baseline measurements. Optionally, different thresholds may be utilized to detect different types of irregular physiological responses, to detect irregular physiological responses to different types of sensitive data, and/or to detect irregular physiological responses when the user is in certain emotional states and/or under certain environmental conditions” the environmental conditions causing the change in measurements is the cause. The threshold is determined based on the baseline measurements.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Tzvieli in view of Yamamoto (M. Yamamoto, T. Mitani, N. Ichiguchi, T. Tezuka and H. Yoshikawa, "An experimental study on distributed virtual environment for integrated training system on machine maintenance," SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218), San Diego, CA, USA, 1998, pp. 1479-1484 vol.2, doi: 10.1109/ICSMC.1998.728094.).
As per claim 9, Tzvieli alone does not explicitly teach the claimed limitations.
However, Tzvieli in combination with Yamamoto teaches the claimed:
9. The information processing system according to claim 1, wherein:
,the at least one processor further comprising carries out a virtual space generation process for generating that generates the virtual space,; (Yamamoto 1.: “1. INTRODUCTION Maintenance work in large-scaled plants, such as nuclear power plants, is indispensable to keep the reliability of their operation. Therefore, training facilities for maintenance workers have been constructed to give them the basic knowledge and skills necessary for maintenance work in the plants. However, the existing training facilities couldn’t provide enough chance for all the maintenance workers. With the rapid progress in Virtual Reality (VR) technology of Collaborative work of with communication I instructorTzvieli [ for multi-users Figure 1. Maintenance work and DVE today, studies on DVETzvieli [l] have been stimulating the development of the potential for innovative cooperative work environment over Intemet.”).
And in the virtual space generation process, the at least one processor generates generating the virtual space that allows the task to be carried out therein again in a case where information that gives an instruction to carry out the task again is inputted in response to output of the information indicating the relevance. (Yamamoto 1.: “1. INTRODUCTION Maintenance work in large-scaled plants, such as nuclear power plants, is indispensable to keep the reliability of their operation. Therefore, training facilities for maintenance workers have been constructed to give them the basic knowledge and skills necessary for maintenance work in the plants. However, the existing training facilities couldn’t provide enough chance for all the maintenance workers. With the rapid progress in Virtual Reality (VR) technology of Collaborative work of with commuiiication I instructorTzvieli [ for multi-users Figure 1. Maintenance work and DVE today, studies on DVETzvieli [l] have been stimulating the development of the potential for innovative cooperative work environment over Intemet. In this paper, we propose a new type of DVE with multi-modal communication for the integrated training system on machine maintenance, which will enable us to realize various kinds of effective training by using virtual realityTzvieli [2] through computer network”
Multi-modal communication provides a highly effective training, by offering proper indication. This results in effective communication between trainees. For example. the indications of the users’ viewpoint by camera-shaped objects is useful to know the position and direction of each trainee, and the display of hand-shaped objects in combination with camera-shaped objects is also helpful to grasp the trainees’ will. The oral instruction by the instructor is effective for activating the collaborative work by the trainees.” The trainees’ will can correspond to the emotional state. The instruction for the tasks is the instruction and the task information.
“The photograph in the center of Figure 1 shows the scene of the training in disassembling the pump, which consists of motor- and-pump set. The pump and motor in the training facility are real machines the same as used in the plant. During the training, an instructor guides the teaching course by giving oral instructions and the trainees follow his guidance with the collaboration by each other. Here, to simulate the maintenance workTzvieli [3], we use DVE, which is an extended multi-user virtual environment over Intemet. Figure 1 shows the relationship between DVE users and maintenance training simulation in virtual space.” The relationship between the DVE users and the maintenance training is the line of sight. The user’s physiological measurements related to the task as described in the rejection to claim 1 are the emotional information. The task, emotion, and line of sight information comprise the relevance information.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the instructions for a work task as taught by Yamamoto with the system of Tzvieli in order to monitor the emotions of the worker as they are practicing a task at work and interface those emotions with the training they receive.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/Examiner, Art Unit 2616
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights