DETAILED ACTION
This action is in response to the Amendment dated 28 May 2025. Claims 1, 6, 8 and 11 are amended. No claims have been added or cancelled. Claims 1-20 remain pending and have been considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ito et al. (US 2023/0009322 A1) in view of Ihara (US 2018/0309703 A1) and further in view of Chow (US 2021/0402292 A1).
As for independent claim 1, Ito teaches a system comprising:
a head-mounted device comprising a display to display content to a first user, one or more sensors to capture input data including images of a visual field of the first user [(e.g. see Ito paragraphs 0100, 0127, 0140 and Figs. 2 and 7) ”the user terminal 300 can be the smartphone 300a, the tablet personal computer (PC) 300b, a cell phone, a laptop PC, or a wearable device such as a head mounted display (HMD) … The display unit 302 can display, for the viewer 600, a selection screen for selecting a stamp (control command) 700 with tactile stimulus effect … the camera 322 captures an image of motion or the like of the viewer 600 to output the captured image to the control unit 310. The control unit 310 can then extract predetermined motion of the viewer 600 from the image captured by the camera 322 and acquire a control command related to the extracted motion”].
one or more processors [(e.g. see Ito paragraph 0125) ”the detailed configuration of the user terminal 300 according to the embodiment of the present disclosure is described with reference to FIG. 7 … the user terminal 300 mainly includes … a control unit 310”].
one or more memories accessible to the one or more processors, the one or more memories storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising: [(e.g. see Ito paragraph 0137) ”The storage unit 312 is implemented by a storage device such as ROM that stores programs, operation parameters, and the like used for the processing of the control unit 310, and RAM that temporarily stores parameters and the like changing as appropriate”].
capturing, using the one or more sensors, the input data from the first user [(e.g. see Ito paragraph 0140) ”the camera 322 captures an image of motion or the like of the viewer 600 to output the captured image to the control unit 310. The control unit 310 can then extract predetermined motion of the viewer 600 from the image captured by the camera 322 and acquire a control command related to the extracted motion”].
obtaining digital assets for the emoji, wherein the digital assets comprise a haptic signal configured with parameter information to generate patterns for haptic output [(e.g. see Ito paragraphs 0113, 0116, 0118, 0170 and Fig. 2) ”display the screen as illustrated in FIG. 2 as a screen on which to select the stamp 700 with tactile stimulus effect. A large number of stamps 700 with tactile stimulus effect are displayed on the selection screen (stamp selection screen). Each stamp 700 with tactile stimulus effect is correlated with a control command for the tactile presentation device 100, and the viewer 600 can select the stamp 700 displayed on the selection screen to input a control command. In the embodiment, it is preferable that an image of the stamp 700 with tactile stimulus effect intuitively evokes the tactile stimuli, thoughts (message), and so on that the viewer 600 wishes to send to the artist 602 … presenting a predetermined image or a predetermined sound to the viewer 600 or the artist 602. For example, the predetermined image can be a virtual object such as an icon, an animation, or text that is superimposed on an image of the artist 602 or an image of the real space around the artist 602, or is projected onto the artist 602; and more specifically, the predetermined image can be a trajectory of a social tip moving from the viewer 600 to the artist 602, an animation that looks like a bomb exploding, which is displayed at the same time as the tactile stimulus is presented, or an icon corresponding to the stamp 700 with tactile stimulus effect inputted by the viewer 600 … The generation unit 218 generates a tactile control signal for controlling the tactile presentation device 100 on. the basis of the control command correlated with the stamp 700 with tactile stimulus effect inputted by the viewer 600, and outputs the tactile control signal to the determination unit 220 described later. Specifically, the generation unit 218 generates, on the basis of the control command including a perceptual location of a vibration stimulus (positional information), a waveform type of the vibration stimulus, and intensity information on the vibration stimulus (mode information), waveform data to be inputted to the individual tactile stimulus units of the tactile presentation device 100 so as to present a vibration stimulus with the intensity and waveform designated in the control command at the perceptual location designated therein. Note that the tactile control signal may include information for giving a command to present the vibration stimulus at a presentation timing, frequency, interval, and presentation time or a tactile stimulus based on the control command … set also a waveform type, intensity, presentation timing, frequency, interval, presentation time, and the like of the vibration stimulus”].
transmitting the digital assets and the additional information to a device of a second user [(e.g. see Ito paragraph 0082) ”a viewer 600 selects one stamp 700 with tactile stimulus effect from the stamps 700 with tactile stimulus effect displayed on a display unit 302 of the user terminal 300 to send the selected stamp 700 with tactile stimulus effect; thereby, a predetermined tactile control signal corresponding to the tactile stimulus given to the stamp 700 is sent to the tactile stimulus unit (so-called “social tipping” with the tactile stimulus effect is made). Furthermore, the tactile stimulus unit presents, to the wearer, the tactile stimulus corresponding to the selected stamp 700 with tactile stimulus effect on the basis of the predetermined tactile control signal received. Therefore, if the artist 602 perceives the presented tactile stimulus and takes action, the viewer 600 can confirm, in real time, the action induced by the tactile stimulus presented by the stamp 700 that the viewer 600 has sent”].
Ito does not specifically teach extracting features from the input data that correspond to an electronic communication or identifying an emoji from a lexicon of emojis based on the extracted features. However, in the same field of invention, Ihara teaches:
extracting features from the input data that correspond to an electronic communication [(e.g. see Ihara paragraph 0086 and Fig. 6) ”the love expression extraction unit 102 refers to the love expression storage unit 108, and determines whether there is a love expression that matches a versatile love expression/limited love expression on the basis of sensor data (Step S139). The love expressions may be assumed to be movements such as blowing a kiss, clutching the information processing device 1, allowing a partner to listen to heartbeats (attaching the information processing device 1 to his/her chest and collecting heartbeat sound via the microphone 12), or waving a hand. In the case of blowing a kiss, movements of a hand of a user is detected from a video taken by the camera 11 of the information processing device 1, and it is determined that this movement is a love expression of blowing a kiss”].
identifying an emoji from a lexicon of emojis based on the extracted features [(e.g. see Ihara paragraphs 0075, 0076 and Fig. 6) ”the love expression table stored in the love expression storage unit 108. Specifically, the text/love expression conversion unit 106 decodes the code, and extracts a corresponding animation image from the love expression storage unit 108 … The communication AP execution unit 110 displays the love expression information (specifically, animation image) on the chatting screen”].
Therefore, considering the teachings of Ito and Ihara, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add extracting features from the input data that correspond to an electronic communication and identifying an emoji from a lexicon of emojis based on the extracted features, as taught by Ihara, to the teachings of Ito because it allows a user to transmit physical expression information in a versatile and convenient manner (e.g. see Ihara paragraphs 0009, 0013).
Ito and Ihara do not specifically teach determining, by a learning module, addition information based on the haptic signal, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output such that a respective user can learn the haptic output. However, in the same field of invention, Chow teaches:
determining, by a learning module, addition information based on the haptic signal, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output such that a respective user can learn the haptic output [(e.g. see Chow paragraphs 0044, 0079, 0099, 0100 and Figs. 5B-D) ”The haptic learning engine 403 is a machine learning algorithm that includes a plurality of classifiers and a haptic learning model, which is an artificial intelligence (AI) model generated by the machine learning algorithm. The haptic learning model is generated by the haptic profiler 400 to learn the play style of the player in order to determine if the player needs assistance during game play, and to identify a haptic setting output that can be used to notify the player or provide assistance … These types of coordinated haptic responses can be customized for the player based on the … learned behavior of the player. For instance, if the player needs more assistance in deciding which button to press, or which direction to move the game object or which direction to turn or which direction to direct the player's attention on the screen, the vibrational cues can be provided using specific components of the controller. In some specific instances, the vibration may be provided with more magnitude or for a longer period of time. If the learning algorithm learns that the player is more comfortable playing the game and does not need additional assistance, the vibrational cues may be automatically adjusted downward … In addition to vibrational cues, the haptic responses may be provided as audio cues, textual cues, visual cues, etc. … FIGS. 5B-5D illustrate some of the setting options that may be used for customizing notifications using controls of a hand-held controller, in some implementations. The customization, as mentioned previously, is done using haptic setting output identified by the haptic learning model 403b. The customization settings can be defined for the controller and for each control button or input interface of the controller 120. FIG. 5B illustrates one such option, option A, wherein the notification setting is defined for the controller, in one implementation. The notification setting is defined in a manner that allows haptic response to be provided on a side of the controller 120 that corresponds with a side the player has to proceed. In the example illustrated in FIG. 5B, the player is nudged to proceed on the right side of the fork toward path C (illustrated by solid line 520) instead of continuing on path A (illustrated by broken line 510) by causing the controller or handle of the controller to vibrate on the right side. Similar notification may be provided to the player if the haptic profiler wants the player to direct their attention to the right side of the screen. FIG. 5C illustrates another option, option B, wherein the notification setting is defined for the input interface of the input device (e.g., touch screen of the controller 120), in one implementation. In the example illustrated in FIG. 5C, a directional cue is provided on the touch screen using arrow heads moving from left to right (shown by the directional arrow) to indicate the direction the haptic profiler wants the player to proceed or pay attention to in the game scenario rendered on the display screen 100. In one implementation, the intensity of the directional cue may increase from left to right to provide additional feedback to the player. In addition to visual cue, the notification may also include audio cues. For example, as the arrow heads move from left to right the audio cue may increase in volume or provide a distinct sound. Alternatively, special audio tunes may be set up either by the player or for the player to indicate the direction the player has to direct their attention to in the game or on the display screen rendering the game play. In some implementation, separate audio tunes may be set to indicate a correct direction or an incorrect direction … FIG. 5D illustrates yet another setting option, option C, wherein each button on the controller 120 is set to provide a notification/feedback, in one implementation. The setting of each button may be defined differently to indicate different feedback responses to the player. For example, each button may be set to provide a haptic feedback. In place of or in addition to haptic feedback, each button may also include visual setting by allowing each button to light up in a different color or in different sequence of colors to indicate to the player the different button presses (single press vs. multiple presses) that the player is performing or has to perform. It should be noted that the various setting options described with reference to FIGS. 5B-5D have been described in relation to various settings for different controls on a controller for providing haptic and/or visual feedback”].
Therefore, considering the teachings of Ito, Ihara and Chow, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add determining, by a learning module, addition information based on the haptic signal, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output such that a respective user can learn the haptic output, as taught by Chow, to the teachings of Ito and Ihara because it allows the user to be provided with sufficient cues to detect the interactive task (e.g. see Chow paragraph 0006).
As for dependent claim 2, Ito, Ihara and Chow teach the system as described in claim 1, but Ito does not specifically teach the following limitations. However, Ihara teaches:
wherein the extracting the features comprises: determining characteristics of the input data and identifying patterns within the input data that correspond to a key or attributes of electronic communication based on the characteristics, the key or attributes being the extracted features and [(e.g. see Ihara paragraph 0086) ”the love expression extraction unit 102 refers to the love expression storage unit 108, and determines whether there is a love expression that matches a versatile love expression/limited love expression on the basis of sensor data (Step S139). The love expressions may be assumed to be movements such as blowing a kiss, clutching the information processing device 1, allowing a partner to listen to heartbeats (attaching the information processing device 1 to his/her chest and collecting heartbeat sound via the microphone 12), or waving a hand. In the case of blowing a kiss, movements of a hand of a user is detected from a video taken by the camera 11 of the information processing device 1, and it is determined that this movement is a love expression of blowing a kiss”].
wherein the identifying the emoji comprises: constructing a query using the extracted features as parameters of the query and executing the query on the lexicon of emojis [(e.g. see Ihara paragraphs 0066, 0067, 0070, 0075, 0086 and Fig. 6) ”the love expression extraction unit 102 extracts love expression information from the input sensor data with reference to love expression information stored in the love expression storage unit 108 … versatile love expression information table. As illustrated in FIG. 6, love expression codes, sensor detection conditions, natural languages, love expression contents, animation images, sound effects, and vibrations are included … The text/love expression conversion unit 106 converts the acquired text information into a love expression with reference to the love expression table stored in the love expression storage unit 108. Specifically, the text/love expression conversion unit 106 decodes the code, and extracts a corresponding animation image from the love expression storage unit 108 … the love expression extraction unit 102 refers to the love expression storage unit 108, and determines whether there is a love expression that matches a versatile love expression/limited love expression on the basis of sensor data (Step S139)”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 3, Ito, Ihara and Chow teach the system as described in claim 1 and Ito further teaches:
wherein the haptic signal is configured with the parameter information for interval, pitch, and amplitude to generate the patterns for the haptic output [(e.g. see Ito paragraphs 0116, 0170) ”The generation unit 218 generates a tactile control signal for controlling the tactile presentation device 100 on. the basis of the control command correlated with the stamp 700 with tactile stimulus effect inputted by the viewer 600, and outputs the tactile control signal to the determination unit 220 described later. Specifically, the generation unit 218 generates, on the basis of the control command including a perceptual location of a vibration stimulus (positional information), a waveform type of the vibration stimulus, and intensity information on the vibration stimulus (mode information), waveform data to be inputted to the individual tactile stimulus units of the tactile presentation device 100 so as to present a vibration stimulus with the intensity and waveform designated in the control command at the perceptual location designated therein. Note that the tactile control signal may include information for giving a command to present the vibration stimulus at a presentation timing, frequency, interval, and presentation time or a tactile stimulus based on the control command … set also a waveform type, intensity, presentation timing, frequency, interval, presentation time, and the like of the vibration stimulus”].
As for dependent claim 4, Ito, Ihara and Chow teach the system as described in claim 1 and Ito further teaches:
wherein the digital assets further comprise an image or video asset, an audio asset, or both [(e.g. see Ito paragraphs 0113, 0118 and Fig. 2) ”display the screen as illustrated in FIG. 2 as a screen on which to select the stamp 700 with tactile stimulus effect. A large number of stamps 700 with tactile stimulus effect are displayed on the selection screen (stamp selection screen). Each stamp 700 with tactile stimulus effect is correlated with a control command for the tactile presentation device 100, and the viewer 600 can select the stamp 700 displayed on the selection screen to input a control command. In the embodiment, it is preferable that an image of the stamp 700 with tactile stimulus effect intuitively evokes the tactile stimuli, thoughts (message), and so on that the viewer 600 wishes to send to the artist 602 … presenting a predetermined image or a predetermined sound to the viewer 600 or the artist 602. For example, the predetermined image can be a virtual object such as an icon, an animation, or text that is superimposed on an image of the artist 602 or an image of the real space around the artist 602, or is projected onto the artist 602; and more specifically, the predetermined image can be a trajectory of a social tip moving from the viewer 600 to the artist 602, an animation that looks like a bomb exploding, which is displayed at the same time as the tactile stimulus is presented, or an icon corresponding to the stamp 700 with tactile stimulus effect inputted by the viewer 600”].
As for dependent claim 5, Ito, Ihara and Chow teach the system as described in claim 1 and Ito further teaches:
wherein the processing further comprises obtaining additional information based on the emoji or the haptic signal, the additional information includes a text description of the haptic output conveyed by the haptic signal, an audio component corresponding to the haptic signal, an image component corresponding to the haptic signal, or a combination thereof, and transmitting the additional information to the device of the second user [(e.g. see Ito paragraphs 0113, 0118 and Fig. 2) ”display the screen as illustrated in FIG. 2 as a screen on which to select the stamp 700 with tactile stimulus effect. A large number of stamps 700 with tactile stimulus effect are displayed on the selection screen (stamp selection screen). Each stamp 700 with tactile stimulus effect is correlated with a control command for the tactile presentation device 100, and the viewer 600 can select the stamp 700 displayed on the selection screen to input a control command. In the embodiment, it is preferable that an image of the stamp 700 with tactile stimulus effect intuitively evokes the tactile stimuli, thoughts (message), and so on that the viewer 600 wishes to send to the artist 602 … presenting a predetermined image or a predetermined sound to the viewer 600 or the artist 602. For example, the predetermined image can be a virtual object such as an icon, an animation, or text that is superimposed on an image of the artist 602 or an image of the real space around the artist 602, or is projected onto the artist 602; and more specifically, the predetermined image can be a trajectory of a social tip moving from the viewer 600 to the artist 602, an animation that looks like a bomb exploding, which is displayed at the same time as the tactile stimulus is presented, or an icon corresponding to the stamp 700 with tactile stimulus effect inputted by the viewer 600”].
As for independent claim 6, Ito teaches a system comprising:
a head-mounted device comprising a display to display content to a first user, one or more sensors to capture input data including images of a visual field of the first user [(e.g. see Ito paragraphs 0100, 0127, 0140 and Figs. 2 and 7) ”the user terminal 300 can be the smartphone 300a, the tablet personal computer (PC) 300b, a cell phone, a laptop PC, or a wearable device such as a head mounted display (HMD) … The display unit 302 can display, for the viewer 600, a selection screen for selecting a stamp (control command) 700 with tactile stimulus effect … the camera 322 captures an image of motion or the like of the viewer 600 to output the captured image to the control unit 310. The control unit 310 can then extract predetermined motion of the viewer 600 from the image captured by the camera 322 and acquire a control command related to the extracted motion”].
one or more processors [(e.g. see Ito paragraph 0125) ”the detailed configuration of the user terminal 300 according to the embodiment of the present disclosure is described with reference to FIG. 7 … the user terminal 300 mainly includes … a control unit 310”].
one or more memories accessible to the one or more processors, the one or more memories storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors to perform processing comprising: [(e.g. see Ito paragraph 0137) ”The storage unit 312 is implemented by a storage device such as ROM that stores programs, operation parameters, and the like used for the processing of the control unit 310, and RAM that temporarily stores parameters and the like changing as appropriate”].
capturing, using the one or more sensors, the input data from the first user [(e.g. see Ito paragraph 0140) ”the camera 322 captures an image of motion or the like of the viewer 600 to output the captured image to the control unit 310. The control unit 310 can then extract predetermined motion of the viewer 600 from the image captured by the camera 322 and acquire a control command related to the extracted motion”].
[predicting … based on] model parameters learned from historical input data and context data [(e.g. see Ito paragraphs 0119, 0123) ”For example, the server 200 may further include a recommendation unit (not illustrated) that refers to the viewer profile 242 and the distributor profile 244 to select a control command that the viewer 600 or the artist 602 uses frequently, or conversely, a control command that the viewer 600 or the artist 602 uses less frequently, and recommends, to the viewer 600, a stamp 700 with tactile stimulus effect that is correlated. with the selected control command. Furthermore, the recommendation unit may recommend, to the viewer 600, a stamp 700 with tactile stimulus effect recommended by the artist 602 … in a case where different viewers 600 designate the same presentation position consecutively, or in a case where the same viewer 600 inputs control commands continuously, the determination unit 220 may determine that … the generation unit 218 may change the presentation position of the tactile stimulus”].
transmitting the haptic signal or digital assets for the haptic emoji and the additional content to a device of a second user [(e.g. see Ito paragraph 0082) ”a viewer 600 selects one stamp 700 with tactile stimulus effect from the stamps 700 with tactile stimulus effect displayed on a display unit 302 of the user terminal 300 to send the selected stamp 700 with tactile stimulus effect; thereby, a predetermined tactile control signal corresponding to the tactile stimulus given to the stamp 700 is sent to the tactile stimulus unit (so-called “social tipping” with the tactile stimulus effect is made). Furthermore, the tactile stimulus unit presents, to the wearer, the tactile stimulus corresponding to the selected stamp 700 with tactile stimulus effect on the basis of the predetermined tactile control signal received. Therefore, if the artist 602 perceives the presented tactile stimulus and takes action, the viewer 600 can confirm, in real time, the action induced by the tactile stimulus presented by the stamp 700 that the viewer 600 has sent”].
Ito does not specifically teach predicting a haptic emoji or a haptic signal based on the input data. However, in the same field of invention, Ihara teaches:
predicting a haptic emoji or a haptic signal based on the input data [(e.g. see Ihara paragraphs 0075, 0076, 0086 and Fig. 6) ”the love expression extraction unit 102 refers to the love expression storage unit 108, and determines whether there is a love expression that matches a versatile love expression/limited love expression on the basis of sensor data (Step S139). The love expressions may be assumed to be movements such as blowing a kiss, clutching the information processing device 1, allowing a partner to listen to heartbeats (attaching the information processing device 1 to his/her chest and collecting heartbeat sound via the microphone 12), or waving a hand. In the case of blowing a kiss, movements of a hand of a user is detected from a video taken by the camera 11 of the information processing device 1, and it is determined that this movement is a love expression of blowing a kiss … the love expression table stored in the love expression storage unit 108. Specifically, the text/love expression conversion unit 106 decodes the code, and extracts a corresponding animation image from the love expression storage unit 108 … The communication AP execution unit 110 displays the love expression information (specifically, animation image) on the chatting screen”].
Therefore, considering the teachings of Ito and Ihara, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add predicting a haptic emoji or a haptic signal based on the input data, as taught by Ihara, to the teachings of Ito because it allows a user to transmit physical expression information in a versatile and convenient manner (e.g. see Ihara paragraphs 0009, 0013).
Ito and Ihara do not specifically teach determining, by a learning module, additional information based on the haptic signal or the haptic emoji, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output or the haptic emoji such that a respective user can learn the haptic output or the haptic emoji. However, in the same field of invention, Chow teaches:
determining, by a learning module, additional information based on the haptic signal or the haptic emoji, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output or the haptic emoji such that a respective user can learn the haptic output or the haptic emoji [(e.g. see Chow paragraphs 0044, 0079, 0099, 0100 and Figs. 5B-D) ”The haptic learning engine 403 is a machine learning algorithm that includes a plurality of classifiers and a haptic learning model, which is an artificial intelligence (AI) model generated by the machine learning algorithm. The haptic learning model is generated by the haptic profiler 400 to learn the play style of the player in order to determine if the player needs assistance during game play, and to identify a haptic setting output that can be used to notify the player or provide assistance … These types of coordinated haptic responses can be customized for the player based on the … learned behavior of the player. For instance, if the player needs more assistance in deciding which button to press, or which direction to move the game object or which direction to turn or which direction to direct the player's attention on the screen, the vibrational cues can be provided using specific components of the controller. In some specific instances, the vibration may be provided with more magnitude or for a longer period of time. If the learning algorithm learns that the player is more comfortable playing the game and does not need additional assistance, the vibrational cues may be automatically adjusted downward … In addition to vibrational cues, the haptic responses may be provided as audio cues, textual cues, visual cues, etc. … FIGS. 5B-5D illustrate some of the setting options that may be used for customizing notifications using controls of a hand-held controller, in some implementations. The customization, as mentioned previously, is done using haptic setting output identified by the haptic learning model 403b. The customization settings can be defined for the controller and for each control button or input interface of the controller 120. FIG. 5B illustrates one such option, option A, wherein the notification setting is defined for the controller, in one implementation. The notification setting is defined in a manner that allows haptic response to be provided on a side of the controller 120 that corresponds with a side the player has to proceed. In the example illustrated in FIG. 5B, the player is nudged to proceed on the right side of the fork toward path C (illustrated by solid line 520) instead of continuing on path A (illustrated by broken line 510) by causing the controller or handle of the controller to vibrate on the right side. Similar notification may be provided to the player if the haptic profiler wants the player to direct their attention to the right side of the screen. FIG. 5C illustrates another option, option B, wherein the notification setting is defined for the input interface of the input device (e.g., touch screen of the controller 120), in one implementation. In the example illustrated in FIG. 5C, a directional cue is provided on the touch screen using arrow heads moving from left to right (shown by the directional arrow) to indicate the direction the haptic profiler wants the player to proceed or pay attention to in the game scenario rendered on the display screen 100. In one implementation, the intensity of the directional cue may increase from left to right to provide additional feedback to the player. In addition to visual cue, the notification may also include audio cues. For example, as the arrow heads move from left to right the audio cue may increase in volume or provide a distinct sound. Alternatively, special audio tunes may be set up either by the player or for the player to indicate the direction the player has to direct their attention to in the game or on the display screen rendering the game play. In some implementation, separate audio tunes may be set to indicate a correct direction or an incorrect direction … FIG. 5D illustrates yet another setting option, option C, wherein each button on the controller 120 is set to provide a notification/feedback, in one implementation. The setting of each button may be defined differently to indicate different feedback responses to the player. For example, each button may be set to provide a haptic feedback. In place of or in addition to haptic feedback, each button may also include visual setting by allowing each button to light up in a different color or in different sequence of colors to indicate to the player the different button presses (single press vs. multiple presses) that the player is performing or has to perform. It should be noted that the various setting options described with reference to FIGS. 5B-5D have been described in relation to various settings for different controls on a controller for providing haptic and/or visual feedback”].
Therefore, considering the teachings of Ito, Ihara and Chow, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add determining, by a learning module, additional information based on the haptic signal or the haptic emoji, wherein i) the additional information is configured with supplemental content to be presented and ii) the supplemental content is descriptive of the haptic output or the haptic emoji such that a respective user can learn the haptic output or the haptic emoji, as taught by Chow, to the teachings of Ito and Ihara because it allows the user to be provided with sufficient cues to detect the interactive task (e.g. see Chow paragraph 0006).
As for dependent claim 7, Ito, Ihara and Chow teach the system as described in claim 6 and Ito further teaches:
wherein the processing further comprises obtaining the digital assets for the haptic emoji, and the digital assets comprise the haptic signal configured with parameter information to generate patterns for haptic output [(e.g. see Ito paragraphs 0113, 0118 and Fig. 2) ”display the screen as illustrated in FIG. 2 as a screen on which to select the stamp 700 with tactile stimulus effect. A large number of stamps 700 with tactile stimulus effect are displayed on the selection screen (stamp selection screen). Each stamp 700 with tactile stimulus effect is correlated with a control command for the tactile presentation device 100, and the viewer 600 can select the stamp 700 displayed on the selection screen to input a control command. In the embodiment, it is preferable that an image of the stamp 700 with tactile stimulus effect intuitively evokes the tactile stimuli, thoughts (message), and so on that the viewer 600 wishes to send to the artist 602 … presenting a predetermined image or a predetermined sound to the viewer 600 or the artist 602. For example, the predetermined image can be a virtual object such as an icon, an animation, or text that is superimposed on an image of the artist 602 or an image of the real space around the artist 602, or is projected onto the artist 602; and more specifically, the predetermined image can be a trajectory of a social tip moving from the viewer 600 to the artist 602, an animation that looks like a bomb exploding, which is displayed at the same time as the tactile stimulus is presented, or an icon corresponding to the stamp 700 with tactile stimulus effect inputted by the viewer 600”].
As for dependent claim 8, Ito, Ihara and Chow teach the system as described in claim 6; further, claim 8 discloses substantially the same limitations as claim 3. Therefore, it is rejected with the same rational as claim 3.
As for dependent claim 9, Ito, Ihara and Chow teach the system as described in claim 6; further, claim 9 discloses substantially the same limitations as claim 4. Therefore, it is rejected with the same rational as claim 4.
As for dependent claim 10, Ito, Ihara and Chow teach the system as described in claim 7; further, claim 10 discloses substantially the same limitations as claim 5. Therefore, it is rejected with the same rational as claim 5.
As for independent claim 11, Ito teaches a system comprising:
a head-mounted device comprising a display to display content to a first user, one or more sensors to capture input data including images of a visual field of the first user [(e.g. see Ito paragraphs 0100, 0127, 0140 and Figs. 2 and 7) ”the user terminal 300 can be the smartphone 300a, the tablet personal computer (PC) 300b, a cell phone, a laptop PC, or a wearable device such as a head mounted display (HMD) … The display unit 302 can display, for the viewer 600, a selection screen for selecting a stamp (control command) 700 with tactile stimulus effect … the camera 322 captures an image of motion or the like of the viewer 600 to output the captured image to the control unit 310. The control unit 310 can then extract predetermined motion of the viewer 600 from the image captured by the camera 322 and acquire a control command related to the extracted motion”].
one or more processors [(e.g. see Ito paragraph 0125) ”the detailed configuration of the user terminal 300 according to the embodiment of the present disclosure is described with reference to FIG. 7 … the user terminal 300 mainly includes … a control unit 310”].
one or more memories accessible to the one or more processors, the one or more memories storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising: [(e.g. see Ito paragraph 0137) ”The storage unit 312 is implemented by a storage device such as ROM that stores programs, operation parameters, and the like used for the processing of the control unit 310, and RAM that temporarily stores parameters and the like changing as appropriate”].
receiving, at the head-mounted device, a haptic signal and additional information from a second user [(e.g. see Ito paragraph 0162) ”for example, as with the first embodiment, when the artist 602 selects and sends the stamp 700 with tactile stimulus effect, the tactile presentation device 100 of the viewer 600 presents the tactile stimulus also to the viewer 600. Specifically, in the modification example, the server 200 acquires the stamp 700 with tactile stimulus effect inputted by the artist 602, generates a tactile control signal according to a control command correlated with the acquired stamp 700, and sends the generated tactile control signal to the tactile presentation device 100 worn on the body of the viewer 600”].
wherein the haptic signal is configured with parameter information on interval, pitch, amplitude, or a combination thereof for a touch message to be perceived by the first user’s body [(e.g. see Ito paragraphs 0116, 0170) ”The generation unit 218 generates a tactile control signal for controlling the tactile presentation device 100 on. the basis of the control command correlated with the stamp 700 with tactile stimulus effect inputted by the viewer 600, and outputs the tactile control signal to the determination unit 220 described later. Specifically, the generation unit 218 generates, on the basis of the control command including a perceptual location of a vibration stimulus (positional information), a waveform type of the vibration stimulus, and intensity information on the vibration stimulus (mode information), waveform data to be inputted to the individual tactile stimulus units of the tactile presentation device 100 so as to present a vibration stimulus with the intensity and waveform designated in the control command at the perceptual location designated therein. Note that the tactile control signal may include information for giving a command to present the vibration stimulus at a presentation timing, frequency, interval, and presentation time or a tactile stimulus based on the control command … set also a waveform type, intensity, presentation timing, frequency, interval, presentation time, and the like of the vibration stimulus”].
determining parameters of one or more actuator signals based on the haptic signal [(e.g. see Ito paragraph 0097) ” generate a tactile control signal in accordance with the stamp 700, and send the generated tactile control signal to the tactile presentation device 100. Each stamp 700 with tactile stimulus effect is correlated with each predetermined control command, and each control command includes information for designating a perceptual location where a vibration. stimulus is to be presented (positional information.), and information for designating a waveform type and intensity of the vibration stimulus for example (mode information). Further, the control command can include identification information (ID) for identifying the tactile stimulus unit of the tactile presentation device 100 and the mode information. Specifically, the server 200 generates a tactile control signal (waveform data) to be inputted to the individual tactile stimulus units of the tactile presentation device 100 so as to present a vibration stimulus with the intensity and waveform designated in the control command at the perceptual location designated therein, and sends the tactile control signal to the tactile presentation device 100”].
generating the one or more actuator signals based on the parameters determined for the one or more actuator signals; transmitting the one or more actuator signals to one or more corresponding cutaneous actuators [(e.g. see Ito paragraphs 0082, 0273) ”The tactile presentation device 100 includes a plurality of tactile stimulus units (for example, actuators or the like) the Furthermore, in response to a predetermined signal received, the tactile stimulus units can present tactile stimuli to the wearer … The drive amplifier/interface 140 is an interface that sends and receives a tactile control signal between the tactile presentation device 100 and the haptic server 250a. For example, the drive amplifier/interface 140 acquires the profile information (functional information or the like) of the tactile presentation device 100 from the tactile presentation device 100, or converts and amplifies a tactile control signal generated by the haptic: server 250a, and sends the signal to the tactile presentation device 100”].
generating, by the one or more cutaneous actuators, haptic output in accordance with the corresponding one or more actuator signals, wherein the haptic output conveys the touch message to the first user’s body [(e.g. see Ito paragraphs 0082, 0103, 0113) ”selects one stamp 700 with tactile stimulus effect from the stamps 700 with tactile stimulus effect displayed on a display unit 302 of the user terminal 300 to send the selected stamp 700 with tactile stimulus effect; thereby, a predetermined tactile control signal corresponding to the tactile stimulus given to the stamp 700 is sent to the tactile stimulus unit (so-called “social tipping” with the tactile stimulus effect is made). Furthermore, the tactile stimulus unit presents, to the wearer, the tactile stimulus corresponding to the selected stamp 700 with tactile stimulus effect on the basis of the predetermined tactile control signal received … The tactile stimulus unit 106 includes, for example, an actuator, is driven by a tactile control signal generated by the server 200 to generate vibrations, and presents such vibrations as a tactile stimulus … it is preferable that an image of the stamp 700 with tactile stimulus effect intuitively evokes the tactile stimuli, thoughts (message), and so on that the viewer 600 wishes to send”].
Ito does not specifically teach the haptic signal is determined by one or more of a lexicon converter and an artificial intelligence system using input data from the second user. However, in the same field of invention, Ihara teaches:
the haptic signal is determined by one or more of a lexicon converter and an artificial intelligence system using input data from the second user [(e.g. see Ihara paragraphs 0075, 0076, 0086 and Fig. 6) ”the love expression extraction unit 102 refers to the love expression storage unit 108, and determines whether there is a love expression that matches a versatile love expression/limited love expression on the basis of sensor data (Step S139). The love expressions may be assumed to be movements such as blowing a kiss, clutching the information processing device 1, allowing a partner to listen to heartbeats (attaching the information processing device 1 to his/her chest and collecting heartbeat sound via the microphone 12), or waving a hand. In the case of blowing a kiss, movements of a hand of a user is detected from a video taken by the camera 11 of the information processing device 1, and it is determined that this movement is a love expression of blowing a kiss … the love expression table stored in the love expression storage unit 108. Specifically, the text/love expression conversion unit 106 decodes the code, and extracts a corresponding animation image from the love expression storage unit 108 … The communication AP execution unit 110 displays the love expression information (specifically, animation image) on the chatting screen”].
Therefore, considering the teachings of Ito and Ihara, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add the haptic signal is determined by one or more of a lexicon converter and an artificial intelligence system using input data from the second user, as taught by Ihara, to the teachings of Ito because it allows a user to transmit physical expression information in a versatile and convenient manner (e.g. see Ihara paragraphs 0009, 0013).
Ito and Ihara do not specifically teach the additional information is determined by a learning module using the haptic signal determined by one or more of the lexicon converter and the artificial intelligence system, the additional information is configured with supplemental content to be presented at the head-mounted device; transmitting the supplemental content to one or more corresponding output components of the head-mounted device; and causing presentation of the supplemental content at the one or more corresponding output components of the head-mounted device, wherein the supplemental content is descriptive of the haptic output such that a respective user can learn the haptic output. However, in the same field invention, Chow teaches:
the additional information is determined by a learning module using the haptic signal determined by one or more of the lexicon converter and the artificial intelligence system, the additional information is configured with supplemental content to be presented at the head-mounted device; transmitting the supplemental content to one or more corresponding output components of the head-mounted device; and causing presentation of the supplemental content at the one or more corresponding output components of the head-mounted device, wherein the supplemental content