DETAILED ACTION
DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
2. The information disclosure statements (IDS) submitted on the following dates are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner: 11/20/2024.
Specification
3. The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested:
INFORMATION PROCESSING SYSTEM, APPARATUS, AND METHOD FOR IDENTIFYING THE VIRTUAL SPACE IMAGE WITH GAZING POINT AND FEATURE POINT.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 1-2, 5, 8-11 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Sasa et al. (machine translation of JP-2005058449-A cited by applicant, hereinafter "Sasa") in view of Ukiyo et al. (machine translation of JP-2018097664-A cited by applicant, hereinafter "Ukiyo")
Regarding claim 1, Sasa discloses an information processing system (Sasa- Fig. 1 and ¶0022, at least disclose an imaging information recording / reproducing apparatus 100) comprising:
an information terminal (Sasa- Fig. 1 and ¶0022, at least disclose an imaging information recording / reproducing apparatus 100 as an example of a visualization apparatus); and
an information processing apparatus that is capable of communicating with the information terminal (Sasa- Fig. 1 and ¶0022, at least disclose an imaging information recording / reproducing apparatus 100; ¶0024, at least discloses with respect to the system bus 102, an imaging unit controller 105, an imaging signal processing unit 106, a biological information input unit controller 107, an emotion analysis processing unit 108, a data storage unit controller 109, a reproduction processing unit 110, A timer unit 111, a visual data processing unit 112, a display unit controller 113, a key input interface (in the figure, the interface is described as I / F; the same applies hereinafter) 114 and a network interface 115 are connected), wherein
the information terminal includes:
an operation receiving part that receives an operation of a user (Sasa- Fig. 1 and ¶0026-0027, at least disclose The imaging unit 121 includes a lens system and an imaging device configured by, for example, a CCD (Charge Coupled Device), receives an imaging unit control signal from the imaging unit controller 105, and starts and stops imaging recording, zoom magnification, The exposure value, auto focus, etc. are controlled […] The captured image signal from the imaging unit 121 is converted into a digital signal by an A / D (Analog to Digital) converter 122 and then supplied to the imaging signal processing unit 106; Fig. 1 and ¶0043, at least disclose A key input operation unit 128 is connected to the key input interface 114. The key input operation unit 128 includes keys such as an imaging recording start key, a reproduction start key, a fast forward key, a rewind key, a jump key, and a stop key. When any key of the key input operation unit 128 is operated, the operation input information is supplied to the system bus 102 through the key input interface 114; ¶0130, at least discloses the user performs an event operation for changing the playback position such as fast forward, rewind, and jump through the key input operation unit 128), and
a display processing part that displays a space image by changing a displayed region and a displaying orientation of space image data provided from the information processing apparatus, on the basis of an operation of the user received by the operation receiving part (Sasa- Fig. 1 and ¶0042, at least disclose The display unit 127 [display processing part] is a display such as an LCD (Liquid Crystal Display), and displays an image [space image data] corresponding to an image signal from the display unit controller 113 on the display screen; ¶0072, at least discloses when evaluating emotional states by judging the user's posture [operation of the user] by using an acceleration sensor or tilt sensor and detecting whether the chest is worn or bent forward; Fig. 4 shows color output as space image data; ¶0082, at least discloses As the color change from the lower right corner to the upper right corner in FIG. 4, the ratio of the mixed color of red gradually increases from pure yellow to pure red. Here, the mixed color of yellow and red is orange [changing a displayed region] […] Similarly, the color change from the upper left corner to the upper right corner in FIG. 4 is as follows: dark magenta color → magenta color that gradually fades with blue → red that gradually fades with yellow → dark red It will change; Fig. 5C and ¶0094, at least disclose FIG. 5C shows a circle type among the time-series display types. A predetermined period in which emotion is to be displayed is assigned to one round of the annular area, and the color output corresponding to emotion is This is a display mode in which images are sequentially displayed in a predetermined width unit corresponding to the unit time interval in the circumferential direction [displaying orientation]; ¶0130, at least discloses the user performs an event operation for changing the playback position such as fast forward, rewind, and jump through the key input operation unit 128), and
the information processing apparatus includes:
a data acquisition part that acquires display state data for identifying the space image viewed by the user (Sasa- ¶0015, at least discloses when the predetermined period corresponds to, for example, a period during which a predetermined broadcast program or movie content is viewed, the state distribution of emotion when the predetermined broadcast program or movie content is viewed […] it is possible to visualize in what emotional state the broadcast program or movie content is viewed; Fig. 1 and ¶0030, at least disclose the biological information input unit 124 [data acquisition part] constitutes means for acquiring biological information of the user (photographer) of the imaging information recording / reproducing apparatus 100; Fig. 2 and ¶0047, at least disclose the biological information input unit 124 of this example is worn by a person and corresponds to a plurality of biological information, in this example, brain waves, electrocardiograms, pulse waves, body temperature, myoelectricity, and the like. , 1241e and a plurality of electrodes 1241a, 1241b, 1241c, 1241d,..., 1241e for collecting biological signals, and for amplifying the biological signals from the respective electrodes 1241a, 1241b, 1241c, 1241d,. The plurality of amplifiers 1242a, 1242b, 1242c, 1242d),
a gazing identification part that identifies a gazing time point at which the user viewing the space image gazes at least at a portion of the space image for a
predetermined time or longer, on the basis of the display state data (Sasa- Figs. 5A-5C shows several examples of time-series display types as time point; ¶0012, at least discloses in the emotion visualization device described in claim 1, the emotion analysis processing unit averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals [predetermined time] to obtain emotion information representing the average emotion for each unit time interval, and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0015, at least discloses when the predetermined period [predetermined time] corresponds to, for example, a period [time point ] during which a predetermined broadcast program or movie content is viewed [gazing], the state distribution of emotion [display state data] when the predetermined broadcast program or movie content is viewed […] it is possible to visualize in what emotional state the broadcast program or movie content is viewed; ¶0018, at least discloses In addition, when the predetermined period corresponds to, for example, a period during which a predetermined broadcast program or movie content is viewed [gazing time point at which the user viewing the space image gazes at least at a portion of the space image for a predetermined time], the state distribution of emotion when the predetermined broadcast program or movie content is viewed It can be visualized as a color distribution. Accordingly, it is possible to visualize in what emotional state the broadcast program or movie content is viewed),
a feature identification part that identifies a feature manifestation time point that
is a point in time at which a specific feature is included in the virtual space image displayed on the information terminal (Sasa- Figs. 5A-5C shows several examples of time-series display types as time point; ¶0012, at least discloses in the emotion visualization device described in claim 1, the emotion analysis processing unit [a feature identification part] averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information [feature manifestation] representing the average emotion for each unit time interval [feature manifestation time point], and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0021, at least discloses An example of an emotion visualization device is an imaging information recording and playback device which includes an imaging unit, and has a function to capture the photographer's biological information while imaging is being taken by the imaging unit, perform emotion analysis, convert the result of the emotion analysis into color data, and record it together with the imaging information and in relation to time information), and
an output part that outputs the gazing time point and the feature manifestation
time point in association with each other (Sasa- Figs. 5A-5C shows several examples of time-series display types as time point; ¶0012, at least discloses in the emotion visualization device described in claim 1, the emotion analysis processing unit averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information [feature manifestation] representing the average emotion for each unit time interval [feature manifestation time point], and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information [outputs] representing the average emotion for each unit time interval from the emotion analysis processing unit; Fig. 1 and ¶0042, at least disclose The display unit 127 [an output part] is a display such as an LCD (Liquid Crystal Display), and displays an image [space image data] corresponding to an image signal from the display unit controller 113 on the display screen; ¶0085-0086, at least disclose The visual data processing unit 112 does not output the color information obtained by converting the emotion parameter value using the color conversion table information as it is, but changes the emotion corresponding to the passage of time in chronological order. The output is performed in a display output manner that can be understood (hereinafter, this type of display output manner is referred to as a time-series display type) […] the visual data processing unit 112 determines how much of each emotion such as excitement, relaxation, comfort, and discomfort is included in the predetermined time. It is also possible to output in a display output mode (hereinafter, this type of display output mode is referred to as a distribution display type) represented by a color distribution corresponding to each emotion in the region; Fig. 5A-5C and ¶0089, at least disclose a predetermined unit time is used as the averaging interval, the emotional data obtained within the unit time interval of the averaging interval is averaged, and the averaged emotional data is converted into color information using a color conversion table. Therefore, in the examples of Figures 5(A), (B), and (C), the color information output corresponding to the emotion data is performed in predetermined width units corresponding to the unit time interval).
Sasa does not explicitly disclose, but Ukiyo discloses
a virtual space image (Ukiyo- ¶0013, at least discloses The image used by the information processing system 1 is used for allowing the user to experience virtual reality. The user can experience virtual reality by viewing an environment that feels substantially like reality through the information processing system 1 (more specifically, a display). An example of an image for allowing a user to experience virtual reality is an image of a landscape including at least one of a natural object and an artificial object, but the object represented by the image is not limited at all. For example, the image may be a natural scene, a competition venue, or a concert hall. In view of effective virtual reality, the image may be a 360 ° image (360 ° still image or 360 ° video) that surrounds the user. Of course, the image need not be a 360 ° image.);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa to incorporate the teachings of Ukiyo, and apply the image for allowing a user to experience virtual reality into Sasa’s teachings in order a display processing part that displays a virtual space image by changing an displayed region and a displaying orientation of virtual space image data provided from the information processing apparatus, on the basis of an operation of the user received by the operation receiving part, and the information processing apparatus includes: a data acquisition part that acquires display state data for identifying the virtual space image viewed by the user, a gazing identification part that identifies a gazing time point at which the user viewing the virtual space image gazes at least at a portion of the virtual space image for a predetermined time or longer, on the basis of the display state data, a feature identification part that identifies a feature manifestation time point that is a point in time at which a specific feature is included in the virtual space image displayed
Doing so would effectively convey to a user an image and sound suitable for a desired scene using a virtual reality mechanism.
Ukiyo further discloses
changing a displayed region and a displaying orientation of virtual space image data (Ukiyo- ¶0034, at least discloses When the image displayed on the display 20 is a 360 ° image, the data processing unit 17 changes the angle of the image so that the portion including the gazing point is reflected in the reproduced image area 112);
a gazing time point at which the user viewing the virtual space image gazes at least at a portion of the virtual space image for a predetermined time or longer (Ukiyo- ¶0016, at least discloses The viewpoint data is data indicating the portion of the image that the user is viewing, that is, the user's gaze point; ¶0034, at least discloses The data processing unit 17 displays the gazing point in the reproduced image area 112 based on the viewpoint data. When the image displayed on the display 20 is a 360 ° image, the data processing unit 17 changes the angle of the image so that the portion including the gazing point is reflected in the reproduced image area 112. In addition, the data processing unit 17 draws the graph 113 based on the biometric data. The data processing unit 17 updates the display of the gazing point and the graph every time [gazing time point] the viewpoint data and the biological data are input from the receiving unit 16. Therefore, the user's viewpoint and biometric data are displayed on the interface screen 110 in real time).
Regarding claim 2, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the data acquisition part (As discussed above) acquires end data indicating that the user ends viewing the virtual space image, from the information terminal (Sasa- ¶0112-0113, at least disclose it is determined whether or not the user has entered an operation to end recording (step S207). If it is determined that the user has not entered an operation to end recording, the process returns to step S203 and repeats the process from step S203 onward […] when it is determined in step S207 that the user has given an input to end recording, the recorded emotion data for the entire recorded time interval is read from the data storage unit 123 via the data storage unit controller 109, and as shown in Figure 6, emotion visual data of the display type set in step S202 is created and stored in the data storage unit 123 in association with the information recorded by imaging (step S208); Ukiyo- ¶0035, at least discloses When the ZIP button 116 is pressed, the data processing unit 17 generates medical record data including the input set of biological data (a series of biological data obtained from the start to the end of measurement by the biological measuring instrument 40) and a user identifier, and stores the medical record data in the medical record database 18), and
the output part (As discussed above) outputs information indicating a viewing end time point indicated by the end data in association with the gazing time point and the feature manifestation time point (Sasa- ¶0015, at least discloses when the predetermined period corresponds to, for example, a period during which a predetermined broadcast program or movie content is viewed [gazing], the state distribution of emotion when the predetermined broadcast program or movie content is viewed […] it is possible to visualize in what emotional state the broadcast program or movie content is viewed; ¶0112-0113, at least disclose it is determined whether or not the user has entered an operation to end recording (step S207). If it is determined that the user has not entered an operation to end recording, the process returns to step S203 and repeats the process from step S203 onward […] when it is determined in step S207 that the user has given an input to end recording, the recorded emotion data for the entire recorded time interval is read from the data storage unit 123 via the data storage unit controller 109, and as shown in Figure 6, emotion visual data of the display type set in step S202 is created and stored in the data storage unit 123 in association with the information recorded by imaging (step S208)).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa to incorporate the teachings of Ukiyo, and apply the start to the end of measurement by the biological into Sasa’s teachings for indicating that the user ends viewing the virtual space image, from the information terminal, and indicating a viewing end time point indicated by the end data in association with the gazing time point and the feature manifestation time point.
The same motivation that was utilized in the rejection of claim 1 applies equally to this claim.
Regarding claim 5, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the feature identification part identifies the feature manifestation time point (see Claim 1 rejection for detailed analysis) including at least one of a type, a size, or a position of guidance information indicating a route along which the user can follow in the virtual space image (Sasa- ¶0163, at least discloses when recording while viewing a broadcast program, biometric information of the viewer is collected, emotion data is analyzed from the collected biometric information, and emotion visual data is generated from the emotion data. It is also possible to record the recorded information together with the recorded information and display the emotion visualized image in accordance with the reproduced image when the recorded information is reproduced).
Regarding claim 8, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the feature identification part (As discussed above) identifies a proportion of time during which each of a plurality of objects is displayed in the virtual space image within a predetermined time including the gazing time point (Sasa- ¶0008, at least discloses only the average emotion expressed for a certain period expressed in the sentence is represented. It is not possible to represent the state distribution of emotions such as how much each of various emotions such as “relaxed”, “uncomfortable”, and “comfortable” was distributed during the period; ¶0016, at least discloses Emotion distribution visualization means for visualizing the state distribution of the emotion within the predetermined period by visualizing a ratio of each color output by the color information from the color conversion processing unit for a predetermined period), and
the output part outputs information indicating the proportion of time in association with the virtual space image (Sasa- ¶0008, at least discloses only the average emotion expressed for a certain period expressed in the sentence is represented. It is not possible to represent the state distribution of emotions such as how much each of various emotions such as “relaxed”, “uncomfortable”, and “comfortable” was distributed during the period; ¶0016, at least discloses Emotion distribution visualization means for visualizing the state distribution of the emotion within the predetermined period by visualizing a ratio of each color output by the color information from the color conversion processing unit for a predetermined period).
Regarding claim 9, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the output part (As discussed above) outputs information indicating a time during which the user continues viewing the virtual space image (Sasa- ¶0012, at least discloses the emotion analysis processing unit averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information representing the average emotion for each unit time interval, and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0164, at least discloses after viewing the broadcast program, a time series display type or distribution display type icon-like emotion visualization image that visualizes the emotion during the period of watching the broadcast program is printed on a recording sheet or sticker).
Regarding claim 10, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the information processing apparatus (As discussed above) further includes a storage that stores an attribute of the user (Sasa- ¶0118, at least discloses in the management information, the title, the owner, the disclosure attribute, the recording start date and time, and the recording time are recorded for the entire captured and recorded information. For emotional visual data, the address ( (Location information), data attributes, and display type are recorded. For captured image data and audio data, their addresses (location information) and data attributes are recorded, and for emotion data, their addresses (location information). Information attributes, emotion models, emotion parameters, and sampling intervals are recorded.), and
the output part (As discussed above) outputs the gazing time point and the feature manifestation time point (see Claim 1 rejection for detailed analysis) in association with the attribute of the user (Sasa- ¶0103, at least discloses the generated emotion visual data (emotion visualized image data) is stored together with the attribute data (step S108); ¶0118, at least discloses in the management information, the title, the owner, the disclosure attribute, the recording start date and time, and the recording time are recorded for the entire captured and recorded information. For emotional visual data, the address ( (Location information), data attributes, and display type are recorded. For captured image data and audio data, their addresses (location information) and data attributes are recorded, and for emotion data, their addresses (location information). Information attributes, emotion models, emotion parameters, and sampling intervals are recorded.).
Regarding claim 11, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the output part (As discussed above) displays a plurality of images (Sasa- ¶0094, at least discloses FIG. 5C shows a circle type among the time-series display types. A predetermined period in which emotion is to be displayed is assigned to one round of the annular area, and the color output corresponding to emotion is This is a display mode in which images are sequentially displayed in a predetermined width unit corresponding to the unit time interval in the circumferential direction.), each showing the gazing time point and the feature manifestation time point (see Claim 1 rejection for detailed analysis) in a time sequence manner on the same time axis (Sasa- ¶0041, at least discloses at the time of reproduction, the visual data processing unit 112 reproduces the captured image data and emotion visualization data read from the data storage unit 123 in synchronization with the time axis, and through the display unit controller 113, the visual data processing unit 112 reproduces the captured image data and emotion visualization data; ¶0087, at least discloses 5A, 5B, and 5C show some examples of the time series display type; ¶0094, at least discloses FIG. 5C shows a circle type among the time-series display types. A predetermined period in which emotion is to be displayed is assigned to one round of the annular area, and the color output corresponding to emotion is This is a display mode in which images are sequentially displayed in a predetermined width unit corresponding to the unit time interval in the circumferential direction.), the images corresponding to different user viewing durations during which the user continues viewing the virtual space images (Sasa- ¶0094, at least discloses FIG. 5C shows a circle type among the time-series display types. A predetermined period in which emotion is to be displayed is assigned to one round of the annular area, and the color output corresponding to emotion is This is a display mode in which images are sequentially displayed in a predetermined width unit corresponding to the unit time interval in the circumferential direction; ¶0164, at least discloses after viewing the broadcast program, a time series display type or distribution display type icon-like emotion visualization image that visualizes the emotion during the period of watching the broadcast program is printed on a recording sheet or sticker).
Regarding claim 17, Sasa in view of Ukiyo, discloses an information processing device (Sasa- Fig. 1 and ¶0022, at least disclose an imaging information recording / reproducing apparatus 100) comprising:
a data acquisition part that acquires display state data for identifying a virtual
space image, viewed by a user (see Claim 1 rejection for detailed analysis), in which a displayed region and a displaying orientation are changed by an operation of the user (see Claim 1 rejection for detailed analysis);
a gazing identification part that identifies a gazing time point at which the user gazes at least at a portion of the virtual space image for a predetermined time or more (see Claim 1 rejection for detailed analysis);
a feature identification part that identifies a feature manifestation time point that is
a point in time at which specific feature is contained in the virtual space image displayed on the information terminal used by the user (see Claim 1 rejection for detailed analysis); and
an output part that outputs the gazing time point and the feature manifestation time point in association with each other (see Claim 1 rejection for detailed analysis).
Regarding claim 18, Sasa in view of Ukiyo, discloses an information processing method, executed by a computer, the method (Sasa- ¶0001, at least discloses a device and method for visualizing a person's daily emotions and emotions when viewing content, and a visualized output of emotions visualized by the devices and methods; Fig. 1 and ¶0023, at least disclose The imaging information recording / reproducing apparatus 100 includes a computer in which a ROM (Read Only Memory) 103 and a RAM (Random Access Memory) 104 are connected to a CPU (Central Processing Unit) 101 via a system bus 102; Fig. 6 and ¶0098, at least disclose is a flowchart for explaining visual data creation processing in the visual data processing unit 112) comprising the steps of:
acquiring display state data for identifying a virtual space image, viewed by a user, in which a displayed region and a displaying direction are changed by an operation of the user (see Claim 1 rejection for detailed analysis);
identifying a gazing time point at which the user gazes at least at a portion of the virtual space image for a predetermined time or longer (see Claim 1 rejection for detailed analysis);
identifying a feature manifestation time point at which specific feature is contained in the virtual space image displayed on the information terminal used by the user (see Claim 1 rejection for detailed analysis); and
outputting the gazing time point and the feature manifestation time point m association with each other (see Claim 1 rejection for detailed analysis).
6. Claims 3 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Sasa in view of Ukiyo, further in view of Inagaki et al., (“Inagaki”) [WO-2018034113-A1]
Regarding claim 3, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the output part outputs state data indicating a result of a biological state of the user on the basis of the gazing time point and the feature manifestation point (Sasa- Figs. 5A-5C shows several examples of time-series display types as time point; ¶0012, at least discloses in the emotion visualization device described in claim 1, the emotion analysis processing unit averages and analyzes the biological information [biological state] from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information [feature manifestation] representing the average emotion for each unit time interval [feature manifestation time point], and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information [outputs] representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0015, at least discloses emotional changes over a predetermined period, for example, a specific day, are visualized as changes in color corresponding to changes in time, making it possible to visualize what kind of emotional state the person was in during that day [..,] visualizing the distribution of the emotional state within a predetermined period by visualizing the proportion of each color output from the color conversion processing unit based on the color information over a predetermined period; ¶0085-0086, at least disclose The output is performed in a display output manner that can be understood (hereinafter, this type of display output manner is referred to as a time-series display type) […] the visual data processing unit 112 determines how much of each emotion such as excitement, relaxation, comfort, and discomfort is included in the predetermined time. It is also possible to output in a display output mode (hereinafter, this type of display output mode is referred to as a distribution display type) represented by a color distribution corresponding to each emotion in the region; Fig. 5A-5C and ¶0089, at least disclose a predetermined unit time is used as the averaging interval, the emotional data obtained within the unit time interval of the averaging interval is averaged, and the averaged emotional data is converted into color information using a color conversion table. Therefore, in the examples of Figures 5(A), (B), and (C), the color information output corresponding to the emotion data is performed in predetermined width units corresponding to the unit time interval; Ukiyo- ¶0016, at least discloses Biometric data is data indicating the physical or physiological state of a user. The biological data processed by the information processing system 1 is not limited at all. For example, the physiological data may be data indicating heartbeat, pulse, blood pressure, body temperature, electroencephalogram, sweating, eye movement, respiration, and the like).
The prior art does not explicitly disclose, but Inagaki discloses
a result of estimating a biological state of the user (Inagaki- page 5, 2nd - 3rd paragraphs, at least discloses The biological signal acquisition unit 12 is a noise removal process that removes noise from the biological signal, a prediction process that estimates the value of the biological signal in terms of time […] the feature-value calculation part 14 can also estimate the physical state and mental state of the user 3 by analyzing a biological signal. Here, the feature amount is data that characterizes the biological signal, and includes, for example, the magnitude of the biological signal, the maximum value of the biological signal, the minimum value of the biological signal, and the average value of the biological signal (for example, within a predetermined period). The time average value of the magnitude and intensity of the acquired biological signal, etc.), the variance value of the biological signal, the differential value of the biological signal, the amount based on the temporal change of the biological signal, and the like).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Inagaki, and apply testimating the value of the biological signal in terms of time into Sasa/Ukiyo’s teachings in order the output part outputs state data indicating a result of estimating a biological state of the user on the basis of the gazing time point and the feature manifestation point.
Doing so would influence the physical state, mental state, etc. of the user and lead the user to a predetermined state.
Regarding claim 13, Sasa in view of Ukiyo, discloses the information processing system according to claim 11, and does not explicitly disclose, but Inagaki discloses wherein
the output part (As discussed above) outputs a result of identifying a pattern of manifestation of each of a plurality of features of the virtual space image in which the user viewing duration is longer than an average viewing duration (Inagaki- page 6, 7th paragraph, at least discloses A standard in which the average value of the intensity of the electroencephalogram signal within the time is greater than or less than a predetermined value, and a standard in which the rate of change in the intensity of the electroencephalogram signal within the predetermined time is greater than or less than the predetermined rate The change process is executed based on various criteria).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Inagaki, and apply the average value of the intensity of the electroencephalogram signal within the time is less than a predetermined value into Sasa/Ukiyo’s teachings in order to output a result of identifying a pattern of manifestation of each of a plurality of features of the virtual space image in which the user viewing duration is longer than an average viewing duration.
Doing so would influence the physical state, mental state, etc. of the user and lead the user to a predetermined state.
Regarding claim 14, Sasa in view of Ukiyo and Inagaki, discloses the information processing system according to claim 13, and further discloses wherein
the output part (As discussed above)outputs, as a result of identifying the pattern of manifestation (see Claim 1 rejection for detailed analysis), an interval between positions at which a specific feature is contained in the virtual space image in which the user viewing duration is longer than an average viewing duration (Sasa- ¶0102, at least discloses In the display graphic region corresponding to the display type, image data is generated so as to draw the color based on the color conversion output information obtained in step S104 at a position corresponding to the corresponding unit time interval. Temporary storage is performed corresponding to the unit time interval (step S106); Inagaki- page 6, 7th paragraph, at least discloses A standard in which the average value of the intensity of the electroencephalogram signal within the time is greater than or less than a predetermined value, and a standard in which the rate of change in the intensity of the electroencephalogram signal within the predetermined time is greater than or less than the predetermined rate The change process is executed based on various criteria).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Inagaki, and apply the average value of the intensity of the electroencephalogram signal within the time is less than a predetermined value into Sasa/Ukiyo’s teachings in order the output part outputs, as a result of identifying the pattern of manifestation, an interval between positions at which a specific feature is contained in the virtual space image in which the user viewing duration is longer than an average viewing duration.
The same motivation that was utilized in the rejection of claim 13 applies equally to this claim.
7. Claims 4 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Sasa in view of Ukiyo, further in view of Honda et al., (“Honda”) [WO-2016189633-A1]
Regarding claim 4, Sasa in view of Ukiyo, discloses the information processing system according to claim 1, and further discloses wherein
the feature identification part identifies the feature manifestation time point at which a composition of the virtual space image displayed on the information terminal emotion analysis processing unit [a feature identification part] averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information [feature manifestation] representing the average emotion for each unit time interval [feature manifestation time point], and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0021, at least discloses An example of an emotion visualization device is an imaging information recording and playback device which includes an imaging unit, and has a function to capture the photographer's biological information while imaging is being taken by the imaging unit, perform emotion analysis, convert the result of the emotion analysis into color data, and record it together with the imaging information and in relation to time information).
The prior art does not explicitly disclose, but Honda discloses
a single-point perspective composition (Honda- Fig. 8 shows a line of perspective drawing is specified for an image included in a moving image showing a space route; page 6, 2nd - 3rd paragraphs, at least discloses the element detection unit 142 stores a plurality of images corresponding to the composition of the perspective projection method in advance in the storage unit 13, and the plurality of images included in the moving image acquired by the acquisition unit 141 are stored in the storage unit 13 , The composition of the perspective projection method may be specified […] when the composition of the image included in the moving image is a one-point perspective composition and the pattern of the movement of the line of sigh corresponding to the image is gaze, the element detection unit 142 determines that the element assumed to be recognized by the user is" Path").
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Inagaki, and apply a one-point perspective composition into Sasa/Ukiyo’s teachings in order the feature identification part identifies the feature manifestation time point at which a composition of the virtual space image displayed on the information terminal contains a single-point perspective composition.
Doing so would provide a degree-of-recognition capable of indexing ease of understanding for a user of a city or a building.
Regarding claim 6, Sasa in view of Ukiyo, discloses the information processing system according to claim 5, and does not explicitly disclose, but Honda discloses wherein
the gazing identification part identifies a point in time at which the user gazes at the guidance information (Honda- page 3, 6th paragraph, at least discloses The wearable terminal 2 photographs a space route of a city or the like to generate a moving image, and generates gaze data of the user. Here, the spatial route is assumed to be a route specified by a designer or the like who investigates easiness of understanding of a city or the like in advance. For example, as information indicating a space route, map information such as a guide board or the like in front of a station, map information or route information displayed on a terminal such as a smartphone, route information heard from an acquaintance (for example, A wording indicating a way guidance)).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Inagaki, and apply a way guidance into Sasa/Ukiyo’s teachings in order the gazing identification part identifies a point in time at which the user gazes at the guidance information.
Doing so would provide a degree-of-recognition capable of indexing ease of understanding for a user of a city or a building.
Regarding claim 7, Sasa in view of Ukiyo and Honda, discloses the information processing system according to claim 6, and further discloses wherein
when a plurality of pieces of the guidance information are included in the virtual space image, the gazing identification part identifies the guidance information being gazed at for a longer time from among the plurality of pieces of the guidance information (Honda- page 3, 6th paragraph, at least discloses The wearable terminal 2 photographs a space route of a city or the like to generate a moving image, and generates gaze data of the user. Here, the spatial route is assumed to be a route specified by a designer or the like who investigates easiness of understanding of a city or the like in advance. For example, as information indicating a space route, map information such as a guide board or the like in front of a station, map information or route information displayed on a terminal such as a smartphone, route information heard from an acquaintance (for example, A wording indicating a way guidance)).
8. Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Sasa in view of Ukiyo, further in view of Sawaki et al., (“Sawaki”) [JP-2018-200566-A]
Regarding claim 12, Sasa in view of Ukiyo, discloses the information processing system according to claim 11, and further discloses wherein
the output part (As discussed above) displays information indicating an average viewing duration of the virtual space image (Sasa- ¶0089, at least discloses in the examples of FIGS. 5A, 5B, and 5C, the predetermined predetermined unit time is set as the averaging interval, and the emotion data obtained in the unit time interval of the averaging interval is averaged. The averaged emotion data is converted into color information using a color conversion table; ¶0101, at least discloses a unit time interval for the first averaging interval is set (step S102). Next, the emotion data within the set unit time interval is averaged, and the emotion parameter value of the average emotion within the unit time interval is calculated (step S103)).
The prior art does not explicitly disclose information indicating an average viewing duration of the virtual space image overlaid on the plurality of images.
However, Sawaki discloses
viewing duration overlaid on the plurality of images (Sawaki- Fig. 16 and ¶0192, at least disclose viewpoint object 1620 of user 190 A is superimposed on cat 1610; Fig. 27 and ¶0212, at least disclose the user 190 operates the viewpoint object 2730 to select the panoramic image 22. As an example, the user 190 superimposes the viewpoint object 2730 on a part of the panoramic image 22 of interest for a predetermined time (for example, 3 seconds)).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Sawaki, and apply superimposing the viewpoint object on a part of the panoramic image into Sasa/Ukiyo’s teachings for indicating an average viewing duration of the virtual space image overlaid on the plurality of images.
Doing so would provide a technique for acquiring the interest of a user based on a line of sight.
9. Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Sasa in view of Ukiyo, further in view of Digirolamo et al., (“Digirolamo”) [US-2018/0314327-A1], still further in view of Inagaki et al., (“Inagaki”) [WO-2018034113-A1]
Regarding claim 15, Sasa in view of Ukiyo, discloses the information processing system according to claim 11, and does not explicitly disclose, but Digirolamo discloses wherein the output part (As discussed above) outputs an improvement strategy for the virtual space image by using a machine learning model generated through deep learning, utilizing the pattern of manifestation of a plurality of features in a large number of virtual space imagesas teacher data (Digirolamo- ¶0014, at least discloses the trained system could be used for generalizing eye movement patterns for a new radiologist in the system to classify their eye movements and determine if their pattern of eye movements in a particular location are a match for what has been seen in past eye movement patterns in other radiologists. In other words, once the deep machine learning has occurred across a group of radiologists; ¶0026, at least discloses teachers can view the information product and determine on what subject and what features the student is spending less or more time. This specific and detailed information about where, for how long, and through the use of which technique the students are viewing a subject will help a teacher to develop a more sophisticated understanding of how the student may be learning and doing the viewing; ¶0053-0054, at least disclose the threshold value is dynamic and may be calculated based on the user's own viewing patterns such as a number of fixations related to a particular subject or based on a number of fixations from previous viewings of similar subjects, or based on deep machine learning (FIG. 8) of how many fixations are typically recorded for the current image being viewed or previous similar images […] the threshold value is dynamic and may be calculated based on the user's own viewing patterns, or based on deep machine learning (FIG. 8) of the durations typically recorded for the current image being viewed or previous similar images).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Digirolamo, and apply deep machine learning into Sasa/Ukiyo’s teachings for outputing an improvement strategy for the virtual space image by using a machine learning model generated through deep learning, utilizing the pattern of manifestation of a plurality of features in a large number of virtual space images.
Doing so would permit a user to view a subject, not drawn from archives, analyze it, and provide an evaluation.
The prior art does not explicitly disclose, but Inagaki discloses
the user viewing duration is longer than an average viewing duration (Inagaki- page 6, 7th paragraph, at least discloses A standard in which the average value of the intensity of the electroencephalogram signal within the time is greater than or less than a predetermined value, and a standard in which the rate of change in the intensity of the electroencephalogram signal within the predetermined time is greater than or less than the predetermined rate The change process is executed based on various criteria).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo/Digirolamo to incorporate the teachings of Inagaki, and apply the average value of the intensity of the electroencephalogram signal within the time is less than a predetermined value into Sasa/Ukiyo/Digirolamo’s teachings in order to output an improvement strategy for the virtual space image by using a machine learning model generated through deep learning, utilizing the pattern of manifestation of a plurality of features in a large number of virtual space images in which the user viewing duration is longer than an average viewing duration, as teacher data.
Doing so would influence the physical state, mental state, etc. of the user and lead the user to a predetermined state.
Regarding claim 16, Sasa in view of Ukiyo, Digirolamo and Inagaki, discloses the information processing system according to claim 15, and further discloses wherein
when data indicating a point in time at which one or more specific features are contained in the virtual space image is inputted (Sasa- Figs. 5A-5C shows several examples of time-series display types as time point; ¶0012, at least discloses in the emotion visualization device described in claim 1, the emotion analysis processing unit [a feature identification part] averages and analyzes the biological information from the plurality of biological information acquisition means at predetermined unit time intervals to obtain emotion information [feature manifestation] representing the average emotion for each unit time interval [feature manifestation time point], and the color conversion processing unit uses the conversion table information of the conversion table information holding unit to output a color corresponding to the emotion information representing the average emotion for each unit time interval from the emotion analysis processing unit; ¶0021, at least discloses An example of an emotion visualization device is an imaging information recording and playback device which includes an imaging unit, and has a function to capture the photographer's biological information while imaging is being taken by the imaging unit, perform emotion analysis, convert the result of the emotion analysis into color data, and record it together with the imaging information and in relation to time information), the machine learning model outputs data that indicates a point in time, closest to the inputted data (Digirolamo- ¶0014, at least discloses the trained system could be used for generalizing eye movement patterns for a new radiologist in the system to classify their eye movements and determine if their pattern of eye movements in a particular location are a match for what has been seen in past eye movement patterns in other radiologists. In other words, once the deep machine learning has occurred across a group of radiologists; ¶0026, at least discloses teachers can view the information product and determine on what subject and what features the student is spending less or more time. This specific and detailed information about where, for how long, and through the use of which technique the students are viewing a subject will help a teacher to develop a more sophisticated understanding of how the student may be learning and doing the viewing; ¶0053-0054, at least disclose the threshold value is dynamic and may be calculated based on the user's own viewing patterns such as a number of fixations related to a particular subject or based on a number of fixations from previous viewings of similar subjects, or based on deep machine learning (FIG. 8) of how many fixations are typically recorded for the current image being viewed or previous similar images […] the threshold value is dynamic and may be calculated based on the user's own viewing patterns, or based on deep machine learning (FIG. 8) of the durations typically recorded for the current image being viewed or previous similar images), at which the specific feature is contained and the viewing duration is longer than an average viewing duration (Inagaki- page 6, 7th paragraph, at least discloses A standard in which the average value of the intensity of the electroencephalogram signal within the time is greater than or less than a predetermined value, and a standard in which the rate of change in the intensity of the electroencephalogram signal within the predetermined time is greater than or less than the predetermined rate The change process is executed based on various criteria).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Sasa/Ukiyo to incorporate the teachings of Digirolamo and Inagaki, and apply the deep machine learning and the average value of the intensity of the electroencephalogram signal within the time is less than a predetermined value into Sasa/Ukiyo’s teachings in order to output an improvement strategy for the virtual space image by using a machine learning model generated through deep learning, utilizing the pattern of manifestation of a plurality of features in a large number of virtual space images in which the user viewing duration is longer than an average viewing duration, as teacher data.
Doing so would influence the physical state, mental state, etc. of the user and lead the user to a predetermined state.
Conclusion
10. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. They are as recited in the attached PTO-892 form.
11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LE whose telephone number is (571)272-5330. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Primary Examiner, Art Unit 2614
Prosecution Insights