DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on January 16, 2026 has been entered.
Status of Claims
Claims 1, 18, and 19 have been amended. Claim 15 has been cancelled. No claims have been newly added. Therefore, claims 1-14 and 17-21 are currently pending.
Response to Remarks/Arguments
Applicant’s amendments and remarks, filed on January 16, 2026, with respect to the previous 35 U.S.C. 103 rejections have been fully considered and are partially persuasive for the following reasons:
Regarding the statement that Sommer does not disclose “adapting the audio output while maintaining modulation based on the same spatial signal as the previously output audio signal” (Remarks, Page 13) in reference to the subject matter from original claim 15 (which has now been incorporated into independent claims 1/18/19), while the examiner understands the concerns on whether Sommer explicitly recites features for anchoring a spatial audio signal to a specific location rather than adjusting how a signal is perceived based on a reaction of the user after missing a turn, the examiner notes that as currently claimed, the only adaptation of the signal described broadly described is to “adapt a parameter of the modulated audio output signal based on the determined reaction or lack of reaction of the assisted person”. While claims are interpreted in light of the specification, limitations from the specification are not read into the claims (MPEP 2145(VI)). Therefore, as Sommer recites features for at least adapting a parameter (e.g. frequency, reverberation, directionality, volume etc.) of the audio output signal, the examiner respectfully considers Sommer as disclosing the subject matter from original claim 15.
However, regarding the statements that Sommer (Remarks, Page 13) and Koga (Remarks, Pages 14-15) do not teach the cognitive feedback described by at least the amended limitations “state of attention” and “attention history”, the examiner agrees. Accordingly, upon further search and consideration of the amended claims, a new ground(s) of 35 U.S.C. 103 rejection in view of newly found art Balsam et al. (US20240311076A1) has been made, as shown in the updated mapping below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 11, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Sommer et al. (US20200264006A1; hereinafter Sommer) in view of Koga (US20120314871A1; hereinafter Koga) and Balsam et al. (US20240311076A1, filed on March 16, 2023; hereinafter Balsam).
Regarding Claims 1 and 18 (independent), Sommer discloses:
a computer-implemented method for assisting a person to perform a task (Abstract, “Methods and apparatus for spatial audio navigation that may, for example, be implemented by mobile multipurpose devices”);
a device for assisting a person to perform a task, the device comprising [at least one sensor, at least one interface, at least one processor] (Paragraph 0043, “A mobile multipurpose device 100 such as a smartphone, tablet, or pad device may include, but is not limited to, one or more processors 104, a memory 130, one or more sensors 120, and a touch-enabled display 102”) to perform the following steps, comprising:
obtaining an audio output signal for reproduction to the assisted person (Paragraph 0036, “The audio source may, for example, include ambient noise sampled from or simulated for the environment, an audio source that the user is listening to (e.g., music, podcasts, audio books, radio broadcasts, phone calls, etc.), or various pre-recorded or generated sounds”);
generating a spatial signal [wherein the spatial signal includes a position information or a direction information] (Paragraph 0069, “The spatial audio navigation system 300 may output right 312A and left 312B audio channels to the headset via a wired or wireless connection so that the user hears the sound in a spatial audio sense. In other words, the user hears the sound as if the sound were coming from a real world location with accurate distance and direction. For example, the system 300 may play a sound through the headset so that the user hears the sound coming from their left, their right, straight ahead, behind, or at some angle. The direction that the sound seems to be coming from may change to guide the user on the path 304. For example, as the user approaches a left turn on the path 304, the sound may be moved to the left…The navigation component 306 may use one or more spatial audio navigation methods to guide the user on the path 304”; Examiner notes that simulating a sound to originate from a target position in order to guide a user indicates that a spatial signal with position information was generated)
modulating the obtained audio output signal with the spatial signal, wherein a perceived position of origin or a direction of the modulated audio output signal corresponds to the position information or the direction information included in the spatial signal (Paragraph 0068, “The user’s current location may be tracked by the device and input to the navigation component 306. In addition, head orientation and motion may be tracked by sensors of the headset or HMD and input to the navigation component 306. The navigation component 306 then adjust the directionality of audio provided by an audio source as the user moves in the real world to guide the user along the path 304 to the target location. In some embodiments, the virtual directionality of the audio may be adjusted by attenuating one or more aspects of audio output to the left and/or right audio channels, including but not limited to volume, frequency, and reverberation. Other aspects of the audio may be adjusted based on the inputs, such as volume and type of audio, to assist in guiding the user along the path 304 to the target location”); and
outputting the modulated audio output signal to the assisted person [via plural speakers or via earphones worn by the assisted person] (Paragraph 0069, “The spatial audio navigation system 300 may output right 312A and left 312B audio channels to the headset via a wired or wireless connection so that the user hears the sound in a spatial audio sense”).
determining a reaction or a lack of reaction of the assisted person to a previously output modulated audio output signal modulated based on a same spatial signal; adapting a parameter of the modulated audio output signal based on the determined reaction or lack of reaction of the assisted person (Paragraph 0086, “As shown in FIG. 7C, if the user misses the turn, the sound may decrease in volume and be moved so that it seems to be coming from behind the user to prompt the user that they have missed the turn. Alternatively, in some embodiments, an alert sound may be played when the user misses a turn. In some embodiments, if a user continues past a turn or otherwise goes off a path determined by the system, the system may recalculate a new path to the target location and begin directing the user on the new path using the spatial audio navigation methods. Instead of or in addition to volume, other aspects of the audio may be attenuated to affect the virtual distance and directionality of the audio, including but not limited to frequency and reverberation”; Examiner notes that the audio output signal is adapted (e.g. frequency, reverberation, directionality, volume etc.) when a user does not react to a missed turn).
While Sommer further discloses that the generated spatial signals are based on information related to the assisted person (Paragraphs 0067-0069 describe location and target location information), Sommer does not explicitly recite: acquiring state information on a state of the assisted person; acquiring target state information on a target state of the assisted person; wherein the state of the assisted person includes an ego-velocity of the assisted person, and the target state includes a predetermined velocity target and generating a spatial signal based on the acquired state information and the acquired target state information.
Nevertheless, Koga teaches an apparatus for generating spatial audio (see at least Abstract) comprising:
acquiring state information on a state of the assisted person (Paragraphs 0007-0008, “According to an embodiment of the present disclosure, there is provided an information processing apparatus including a storage, a sensor, a controller, and a sound output unit. The storage is capable of storing a plurality of sound information items associated with respective positions. The sensor is capable of detecting a displacement of one of the information processing apparatus and a user of the information processing apparatus...The "displacement" is a concept including various changes of a position, direction, velocity, and the like”);
acquiring target state information on a target state of the assisted person (Paragraph 0060, “A position of sound of a target (virtual object) that moves at a target velocity”);
wherein the state of the assisted person includes an ego-velocity of the assisted person, and the target state includes a predetermined velocity target (Paragraphs 0007-0008 describes acquiring “velocity”; Paragraphs 0060-0061 describe the target velocity as being predetermined (“In the processing, the user U sets a target at a target velocity via the display unit 14 of the portable terminal 10”));
[generating a spatial signal] based on the acquired state information and the acquired target state information (Paragraph 0060 describes generating a spatial signal (“position of sound of a target (virtual object)…”) based on the acquired state and target state information (“…that moves at a target velocity changes according to the movement velocity of the user during exercise”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Sommer invention to expand the features for modulating audio signals based on information of an assisted person (see at least Paragraphs 0067-0069) by including features for acquiring and using state information for generating a spatial signal, as taught by Koga, for the benefit of maintaining the assisted person at a target velocity (Koga, Paragraph 0060).
While Sommer further discloses features for collecting information related to user attentiveness (Paragraph 0052, “The HMD 200 may include sensors that collect…information about the user 290 (e.g., the user's expressions, eye movement, head movement, gaze direction, hand gestures, etc.)”), Sommer does not explicitly recite: determining a state of attention and an attention history of the assisted person; generating the spatial signal further based on at least one of the determined state of attention and attention history of the assisted person.
Nevertheless, Balsam teaches features that allow sounds to be adjusted based on a state of attention (see at least Abstract) comprising:
determining a state of attention and an attention history of the assisted person (Paragraphs 0072-0074, “…attention determination module 410 determines whether a user's attention has shifted…based on historical information including attributes associated with the user that may change over time. Examples of such attributes include a gaze point of the user, a position and/or an orientation of the user's eye(s), one or more of the user's biomarkers and/or vital signs, information describing sounds or objects detected in an environment of the user, or any other suitable attributes associated with the user”));
generating the spatial signal further based on at least one of the determined state of attention and attention history of the assisted person (Paragraph 0081 describes a spatialized signal based on user attentiveness (“…a sound detected by the audio system 114 should be modified if the user attention determination module 410 determines that a user's attention has shifted. In this example, the set of rules may indicate that the sound should be modified by enhancing or emphasizing it (e.g., by increasing its volume or changing its spatialization so that it appears to originate from a region closer to the user) if it originates from an object of interest of the user”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to expand the features for collecting sensor information (Paragraph 0052) and generating a spatial signal based on information of an assisted person (see at least Paragraphs 0067-0069) to further factor in attentiveness, as taught by Balsam, for the benefit of optimizing audio to ensure user’s perception of critical information.
Regarding Claim 2, Sommer as currently modified teaches claim 1. Sommer does not explicitly recite: in the step of generating the spatial signal, the spatial signal is generated based on a deviation of the acquired state information and the acquired target state information.
Nevertheless, Koga further teaches:
in the step of generating the spatial signal, the spatial signal is generated based on a deviation of the acquired state information and the acquired target state information (Paragraph 0060 describes the spatial signal (“virtual object”) as being based on the user being slower or faster than the target velocity (“A position of sound of a target (virtual object) that moves at a target velocity changes according to the movement velocity of the user during exercise, and the user U overtakes the target or the target catches up with the user U. Accordingly, the user can virtually compete with the target”) which is reasonably indicative of being based on a deviation between a user velocity and the target velocity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to expand the features for modulating audio signals based on information of an assisted person (see at least Paragraphs 0067-0069) by including features for acquiring and using state information for generating a spatial signal, as taught by Koga, for the benefit of maintaining the assisted person at a target velocity (Koga, Paragraph 0060).
Regarding Claim 3, Sommer as currently modified teaches claim 1. Sommer does not explicitly recite: the position information corresponds to the current location of the assisted person in case of determining that the deviation of the acquired state information and the acquired target state information is below a first predetermined threshold or zero (Examiner notes that this limitation is being interpreted as adjusting the position information of the audio output to induce an assisted person to maintain a specific velocity or pace, as supported by Paragraph 0028 of the instant specification).
Nevertheless, Koga further teaches:
the position information corresponds to the current location of the assisted person in case of determining that the deviation of the acquired state information and the acquired target state information is below a first predetermined threshold or zero (Paragraph 0064 describes adjusting the position/distance of the spatial signal (“…For calculating the relative distance…while the running distance of the user U from the exercise start time point to the calculated time point is calculated from the output of the position sensor 15, a virtual running distance of the target at a certain time point is calculated from the elapse time and the set target velocity, and thus a difference between the two distances is calculated as the relative distance”) to induce the user to maintain a target velocity (“A position of sound of a target (virtual object) that moves at a target velocity changes according to the movement velocity of the user during exercise, and the user U overtakes the target or the target catches up with the user U”); Examiner notes that Paragraph 0066 describes the sound source as being allowed to be spatially centered along a traveling direction (“In other words, the sound source may exist at any position in a traveling direction (front-back direction) of the user U”), which reasonably indicates that the sound position corresponds with the current location of the assisted person when there is no deviation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to expand the features for modulating audio signals based on information of an assisted person (see at least Paragraphs 0067-0069) by including features for acquiring and using state information for generating a spatial signal, as taught by Koga, for the benefit of maintaining the assisted person at a target velocity (Koga, Paragraph 0060).
Regarding Claim 4, Sommer as currently modified teaches claim 1. Sommer further discloses:
generating the spatial signal includes setting the position information to correspond to a location ahead of the location of the assisted person in a moving direction of the assisted person (Paragraph 0069, “The spatial audio navigation system 300 may output right 312A and left 312B audio channels to the headset via a wired or wireless connection so that the user hears the sound in a spatial audio sense. In other words, the user hears the sound as if the sound were coming from a real world location with accurate distance and direction. For example, the system 300 may play a sound through the headset so that the user hears the sound coming from their left, their right, straight ahead, behind, or at some angle. The direction that the sound seems to be coming from may change to guide the user on the path 304”).
Regarding Claim 11, Sommer as currently modified teaches claim 1. Sommer further discloses:
outputting the modulated output signal via plural speakers or earphones worn by the assisted person (Paragraph 0066, “The HMD 200 may provide binaural audio output (e.g., via right audio 210A and left audio 210B output components). For example, right audio 210A and left audio 210B output components may be over-the ear speakers or ear pieces integrated in the HMD 200 and positioned at or over the user's right and left ears, respectively”).
Regarding Claim 17, Sommer as currently modified teaches claim 1. Sommer further discloses:
a computer program product comprising machine-readable instructions executable by a digital processor to cause the digital processor to control a method for assisting a person to perform a task (Paragraph 0046, “Processors 104 may include central processing units (CPUs) configured to implement any suitable instruction set architecture, and may be configured to execute instructions defined in that instruction set architecture”).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, and Smus et al. (US20160356620A1; hereinafter Smus).
Regarding Claim 5, Sommer as currently modified teaches claim 1. Sommer does not explicitly disclose: generating the spatial signal includes setting the position information to correspond to a location in a direction of the target location from the location of the assisted person in case of determining that a moving velocity of the assisted person decreases or that the moving velocity of the assisted person is below a second predetermined threshold or the assisted person stops.
Nevertheless, Smus teaches binaural navigational cues (Abstract, “Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing binaural navigational [cues]”) comprising:
generating the spatial signal includes setting the position information to correspond to a location in a direction of the target location from the location of the assisted person in case of determining that a moving velocity of the assisted person decreases or that the moving velocity of the assisted person is below a second predetermined threshold or the assisted person stops (Paragraph 0034, “As the user navigates along the navigational route towards the waypoint 1, the binaural navigational system iteratively determines a navigational heading relative to a current navigational course, the navigational heading indicative of a direction to face to proceed along a navigational route (184)”; Paragraph 0036, “For each determination that a change is required to the current navigational course, the system 100 changes the non-directional playback state to a directional playback state that presents the audible media in a modified format so that playback is directional from the navigational heading (188)”; Paragraphs 0038, “In implementations in which the system 100 can determine a user's current facing, e.g., by using gyroscopic and inertia sensors, navigational cue may be adjusted more quickly. For example, if a user becomes confused about the direction in which she needs to be heading, she can pause and slowly turn until she determines the correct heading from the changing navigation cues”).
Smus is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of generating a spatial signal when a user is stopped. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to incorporate the teachings of Smus by including features that allow a spatial signal to be perceived from the direction of a target location when a user is stopped. Doing so would provide a reminder of the correct navigational heading when a user becomes confused (Smus, Paragraph 0036, “For example, if a user becomes confused about the direction in which she needs to be heading, she can pause and slowly turn until she determines the correct heading from the changing navigation cues”).
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, and Paggel (DE102018110890A1; hereinafter Paggel).
Regarding Claim 6, Sommer as currently modified teaches claim 2. While Sommer further discloses generating the spatial signal (see at least Paragraph 0069), Sommer does not explicitly disclose: the state of the assisted person includes a safety-related variable associated with the assisted person, and the target state includes a predetermined target value of the safety-related variable, or the state of the assisted person includes a fitness-related variable associated with the assisted person, and the target state includes a predetermined target value of the fitness-related variable, or the state of the assisted person includes a health-related variable associated with the assisted person, and the target state includes a predetermined target value of the health-related variable.
Nevertheless, Paggel teaches a method for determining a user’s level of attention (Paragraph 0007 of the previously attached English Translation, “In the simplest case, a reduced level of attention on the part of the driver is detected and counteracted with the signal according to the invention”) comprising:
the state of the assisted person includes a safety-related variable associated with the assisted person, and the target state includes a predetermined target value of the safety-related variable, or the state of the assisted person includes a fitness-related variable associated with the assisted person, and the target state includes a predetermined target value of the fitness-related variable, or the state of the assisted person includes a health-related variable associated with the assisted person, and the target state includes a predetermined target value of the health-related variable (Paragraph 0006 of the previously attached English Translation, “The process is designed to measure the driver’s heart rate. This is done using suitable sensors known in the state of the art”; Paragraph 0007 of the attached English Translation, “Known methods for fatigue detection can be used in the state of the art, e.g. For example, reduced driver attention can be detected by a reduced blinking frequency, by evaluating typical steering movements, or by a heart rate that is evenly reduced or close to the resting heart rate. Preferably, the detection of reduced driver attention and the driving condition are used in combination as triggers for the attention-enhancing signal…Furthermore, the criticality of the traffic situation can be taken into account. So that a seemingly only slightly critical traffic scenario leads to the triggering of the driver's attention-enhancing signal when the driver's attention level is greatly reduced, while this is prevented at a higher level of attention. For this purpose, the driver's attention must then be classified additionally, e.g. For example, different heart rate zones can be used”; Examiner notes that determining whether a heart rate is evenly reduced necessarily requires a target heart rate (e.g. resting heart rate or heart rate zone that is dependent on situational context)).
Paggel is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of determining a state of a user based on a health-related variable. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified the Sommer invention to incorporate the teachings of Paggel by including a measure of a health-related variable associated with the user, such as heart rate, in order to determine the state of the user. Doing so would allow a determination of the user’s attentiveness (Paggel, Paragraph 0011 of the previously attached English Translation, “Here, a physiological state of the driver is preferably detected in which reduced attention is to be expected. It is known from the state of the art to use the respiratory rate, heart rate, eyelid blinking or skin conductivity for this purpose, for example. B. to detect driver fatigue”).
Regarding Claim 7, Sommer as currently modified teaches claim 6. While Sommer further discloses generating the spatial signal (Paragraph 0069), Sommer does not explicitly disclose: the fitness-related variable or the health-related variable includes a heart rate of the assisted person, and the predetermined target value of the fitness-related variable or the predetermined target value of the health-related variable includes a predetermined heart rate target, and generating the spatial signal based on the heart rate and the acquired target heart rate, wherein the signal includes setting the position information or the direction information based on the deviation of the heart rate and the acquired target heart rate (Examiner notes the limitation “generating the spatial signal based on the heart rate and the acquired target heart rate…” is being treated as generating the spatial signal based on the attentiveness of the assisted person, which is based on the deviation of the heart rate and the acquired target heart rate, as supported by Paragraphs 0094 of the instant specification).
Nevertheless, Paggel further teaches:
the fitness-related variable or the health-related variable includes a heart rate of the assisted person, and the predetermined target value of the fitness-related variable or the predetermined target value of the health-related variable includes a predetermined heart rate target (Paragraph 0006 of the attached English Translation, “The process is designed to measure the driver’s heart rate. This is done using suitable sensors known in the state of the art”; Paragraph 0007 of the attached English Translation, “Known methods for fatigue detection can be used in the state of the art, e.g. For example, reduced driver attention can be detected by a reduced blinking frequency, by evaluating typical steering movements, or by a heart rate that is evenly reduced or close to the resting heart rate. Preferably, the detection of reduced driver attention and the driving condition are used in combination as triggers for the attention-enhancing signal…Furthermore, the criticality of the traffic situation can be taken into account. So that a seemingly only slightly critical traffic scenario leads to the triggering of the driver's attention-enhancing signal when the driver's attention level is greatly reduced, while this is prevented at a higher level of attention. For this purpose, the driver's attention must then be classified additionally, e.g. For example, different heart rate zones can be used”; Examiner notes that determining whether a heart rate is evenly reduced necessarily requires a heart rate target (e.g. resting heart rate or heart rate zone that is dependent on situational context)) and
generating the [signal] based on the heart rate and the acquired target heart rate, wherein the spatial signal includes setting the position information or the direction information based on the deviation of the heart rate and the acquired target heart rate (Paragraphs 0010-0011 of the attached English Translation, “Furthermore, the signal is generated alternatively or additionally depending on a driver condition. In the simplest case, a reduced level of attention on the part of the driver is detected and counteracted with the signal according to the invention… Furthermore, the use of signals that periodically change in amplitude is possible, whereby the amplitude sequence of these signals should reflect the described period of the attention-enhancing signal”; Examiner notes that a signal is generated based on the driver’s reduced level of attention, as determined by a deviation of the driver’s heart rate and the target heart rate).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified the Sommer invention to further incorporate the teachings of Paggel by including a measure of a health-related variable associated with the user, such as heart rate, and a predetermined target value, such as a heart rate zone. Doing so would allow for automatic adjustments to how signals are being outputted to the user based on the user’s attentiveness (Paragraph 0011 of the attached English Translation, “According to the invention, the driver's condition can be advantageously evaluated as the trigger for the attention-enhancing signal. Here, a physiological state of the driver is preferably detected in which reduced attention is to be expected. It is known from the state of the art to use the respiratory rate, heart rate, eyelid blinking or skin conductivity for this purpose, for example. B. to detect driver fatigue”).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, Paggel and Prakah-Asante (US20190263419A1; hereinafter Prakah-Asante).
Regarding Claim 8, Sommer as currently modified teaches claim 7. While Sommer further discloses generating the spatial signal (Paragraph 0069), Sommer does not explicitly disclose: generating the spatial signal based on the heart rate and the acquired target heart rate, determining the position information or the direction information based on the deviation of the heart rate and the acquired target heart rate, and further based on a pace target value.
Nevertheless, Paggel further teaches:
generating the [signal] based on the heart rate and the acquired target heart rate, determining the position information or the direction information based on the deviation of the heart rate and the acquired target heart rate (Paragraph 0007 of the attached English Translation, “Known methods for fatigue detection can be used in the state of the art, e.g. For example, reduced driver attention can be detected by a reduced blinking frequency, by evaluating typical steering movements, or by a heart rate that is evenly reduced or close to the resting heart rate. Preferably, the detection of reduced driver attention and the driving condition are used in combination as triggers for the attention-enhancing signal”; Paragraphs 0010-0011 of the attached English Translation, “Furthermore, the signal is generated alternatively or additionally depending on a driver condition. In the simplest case, a reduced level of attention on the part of the driver is detected and counteracted with the signal according to the invention… Furthermore, the use of signals that periodically change in amplitude is possible, whereby the amplitude sequence of these signals should reflect the described period of the attention-enhancing signal”; Examiner notes that a signal is generated based on the driver’s reduced level of attention, as determined by a deviation of the driver’s heart rate and the target heart rate).
It would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified the Sommer invention to further incorporate the teachings of Paggel by including a measure of a health-related variable associated with the user, such as heart rate, and a predetermined target value, such as a heart rate zone. Doing so would allow for automatic adjustments to how signals are being outputted to the user based on the user’s attentiveness (Paragraph 0011 of the attached English Translation, “According to the invention, the driver's condition can be advantageously evaluated as the trigger for the attention-enhancing signal. Here, a physiological state of the driver is preferably detected in which reduced attention is to be expected. It is known from the state of the art to use the respiratory rate, heart rate, eyelid blinking or skin conductivity for this purpose, for example. B. to detect driver fatigue”).
However, Sommer as currently modified does not explicitly teach that the signal may further be based on: a pace target value.
Nevertheless, Prakah-Asante teaches a comparative transition prediction system for measuring a user’s attentiveness (Paragraph 0023, “FIG. 2 is a diagram of a comparative transition prediction system 200...Comparative transition prediction system 200 can include a heart rate monitor 202. Heart rate monitor 202 can acquire heart rate data from vehicle 110 occupant. Acquire means to receive, obtain, measure, gauge, read, or in any manner whatsoever acquire”) comprising
a pace target value (Paragraph 0030, “Returning to FIG. 2, heart rate monitor can also output the heart rate data 300, 400 to the Transitional Engagement Value (TEV) computation process 206. TEV is a measure of an occupant's attentiveness to piloting activities or virtual driver supervision. TEV computation process 206 determines a Transitional Engagement Value (TEV) based on the baseline range Pmin and Prange and a norm heart rate xk. The norm heart rate in BPM at time k, can be calculated by [Equation 1] wherein the norm heart rate 4 is calculated by weighting the previous norm heart rate with a tunable constant α and adding it to the current heart rate xk weighted by 1−α. The tunable constant α is a value between 0 and 1 and may be chosen based on the desired time constant or response time to alert the occupant or advice the virtual driver”).
Prakah-Asante is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of setting a pace target value. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified the Sommer invention to incorporate the teachings of Prakah-Asante by including features that allow a situation-dependent pace target value to be determined. Doing so would allow for a more accurate determination of the user’s attentiveness by establishing different target heart rate values based on the situational context (Prakah-Asante, Paragraph 0030, “A typical value of a may be 0.97. For a faster response a lower value of a may be selected. For example, a may be relatively chosen as 0.85. A faster response may be required to alert the user during situational contexts including the time-of-day or traffic conditions”).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, Paggel and Tran et al. (US20200277008A1; hereinafter Tran).
Regarding Claim 9, Sommer as currently modified teaches claim 6. Sommer does not explicitly disclose: the assisted person moves in a dynamic traffic environment including a plurality of other participants and the safety-related variable includes a distance of the assisted person to the other traffic participants, and the predetermined target value of the safety-related variable includes a minimum distance of the assisted person to the other traffic participants.
Nevertheless, Tran teaches a method for maintaining a safe distance from other traffic participants (Paragraph 0002, “Navigating a vehicle in proximity to another vehicle may require determining and maintaining a safe distance from the other vehicle. Determining and maintaining such a safe distance may be facilitated”) comprising:
the assisted person moves in a dynamic traffic environment including a plurality of other participants [and] the safety-related variable includes a distance of the assisted person to the other traffic participants, and the predetermined target value of the safety-related variable includes a minimum distance of the assisted person to the other traffic participants (Paragraph 0021, “By monitoring transit-condition data, i.e., data corresponding to transit conditions of a user vehicle (e.g. roadway-characteristic data, ambient-environment data, user-vehicle data, and roadway-vehicle data of one or more roadway vehicles near the user vehicle), embodiments of the present disclosure may provide inter-vehicular proximity information that is tailored to the specific transit conditions experienced by a vehicle operator at a given time”; Paragraph 0028 and Figure 1, “System 100 may include monitored transit-condition data 190 obtained from sources such as such as sensing devices, cameras, global positioning system (“GPS”) devices, speedometers, accelerometers, in-vehicle data storage/processing devices, and/or web-based resources…A discrete or integrated visual representation generator 160 may determine the specific design of one or more visual representations 170 to be displayed. Additionally, the visual representation 170 may mark one or more threshold distances, and the presence of a roadway vehicle within the one or more threshold distances may trigger an audio alert 180 to vehicle operators to warn of a dangerous proximity or a potential collision”).
Tran is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of determining a minimum safe distance from other traffic participants. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to incorporate the teachings of Tran by including features that allow a determination of a minimum distance between a user and other traffic participants. Doing so would improve the safety of the user (Paragraph 0021, “Accordingly, embodiments of the present disclosure may facilitate improved vehicle safety by permitting vehicle operators to better gauge and maintain safe inter-vehicular proximities”).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, Paggel and Li (CN108834068B; hereinafter Li).
Regarding Claim 10, Sommer as currently modified teaches claim 6. Sommer further discloses:
the assisted person wears a virtual reality headset for moving in a virtual representation and reproducing the audio output signal associated with the virtual representation (Paragraphs 0051-0052, “FIG. 2A illustrates a VR/MR system, according to at least some embodiments. In some embodiments, a VR/MR system may include a HMD 200 such as a helmet, goggles, or glasses that may be worn by a user 290…The HMD 200 may include sensors that collect information about the user 290's environment (video, depth information, lighting information, etc.) and information about the user 290 (e.g., the user's expressions, eye movement, head movement, gaze direction, hand gestures, etc.). Virtual content may be rendered based at least in part on the various information obtained from the sensors for display to the user 290. The virtual content may be displayed by the HMD 200 to the user 290 to provide a virtual reality view (in VR applications) or to provide an augmented view of reality (in MR applications). HMD 200 may implement any of various types of display technologies”).
However, Sommer does not explicitly disclose: the safety-related variable includes a location of the assisted person in a real environment of the assisted person, and the predetermined target value of the safety-related variable includes a safe location of the assisted person (Examiner notes that this limitation is being interpreted as determining whether the location of the assisted person is within a specified region, as supported by Paragraphs 0034-0035 of the instant specification).
Nevertheless, Li teaches a wearable device for navigating a user (Paragraph 0002 of the attached English Translation, “The present invention relates to the technical field of wearable devices, and in particular to a positioning method based on a wearable device”) comprising:
the safety-related variable includes a location of the assisted person in a real environment of the assisted person, and the predetermined target value of the safety-related variable includes a safe location of the assisted person (Paragraph 0013 of the attached English Translation, “The wearable device determines whether the current location is within a preset safety area”).
Li is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of determining whether a user is within a safe location. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified the Sommer invention to incorporate the teachings of Li by including features that allow a determination on whether a user is within a preset safety area. Doing so would help improve safety for the user (Li, Paragraph 0129 of the attached English translation, “It can be seen that the embodiments of the present invention can provide a variety of navigation routes to the wearer of the wearable device…and then output the navigation route to help the wearer of the wearable device return to a safe area, thereby reducing the probability of the personal safety of the wearer of the wearable device being threatened”).
Claims 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Koga, Balsam, and Polanowski (US20200003569A1; hereinafter Polanowski).
Regarding Claim 12, Sommer as currently modified teaches claim 1. Sommer does not explicitly disclose: wherein the method comprises a step of unified encoding of multiple objectives in the target state information, wherein individual contributions of the objectives are weighted differentially based on at least one of priority constraints or preset user preferences.
Nevertheless, Polanowski teaches a navigation system that can prioritize safety (Paragraph 0001, “More particularly, the present disclosure relates to systems, devices, and methods for facilitating safer navigation from an origin to a destination and any intermediate point therebetween”) comprising:
wherein the method comprises a step of unified encoding of multiple objectives in the target state information, wherein individual contributions of the objectives are weighted differentially based on at least one of priority constraints or preset user preferences (Paragraphs 0129-0130, “In some embodiments, the optimized route may be determined by maximizing the function of the variables: route safety rating, travel duration, and travel distance…the function may provide a predetermined weight (e.g., an importance factor or a multiplier) to at least one of the variables. By assigning predetermined weight(s), the function may also favor one of the variables over the remaining variables. For example, the function may favor a safer route (i.e., a route that has a higher route safety rating) over an alternative route that has a shorter travel duration and/or shorter travel distance. The route for which the function is maximized may be selected as the optimized route”).
Polanowski is considered analogous art to the claimed invention because it is reasonably pertinent to the problem of incorporating multiple weighted objectives. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to incorporate the teachings of Polanowski by including multiple objectives weighted based on predetermined constraints. Doing so would allow a user to optimize a route based on multiple factors (Polanowski, Paragraph 0129, “In other words, the function of the variables may be maximized (and thus the route may be optimized) when the combination of variables that are associated with the selected route include the best combination of the available variables (i.e., route safety rating, travel duration, and travel distance) of an identified set of routes”).
Regarding Claim 13, Sommer as currently modified teaches claim 12. Sommer further discloses:
target state information encodes objectives that include at least one of safety objectives, navigation objectives, sports objectives, health objectives, and convenience objectives (Paragraph 0067, “In embodiments of the spatial audio navigation system 300, a current location, target location, and map information may be input to a pathfinding component 302 to determine a real world path 304 between the user's current location and the target location”; Examiner notes that a target location is a navigation objective).
Regarding Claim 14, Sommer as currently modified teaches claim 12. Sommer does not explicitly disclose: wherein the priority constraints include a preference for safety objectives over convenience objectives.
Nevertheless, Polanowski further teaches:
wherein the priority constraints include a preference for safety objectives over convenience objectives (Paragraph 0130, “In some embodiments, the function may provide a predetermined weight (e.g., an importance factor or a multiplier) to at least one of the variables. By assigning predetermined weight(s), the function may also favor one of the variables over the remaining variables. For example, the function may favor a safer route (i.e., a route that has a higher route safety rating) over an alternative route that has a shorter travel duration and/or shorter travel distance. The route for which the function is maximized may be selected as the optimized route. Furthermore, when the server system 101 has received an indication of a minimum safety threshold as discussed above with regard to operation 404, the function may reject and not consider any route that does not satisfy the minimum safety threshold”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Sommer invention to further incorporate the teachings of Polanowski by including a predetermined prioritization of safety objectives over convenience objectives. Doing so would allow a user to optimize a route based on safety (Paragraph 0131, “In some embodiments, the function that determines the optimized route may select the optimized route from a set of routes identified by the server system 101, such that the function maximizes the route safety rating of the optimized route while minimizing at least one of the travel duration and/or the travel distance of the optimized route”).
Claims 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Sommer in view of Beaurepaire (US20210389152A1; hereinafter Beaurepaire) and Balsam et al. (previously cited above).
Regarding Claim 19 (independent), Sommer discloses:
a computer-implemented method for assisting a person (Abstract, “Methods and apparatus for spatial audio navigation that may, for example, be implemented by mobile multipurpose devices”), wherein the method comprises steps of:
acquiring movement direction information on a movement direction (Paragraph 0037 describes determining a movement direction (“The current location, target location, and map information may be input to pathfinding algorithms to determine a real world path between the user's current location and the target location. The spatial audio navigation system may then use directional audio played through the headset to guide the user on the path from the current location to the target location. The user's current location and direction of motion as determined by the mobile multipurpose device and head orientation and movement as determined by the headset are…”));
acquiring information on a target direction or on a correct transport system line; and generating a spatial signal based on the acquired movement direction information and the acquired information on the target direction or the correct transport system line, wherein the spatial signal includes a position information or a direction information (Paragraph 0037 describes guiding a user by spatial audio navigation to a target location based on the direction of motion (“…used by the spatial audio navigation system to adjust the virtual directionality and distance of the audio as the user moves in the real world to guide the user along the path to the target location”) which reasonably indicates that acquired information on a target direction is used (e.g. to allow adjusting the “virtual directionality” towards a target location));
obtaining an audio output signal for reproduction to the assisted person (Paragraph 0036, “The audio source may, for example, include ambient noise sampled from or simulated for the environment, an audio source that the user is listening to (e.g., music, podcasts, audio books, radio broadcasts, phone calls, etc.), or various pre-recorded or generated sounds”);
modulating the obtained audio output signal with the spatial signal, wherein a perceived position of origin or a direction of the modulated audio output signal corresponds to the position information or the direction information included in the spatial signal (Paragraph 0068, “The user’s current location may be tracked by the device and input to the navigation component 306. In addition, head orientation and motion may be tracked by sensors of the headset or HMD and input to the navigation component 306. The navigation component 306 then adjust the directionality of audio provided by an audio source as the user moves in the real world to guide the user along the path 304 to the target location. In some embodiments, the virtual directionality of the audio may be adjusted by attenuating one or more aspects of audio output to the left and/or right audio channels, including but not limited to volume, frequency, and reverberation. Other aspects of the audio may be adjusted based on the inputs, such as volume and type of audio, to assist in guiding the user along the path 304 to the target location”);
outputting the modulated audio output signal to the assisted person (Paragraph 0069, “The spatial audio navigation system 300 may output right 312A and left 312B audio channels to the headset via a wired or wireless connection so that the user hears the sound in a spatial audio sense”);
determining a reaction or a lack of reaction of the assisted person to a previously output modulated audio output signal modulated based on a same spatial signal; adapting a parameter of the modulated audio output signal based on the determined reaction or lack of reaction of the assisted person (Paragraph 0086, “As shown in FIG. 7C, if the user misses the turn, the sound may decrease in volume and be moved so that it seems to be coming from behind the user to prompt the user that they have missed the turn. Alternatively, in some embodiments, an alert sound may be played when the user misses a turn. In some embodiments, if a user continues past a turn or otherwise goes off a path determined by the system, the system may recalculate a new path to the target location and begin directing the user on the new path using the spatial audio navigation methods. Instead of or in addition to volume, other aspects of the audio may be attenuated to affect the virtual distance and directionality of the audio, including but not limited to frequency and reverberation”; Examiner notes that the audio output signal is adapted (e.g. frequency, reverberation, directionality, volume etc.) when a user does not react to a missed turn).
However, Sommer does not explicitly recite: wherein the person is travelling on a public transport system.
Nevertheless, Beaurepaire teaches navigational device compatible with augmented reality and spatial audio (see at least Paragraph 0038, “By way of example, a 3D audio cue can be a spatial audio representation of the navigation guidance information (e.g., turn right after 100 m″) as if originating from the at least one of the one or more surface candidates (e.g., Instance 511a). In another instance, the user may pre-select audio cues as a preferred format over the AR overlays”) comprising:
wherein the person is travelling on a public transport system (Paragraph 0087, “Although various embodiments are described with respect to a walking mode or a driving mode, it is contemplated that the AR overlay approach described herein may be used with other modes of transport (e.g., public transport, etc.). By way of example, when riding on trains/subways…”); Examiner notes that claim 12 further describes the augmented reality overlay features as being represented by spatial audio (“[claim] 12. The method of claim 1, wherein the rendering of the augmented reality overlay is a visual representation of the navigation guidance information anchored to the at least one of the one or more surface candidates, a spatial audio representation of the navigation guidance information originating from the at least one of the one or more surface candidates, or a combination thereof”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to expand the spatial audio navigation compatibility with real-world locations (see at least Paragraphs 0036-0037) by including public transport systems, as taught by Beaurepaire, for the benefit of increasing compatibility with various forms of transportation (see at least Beaurepaire, Paragraph 0027).
While Sommer further discloses features for collecting information related to user attentiveness (Paragraph 0052, “The HMD 200 may include sensors that collect…information about the user 290 (e.g., the user's expressions, eye movement, head movement, gaze direction, hand gestures, etc.)”), Sommer does not explicitly recite: determining a state of attention and an attention history of the assisted person; generating the spatial signal further based on at least one of the determined state of attention and attention history of the assisted person.
Nevertheless, Balsam teaches features that allow sounds to be adjusted based on a state of attention (see at least Abstract) comprising:
determining a state of attention and an attention history of the assisted person (Paragraphs 0072-0074, “…attention determination module 410 determines whether a user's attention has shifted…based on historical information including attributes associated with the user that may change over time. Examples of such attributes include a gaze point of the user, a position and/or an orientation of the user's eye(s), one or more of the user's biomarkers and/or vital signs, information describing sounds or objects detected in an environment of the user, or any other suitable attributes associated with the user”));
generating the spatial signal further based on at least one of the determined state of attention and attention history of the assisted person (Paragraph 0081 describes a spatialized signal based on user attentiveness (“…a sound detected by the audio system 114 should be modified if the user attention determination module 410 determines that a user's attention has shifted. In this example, the set of rules may indicate that the sound should be modified by enhancing or emphasizing it (e.g., by increasing its volume or changing its spatialization so that it appears to originate from a region closer to the user) if it originates from an object of interest of the user”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Sommer invention to expand the features for collecting sensor information (Paragraph 0052) and generating a spatial signal based on information of an assisted person (see at least Paragraphs 0067-0069) to further factor in attentiveness, as taught by Balsam, for the benefit of optimizing audio to ensure user’s perception of critical information.
Regarding Claim 20, Sommer as currently modified teaches claim 19. Sommer further discloses:
acquiring orientation information of an orientation of the assisted person in relation to at least one of the location of the assisted person and the target direction of the assisted person, wherein in the step of generating the spatial signal, generating the spatial direction signal further based on the acquired orientation information (Paragraph 0037 describes generating a spatial signal based on orientation of the user in relation to at least a current location of the user (“The current location, target location, and map information may be input to pathfinding algorithms to determine a real world path between the user's current location and the target location. The spatial audio navigation system may then use directional audio played through the headset to guide the user on the path from the current location to the target location. The user's current location and direction of motion as determined by the mobile multipurpose device and head orientation and movement as determined by the headset are used by the spatial audio navigation system to adjust the virtual directionality and distance of the audio as the user moves in the real world to guide the user along the path to the target location”)).
Regarding Claim 21, Sommer as currently modified teaches claim 19. Sommer further discloses:
determining the information on a target direction or the correct transport system line based on navigation instructions input by the assisted person, or based on an assumed navigation of the assisted person based on a stored navigation history, or based on acquired schedule data, or based on a deviation of predicted location information from the corresponding acquired target location information (Paragraph 0037 describes guiding a user by spatial audio navigation to a target location based on direction of motion of the user (“The user's current location and direction of motion as determined by the mobile multipurpose device and head orientation and movement as determined by the headset are used by the spatial audio navigation system to adjust the virtual directionality and distance of the audio as the user moves in the real world to guide the user along the path to the target location”) which reasonably indicates that acquired information on a target direction is used (e.g. to allow adjusting the “virtual directionality” towards a target location), wherein information on a target direction is based on a target location from navigation instructions inputted by the user (e.g. Paragraph 0036, “Inputs to the spatial audio navigation system may include a real-world target location (typically provided by the user through an interface to an application on the mobile multipurpose device)”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EISEN YIM whose telephone number is (703)756-5976. The examiner can normally be reached M-F 9:30 AM - 5:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at (571) 270-7429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EISEN YIM/Examiner, Art Unit 3669
/Erin M Piateski/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights