DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This is in response to applicant’s amendment/response filed on 08/14/2026, which has been entered and made of record. Claims 1, 9 and 14 have been amended. Claims 1-20 are pending in the application.
Response to Arguments
Applicant's arguments filed on 08/14/2026 have been fully considered but they are not persuasive or are rendered moot in view of the new grounds of rejection presented below (as necessitated by the amendment to claims 1, 9, and 14).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6, 9, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al..
Regarding claim 1, Valkonen teaches a method (abstract) comprising: at an electronic device including a non-transitory memory, one or more processors, and a display (par 0010, par 0030, “The data processing unit 109 comprises at least one processor, at least one memory including computer program code for one or more program units and means for receiving real-time video image data wirelessly or via wired connection from the cameras 103, 104, for example, a receiver or a transceiver. There may be multiple processors e.g. a general purpose processor, a graphics processor and/or a DSP processor and/or multiple different memories e.g. volatile memory for storing data and programs at run-time and non-volatile memory such as a hard disk for permanently storing data and programs”):
measuring an environmental parameter from signals captured by an environmental sensor, wherein the environmental parameter indicates a sensory condition at a location of the electronic device within a physical environment (par 0010-0012, “the at least one camera is configured to capture real-time image data of a target object located within a field of view area of the at least one camera, and which target object is located in the same location wherein the virtual tour data was captured, and wherein the data processing unit is further configured to analyse the real-time image data for detecting a deviation and overlay the real-time image data on an area of a virtual tour view for forming a mixed image virtual tour data, which area of the virtual tour view corresponds the field of view area, if the deviation is detected …. an audio sensor configured to capture acoustic data in the location and to transmit the captured acoustic data to the data processing unit analysis, and if a malfunction is detected, playing the real-time acoustic data in an area of the mixed image virtual tour view corresponding the location of the audio sensor or its surroundings in a real environment”);
determining whether the environmental parameter is within an acceptable range (par 0055, par 0057-0059, “The real-time video data of the field of view area 504 of the second video camera is shown on the kitchen view 501 of the mixed image virtual tour, only when the data processing unit of a camera or an external data processing unit when analysing captured image data of the second video camera detects a predetermined deviation on the floor. In this example, the deviation 502 is water on the floor i.e. a water leak from the dishwasher. In other words, image data of this field of view area 504 is overlayed on the kitchen view 501 of the mixed image virtual tour after a deviation 502 is detected in the captured image data of the second camera “, par 0069-0071, “ This real-time video stream of the second field of view area 702 of the second real-time video camera is shown on the living room view 710 of the mixed image virtual tour, only when the data processing unit when analysing captured image data of the second target object detects that there is a person or something else that does not belong on the couch i.e. it detects a deviation, because the couch should be non-occupied, and the deviation is in this example something overlaying on the couch. In a case of a detected deviation, real-time image data of the second field of view area 702 of the second video camera is overlayed on the living room view 710 of the mixed image virtual tour, more precisely it overlayed on the couch as a pop-up image data ….The deviations which may trigger displaying of image data of one or more cameras on top of one or more views of a mixed image virtual tour are not limited to the examples given above, but a deviation may be any change, anomaly, or defect that may be detected when analysing image data”); and
in response to determining that the environmental parameter is not within the acceptable range, triggering presentation of augmented content in order to enhance the sensory condition at the location of the electronic device (par 0010, “receiving real-time acoustic data captured by an audio sensor in the location, analysing the received acoustic data for detecting a malfunction of the machine, and playing the acoustic data in an area of the mixed image virtual tour view corresponding the location of the audio sensor or its surroundings in the location if a malfunction is detected for further augmenting the mixed image virtual tour by real-time acoustic data. According to an embodiment, the method further comprises obtaining additional information about the location, analysing the additional information for detecting a deviation, and overlaying the additional information on the mixed image virtual tour view if a deviation is detected. According to an embodiment, the real-time image data is real-time video stream or frequently captured real-time images. According to an embodiment, the method further comprises capturing real-time image data to be overlayed on areas of the virtual tour view corresponding the field of view areas by more than one cameras “, par 0055, “It is also possible that a data processing unit analyses the additional information by comparing it to reference values and if it detects a deviation i.e. for example, that value of the additional information is under or over a predetermined reference value, the additional information is overlayed on a view, otherwise not “, par 0072, “FIG. 8 shows a flow chart of a computer implemented method 800 for augmenting a virtual tour to a mixed image virtual tour if a deviation is detected. In step 810 a previously captured virtual tour data of a location is obtained, in step 820, real-time image data of a target object located within a field of view area of a camera, which target object is located in the same location wherein the virtual tour data was captured is received, in step 830, the real-time image data for detecting a deviation is analysed, and in step 840, if the deviation is detected, the real-time image data is overlayed on an area of a virtual tour view for forming a mixed image virtual tour data, which area of the virtual tour view corresponds the field of view area”).
Valkonen keeps silent for teaching wherein the augmented content simulates the signal to amply or reduce the environmental parameter.
In related endeavor, Clark et al. teach wherein the augmented content simulates the signal to amply or reduce the environmental parameter (par 0099-0101, “if the VR environment is a driving simulator, and the detected sound is a sound of a plane, the VR application 345 can present a plane flying in the VR environment 1010, and enhance the detected sound of the plane in the VR environment 1010, for example by selectively amplifying and/or applying sound effects to the detected sound. In another example, if the VR environment is a medieval fantasy game, and the detected sound is a sound of a plane, the VR application 345 can present a dragon in the VR environment 1010, and mask and/or enhance the detected sound of the plane in the VR environment 1010, for example by generating fire type sounds and/or selectively amplifying and/or applying sound effects to the detected sound. Further, the VR application 345 can selectively adjust a volume (i.e., amplitude) of a generated sound based on a sound pressure level of the detected sound that is being masked or enhanced”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen to include wherein the augmented content simulates the signal to amply or reduce the environmental parameter as taught by Clark et al. to present and enhance virtual visual/audio content in a virtual reality environment corresponding to the determined real world environment coordinates of where the physical object is located relative to the user in the real world environment.
Regarding claim 6, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, and Valkonen further teaches wherein the environmental parameter includes an acoustic parameter and the acceptable range includes an acceptable acoustic range (par 0010, “receiving real-time acoustic data captured by an audio sensor in the location, analysing the received acoustic data for detecting a malfunction of the machine, and playing the acoustic data in an area of the mixed image virtual tour view corresponding the location of the audio sensor or its surroundings in the location if a malfunction is detected for further augmenting the mixed image virtual tour by real-time acoustic data”, par 0012, “the system further comprises an audio sensor configured to capture acoustic data in the location and to transmit the captured acoustic data to the data processing unit analysis, and if a malfunction is detected, playing the real-time acoustic data in an area of the mixed image virtual tour view corresponding the location of the audio sensor or its surroundings in a real environment”, par 0067, “the data processing unit analyses the acoustic data by comparing the formed audio profile to pre-stored audio profiles of different operational states. On the basis of the comparison, the data processing unit may determine an operational state of the machine around the acoustic sensor. If it determines that the formed audio profile is equivalent to an operational state that indicates malfunction of the machine, the data processing unit is arranged to play the acoustic data in the wet end view of the mixed image virtual tour corresponding the place in the real paper machinery environment, wherein the acoustic sensor is positioned”).
Regarding claim 9, Valkonen teaches an electronic device comprising: a display; an environmental sensor; one or more processors; a non-transitory memory; and one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the electronic device to (par 0030, par 0037, par 0060). The remaining limitations of the claim are similar in scope to claim 1 and rejected under the same rationale.
Regarding claim 14, Valkonen teaches a non-transitory memory storing one or more programs, which, when executed by one or more processors of [[a]]an electronic device with an environmental sensor and a display, cause the electronic device to (par 0030, par 0037, par 0060). The remaining limitations of the claim are similar in scope to claim 1 and rejected under the same rationale.
Claim(s) 2-3, 15-16, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. PGPubs 2022/0358678 to Lenke et al..
Regarding claim 2, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein the augmented content includes a combination of acoustic content, visual content and haptic content.
In related endeavor, Lenke et al. teach wherein the augmented content includes a combination of acoustic content, visual content and haptic content (par 0045-0050, “the first augmented-reality system 28 is a multi-modal system that features modes for stimulating one or more of a human's various senses. These include vision, hearing, touch, and smell. To implement the first augmented-reality system 28, the vehicle 10 takes control of various subsystems. These subsystems include an audio system 32, a lighting system 34, an olfactory-stimulator 36, a temperature controller 38, a precipitator 40, a haptic system 42, a video system 44, and a projection system 46. The touring circuitry 16 causes the audio system 32 to play stored audio files that may be called upon as needed. Examples include topical music and environmental noises, such as gunshots, barking dogs, sounds of horses' hooves on cobblestone, explosions, sirens, and the like.”, par 0053-0056, “The touring circuitry 16 also activates the haptic system 42 at times that depend on the vehicle's location within the theater 12. By suitable activation, the touring circuitry 16 causes the haptic system 42 to simulate a rough ride. For example, if an immersive tour concerns a colonial-era city, the haptic system 42 simulates the vibrations associated with driving over cobblestone, including adjusting the vibrations based on the vehicle's speed, while the audio system outputs the sound of driving over cobblestone, perhaps incorporating the sounds of horse-drawn carriages with a suitable Doppler shift superimposed to match the vehicle's speed …. Examples of projection systems 46 includes goggles and heads-up displays. The touring circuitry 16 uses the projection system 46 to project images that augment the scenery outside. For example, in the case in which the immersive tour concerns the theater's state during an ice age, the augmented-reality projection system 46 may project an image of a mastodon”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein the augmented content includes a combination of acoustic content, visual content and haptic content as taught by Lenke et al. to cause the augmented-reality system to stimulate at least two sensory inputs of a passenger to deliver information of more significant interest to engage the passengers' full attention.
Regarding claim 3, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein the augmented content augments a live performance in the physical environment.
In related endeavor, Lenke et al. teach wherein the augmented content augments a live performance in the physical environment (par 0012, “the touring circuitry carries out one or more of cueing a live actor in the theater, triggering a special effect in the theater, and triggering a lighting system in the theater.”, par 0040, “the theater 12 may include specially built dressing 20 or live actors 22 to enhance the immersive tour. In some cases, the touring circuitry 16 communicates with the theater dressing 20 and actors 22 via a communication link. As used herein, the term “theater dressing” includes props, set pieces, staging, lighting, and other features, both stationary and moving, that are used in the theater industry to promote a more immersive experience “, par 0058, “Referring to FIG. 4, The second-augmented reality system 30 features include a cast 48 of one or more live actors 22 that are made to enter on cue as the vehicle approaches, lighting units 50 associated with buildings or other features, machinery 52 that can be made to operate, including such animatronic units to simulate animals or other animated beings, special-effects units 54 that are cued to simulate explosions, lightning strikes, or other unexpected events, scenery 56, and a crew of handlers 58 that may be required to intervene in the management of the theater 12 in an effort to choreograph the various activities in the theater 12 as the vehicle passes through. The second augmented-reality system 30 orchestrates the activity of the various features of the theater 12 based on the vehicle's location along the path 14 through the theater 12”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein the augmented content augments a live performance in the physical environment as taught by Lenke et al. to cause the augmented-reality system to stimulate at least two sensory inputs of a passenger to deliver information of more significant interest to engage the passengers' full attention.
Regarding claim 15, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, the claim 15 is similar in scope to claim 2 and is rejected under the same rational.
Regarding claim 16, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, but keeps silent for teaching wherein triggering presentation of the augmented content comprises causing another electronic device to output an audible signal or display visual content.
In related endeavor, Lenke et al. teach wherein triggering presentation of the augmented content comprises causing another electronic device to output an audible signal or display visual content (par 0040, “the theater 12 may include specially built dressing 20 or live actors 22 to enhance the immersive tour. In some cases, the touring circuitry 16 communicates with the theater dressing 20 and actors 22 via a communication link. As used herein, the term “theater dressing” includes props, set pieces, staging, lighting, and other features, both stationary and moving, that are used in the theater industry to promote a more immersive experience”, par 0056, “The touring circuitry 16 also controls a projection system 46. Examples of projection systems 46 includes goggles and heads-up displays. The touring circuitry 16 uses the projection system 46 to project images that augment the scenery outside. For example, in the case in which the immersive tour concerns the theater's state during an ice age, the augmented-reality projection system 46 may project an image of a mastodon”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein triggering presentation of the augmented content comprises causing another electronic device to output an audible signal or display visual content as taught by Lenke et al. to cause the augmented-reality system to stimulate at least two sensory inputs of a passenger to deliver information of more significant interest to engage the passengers' full attention.
Regarding claim 19, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, but keeps silent for teaching wherein the environmental parameter includes a haptic parameter and the acceptable range includes an acceptable haptic level at the location.
In related endeavor, Lenke et al. teach wherein the environmental parameter includes a haptic parameter and the acceptable range includes an acceptable haptic level at the location (par 0053-0054, “The touring circuitry 16 also activates the haptic system 42 at times that depend on the vehicle's location within the theater 12. By suitable activation, the touring circuitry 16 causes the haptic system 42 to simulate a rough ride. For example, if an immersive tour concerns a colonial-era city, the haptic system 42 simulates the vibrations associated with driving over cobblestone, including adjusting the vibrations based on the vehicle's speed, while the audio system outputs the sound of driving over cobblestone, perhaps incorporating the sounds of horse-drawn carriages with a suitable Doppler shift superimposed to match the vehicle's speed. In other examples, the haptic system 42 carries out a seat massage, simulates a rough ride, or simulates the effect of a nearby virtual explosion”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein the environmental parameter includes a haptic parameter and the acceptable range includes an acceptable haptic level at the location as taught by Lenke et al. to cause the augmented-reality system to stimulate at least two sensory inputs of a passenger to deliver information of more significant interest to engage the passengers' full attention.
Regarding claim 20, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, the claim 20 is similar in scope to claim 3 and is rejected under the same rational.
Claim(s) 4 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. PGPubs 2005/0086058 to Lemelson et al..
Regarding claim 4, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein triggering presentation of the augmented content comprises outputting an audible signal that cancels an undesirable audible signal received at the electronic device and boosts a desirable audible signal received at the electronic device.
In related endeavor, Lemelson et al. teach wherein triggering presentation of the augmented content comprises outputting an audible signal that cancels an undesirable audible signal received at the electronic device (par 0016, “An effective technique is the use of headphones with an adaptive filter implemented to introduce "anti-noise" to cancel ambient background noise”, par 0061, “Another adaptive filtering channel can be used in conjunction with headphones to minimize ambient background noise. Microphones located near the headphones and inside the ear cups provide feedback to the adaptive algorithm. The ambient background noise reduction aigorithm is run with no signals applied except those picked-up by the microphones. The external microphone picks up the ambient noise that is then processed by the adaptive filter to create "anti-noise" that is reproduced by the speakers in the headphone cups. When the anti-noise is at the desired amplitude and phase relationship it cancels the ambient noise. When the noise inside the headphone cups is attenuated, the adaptive process is halted and the regular audio signal (or the desired audio signal, which may not necessarily be speech) is applied to the headphones”, par 0068, “The noise estimator 46 provides for determination of the noise content of the initial signal entering the noise cancellation apparatus 40”) and boosts a desirable audible signal received at the electronic device (par 0008, “Complicated signal processing algorithms, such as hidden Markov modeling (HMM), are implemented to resolve the speech in the presence of other speakers or other types of background noise. Once a speech signal is isolated it can be displayed as sub-titling or amplified to stand out from the other sounds in the audio signal “, par 0085, “ The synthetic speech can be boosted in amplitude or processed in other ways to make it more intelligible to the hearing impaired person, including compensation for particular hearing loss profiles and environmental acoustical anomalies as described above. Such compensated synthesized speech for the hearing impaired person improves listening capability without having to read the spoken words as text, making for a more relaxed and natural listening experience. If other listeners with normal hearing are present, the hearing impaired person may use headphones attached to the speech enhancement system to provide independent listening of the modified audio signal with synthesized speech”, par 0091, “he speech enhancement system either processes the speech signal in the speech processing unit to enhance the intelligibility of the speech signals for the hearing impaired before connection to the output signal amplifier and one or more output speakers. Alternatively, the speech enhancement system bypasses the audio signals directly to the output signal amplifier and one or more speakers. The alternative selections are, in one embodiment, user controlled. More specifically the signal processing system may include remote control and audio/video input signal connected to an input circuit, a central processing unit, an output audio circuit for connection to external amplifiers, an adaptive filter for suppression of system, ambient, and/or interfering background noise present in the input audio signal, and/or for compensating for specific hearing loss parameters for selected hearing impaired listeners, and a hearing test system. An equalization/compensation system may be incorporated into the system to optimize the audio signal for selected room acoustics and hearing impairment profiles of particular users”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein triggering presentation of the augmented content comprises outputting an audible signal that cancels an undesirable audible signal received at the electronic device and boosts a desirable audible signal received at the electronic device as taught by Lemelson et al. to improve speech intelligibility .
Regarding claim 10, Valkonen as modified by Clark et al. teaches all the limitation of claim 9, the claim 10 is similar in scope to claim 4 and is rejected under the same rational.
Claim(s) 5, 7, 13, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. PGPubs 2017/0076653 to Bell et al..
Regarding claim 5, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein triggering presentation of the augmented content comprises: increasing a brightness of the display in order to compensate for low ambient lighting at the location of the electronic device; or decreasing the brightness of the display in order to compensate for high ambient lighting at the location of the electronic device.
In related endeavor, Bell et al. teach wherein triggering presentation of the augmented content comprises: increasing a brightness of the display in order to compensate for low ambient lighting at the location of the electronic device; or decreasing the brightness of the display in order to compensate for high ambient lighting at the location of the electronic device (par 0047-0048, “If it is determined that a threshold brightness increase or decrease in the recent portion of the ambient light history has occurred (Y), the method proceeds to 622. If it is determined that a threshold brightness increase or decrease in the recent portion of the ambient light history has not occurred (N), the method proceeds to 626. At 622, a brightening or dimming pattern appropriate to the increase or decrease, respectively, in the brightness of the ambient light determined at 620 is determined to be applied. Brightening and dimming patterns indicate how the brightness of the display (e.g., display 12 of FIG. 4) of the portable electronic device should be modified for a given change in the brightness of the ambient light according to the second predetermined portion of the ambient light history”, par 0061, “As the user emerges into the midday sun from the office, the sudden spike in brightness triggers a brightening pattern such as brightening pattern 906 illustrated in FIG. 9, and as the user 5 returns from the bright sun to the dim office after lunch, a sudden decrease in ambient luminance triggers a dimming pattern, such as dimming pattern 806 shown in FIG. 8. The user 5 commutes home before dusk, and eats dinner at a dimly lit restaurant. A dimming pattern is applied upon entry to the restaurant. The user 5 travels home in the twilight, and enters his home. A brightening pattern is applied to the display as the user 5 turns on the incandescent lights in his home. The display in turn adjusts to the fluorescent lights of the user's changing room environment, and the bedside table lighting prior to the user falling asleep”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein triggering presentation of the augmented content comprises: increasing a brightness of the display in order to compensate for low ambient lighting at the location of the electronic device; or decreasing the brightness of the display in order to compensate for high ambient lighting at the location of the electronic device as taught by Bell et al. to adjust display brightness based on a current ambient brightness level to improve the display visibility based on the detected current brightness of the ambient light.
Regarding claim 7, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, and further teaches wherein the environmental parameter includes a visual parameter (par 0055, par 0057-0059, “The real-time video data of the field of view area 504 of the second video camera is shown on the kitchen view 501 of the mixed image virtual tour, only when the data processing unit of a camera or an external data processing unit when analysing captured image data of the second video camera detects a predetermined deviation on the floor. In this example, the deviation 502 is water on the floor i.e. a water leak from the dishwasher. In other words, image data of this field of view area 504 is overlayed on the kitchen view 501 of the mixed image virtual tour after a deviation 502 is detected in the captured image data of the second camera “, par 0069-0071, “ This real-time video stream of the second field of view area 702 of the second real-time video camera is shown on the living room view 710 of the mixed image virtual tour, only when the data processing unit when analysing captured image data of the second target object detects that there is a person or something else that does not belong on the couch i.e. it detects a deviation, because the couch should be non-occupied, and the deviation is in this example something overlaying on the couch. In a case of a detected deviation, real-time image data of the second field of view area 702 of the second video camera is overlayed on the living room view 710 of the mixed image virtual tour, more precisely it overlayed on the couch as a pop-up image data ….The deviations which may trigger displaying of image data of one or more cameras on top of one or more views of a mixed image virtual tour are not limited to the examples given above, but a deviation may be any change, anomaly, or defect that may be detected when analysing image data”), but keeps silent for teaching the acceptable range includes an acceptable lighting level.
In related endeavor, Bell et al. teach the acceptable range includes an acceptable lighting level (par 0003, “to address this issue uses an ambient light detector to measure a current brightness level of the ambient light. The mobile phone then bins the current brightness level into one of a plurality of brightness ranges, and adjusts a backlight brightness value for the display based on the ambient brightness range in which the detected current ambient brightness level falls”, par 0042-0043, “it is determined whether an ambient light matching mode is activated, which may be the ambient light matching mode activated by a user in FIG. 5 and encoded in mode settings 44 of FIG. 4. …. Since the extent to which the human visual system perceives changes in brightness depends on the brightness level itself, this threshold value may be a proportional, scaled threshold adapted to the brightness recorded in the recent portion of the ambient light history”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include the acceptable range includes an acceptable lighting level as taught by Bell et al. to adjust display brightness based on a current ambient brightness level to improve the display visibility based on the detected current brightness of the ambient light.
Regarding claim 13, Valkonen as modified by Clark et al. teaches all the limitation of claim 9, the claim 13 is similar in scope to claim 7 and is rejected under the same rational.
Regarding claim 18, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, the claim 18 is similar in scope to claim 5 and is rejected under the same rational.
Claim(s) 8 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. Patent 6490359 to Gibson.
Regarding claim 8, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein triggering presentation of the augmented content comprises displaying visual effects that correspond to an audible signal received at the electronic device.
In related endeavor, Gibson teaches wherein triggering presentation of the augmented content comprises displaying visual effects that correspond to an audible signal received at the electronic device (Fig 6, abstract, “Each audio signal is digitized and then transformed into a predefined visual image, which is displayed in a three-dimensional space. Audio effects added to the audio signals are transformed into predefined visual effects images, which are also displayed in a three-dimensional space”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein triggering presentation of the augmented content comprises displaying visual effects that correspond to an audible signal received at the electronic device as taught by Gibson to transform audio signals into predefined visual effects images to achieve a desired sound product to represent the intuitive spatial characteristics of all types of sound with visuals and using those spatial characteristics as a control device for the mix.
Regarding claim 17, Valkonen as modified by Clark et al. teaches all the limitation of claim 14, the claim 17 is similar in scope to claim 8 and is rejected under the same rational.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. PGPubs 2023/0061896 to Davis et al.
Regarding claim 11, Valkonen as modified by Clark et al. teaches all the limitation of claim 1, but keeps silent for teaching wherein the augmented content compensates for reverberation at the location of the electronic device.
In related endeavor, Davis et al. teach wherein the augmented content compensates for reverberation at the location of the electronic device (par 0089, “ the method 900 enables calibration of the second device's audio playback based on output sound that is received at the first device's microphones and to least partially compensate for distortion at the location of the first device, such as distortion due to room geometry, materials, and furniture and also distortion due to non-ideal performance of components in the second device. For example, the location-based compensation may enable a sound quality associated with relatively expensive audio components to be provided using lower-cost components in the second device, and speech intelligibility may be improved though reverberation reduction”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein the augmented content compensates for reverberation at the location of the electronic device as taught by Davis et al. to at least partially compensate for distortion associated with sound propagation from the second device to the position of the device to improve audio quality.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. PGPubs 2024/0233275 to Valkonen in view of U.S. PGPubs 2019/0221035 to Clark et al., further in view of U.S. PGPubs 2024/0362983 to Kim et al..
Regarding claim 12, Valkonen as modified by Clark et al. teaches all the limitation of claim 9, but keeps silent for teaching wherein triggering presentation of the augmented content comprises: outputting haptic responses via a haptic device based on an audible signal data received at the electronic device; or generating haptic responses to cancel haptic responses detected at the location of the electronic device.
In related endeavor, Kim et al. teach wherein triggering presentation of the augmented content comprises: outputting haptic responses via a haptic device based on an audible signal data received at the electronic device; or generating haptic responses to cancel haptic responses detected at the location of the electronic device (abstract, “include a haptic module including at least a motor and outputting a tactile signal; a sound output module including at least one of a speaker and a receiver; and a processor operatively connected to the haptic module and the sound output module, wherein the processor is configured to obtain audio data and a signal related to audio data reproduction; identify whether to generate haptic data based on the audio data; in response to the identification, generate haptic data based on the audio data; reproduce the audio data based on a signal related to the audio data reproduction through the sound output module; control the haptic module; and output the tactile signal based on the generated haptic data”, par 0066, “the processor 120 may identify whether to generate audio data-based haptic data based on the obtained audio data …. The processor 120 may proceed to operation 405 in the case of generating the haptic data based on the audio data as a result of the identification. Or, the processor 120 may proceed to operation 404 in the case of not generating the haptic data based on the audio data as a result of identification …. The processor 120 may generate haptic data by analyzing one or more data parameters (e.g., frequency, duration, amplitude, or waveform) that are included in the audio data. The processor 120, in order to generate the haptic data based on the audio data, may modulate by changing the characteristics such as the amplitude and/or frequency of the audio data or may use various techniques such as sampling and/or filtering. According to an embodiment, the processor 120 may store a library containing data pertaining to one or more vibration waveforms in a memory (e.g., the memory 130 of FIG. 1) and may generate the haptic data by combining haptic signals of corresponding waveforms based on audio data. The audio data and/or the haptic data generated based on the audio data may include one or more markers for synchronization with each other. The processor 120 may generate the haptic data by including one or more markers in the case of generating the haptic data”).
It would have been obvious to a person of ordinary skill in the art at the time before the effective filing data of the claimed invention to modified Valkonen as modified by Clark et al. to include wherein triggering presentation of the augmented content comprises: outputting haptic responses via a haptic device based on an audible signal data received at the electronic device; or generating haptic responses to cancel haptic responses detected at the location of the electronic device as taught by Kim et al. to generate a visual effect, an auditory effect, and/or a tactile effect (a haptic signal, a tactile signal) to notify a user of a designated event to provide a rich user experience by reproducing tactile signals in response to media content have been developed.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jin Ge whose telephone number is (571)272-5556. The examiner can normally be reached 8:00 to 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIN . GE
Examiner
Art Unit 2619
/JIN GE/Primary Examiner, Art Unit 2619
