Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 07/29/2025 has been entered. Claims 1-18 remain pending in this application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-11 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over CHOI et al. (US 20110276659 A1 hereinafter Choi) in view of Jung et al. (US 20100077261 A1 hereinafter Jung)
As to independent claim 1, Choi teaches a content interaction apparatus, comprising: a processor; a memory; a plurality of lines of instructions executed by the processor that configure the processor to: [a home server with user devices having memory ¶38]
receive an encoded piece of content having at least one scene, [user receive a scene with multimedia contents and sensory effects (hot spots) via a receiver Fig. 3 310 for decoding encoded media scene ¶30, ¶54 "receiver 1 310 configured to receive the multimedia data from the service provider 110, a decoder 1 320 configured to decode the sensory effect information encoded by the binary representation in the received multimedia data as described above"]
the encoded piece of content having a hotspot and a sense bug associated with the hotspot embedded in the piece of content at the at least one scene, [Sensory effect information (hot spot) with commands (sense bug) ¶29-¶30 "encodes the sensory effect information using the binary representation, converts the encoded sensory effect information into command information for device command of each user device"]
wherein the hotspot indicates that a sense response occurs in synchronization with the at least one scene by user interaction with the hotspot, and [synchronized and real-time ¶12, ¶30 "home server provides the sensory effect like the wind blows to the user by extracting the sensory effect information from the multimedia data and then, being synchronized with a user device capable of outputting the wind effect like a fan"]
wherein the sense bug has data that identifies a sense device of a plurality of sense devices of a sense interaction system to be activated at the hotspot when the sense interaction occurs and an expected sense response from the sense device; [sensory effect information includes commands (sense bug) for commanding devices that provides senses (fan) ¶54 "converter 330 configured to convert the decoded sensory effect information into the command information for commanding the devices of each user devices 152, 154, 156, and 158], [Command attributes include IDs like "deviceIdRef" which is of type "anyURI" (data that identifies devices) ¶91 and Table 17]
the sense response for the at least one scene on a sense device separate from the display device. [user devices include fans, humidifier and video display ¶43 "user devices 152, 154, 156, and 158 may be a video display and a speaker that outputs video and audio, various devices outputting the various sensory effects, for example, home appliances such as a fan, an air conditioner, a humidifier, a heat blower, a boiler, or the like"]
wherein the sense response is one of a taste sense response, a smell sense response, an auditory sense response and a sight sense response [scent (smell), speaker (auditory) and video/light (sight) devices providing sensory effects ¶43 "video display and a speaker"…"various sensory effects of the multimedia contents may be, for example, a light effect, a colored light effect, a flash light effect, a temperature effect, a wind effect, a vibration effect, a water sprayer effect as a spraying effect, a scent effect, a fog effect, a color correction effect, a motion and feeling effect"]
Choi does not specifically teach a taste sense response and receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content and trigger as a result of selecting;
However, Jung teaches a taste sense response [taste devices outputs a taste for response ¶63 " a taste device for outputting tastes"]
receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content and trigger as a result of selecting; [user selects an object that includes sense data while viewing a scene that gets extracted and output Fig. 5 S500, S510, S590 ¶65-71 "selects a subject or object that a user has expressed the desire to experience by using audio and gestures"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the enhanced content disclosed by Choi by incorporating the a taste sense response and receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content disclosed by Jung because both techniques address the same field of providing supplemental content to video and by incorporating Jung into Choi enables experiences to be more realistic and useful interface of the five senses [Jung ¶3-4]
As to dependent claim 2, the rejection of claim 1 is incorporated, Choi and Jung further teach wherein the processor is further configured to encode the hotspot and sense bug in the at least one scene of the encoded piece of content. [Choi encode sensory effect information (hot spot) with commands (sense bug) ¶29-¶30 "encodes the sensory effect information using the binary representation, converts the encoded sensory effect information into command information for device command of each user device"]
As to dependent claim 7, the rejection of claim 1 is incorporated, Choi and Jung further teach wherein the encoded piece of content is one of a new piece of content and a third party piece of content. [Choi users receive new (real-time content) according to requests ¶10-11]
As to dependent claim 8, the rejection of claim 1 is incorporated, Choi and Jung further teach wherein the processor configured to trigger the sense response is further configured to trigger a smell sense device for the smell sense response and a taste sense device for the taste sense response. [Choi scent effect and water can be tasted ¶43]
As to dependent claim 9, the rejection of claim 8 is incorporated, Choi and Jung further teach wherein the processor configured to trigger the sense response is further configured to trigger a hearing device for the auditory sense response and a visual sense device for the sight sense response. [Choi light ¶24], [Jung speaker ¶63]
As to independent claim 10, Choi teaches a method, comprising:
receiving, by a processor, an encoded piece of content having at least one scene [user receive a scene with multimedia contents and sensory effects (hot spots) via a receiver Fig. 3 310 for decoding encoded media scene ¶30, ¶54 "receiver 1 310 configured to receive the multimedia data from the service provider 110, a decoder 1 320 configured to decode the sensory effect information encoded by the binary representation in the received multimedia data as described above"]
the encoded piece of content having a hotspot and a sense bug associated with the hotspot embedded in the piece of content at the at least one scene, [Sensory effect information (hot spot) with commands (sense bug) ¶29-¶30 "encodes the sensory effect information using the binary representation, converts the encoded sensory effect information into command information for device command of each user device"]
wherein the hotspot indicates that a sense response occurs in synchronization with the at least one scene by user interaction with the hotspot, and [synchronized and real-time ¶12, ¶30 "home server provides the sensory effect like the wind blows to the user by extracting the sensory effect information from the multimedia data and then, being synchronized with a user device capable of outputting the wind effect like a fan"]
wherein the a sense bug has data that identifies a sense device of a plurality of sense devices of a sense interaction system to be activated at the hotspot when the sense interaction occurs and an expected sense response from the sense device; [sensory effect information includes commands (sense bug) for commanding devices that provides senses (fan)¶54 "converter 330 configured to convert the decoded sensory effect information into the command information for commanding the devices of each user devices 152, 154, 156, and 158], [Command attributes include IDs like "deviceIdRef" which is of type "anyURI" (data that identifies devices) ¶91 and Table 17]
displaying, on a device, the encoded piece of content; [video display ¶43 " video display and a speaker that outputs video and audio, various devices outputting the various sensory effects"]
the sense response for the at least one scene on a sense device separate from the display device. [user devices include fans, humidifier and video display ¶43 "user devices 152, 154, 156, and 158 may be a video display and a speaker that outputs video and audio, various devices outputting the various sensory effects, for example, home appliances such as a fan, an air conditioner, a humidifier, a heat blower, a boiler, or the like"]
wherein the sense response is one of a taste sense response, a smell sense response, an auditory sense response and a sight sense response [scent (smell), speaker (auditory) and video/light (sight) devices providing sensory effects ¶43 "video display and a speaker"…"various sensory effects of the multimedia contents may be, for example, a light effect, a colored light effect, a flash light effect, a temperature effect, a wind effect, a vibration effect, a water sprayer effect as a spraying effect, a scent effect, a fog effect, a color correction effect, a motion and feeling effect"]
Choi does not specifically teach a taste sense response and receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content;
However, Jung teaches a taste sense response [taste devices outputs a taste for response ¶63 " a taste device for outputting tastes"]
receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content and trigger as a result of selecting; [user selects an object that includes sense data while viewing a scene that gets extracted and output Fig. 5 S500, S510, S590 ¶65-71 "selects a subject or object that a user has expressed the desire to experience by using audio and gestures"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the enhanced content disclosed by Choi by incorporating the a taste sense response and receive an activation of the hotspot as a result of a user selecting the hotspot and viewing the at least one scene of the encoded piece of content disclosed by Jung because both techniques address the same field of providing supplemental content to video and by incorporating Jung into Choi enables experiences to be more realistic and useful interface of the five senses [Jung ¶3-4]
As to dependent claim 11, the rejection of claim 10 is incorporated, Choi and Jung further teach encoding the hotspot and sense bug in the at least one scene of the encoded piece of content. [Choi encode sensory effect information (hot spot) with commands (sense bug) ¶29-¶30 "encodes the sensory effect information using the binary representation, converts the encoded sensory effect information into command information for device command of each user device"]
As to dependent claim 16, the rejection of claim 10 is incorporated, Choi and Jung further teach wherein the encoded piece of content is one of a new piece of content and a third party piece of content. [Choi users receive new (real-time content) according to requests ¶10-11]
As to dependent claim 17, the rejection of claim 1 is incorporated, Choi and Jung further teach wherein triggering the sense response further comprises triggering a smell sense device for the smell sense response and triggering a taste sense device for the taste sense response. [Choi scent effect and water can be tasted ¶43]
As to dependent claim 18, the rejection of claim 8 is incorporated, Choi and Jung further teach wherein triggering the sense response further comprises triggering a hearing device for the auditory sense response and triggering a visual sense device for the sight sense response. [Choi light ¶24], [Jung speaker etc. ¶63]
Claims 3-5 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Jung, as applied in the rejection of claim 2 and 11 above, and further in view of FATEH (US 20180286351 A1)
As to dependent claim 3, Choi and Jung teach the rejection of claim 2 above.
Choi and Jung do not specifically teach wherein the processor configured to encode the hotspot and sense bug is further configured to automatically map a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content.
However, Fateh teaches wherein the processor configured to encode the hotspot and sense bug is further configured to automatically map a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content. [creates scented content, identifies candidate keyframes and scents using machine learning ¶104 "Thus, unscented digital content could be manually repurposed into scented digital content by an administrator 512 and/or automatically repurposed into scented digital content by the repurposing engine 500. The unscented digital content could be non-visual content or visual content, such as two-dimensional (2D) content or three-dimensional (3D) content. The repurposing engine 500 may also engage in machine learning to identify candidate instances and keyframes, identify the appropriate scent(s) to be produced at the keyframes, etc."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein the processor configured to encode the hotspot and sense bug is further configured to automatically map a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
As to dependent claim 4, Choi and Jung teach the rejection of claim 2 above.
Choi and Jung do not specifically teach wherein the processor is further configured to receive a user encoding of the hotspot for the at least one scene.
However, Fateh teaches wherein the processor is further configured to receive a user encoding of the hotspot for the at least one scene. [Fateh manually repurposed into scented content ¶45, ¶49]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein the processor is further configured to receive a user encoding of the hotspot for the at least one scene disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
As to dependent claim 5, Choi and Jung teach the rejection of claim 2 above.
Choi and Jung do not specifically teach wherein the processor configured to encode the hotspots and sense bug is further configured to receive, for an advertiser, an advertisement associated with the hotspot and pay the advertiser when the hotspot is activated.
However, Fateh teaches wherein the processor configured to encode the hotspots and sense bug is further configured to receive, for an advertiser, an advertisement associated with the hotspot and pay the advertiser when the hotspot is activated. [Fateh frames with advertisements and payment through advertisement ¶46, ¶115-117 "administrator may identify keyframes in existing digital content (e.g., a cinematic film) in order to create an indirect advertisement"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein the processor configured to encode the hotspots and sense bug is further configured to receive, for an advertiser, an advertisement associated with the hotspot and pay the advertiser when the hotspot is activated disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
As to dependent claim 12, Choi and Jung teach the rejection of claim 11 above.
Choi and Jung do not specifically teach wherein encoding the hotspot and sense bug further comprises automatically mapping a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content.
However, Fateh teaches wherein encoding the hotspot and sense bug further comprises automatically mapping a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content. [creates scented content, identifies candidate keyframes and scents using machine learning ¶104 "Thus, unscented digital content could be manually repurposed into scented digital content by an administrator 512 and/or automatically repurposed into scented digital content by the repurposing engine 500. The unscented digital content could be non-visual content or visual content, such as two-dimensional (2D) content or three-dimensional (3D) content. The repurposing engine 500 may also engage in machine learning to identify candidate instances and keyframes, identify the appropriate scent(s) to be produced at the keyframes, etc."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein encoding the hotspot and sense bug further comprises automatically mapping a sense associated with one or more of a plurality of sense devices onto the at least one scene of the piece of content based on an emotion using a learning algorithm based on historical manual mappings generated from previous pieces of content disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
As to dependent claim 13, Choi and Jung teach the rejection of claim 11 above.
Choi and Jung do not specifically teach wherein encoding the hotspot and sense bug further comprises receiving a user encoding of the hotspot for the at least one scene.
However, Fateh teaches wherein encoding the hotspot and sense bug further comprises receiving a user encoding of the hotspot for the at least one scene. [Fateh manually repurposed into scented content ¶45, ¶49]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein encoding the hotspot and sense bug further comprises receiving a user encoding of the hotspot for the at least one scene disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
As to dependent claim 14, Choi and Jung teach the rejection of claim 11 above.
Choi and Jung do not specifically teach wherein encoding the hotspot and sense bug further comprises receiving, for an advertiser, an advertisement associated with the hotspot and paying the advertiser when the hotspot is activated.
However, Fateh teaches wherein encoding the hotspot and sense bug further comprises receiving, for an advertiser, an advertisement associated with the hotspot and paying the advertiser when the hotspot is activated. [Fateh frames with advertisements and payment through advertisement ¶46, ¶115-117 "administrator may identify keyframes in existing digital content (e.g., a cinematic film) in order to create an indirect advertisement"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein encoding the hotspot and sense bug further comprises receiving, for an advertiser, an advertisement associated with the hotspot and paying the advertiser when the hotspot is activated disclosed by Fateh because all techniques address the same field of enhanced media and by incorporating Fateh into Choi and Jung improves marketing and video by evoking powerful smell senses tied to feelings and memories for impacting customers more [Fateh ¶6-7]
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Jung, as applied in the rejection of claim 1 and 10 above, and further in view of Bogdon et al. (US20160371721 hereinafter Bogdon)
As to dependent claim 6, Choi and Jung teach the rejection of claim 1 above.
Choi and Jung do not specifically teach wherein the processor is further configured to receive a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content.
However, Bogdon teaches wherein the processor is further configured to receive a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content. [helmet that reads brain activity and detects interest via monitoring if they absorbed content ¶6-8 " detecting a change (or lack thereof) in brain activity, and/or based on a predetermined schedule (e.g., when an advertisement is scheduled to be presented)"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the wherein the processor is further configured to receive a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content disclosed by Bogdon because all techniques address the same field of enhanced media and by incorporating Bogdon into Choi and Jung better estimates viewer engagement with content confirming user as watched content and enhancing marketing further [Bogdon ¶1-2]
As to dependent claim 15, Choi and Jung teach the rejection of claim 1 above.
Choi and Jung do not specifically teach receiving, by the processor of the content interaction system, a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content.
However, Bogdon teaches receiving, by the processor of the content interaction system, a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content. [helmet that reads brain activity and detects interest via monitoring if they absorbed content ¶6-8 " detecting a change (or lack thereof) in brain activity, and/or based on a predetermined schedule (e.g., when an advertisement is scheduled to be presented)"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the video enhancing disclosed by Choi and Jung by incorporating the receiving, by the processor of the content interaction system, a pulse of the user from a neuroscience helmet to determine an interest of the user in the piece of encoded content disclosed by Bogdon because all techniques address the same field of enhanced media and by incorporating Bogdon into Choi and Jung better estimates viewer engagement with content confirming user as watched content and enhancing marketing further [Bogdon ¶1-2]
Response to Arguments
Applicant's arguments filed 7/29/2025. In the remark, applicant argues that:
(1) Choi and Osman fail to teach “wherein the hotspot is associated with a taste sense response that occurs in synchronization with the at least one scene by user interaction with the hotspot, and trigger, in response to the received activation, the taste sense response for the at least one scene on a sense device separate from the display device”. See Fig. 4 and how Choi discloses content and sensory effect information are sent to each device.
As to point (1), Applicant’s arguments with respect to claim 1 have been considered but are moot in view of a new ground of rejection as set forth above of Choi in view of Jung. Further to Choi, see ¶30 where Choi teaches a server that enables the sensory effects being synchronized with user devices according to a scene. Choi (¶43 and ¶91) discusses video displays, separate user devices and data that identifies device within the command (sense bug) encoded with content. In response to Applicants’ arguments against the references individually, one cannot show non obviousness by attacking references individually where the rejections are based on combinations of references. According to MPEP 2111, examiner is obliged to give the terms or phrases their broadest interpretation definition awarded by one of an ordinary skill in the art unless applicant has provided some indication of the definition of the claimed terms or phrases.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
WON et al. (US 20150199975 A1) teaches extracting effect data and mapping it to a timeline for controlling peripheral devices (see ¶8 and ¶31)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEAU SPRATT whose telephone number is (571)272-9919. The examiner can normally be reached M-F 8:30-5 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on 5712127212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEAU D SPRATT/ Primary Examiner, Art Unit 2143