DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/09/2025 has been placed in record and considered by the examiner.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 7-13, 15, and 17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sanger et al. (US 2020/0117269 hereinafter Sanger).
Referring to claim 1, Sanger discloses a method for adjusting a service consumed by a human, the method comprising capturing optical event data of the human using an event-based sensor ([0011]; The sensed biometric parameter(s) may comprise at least one facial expression (e.g., one or both of an attitude of the mouth and crow's feet around the eyes of the end user), hunching of shoulders, respiration rate, heart rate, body temperature, blood pressure, frequency and/or location of hand movements, frequency and/or location of body twitches, elapsed time between eye movements, and a micro-expression.);
analyzing the optical event data to detect predefined emotional patterns ([0012]; The method further comprises determining if the end user is in at least one specific emotional state (e.g., one of anger, contempt, disgust, fear, happiness, sadness, surprise, confusion, shame, attentiveness, exhaustion, relaxation, frustration, boredom, embarrassment) based on the biometric data for each of the sensed biometric parameter(s). The sensed biometric parameter(s) may comprise a plurality of different sensed biometric parameters, in which case, determining if the end user is in the specific emotional state(s) may comprise performing a pattern recognition analysis on the generated biometric data. For example, if the sensed biometric parameter(s) comprises an attitude of the mouth and crow's feet around the eyes of the end user), the specific emotional state(s) may be determined to comprise happiness.);
determining information on an emotional state of the human based on detected emotional patterns in the optical event data ([0012]; determining if the end user is in the specific emotional state(s) may comprise performing a pattern recognition analysis on the generated biometric data. For example, if the sensed biometric parameter(s) comprises an attitude of the mouth and crow's feet around the eyes of the end user), the specific emotional state(s) may be determined to comprise happiness.); and
adjusting the service based on the information on the emotional state of the human ([0018]; As still another example, the current objective may be to evoke a plurality of different desired emotional states in the end user for a desired duration, and the plurality of different emotional states may be consistent with the plurality of desired emotional states. If the end user is determined to be in the plurality of different emotional states for the desired duration, the action performed may comprise making the stimulus more available to the end user, and in contrast, if the end user is determined to not be in the plurality of different emotional states for the desired duration, and action performed may comprise making the stimulus less available to the end user…. and [0019]; In another embodiment, the stimulus takes the form of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, virtual object, or alert more available or less available to the end user. In still another embodiment, the stimulus takes the form of a feature of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, or alert more available or less available to the end user.).
Referring to claim 2, Sanger discloses wherein the service is a game, a movie, a smart home ser- vice, or a metaverse ([0019]; In another embodiment, the stimulus takes the form of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, virtual object, or alert more available or less available to the end user. In still another embodiment, the stimulus takes the form of a feature of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, or alert more available or less available to the end user.).
Referring to claim 3, Sanger discloses wherein the capturing comprises capturing optical event data from a plurality of event-based sensors ([0048]; More significant to the present inventions, in response to stimuli presented to the end user 50, the sensor(s) 126 are configured for sensing biometric parameters of the end user 50 and generating biometric data representative of these sensed biometric parameters. The stimuli generated by the augmented reality system 100 can be any stimuli that may evoke a physical reaction by the end user 50.).
Referring to claim 4, Sanger discloses wherein the capturing comprises capturing optical event data in 3-dimensional volume comprising a time component ([0047]; Referring back to FIG. 2, the augmented reality system 100 further comprises one or more sensors 126. The sensor(s) 126 may sense, measure, or collect information about movements of the head 54 of the end user 50 (e.g., speed, acceleration, and position) and/or the eye position and inter-ocular distance of the end user 50 (movement, blinking, and depth of focus) to facilitate the rendering of images of a three-dimensional environment from the perspective of the point of view of the end user 50, such that the end user 50 is immersed in the three-dimensional environment.).
Referring to claim 7, Sanger discloses wherein the analyzing comprises determining micro movements in the optical event data to detect the predefined emotional patterns ([0060]; Micro-expressions are the facial expressions that flash on a person's face for a fraction of a second, and can help determine the true feelings or emotions of a human. The fundamental characteristics of a micro-expression are involuntary leakage and fast speed through the mouth, eye movements, and eyebrows, and are often used to determine whether a person is being truthful or not.).
Referring to claim 8, Sanger discloses wherein the micro movements refer to one or more elements of the group of the human's mouth, eye, eyebrow, eye lid, lip, jaw, skin, cheek and nose ([0060]; Micro-expressions are the facial expressions that flash on a person's face for a fraction of a second, and can help determine the true feelings or emotions of a human. The fundamental characteristics of a micro-expression are involuntary leakage and fast speed through the mouth, eye movements, and eyebrows, and are often used to determine whether a person is being truthful or not.).
Referring to claim 9, Sanger discloses wherein the analyzing comprises analyzing a facial expression of the human to detect the emotional state ([0060]; Micro-expressions are the facial expressions that flash on a person's face for a fraction of a second, and can help determine the true feelings or emotions of a human. The fundamental characteristics of a micro-expression are involuntary leakage and fast speed through the mouth, eye movements, and eyebrows, and are often used to determine whether a person is being truthful or not.).
Referring to claim 10, Sanger discloses wherein the analyzing comprises analyzing a body posture or gesture of the human to detect the emotional state ([0011]; The method further comprises presenting a stimulus (e.g., visually or aurally) to the end user in the context of the three-dimensional environment, sensing at least one biometric parameter of the end user in response to the presentation of the stimulus to the end user, and generating biometric data for each of the sensed biometric parameter(s). In one embodiment, the biometric parameter(s) of the end user are sensed at a plurality of different times in response to the presentation of the stimulus, and the biometric data is generated at these different times. The sensed biometric parameter(s) may comprise at least one facial expression (e.g., one or both of an attitude of the mouth and crow's feet around the eyes of the end user), hunching of shoulders, respiration rate, heart rate, body temperature, blood pressure, frequency and/or location of hand movements, frequency and/or location of body twitches, elapsed time between eye movements, and a micro-expression.).
Referring to claim 11, Sanger discloses wherein the adjusting of the service comprises one or more elements of the group of increasing or decreasing an agility of a gaming character, increasing or decreasing a volume of audio, increasing or decreasing a speed of speech or audio content, changing a style of music, adjusting a room temperature, adjusting a brightness and/or color of lighting, offering special effects in the service, adjusting of an avatar of the human, or introducing other humans to the human ([0051]; In still another case, the stimuli may be related to the genre of a background musical piece or a feature thereof. For example, one type of music (e.g., rock) can be played in the background or another type of music (e.g., classical) may be played in the background. In this case, different stimuli are different types of music, which may evoke different emotions in the end user 50. For example, the end user 50 may experience anxiety with rock music, but experience joy with the classical music. Thus, reads on “changing a style of music”).
Referring to claim 12, Sanger discloses a computer program having a program code for performing a method according to claim 1, when the computer program is executed on a computer, a processor, or a programmable hardware component ([0073]; The augmented reality system 100 further comprises a control subsystem that may take any of a large variety of forms. The control subsystem includes a number of controllers, for instance one or more microcontrollers, microprocessors or central processing units (CPUs), digital signal processors, graphics processing units (GPUs), other integrated circuit controllers, such as application specific integrated circuits (ASICs), programmable gate arrays (PGAs), for instance field PGAs (FPGAs), and/or programmable logic controllers (PLUs).).
Referring to claim 13, Sanger discloses an apparatus for adjusting a service consumed by a human comprising circuitry configured to perform the method of claim 1 ([0018]; As still another example, the current objective may be to evoke a plurality of different desired emotional states in the end user for a desired duration, and the plurality of different emotional states may be consistent with the plurality of desired emotional states. If the end user is determined to be in the plurality of different emotional states for the desired duration, the action performed may comprise making the stimulus more available to the end user, and in contrast, if the end user is determined to not be in the plurality of different emotional states for the desired duration, and action performed may comprise making the stimulus less available to the end user…. and [0019]; In another embodiment, the stimulus takes the form of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, virtual object, or alert more available or less available to the end user. In still another embodiment, the stimulus takes the form of a feature of a video game, program, musical background piece, reward, virtual object, or alert, in which case, the action performed may comprise making the video game, program, musical background piece, reward, or alert more available or less available to the end user.).
Referring to claim 15, Sanger discloses a robot comprising the apparatus of claim 12 ([0004]; For example, referring to FIG. 1, an augmented reality scene 4 is depicted wherein a user of an AR technology sees a real-world park-like setting 6 featuring people, trees, buildings in the background, and a concrete platform 8. In addition to these items, the end user of the AR technology also perceives that he or she “sees” a robot statue 10 standing upon the real-world platform 8, and a cartoon-like avatar character 12 flying by which seems to be a personification of a bumble bee, even though these elements 10, 12 do not exist in the real world. As it turns out, the human visual perception system is very complex, and producing a VR or AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging.).
Referring to claim 17, Sanger discloses an entertainment system comprising the apparatus of claim 12 ([0086]; Assume that the current objective of the augmented reality system 100 is to maximize happiness or relaxation with respect to background music. Since music is a matter of taste, reactions to different music will vary; a piece of music can elicit different emotions from various listeners. If, in response to a stimulus presented in the form of a particular musical piece, the emotional state of the end user 50 is determined to be happiness or relaxation continuously during the musical piece, then the CPU 132 may subsequently make that musical piece more available to the end user 50, e.g., by including that musical piece in a list that is played in rotation.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Sanger et al. (US 20200117269 hereinafter Sanger) in view of Kameni et al. (US 11,393,252 hereinafter Kameni).
Referring to claim 5, Sanger as applied above does not specifically disclose wherein the optical event data comprises pixelwise polarity data on brightness or contrast changes.
In an analogous art, Kameni discloses wherein the optical event data comprises pixelwise polarity data on brightness or contrast changes (Kameni- Col. 3 lines 21-32; The enhanced emotion sensing apparatus may include sensors that receive physiological data. The physiological data may include pupil and/or eye information of the subject. The enhanced emotion sensing apparatus may analyze the pupil and/or eye information to determine the subject's emotions. In some embodiments, the enhanced emotion sensing apparatus may use the determined emotions to control real-world changes for the subject, such as adjusting a game that the subject is currently playing, modifying a display that the subject is viewing (e.g., content, color, brightness, volume, etc.), reporting the determined emotions to other parties and/or updating a library.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the technique of Kameni to the system of Sanger in order to provide an enhanced understanding of student emotion to provide insight on how to adapt a learning context, e-learning context, and/or content of lesson.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Sanger et al. (US 20200117269 hereinafter Sanger) in view of Shazly et al. (US 2016/0342206 hereinafter Shazly).
Referring to claim 6, Sanger as applied above does not specifically disclose wherein the optical event data has a sampling rate of at least 500 frames per second.
In an analogous art, Shazly discloses wherein the optical event data has a sampling rate of at least 500 frames per second (Shazly- [0062]; The successive images of the eye 112 captured by the conventional back facing camera 135 are processed by the external device 400 which utilizes a software 600 to generate the absolute co-ordinates of the eye in its socket based on recognizing one or more of the eye features within the captured frame such as the illuminated pupil. The eye tracking data obtained via this method is generated at a much lower rate for example 16-250 frame per second given the low refresh rate of the camera compared to the tracking data obtained via the optical array sensor 102 with frame rates of at least 1500 frame per second. The software can combine the slow absolute co-ordinates of the eye within each frame obtained from the back facing camera with the fast relative eye movement co-ordinates generated by the frame to frame changes obtained from the opto-electrical array sensor 102 to improve the accuracy and the speed of eye tracking.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the technique of Shazly to the system of Sanger in order to determine the amount and direction of head and eye movement.
Claims 14, 16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sanger et al. (US 20200117269 hereinafter Sanger) in view of Kim et al. (US 11501794 hereinafter Kim).
Referring to claim 14, Sanger as applied above does not specifically disclose further comprising a plurality of event-based optical sensors with different field of views of the user.
In an analogous art, Kim discloses further comprising a plurality of event-based optical sensors with different field of views of the user (Kim- Col. 62 lines 32-50; An optical sensor 2360 may provide sensor data 2347 indicative of one or more of a presence or absence of an object, a distance to the object, or characteristics of the object. The optical sensor 2360 may use time-of-flight, structured light, interferometry, or other techniques to generate the distance data. For example, time-of-flight determines a propagation time (or “round-trip” time) of a pulse of emitted light from an optical emitter or illuminator that is reflected or otherwise returned to an optical detector. By dividing the propagation time in half and multiplying the result by the speed of light in air, the distance to an object may be determined. The optical sensor 2360 may utilize one or more sensing elements. For example, the optical sensor 2360 may comprise a 4×4 array of light sensing elements. Each individual sensing element may be associated with a field of view that is directed in a different way. For example, the optical sensor 2360 may have four light sensing elements, each associated with a different 10° field-of-view, allowing the sensor to have an overall field-of-view of 40°.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the technique of Kim to the system of Sanger in order to enable allowing a speech recognition system to identify words spoken by a user based on qualities of received audio input in efficient manner and improving human-computer interactions in effective manner.
Referring to claim 16, Sanger as applied above does not specifically disclose a vehicle comprising the apparatus of claim 12.
In an analogous art, Kim discloses a vehicle comprising the apparatus of claim 12 (Col. 22 line 62 to col. 23 line 5; The other data 709 may include data indicating a type of the device 110. Different types of devices may include, for example, a smart watch, a smart phone, a tablet, and a vehicle. The type of the device 110 may be indicated in a profile associated with the device 110. For example, if the device 110 from which the audio data 211 was received is a smart watch or vehicle belonging to a user A, the fact that the device 110 belongs to user A may increase a user recognition confidence value associated with user A and/or decrease a user recognition confidence value associated with user B.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the technique of Kim to the system of Sanger in order to enable allowing a speech recognition system to identify words spoken by a user based on qualities of received audio input in efficient manner and improving human-computer interactions in effective manner.
Referring to claim 18, Sanger as applied above does not specifically disclose A smart home system comprising the apparatus of claim 12.
In an analogous art, Kim discloses a smart home system comprising the apparatus of claim 12 (Kim-The other data 709 may include data representing activity of a particular user that may be useful in performing user recognition processing. For example, a user may have recently entered a code to disable a home security alarm. A device 110, represented in a group profile associated with the home, may have generated the audio data 211. The other data 709 may reflect signals from the home security alarm about the disabling user, time of disabling, etc. If a mobile device (such as a smart phone, Tile, dongle, or other device) known to be associated with a particular user is detected proximate to (for example physically close to, connected to the same WiFi network as, or otherwise nearby) the device 110, this may be reflected in the other data 709 and considered by the user recognition component 295.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the technique of Kim to the system of Sanger in order to enable allowing a speech recognition system to identify words spoken by a user based on qualities of received audio input in efficient manner and improving human-computer interactions in effective manner.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT D AU whose telephone number is (571)272-5948. The examiner can normally be reached M-F. General 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached at 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT D AU/Examiner, Art Unit 2624
Prosecution Insights