Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2006/0064037 to Shalon et al. (Shalon) (cited by applicant).
In reference to at least claim 1
Shalon discloses a power- and bandwidth-limited system to measure sound-producing behaviors of a subject, the system comprising: a processor (e.g. “a processing unit for processing the acoustic input from the subject into data relatable to the ingestion activity sensed by the device”, para. [0071], “Each microphone can be optimized to receive a specific range of sound frequencies corresponding to the signal to be measured.”, para. [0117]; “a processing unit 14.”, para. [0232]); a power source (e.g. “A rechargeable or disposable battery 34 can power processing unit 14 and all its communication ports” para. [0251]); a microphone communicatively coupled to the processor and configured to generate an audio data signal that represents sound in a vicinity of the subject (e.g. “and ambient noises in the user's immediate surroundings can be monitored through one or more sensors (e.g. microphones) positioned in or around the ear area, on the skull, neck, throat, chest, back or abdomen regions.”, para. [0117], “and therefore the microphone component for picking up the user's voice, internal cranial sounds and/or ambient sounds in the user's environment usually will not need to cancel out the feedback of the speaker sounds”, para. [0166], “The bone conduction microphone is designed to sense the acoustic energy generated within the mouth during eating. The microphone's analogue electrical output is transmitted to processing unit 14 for signal processing.”, para. [0240]); at least one second sensor communicatively coupled to the processor and configured to generate at least a second data signal that represents at least one parameter other than sound (e.g. “at least one addition sensor selected from the group consisting of a heart rate sensor, an accelerometer, a skin conductance sensor, a muscle tone sensor, a blood sugar level sense, a bite sensor and a stomach contraction sensor.”, para. [0018]; “includes one or more sensors for detecting sound, movement, density, light, or other conditions associated with a particular activity”, para. [0115]; “One or more of the user's limbs can be fitted with accelerometers or induction systems which allow the location of the limb in space to be monitored at all times.”, para. [0134]) and a non-transitory computer-readable medium communicatively coupled to the processor and having computer-executable instructions stored thereon that are executable by the processor to perform or control performance of operations (e.g. “As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system.”, para. [0075]) comprising: receiving the second data signal from the at least one second sensor e.g. “at least one addition sensor selected from the group consisting of a heart rate sensor, an accelerometer, a skin conductance sensor, a muscle tone sensor, a blood sugar level sense, a bite sensor and a stomach contraction sensor.”, para. [0018]; “includes one or more sensors for detecting sound, movement, density, light, or other conditions associated with a particular activity”, para. [0115]; “One or more of the user's limbs can be fitted with accelerometers or induction systems which allow the location of the limb in space to be monitored at all times.”, para. [0134]); detecting a wake-up event from the second data signal (e.g. “The system can also wake the user up during the appropriate stage of shallow sleep within a defined window of time.”, para. [0325], “The system would sense when the person falls asleep using breathing, motion, brain wave, or other forms of sleep sensing technologies and then awaken the user after 15 minutes.”, para. [0342]); sending a first signal to the power source to provide power to the microphone (e.g. “The sensors can be powered by a coil and rectifier that generate DC power from transmitted RF signals generated by the system or present in the system environment.”, para. [0116]); receiving the audio data signal from the microphone; detecting one or more sound-producing behaviors of the subject based on: both the audio data signal and the second data signal; or information derived from both the audio data signal and the second data signal; and sending a second signal to the power source to cease providing power to the microphone.
In reference to at least claim 2
Shalon discloses wherein the processor, the microphone, the at least one second sensor, and the non-transitory computer-readable medium are integrated in a wearable electronic device (e.g. “ (a) a sensor mountable on or in a body region of the subject, the sensor being capable of sensing ingestion related motion or acoustic energy;”, para. [0030], [0190], “to require no user intervention other than wearing the system”, para. [0157], “When housed with sensor 12 within a single device (FIG. 1c), processing unit 14 is typically a small embedded microprocessor.”, para. [0238]).
In reference to at least claim 3
Shalon discloses wherein the at least one second sensor comprises: an accelerometer communicatively coupled to the processor and configured to generate an acceleration data signal that represents acceleration of at least a portion of the subject (e.g. “at least one addition sensor selected from the group consisting of a heart rate sensor, an accelerometer”, para. [0018], “One or more of the user's limbs can be fitted with accelerometers or induction systems which allow the location of the limb in space to be monitored at all times.”, para. [0134]); a gyro sensor communicatively coupled to the processor and configured to generate an angular velocity data signal that represents angular velocity of the at least the portion of the subject; a thermometer communicatively coupled to the processor and configured to generate a temperature signal that represents at least one of skin temperature or core body temperature of the subject (e.g. “thermal (e.g. body temperature),”, para. [0109]); an oxygen saturation sensor communicatively coupled to the processor and configured to generate an oxygen saturation signal of the oxygen saturation level of the subject (e.g. “oxygen saturation,”, para. [0299]); a photoplethysmograph (PPG) sensor communicatively coupled to the processor and configured to generate a blood flow data signal that represents blood flow of the subject; an electrocardiograph (ECG) sensor communicatively coupled to the processor and configured to generate an ECG data signal that represents electrical activity of a heart of the subject (e.g. “electrical (e.g. EKG or EMG)”, para. [0109], “EKG electrodes,”, para. [0284]-[0286]); or an electrodermal activity (EDA) sensor communicatively coupled to the processor and configured to generate an EDA data signal that represents EDA of the subject.
In reference to at least claim 4
Shalon discloses wherein the detecting includes: extracting, by the processor, a plurality of audio features from the audio data signal (e.g. “Sensor unit 12 (bone conduction microphone in this case) records the sounds made by chewing, swallowing, biting, sipping, and drinking. The salient acoustic features are extracted using a statistical-based pattern recognition system to classify the sounds into specific events”, para. [0252]); extracting, by the processor, a plurality of second features from the second data signal (e.g. “processing the ingestion related motion or acoustic energy and deriving an ingestion activity related signature therefrom, thereby monitoring ingestion activity of the subject.”, para [0043]); classifying each audio feature of a first subset of the plurality of audio features and each feature of a first subset of the plurality of second features as indicative of a sound-producing behavior (e.g. “a processing unit being capable of processing the ingestion related motion or acoustic energy and deriving an ingestion activity related signature therefrom, thereby monitoring ingestion activity of the subject.”, para. [0030], [0043], “sensors can be combined in an integrated system to increase the reliability or precision of the food volume monitoring.”, para. [0146], “for example, the user is running, system 10 can hear the user's breathing patterns to determine workout intensity. If, for example, the user is lifting weights, system 10 can use the acoustic energy signature of the regular pattern of exhales between each exertion as a sign that the user is actually lifting a weight according to the training plan. “, para. [0301]); and detecting the one or more sound-producing behaviors of the subject based on the classifying (e.g. “a processing unit being capable of processing the ingestion related motion or acoustic energy and deriving an ingestion activity related signature therefrom, thereby monitoring ingestion activity of the subject.”, para. [0030], [0043], “sensors can be combined in an integrated system to increase the reliability or precision of the food volume monitoring.”, para. [0146], “for example, the user is running, system 10 can hear the user's breathing patterns to determine workout intensity. If, for example, the user is lifting weights, system 10 can use the acoustic energy signature of the regular pattern of exhales between each exertion as a sign that the user is actually lifting a weight according to the training plan. “, para. [0301]).
In reference to at least claim 5
Shalon discloses the operations further comprising reporting to a remote server information about sound-producing behaviors of the subject derived from one or both of the audio data signal, the reporting including at least one of: reporting to the remote server one or more of the plurality of audio features extracted from the audio data signal; reporting to the remote server one or more of the plurality of second features extracted from the second data signal (e.g. “When utilized for such purpose, the system can employ any sensor capable of detecting ingestion activity (eating/drinking) and a data transfer unit (e.g. transmitter) for relaying (wirelessly or through a wired connection) sensed ingestion activity to the server or other data sharing mechanisms (examples of transmitters and transmission modes are provided elsewhere herein).”, para. [0068], [0072] [0192], “The spectral analysis and raw signal are utilized to extract features and categorize them with a time stamp and a fitness score.”, para. [0261]); or reporting to the remote server one or more detected sound-producing behaviors and an occurrence time of each of the detected one or more sound-producing behaviors, (e.g. “The spectral analysis and raw signal are utilized to extract features and categorize them with a time stamp and a fitness score.”, para. [0261]), all without reporting to the remote server any of the audio data signal or any of the second data signal.
In reference to at least claim 6
Shalon discloses the operations further comprising: receiving a subsequent audio data signal from the microphone that represents sound measured in the vicinity of the subject within a subsequent period of time (e.g. “and ambient noises in the user's immediate surroundings can be monitored through one or more sensors (e.g. microphones) positioned in or around the ear area, on the skull, neck, throat, chest, back or abdomen regions.”, para. [0117], [0240]; “the microphone component for picking up the user's voice, internal cranial sounds and/or ambient sounds in the user's environment”, para. [0166]); extracting a subsequent plurality of audio features from the subsequent audio data signal (e.g. “The salient acoustic features are extracted using a statistical-based pattern recognition system to classify the sounds into specific events.”, para. [0252]); classifying each audio feature of a first subset of the subsequent plurality of audio features as indicative of a sound-producing behavior (e.g. respectively deriving activity “processing the ingestion related motion or acoustic energy and deriving an ingestion activity related signature therefrom, thereby monitoring ingestion activity of the subject”, para. [0030], [0043], “If, for example, the user is running, system 10 can hear the user's breathing patterns to determine workout intensity. If, for example, the user is lifting weights, system 10 can use the acoustic energy signature of the regular pattern of exhales between each exertion as a sign that the user is actually lifting a weight according to the training plan.”, para. [0301]); determining a quality score of the subsequent audio data signal (e.g. “ Acoustic energy generated by chewing,”, para. [0117], “In another embodiment, the system can use confidence measures to improve chew count accuracy.”, para. [0409]); and one of: in response to determining that the quality score of the subsequent audio data signal exceeds a first threshold quality score, detecting one or more sound-producing behaviors of the subject that occur within the subsequent period of time based exclusively on the first subset of the subsequent plurality of audio features (e.g. “A weighted chew count rather than a hard decision on the existence of a chew would be used to calculate an average of the log-posterior probability to compute a confidence measure of the number of chews. Trailing chews in a bite are expected to be weighted less than the leading chews”, para. [0409]), or in response to determining that the quality score of the subsequent audio data signal is less than a second threshold quality score that is lower than the first threshold quality score, discarding one or both of the subsequent audio data signal and the first subset of the subsequent plurality of audio features without detecting any sound-producing behaviors of the subject that occur within the subsequent period of time based thereon.
In reference to at least claim 7
Shalon discloses the operations further comprising: receiving from the subject annotation input that confirms occurrence of one or more sound-producing behaviors (e.g. “The system can also query the user for confirmation whether he or she is engaged in a specific task that the system believes it is hearing or that appears on the calendar at that time.”, para. [0317]-[0318]); reporting, to a remote server, the annotation input, wherein the remote server is configured to receive annotation inputs and information about sound-producing behaviors from a plurality of subjects (e.g. “relay this information outside of the oral cavity through a wired or wireless connection to an external system that does the data processing and provides the user with feedback.”, para. [0144], “a data transfer unit (e.g. transmitter) for relaying (wirelessly or through a wired connection) sensed ingestion activity to the server or other data sharing mechanisms”, para. [0192]) and to update an algorithm used in the classifying; and receiving, from the remote server, the updated algorithm (e.g. “The system can detect the eating patterns of the user over time, thereby building a database of ingestion behavior and "learning" and customizing the performance of the system to the user.”, para. [0152], “The software and voice feedback files are stored in memory 32 (e.g. EERAM) on board. Both can be updated through one or more external communication ports 36.”, para. [0251], “System 10 may be programmed to measure and analyze eating habits according to specific plans through varying levels of customization of the hardware, software or user interface”, para. [0302]).
In reference to at least claim 8
Shalon discloses wherein the plurality of audio features are time synchronized with the plurality of second features (e.g. “Processing of activity related signatures generated by the present system enables qualification of activities and thus behavior and enables real-time monitoring and modification of such behavior.”, para. [0111], “The above described sensors can be combined in an integrated system”, para. [0146]) and wherein the detecting includes, for at least one of the detected one or more sound-producing behaviors, determining that an audio feature of the first subset of the plurality of audio features coincides chronologically with a second feature of the first subset of the plurality of second features, each of the audio feature and the second feature being indicative of a sound-producing behavior (e.g. “Processing of activity related signatures generated by the present system enables qualification of activities and thus behavior and enables real-time monitoring and modification of such behavior.”, para. [0111], “The above described sensors can be combined in an integrated system”, para. [0146], therefore the acoustic sensors and motion sensors can be combined in a real-time integrate system).
In reference to at least claim 9
Shalon discloses wherein a window size for at least one of the audio data signal and the second data signal are based on a capacity of the non-transitory computer-readable medium (e.g. processing to store and respectively process the acoustically sensed and the motion sensed chew patten for downloading into an external unit in real time or batch mode, abstract; para [0123], [0146]).
In reference to at least claim 10
Shalon discloses wherein the operations further include determining to perform the detecting the one or more sound-producing behaviors on the power- and bandwidth-limited system instead of a remote system based on at least one of a capacity of the power source and a capability of the processor (e.g. “Although the data sensed by the sensor of the present invention can be used to qualify a behavior without any further processing (further described hereinbelow),”, para. [0110], “ Processing unit 14 is selected according to the configuration of system 10. When housed with sensor 12 within a single device (FIG. 1c), processing unit 14 is typically a small embedded microprocessor”, para. [0238]).
In reference to at least claim 11
Shalon discloses wherein: a time-based window size for capturing the audio data signal is adaptively selected based on a talking detection algorithm and if talking is detected, the window size is longer than if talking is not detected (e.g. “The frequency response may be limited to the spectrum required to discriminate the signal sounds”, para. [0123], “The signals received from sensor unit 12 (and other input systems) are preferably first digitized. Preferably, digitization is optimized at twice the highest frequency sound that is to be recorded by applying automatic gain or log amplification to the signal of interest.”, para. [0260]), the audio data signal processed such that words spoken in the audio data cannot be derived from the processed audio data signal (e.g. “The frequency response may be limited to the spectrum required to discriminate the signal sounds”, para. [0123]); and the operations further include discarding the audio data signal after performing the processing if talking is detected (e.g. “ambient sound sensor digitized separately and subtracted in software from the sound sensor, or any other means of noise cancellation known in the art.”, para. [0121], [0260]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 2003/0179888 to Burnett et al. which discloses voice activity detection for use with noise suppression. US 2015/0190110 to Chong et al. which discloses an electronic stethoscope which utilizes a microphone to detect audio sounds in which the microphone has a low power or standby state that transitions to a higher power state using an audio wake-up signal.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNIFER L GHAND whose telephone number is (571)270-5844. The examiner can normally be reached Mon-Fri 7:30AM - 3:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER MCDONALD can be reached at (571)270-3061. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNIFER L GHAND/Examiner, Art Unit 3796
Prosecution Insights